Mark hypertable and chunk as user catalog tables

[zilder]

Mark tables _timescaledb_catalog.hypertable and _timescaledb_catalog.chunk with WITH (user_catalog_table = true) (see doc) so they can be accessed during logical decoding using historic snapshot. This is required for a consistent view of timescaledb catalog from logical decoding plugins.

[github-actions]

@dbeck, @melihmutlu: please review this pull request.

Powered by pull-review

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[svenklemm]

What do you intent to use this for? Why only those 2 tables?

[svenklemm]

I think we need some serious testing to check the implications of this change. This looks innocent at first glance but does have implications.

  Setting ALTER TABLE foo SET (user_catalog_table = true):

  1. WAL logging changes: The table gets full-page WAL logging during checkpoints, same as system catalog tables. This means more WAL volume but better crash safety.
  2. pg_dump behavior: The table is treated as a catalog for dump purposes — its schema is dumped but not its data by default (like pg_class itself).
  3. MVCC snapshot behavior: Queries on the table use SnapshotNow-like catalog snapshot semantics in some code paths, meaning they can see recently committed rows even within a transaction that started earlier.
  4. Vacuum/freeze behavior: The table follows system catalog freezing rules — it may be frozen more aggressively (controlled by vacuum_freeze_min_age catalog defaults).
  5. Cache invalidation: Does not automatically participate in syscache invalidation — that's only for actual system catalogs.
  6. Logical replication: The table may be excluded from logical replication since it's treated as a catalog table rather than user data.
  7. During Hot Standby, writes to user catalog tables are blocked just like system catalogs — the standby treats them as read-only catalog data.

[zilder]

Hi @svenklemm,

What do you intent to use this for? Why only those 2 tables?

I'm working on a custom timescaledb-aware logical decoding plugin based on pgoutput that:
a) writes all DML operations on chunks as if they were performed on the hypertable (consumers don't have to treat hypertables differently from regular vanilla tables);
b) unwrap operations on compressed chunks (direct inserts, complete batch deletes) and present them as a set of regular INSERTs/DELETEs

I have a functional PoC.
I'm using chunk and hypertable catalog tables to attribute an operation on a chunk or on a compressed chunk to its hypertable. Setting user_catalog_table = true allows to get a consistent view of catalog tables using historical snapshot. E.g. a chunk had been dropped before we decoded a DML operation on that chunk. This setting prevents vacuum from removing dead tuples from that catalog table that are younger than catalog_xmin of replication slots.

Will need more time to go though your list.

6. Logical replication: The table may be excluded from logical replication ...

I cannot confirm this. I was able to see all catalog modifications in test_decoding stream:

create table test2 (key int, val int, device_id int);
select create_hypertable('test2', by_range('key', 2000));
insert into test2 values (1, 1, 1);

    lsn     |  xid  |                                                                                                                                                                                                                                      data                                                                                                                                                                                                        
------------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 0/2B9B6EA0 | 14812 | BEGIN
 0/2B9BA588 | 14812 | COMMIT
 0/2B9BA588 | 14813 | BEGIN
 0/2B9BA5F8 | 14813 | table _timescaledb_catalog.hypertable: INSERT: id[integer]:21 schema_name[name]:'public' table_name[name]:'test2' associated_schema_name[name]:'_timescaledb_internal' associated_table_prefix[name]:'_hyper_21' num_dimensions[smallint]:1 chunk_sizing_func_schema[name]:'_timescaledb_functions' chunk_sizing_func_name[name]:'calculate_chunk_interval' chunk_target_size[bigint]:0 compression_state[smallint]:0 compressed_hypertable_id[integer]:null status[integer]:0
 0/2B9BBCE8 | 14813 | table _timescaledb_catalog.dimension: INSERT: id[integer]:13 hypertable_id[integer]:21 column_name[name]:'key' column_type[regtype]:'integer' aligned[boolean]:true num_slices[smallint]:null partitioning_func_schema[name]:null partitioning_func[name]:null interval_length[bigint]:2000 compress_interval_length[bigint]:null integer_now_func_schema[name]:null integer_now_func[name]:null
 0/2B9BC9E8 | 14813 | COMMIT
 0/2B9BC9E8 | 14814 | BEGIN
 0/2B9BC9E8 | 14814 | table _timescaledb_catalog.dimension_slice: INSERT: id[integer]:11 dimension_id[integer]:13 range_start[bigint]:0 range_end[bigint]:2000
 0/2B9BE890 | 14814 | table _timescaledb_catalog.chunk: INSERT: id[integer]:23 hypertable_id[integer]:21 schema_name[name]:'_timescaledb_internal' table_name[name]:'_hyper_21_23_chunk' compressed_chunk_id[integer]:null status[integer]:0 osm_chunk[boolean]:false creation_time[timestamp with time zone]:'2026-03-19 14:52:26.618244+01'
 0/2B9BEB10 | 14814 | table _timescaledb_catalog.chunk_constraint: INSERT: chunk_id[integer]:23 dimension_slice_id[integer]:11 constraint_name[name]:'constraint_11' hypertable_constraint_name[name]:null
 0/2B9BFFF0 | 14814 | table _timescaledb_internal._hyper_21_23_chunk: INSERT: key[integer]:1 val[integer]:1 device_id[integer]:1
 0/2B9C0458 | 14814 | COMMIT
(12 rows)

7. During Hot Standby, writes to user catalog tables are blocked just like system catalogs — the standby treats them as read-only catalog data.

Not sure I understand the problem. The writes are blocked on standby either way for all tables (with exception for hint bits probably).

[zilder]

I took a deeper look into the postgres code regarding user_catalog_table setting. user_catalog_table setting is used through RelationIsAccessibleInLogicalDecoding() and RelationIsUsedAsCatalogTable() macros. What's affected by it:

1. Vacuum, dead tuples pruning, freezing

The only difference between regular tables and catalog tables is that when tuple visibility is calculated the slot's catalog_xmin is taken into account. In practice it means that dead tuples will not be removed until replication slots advance past certain XID.
See the GlobalVisState struct, functions like vacuum_get_cutoffs() (using GetOldestNonRemovableTransactionId()), heap_page_prune_opt() (using GlobalVisTestFor()).

So the vacuum/freeze behavior for catalog tables is actually more conservative than for regular tables.

2. WAL
   It seems in all cases it only affects logical decoding

• It writes extra combo CID WAL records with log_heap_new_cid() (see heapam.c)
• Preserves some extra information in WAL records for user catalog tables to enable logical decoding on standbys. See comment: 6af1793954e8. E.g.
  xl_heap_prune, spgxlogVacuumRedirect, xl_hash_vacuum_one_page, xl_hash_vacuum_one_page, xl_btree_reuse_page etc

	bool		isCatalogRel;	/* to handle recovery conflict during logical
								 * decoding on standby */

I couldn't find anything criminal there, just extra logical decoding related stuff.

4. ALTER TABLE commands requiring table rewrite

This is an actual limitation. Commands like ADD COLUMN ... GENERATED (...) STORED, ALTER COLUMN ... TYPE, SET TABLESPACE would throw an error:

alter table test_catalog alter column val type int8;
ERROR:  cannot rewrite table "test_catalog" used as a catalog table

As far as I understand (at least what I've seen in the past) when a tsdb catalog table requires changes, the table is recreated in the upgrade script, not altered. So shouldn't be a problem?

5. ON CONFLICT clause not supported.

Example:

insert into test_catalog values (1, 50) on conflict (id) do update set val = EXCLUDED.val;
ERROR:  ON CONFLICT is not supported on table "test_catalog" used as a catalog table

[zilder]

2. pg_dump behavior: The table is treated as a catalog for dump purposes — its schema is dumped but not its data by default (like pg_class itself).

Cannot confirm this either. Here's an excerpt from a pg_dump's output (user_catalog_table = true is set for hypertable and chunk):

--
-- Data for Name: hypertable; Type: TABLE DATA; Schema: _timescaledb_catalog; Owner: zilder
--

COPY _timescaledb_catalog.hypertable (id, schema_name, table_name, associated_schema_name, associated_table_prefix, num_dimensions, chunk_sizing_func_schema, chunk_sizing_func_name, chunk_target_size, compression_state, compressed_hypertable_id, status) FROM stdin;
17	_timescaledb_internal	_compressed_hypertable_17	_timescaledb_internal	_hyper_17	0	_timescaledb_functions	calculate_chunk_interval	0	2	\N	0
16	public	test	_timescaledb_internal	_hyper_16	1	_timescaledb_functions	calculate_chunk_interval	0	1	17	0
21	public	test2	_timescaledb_internal	_hyper_21	1	_timescaledb_functions	calculate_chunk_interval	0	0	\N	0
\.


--
-- Data for Name: bgw_job; Type: TABLE DATA; Schema: _timescaledb_catalog; Owner: zilder
--

COPY _timescaledb_catalog.bgw_job (id, application_name, schedule_interval, max_runtime, max_retries, retry_period, proc_schema, proc_name, owner, scheduled, fixed_schedule, initial_start, hypertable_id, config, check_schema, check_name, timezone) FROM stdin;
\.


--
-- Data for Name: chunk; Type: TABLE DATA; Schema: _timescaledb_catalog; Owner: zilder
--

COPY _timescaledb_catalog.chunk (id, hypertable_id, schema_name, table_name, compressed_chunk_id, status, osm_chunk, creation_time) FROM stdin;
20	17	_timescaledb_internal	compress_hyper_17_20_chunk	\N	0	f	2026-03-19 11:51:35.920006+01
15	16	_timescaledb_internal	_hyper_16_15_chunk	20	9	f	2026-03-02 17:27:41.149408+01
23	21	_timescaledb_internal	_hyper_21_23_chunk	\N	0	f	2026-03-19 14:52:26.618244+01
\.