Skip to content

feat: Extend \dt psql command output with shard metadata (#709)#812

Open
Adi-Goll wants to merge 1 commit intopgdogdev:mainfrom
Adi-Goll:adigoll-dt-intercept-shard
Open

feat: Extend \dt psql command output with shard metadata (#709)#812
Adi-Goll wants to merge 1 commit intopgdogdev:mainfrom
Adi-Goll:adigoll-dt-intercept-shard

Conversation

@Adi-Goll
Copy link
Contributor

@Adi-Goll Adi-Goll commented Mar 5, 2026

This PR intercepts the psql \dt command, and add a Shard column to the output.

First a flag to Route that indicates if \dt is being executed so the Shard column is conditionally applied.
A shard_map HashMap is also added to Route that stores table names with their corresponding shard.

This PR also introduces a forward_with_shard() function in backend/pool/connection/binding.rs that
exposes the shard_map property which provides the correct shards be streamed from the query engine.

Finally engine logic was added to populate the new column correctly and handle tables sharded
across multiple databases. The following output uses this config file for testing:

pgdog.toml
[general]
openmetrics_port = 9090
port = 6432
host = "127.0.0.1"
system_catalogs = "sharded"

[[databases]]
name = "mycluster"
database_name = "shard_db_0"
host = "127.0.0.1"
port = 5432
shard = 0

[[databases]]
name = "mycluster"
database_name = "shard_db_1"
host = "127.0.0.1"
port = 5432
shard = 1

[[databases]]
name = "mycluster"
database_name = "shard_db_2"
host = "127.0.0.1"
port = 5432
shard = 2

[[sharded_tables]]
name = "users"
database = "mycluster"
column = "user_id"
data_type = "bigint"
Before this PR:
           List of tables
 Schema |   Name    | Type  | Owner
--------+-----------+-------+--------
 public | only_on_0 | table | ubuntu
 public | only_on_1 | table | ubuntu
 public | only_on_2 | table | ubuntu
 public | users     | table | ubuntu
 public | users     | table | ubuntu
 public | users     | table | ubuntu

Note, the same users table was created on multiple databases, so some deduplication logic was added during streaming, done through an added HashSet property in the QueryEngineContext.
After this PR:

Ex. output:
                List of tables
 Schema |   Name    | Type  | Owner  |  Shard
--------+-----------+-------+--------+---------
 public | only_on_0 | table | ubuntu | 0
 public | only_on_1 | table | ubuntu | 1
 public | only_on_2 | table | ubuntu | 2
 public | users     | table | ubuntu | 0, 1, 2

self.seen_tables.insert(table_lookup.to_string());

let mut new_col = String::new();
for (i, val) in map[table_lookup].iter().enumerate() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking, maybe it would be better to have a more unique key for the HashMap, maybe instead of just the table name I should use the schema + table name? Thoughts?

pending_explain: None,
begin_stmt: None,
router: Router::default(),
seen_tables: HashSet::new(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's reasonable to add this for \dt because the original command doesn't do any aggregating or deduplication handling

@Adi-Goll Adi-Goll force-pushed the adigoll-dt-intercept-shard branch from f94ae29 to 23d3571 Compare March 7, 2026 00:22
)

Intercept \dt command, and add a `Shard`
column to the output.

Add a flag to `Route` that indicates if
\dt is being executed so the Shard column
is conditionally applied. Add `shard_map`
HashMap to `Route` as well that stores
tables with their corresponding shard.

Introduce `forward_with_shard` function in
backend/pool/connection/binding.rs that
exposes the shard_map property to be
streamed in the query engine.

Add engine logic to populate the new
column correctly and handle tables sharded
across multiple databases

Ex. output:
                List of tables
 Schema |   Name    | Type  | Owner  |  Shard
--------+-----------+-------+--------+---------
 public | only_on_0 | table | ubuntu | 0
 public | only_on_1 | table | ubuntu | 1
 public | only_on_2 | table | ubuntu | 2
 public | users     | table | ubuntu | 0, 1, 2

Signed-off-by: Aditya Gollamudi <adigollamudi@gmail.com>
@Adi-Goll Adi-Goll force-pushed the adigoll-dt-intercept-shard branch from 23d3571 to dcbd665 Compare March 10, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant