Skip to content

Feature: add \shards command to track relocations#494

Open
mannreis wants to merge 14 commits intocrate:mainfrom
mannreis:main
Open

Feature: add \shards command to track relocations#494
mannreis wants to merge 14 commits intocrate:mainfrom
mannreis:main

Conversation

@mannreis
Copy link
Copy Markdown

Summary of the changes / Why this is an improvement

As requested in #493 this PR aims adds a new command \shards to conveniently query for shards undergoing relocation. It computes the the aggregated progress per table based on the sum of the sizes of relocating shards divided by the total size of the table shards.

Optionally, state and relocating can be provided as an argument to respectively:

  • get a short summary of shards aggregated by state
  • view details of which ones are currently relocating

Integration testing

I resorted to docker compose and scripting to move shards withing a cluster in order to test the what's implemented here. In order integrate that type of testing it in this code base, I spent sometime looking at cratedb_toolkit.testing.testcontainers.cratedb but I believe it would require another PR. For now, I resort to single-node integration testing.

Checklist

@amotl amotl requested review from mfussenegger and seut March 24, 2026 23:05
Copy link
Copy Markdown
Member

@seut seut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some suggestions related to the used statements, otherwise looks fine.
I had a bit in mind that having some ongoing progress visible with some kind of progress bar, which one could get out e.g. by ctrl-c would be awesome, but also probably better to work on this as a followup, now that you've done the ground work :)

About real integration testing: Right, this would require at least 2 nodes to make it work. Utilizing the testcontainers to make this work would be really great, but agree, lets look into this as followup as well.

@seut
Copy link
Copy Markdown
Member

seut commented Apr 2, 2026

@mannreis any update on this, do you need any further help to continue?

@mannreis
Copy link
Copy Markdown
Author

mannreis commented Apr 2, 2026

Your suggestions were already a great help! I'll apply them early next week. They seem easy enough to have been applied already.

Regarding your original idea of an interactive progress bar. I saw that postgres cli had a \watch command that seemed useful and could be used for queries but I guess other commands as well.

mannreis and others added 8 commits April 6, 2026 12:24
Adds columns `schema_name` and `partition_ident` to the query and improves it by replacing `CASE WHEN` with `FILTER(WHERE ... )`

Co-authored-by: Sebastian Utz <su@rtme.net>
Add `primary` column to the shards query

Co-authored-by: Sebastian Utz <su@rtme.net>
 - table shards view available via "\shards info"
@mannreis mannreis requested a review from seut April 6, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants