Skip to content

Add output formats, interactive table browser, and multi-chunk results#4646

Open
simonfaltum wants to merge 4 commits intomainfrom
simonfaltum/query-output-formats
Open

Add output formats, interactive table browser, and multi-chunk results#4646
simonfaltum wants to merge 4 commits intomainfrom
simonfaltum/query-output-formats

Conversation

@simonfaltum
Copy link
Member

@simonfaltum simonfaltum commented Mar 3, 2026

Why

The aitools tools query command outputs results as JSON to stderr via cmdio.LogString. This means output can't be piped, there's no tabular display for interactive use, and large multi-chunk results are truncated to the first chunk.

Changes

Before: always JSON to stderr, single chunk only.
Now: format-aware output routing with interactive table browser.

Output routing:

  • Interactive terminal (stdout is TTY): table display
    • Small results (<=30 rows): static formatted table, prints and exits
    • Large results (>30 rows): interactive browser with scrolling and search
  • Non-interactive (piped stdout) or --output json: JSON array of objects (backwards compatible)
  • All output goes to stdout, not stderr

Interactive table browser (libs/tableview/):
Reusable shared component using bubbles/viewport. Features:

  • Horizontal + vertical scrolling (arrow keys, pgup/pgdn)
  • Cursor row highlighting (purple background)
  • / search with yellow match highlighting, n/N to navigate matches
  • g/G for top/bottom, q to quit
  • Footer with row count, scroll position, keybinding hints

Multi-chunk fetching:
Queries returning more data than fits in a single response chunk now fetch all chunks via GetStatementResultChunkN and combine them.

Screen.Recording.2026-03-03.at.17.46.00.mov
Screen.Recording.2026-03-03.at.17.43.34.mov

Test plan

  • Unit tests for libs/tableview (table rendering, search, highlighting)
  • Unit tests for renderers (JSON format, static table, extractColumns)
  • Unit tests for fetchAllRows (single chunk, multi-chunk, nil result)
  • All existing query tests pass (polling, cancellation, input resolution)
  • make lintfull clean
  • make checks clean
  • Manual: interactive table with scrolling and search
  • Manual: | cat produces JSON
  • Manual: --output json forces JSON

… results

Replace the single JSON-to-stderr output with format-aware routing:
- Interactive terminal: table browser (viewport-based, with search)
- Non-interactive / --output json: JSON array of objects
- Small results (<=30 rows): static formatted table

Add libs/tableview, a reusable interactive table browser component
using bubbles/viewport. Features: horizontal + vertical scrolling,
cursor row highlighting, / search with match navigation (n/N),
g/G for top/bottom.

Add multi-chunk result fetching via GetStatementResultChunkN for
queries returning more data than fits in a single response chunk.
Avoid opening the interactive table browser when stdin cannot accept input by falling back to static tables, and add output routing tests with updated command help text.
Fix a bug where search highlighting on the cursor line would fail
because cursorStyle.Render() wrapped the line in ANSI codes before
highlightSearch tried to match against it. Now search highlighting
is applied first on clean text, then cursor style wraps the result.

Also merge two const blocks and derive totalRows from lines instead
of storing it as redundant state.
@simonfaltum simonfaltum marked this pull request as ready for review March 3, 2026 17:00
@simonfaltum simonfaltum requested review from a team and lennartkats-db as code owners March 3, 2026 17:00
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 3, 2026

Commit: 7172334

Run: 22633969822

Env 🪲​BUG ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🪲​ aws linux 1 7 2 1 7 265 776 8:19
🪲​ aws windows 1 7 3 1 7 266 774 8:23
🪲​ aws-ucws linux 1 2 7 7 363 691 8:43
🪲​ aws-ucws windows 1 2 7 7 365 689 6:28
🪲​ azure linux 2 8 1 1 9 261 774 146:32
🪲​ azure windows 2 8 1 1 9 263 772 145:51
🪲​ azure-ucws linux 2 8 1 2 9 359 687 148:29
🪲​ azure-ucws windows 2 6 1 4 9 361 685 81:52
🪲​ gcp linux 2 8 1 1 9 257 777 145:15
🪲​ gcp windows 2 8 1 1 9 259 775 145:07
29 interesting tests: 8 FAIL, 7 KNOWN, 7 SKIP, 5 flaky, 2 BUG
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 🔄​f 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🪲​ TestAccept/bundle/deployment/bind/alert 🙈​s 🙈​s 🙈​s 🙈​s 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
❌​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/deployment/bind/alert/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F 🔄​f ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F 🔄​f ❌​F ❌​F
❌​ TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/alerts/with_file/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/ssh/connection 💚​R 💚​R 🔄​f 💚​R 💚​R 🔄​f 🔄​f 🔄​f 💚​R 💚​R
🔄​ TestGenerateFromExistingPipelineAndDeploy ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_14.3_LTS 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS 🔄​f 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🪲​ TestImportFileFormatSource 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
Top 21 slowest tests (at least 2 minutes):
duration env testname
5:06 azure-ucws windows TestAccept/bundle/generate/alert/DATABRICKS_BUNDLE_ENGINE=direct
3:39 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:20 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:16 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:13 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:12 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:06 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:52 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:50 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:48 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:45 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:45 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:44 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:40 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:37 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:15 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:14 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:04 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants