Skip to content

Add per-resource-type state sizes and action counts to deploy telemetry#4629

Open
shreyas-goenka wants to merge 1 commit intomainfrom
deploy-telemetry
Open

Add per-resource-type state sizes and action counts to deploy telemetry#4629
shreyas-goenka wants to merge 1 commit intomainfrom
deploy-telemetry

Conversation

@shreyas-goenka
Copy link
Contributor

@shreyas-goenka shreyas-goenka commented Mar 2, 2026

Summary

  • Add resource_state_size_bytes experimental telemetry field tracking per-resource-type config sizes in bytes, computed from the bundle YAML config (engine-independent, consistent across terraform and direct)
  • Add resource_action_counts experimental telemetry field tracking per-resource-type CRUD action counts from the deploy plan (e.g., jobs.create, pipelines.update, permissions.create)

Test plan

  • make build passes
  • make lintfull passes (0 issues)
  • All telemetry acceptance tests pass for both engine variants (go test ./acceptance -run TestAccept/bundle/telemetry -v)
  • Unit tests pass (go test ./bundle/phases/...)
  • Both engines produce identical resource_state_size_bytes values (config-based, not state-file-based)

🤖 Generated with Claude Code

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 2, 2026

Commit: d04bdb0

Run: 22615520832

Env 🪲​BUG 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🪲​ aws linux 5 7 1 7 263 775 9:22
🪲​ aws windows 5 7 1 7 265 773 8:51
🪲​ aws-ucws linux 5 1 4 7 7 355 691 11:22
🪲​ aws-ucws windows 5 1 7 7 361 689 9:37
🪲​ azure linux 5 1 1 9 266 773 9:36
🪲​ azure windows 5 1 1 9 268 771 7:20
🪲​ azure-ucws linux 5 1 1 9 364 687 9:51
🪲​ azure-ucws windows 5 1 1 9 366 685 9:21
🪲​ gcp linux 5 1 1 9 262 776 8:44
🪲​ gcp windows 5 1 1 9 264 774 7:49
24 interesting tests: 7 KNOWN, 7 SKIP, 5 BUG, 4 flaky, 1 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
🪲​ TestAccept/bundle/resources/dashboards/generate_inplace 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
🪲​ TestAccept/bundle/resources/dashboards/generate_inplace/DATABRICKS_BUNDLE_ENGINE=direct 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
🪲​ TestAccept/bundle/resources/dashboards/generate_inplace/DATABRICKS_BUNDLE_ENGINE=terraform 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🪲​ TestDashboardAssumptions_WorkspaceImport 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
🔄​ TestSparkJarTaskDeployAndRunOnVolumes ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSparkJarTaskDeployAndRunOnVolumes/Databricks_Runtime_13.3_LTS 🙈​s 🙈​s 🔄​f ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_14.3_LTS ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🪲​ TestImportFileFormatSource 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B 🪲​B
Top 27 slowest tests (at least 2 minutes):
duration env testname
5:36 aws-ucws linux TestAccept/ssh/connection
5:15 aws-ucws windows TestAccept/ssh/connection
4:37 azure windows TestSecretsPutSecretStringValue
4:04 azure-ucws windows TestAccept/ssh/connection
3:48 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:45 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:39 azure-ucws linux TestAccept/ssh/connection
3:39 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:13 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:08 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:07 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:02 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:46 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:45 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:44 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:43 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:43 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:42 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:42 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:37 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:26 gcp windows TestSecretsPutSecretStringValue
2:16 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:15 aws windows TestSecretsPutSecretStringValue
2:13 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:10 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:04 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

Track two new experimental metrics in BundleDeployEvent:
- resource_state_size_bytes: per-resource-type config sizes computed from
  the bundle YAML config (engine-independent)
- resource_action_counts: per-resource-type CRUD action counts from deploy
  plan, including permissions and grants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@andrewnester andrewnester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify the reasoning why we need to track these fields?

jobs:
job_one:
name: job one
permissions:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants