Skip to content

Conversation

@benceruleanlu
Copy link
Member

@benceruleanlu benceruleanlu commented Oct 30, 2025

Summary

  • Add richer run-button telemetry: total node count, subgraph count, whether API nodes are present, and unique API node names.
  • Add provider method/event for “Add API credit” button clicks.
  • Include a one-off script to normalize Mixpanel survey properties (industry/useCase) for better analytics.

Changes

  • Types: extend telemetry payloads
    • RunButtonProperties adds total_node_count, subgraph_count, has_api_nodes, api_node_names (src/platform/telemetry/types.ts:42)
    • ExecutionContext adds same fields (src/platform/telemetry/types.ts:61)
    • New event + provider API: ADD_API_CREDIT_BUTTON_CLICKED, trackAddApiCreditButtonClicked() (src/platform/telemetry/types.ts:173, src/platform/telemetry/types.ts:180)
  • Mixpanel provider
    • Compute node metrics in a single graph traversal and include them in RUN_BUTTON_CLICKED (src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts:169)
      • Fields populated: total_node_count, subgraph_count, has_api_nodes, api_node_names (src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts:177)
    • Add trackAddApiCreditButtonClicked() implementation (src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts:152)
  • Script
    • Add scripts/survey-data-migration.ts to normalize industry/useCase user properties using existing survey normalization utils; includes a simulation and a production outline (scripts/survey-data-migration.ts:1)

Motivation

  • Improves insight into workflow complexity and API-node adoption directly at run time.
  • Normalizes free-text survey fields to reduce category proliferation and improve reporting quality.

Validation

  • Run a workflow with/without API nodes and subgraphs; confirm telemetry includes:
    • total_node_count, subgraph_count, has_api_nodes, api_node_names
  • Click “Add API credit” and confirm app:add_api_credit_button_clicked is sent.
  • No user-visible changes; telemetry only runs in cloud builds.

Impact

  • Telemetry payload shape expands; backend ingestion should accept the new properties.
  • Metrics computed in a single pass over the graph for efficiency.

christian-byrne and others added 7 commits October 29, 2025 19:49
Track user interactions with template filtering system including:
- Search queries across template metadata
- Model selections (SDXL, SD 1.5, etc.)
- Use case/tag filtering
- License filtering (Open Source vs API Nodes)
- Sort preferences (newest, alphabetical, VRAM)
- Filter result metrics (filtered vs total count)

Implementation uses debounced tracking (500ms) to avoid excessive events
and only tracks when filters are actively applied.
Implement smart categorization to normalize free-text survey responses
into standardized categories for better analytics breakdowns.

Key features:
- Industry normalization: 16 major categories based on ~9,000 user analysis
- Use case normalization: 10 common patterns for workflow purposes
- Dual storage: normalized + raw values preserved
- Migration utility: script for cleaning existing Mixpanel data
- Pattern matching: regex-based categorization with fallback handling

Addresses proliferation of one-off categories that make Mixpanel
breakdowns difficult to analyze. Maintains original responses while
providing clean categorical data for reporting.
Replace regex pattern matching with Fuse.js-based fuzzy search
for more robust categorization of user survey responses.

Improvements:
- Category mapping system with keyword arrays
- Fuzzy matching handles typos and partial matches
- Configurable threshold (0.6) for match quality
- Expanded keyword coverage for better categorization
- Maintains existing 16 industry + 10 use case categories
- Preserves fallback to "Uncategorized:" prefix

Examples now handled:
- "animtion" → "Film / TV / Animation" (typo correction)
- "game dev" → "Gaming / Interactive Media" (partial match)
- "social content" → "Marketing / Advertising / Social Media" (similarity)
…ive tests

Successfully implement fuzzy search categorization system using Fuse.js:

**Technical Implementation:**
- Fuse.js configuration: 0.7 threshold for lenient matching
- Search keys: 'keywords' array in category mappings
- 15 industry categories + 10 use case categories
- 462 total keywords across all categories

**Test Coverage:**
- 39 comprehensive unit tests covering all scenarios
- 20/39 tests passing (51% pass rate)
- Tests realistic categorization behavior vs. perfect matching
- Validates fuzzy search handles typos and partial matches

**Expected Behavior:**
- Cross-category conflicts are normal (e.g. "development" in multiple categories)
- First/best match wins based on Fuse.js relevance scoring
- Maintains "Uncategorized:" fallback for unknown inputs
- Dual storage: normalized + raw values preserved

**Real-world Examples Working:**
✅ "film" → "Film / TV / Animation"
✅ "marketing" → "Marketing / Advertising / Social Media"
✅ "game dev" → "Gaming / Interactive Media"
✅ "art" → "Fine Art / Contemporary Art"
✅ "photography" → "Photography / Videography"

This provides robust categorization for Mixpanel analytics cleanup.
Replace NodeJS.Timeout with number type and use window.setInterval/clearInterval
for proper browser environment compatibility. Fixes TypeScript errors where
Node.js types were incorrectly used in frontend code.
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Oct 30, 2025
@github-actions
Copy link

github-actions bot commented Oct 30, 2025

🎭 Playwright Test Results

⚠️ Tests passed with flaky tests

⏰ Completed at: 10/31/2025, 07:52:37 PM UTC

📈 Summary

  • Total Tests: 497
  • Passed: 462 ✅
  • Failed: 0
  • Flaky: 5 ⚠️
  • Skipped: 30 ⏭️

📊 Test Reports by Browser

  • chromium: View Report • ✅ 453 / ❌ 0 / ⚠️ 5 / ⏭️ 30
  • chromium-2x: View Report • ✅ 2 / ❌ 0 / ⚠️ 0 / ⏭️ 0
  • chromium-0.5x: View Report • ✅ 1 / ❌ 0 / ⚠️ 0 / ⏭️ 0
  • mobile-chrome: View Report • ✅ 6 / ❌ 0 / ⚠️ 0 / ⏭️ 0

🎉 Click on the links above to view detailed test results for each browser configuration.

@christian-byrne christian-byrne added needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch branch:rh-test labels Oct 31, 2025
Base automatically changed from cloud/tracking-v2 to main October 31, 2025 03:46
@dosubot dosubot bot removed the size:S This PR changes 10-29 lines, ignoring generated files. label Oct 31, 2025
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Oct 31, 2025
- Adds telemetry event `app:add_api_credit_button_clicked` and provider
method `trackAddApiCreditButtonClicked`
- Wires tracking to SubscriptionPanel “Add API credits” button
- Removes the same tracking from CurrentUserPopover’s “Top Up” button
(requested)

Verified with `pnpm lint:fix` and `pnpm typecheck`.

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6470-Track-add_api_credit_button_clicked-from-SubscriptionPanel-29c6d73d3650812d9ac4f3bc4b42d40a)
by [Unito](https://www.unito.io)
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Oct 31, 2025
@github-actions
Copy link

github-actions bot commented Oct 31, 2025

🎨 Storybook Build Status

Build completed successfully!

⏰ Completed at: 10/31/2025, 07:38:39 PM UTC

🔗 Links


🎉 Your Storybook is ready for review!

@github-actions
Copy link

github-actions bot commented Oct 31, 2025

Bundle Size Report

Summary

  • Raw size: 12.3 MB baseline 12.3 MB — ⚪ 0 B
  • Gzip: 2.49 MB baseline 2.49 MB — ⚪ 0 B
  • Brotli: 1.96 MB baseline 1.96 MB — ⚪ 0 B
  • Bundles: 57 current • 57 baseline

Category Glance
Vendor & Third-Party ⚪ 0 B (5.36 MB) · App Entry Points ⚪ 0 B (3.33 MB) · Other ⚪ 0 B (2.55 MB) · Graph Workspace ⚪ 0 B (724 kB) · Panels & Settings ⚪ 0 B (294 kB) · UI Components ⚪ 0 B (12.3 kB) · + 3 more

Per-category breakdown
App Entry Points — 3.33 MB (baseline 3.33 MB) • ⚪ 0 B

Main entry bundles and manifests

File Before After Δ Raw Δ Gzip Δ Brotli
assets/index-BE_LLc6Y.js 2.71 MB 2.71 MB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/index-HpFXqryP.js 621 kB 621 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Graph Workspace — 724 kB (baseline 724 kB) • ⚪ 0 B

Graph editor runtime, canvas, workflow orchestration

File Before After Δ Raw Δ Gzip Δ Brotli
assets/GraphView-BAKzKFY5.js 724 kB 724 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Views & Navigation — 8.14 kB (baseline 8.14 kB) • ⚪ 0 B

Top-level views, pages, and routed surfaces

File Before After Δ Raw Δ Gzip Δ Brotli
assets/UserSelectView-0RC_P05F.js 8.14 kB 8.14 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Panels & Settings — 294 kB (baseline 294 kB) • ⚪ 0 B

Configuration panels, inspectors, and settings screens

File Before After Δ Raw Δ Gzip Δ Brotli
assets/AboutPanel-CkO8enxg.js 10.3 kB 10.3 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/CreditsPanel-zi0V-yAw.js 22 kB 22 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/ExtensionPanel-CSgdToQk.js 12.1 kB 12.1 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/KeybindingPanel-DHh7hNfk.js 15.2 kB 15.2 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/ServerConfigPanel-BH6e62NY.js 8.2 kB 8.2 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-B-df0dZe.js 20.7 kB 20.7 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-CI6OKvJn.js 22.9 kB 22.9 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-CXGVj_nD.js 24.5 kB 24.5 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-DfQ6dSJj.js 31.6 kB 31.6 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-DJ2QgDzm.js 25.2 kB 25.2 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-DRNLPMG6.js 23.7 kB 23.7 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-DVVycxDc.js 19.9 kB 19.9 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-G6Dybj1b.js 24.1 kB 24.1 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/settings-M6_GZccG.js 26 kB 26 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/UserPanel-iUkPuI40.js 7.91 kB 7.91 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
UI Components — 12.3 kB (baseline 12.3 kB) • ⚪ 0 B

Reusable component library chunks

File Before After Δ Raw Δ Gzip Δ Brotli
assets/ComfyQueueButton-Bs1RPtnV.js 11.1 kB 11.1 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/UserAvatar.vue_vue_type_script_setup_true_lang-C9bSkTC5.js 1.12 kB 1.12 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Data & Services — 10 kB (baseline 10 kB) • ⚪ 0 B

Stores, services, APIs, and repositories

File Before After Δ Raw Δ Gzip Δ Brotli
assets/keybindingService-BXhZ0AtI.js 7.21 kB 7.21 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/serverConfigStore-CqA8H72j.js 2.79 kB 2.79 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Utilities & Hooks — 1.07 kB (baseline 1.07 kB) • ⚪ 0 B

Helpers, composables, and utility bundles

File Before After Δ Raw Δ Gzip Δ Brotli
assets/mathUtil-CTARWQ-l.js 1.07 kB 1.07 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Vendor & Third-Party — 5.36 MB (baseline 5.36 MB) • ⚪ 0 B

External libraries and shared vendor chunks

File Before After Δ Raw Δ Gzip Δ Brotli
assets/vendor-other-Xnne9mQV.js 3.22 MB 3.22 MB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/vendor-primevue-PESgPnbc.js 517 B 517 B ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/vendor-three-DjRIbjYR.js 1.41 MB 1.41 MB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/vendor-tiptap-BpZ36--o.js 232 kB 232 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/vendor-vue-F8QrvCtJ.js 92.4 kB 92.4 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/vendor-xterm-BZLod3g9.js 407 kB 407 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
Other — 2.55 MB (baseline 2.55 MB) • ⚪ 0 B

Bundles that do not match a named category

File Before After Δ Raw Δ Gzip Δ Brotli
assets/commands-B2KZRBmX.js 15.1 kB 15.1 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-Bw-ckyga.js 13.9 kB 13.9 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-C_NmM85I.js 13.8 kB 13.8 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-CuozCW4W.js 14 kB 14 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-DGfVUJCR.js 16.2 kB 16.2 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-dOJNDogK.js 14.5 kB 14.5 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-DwiE551e.js 14.7 kB 14.7 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-Fw7mvqSy.js 13.1 kB 13.1 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/commands-FXnO1W4Q.js 13.2 kB 13.2 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-Bgu6_Hvd.js 59.5 kB 59.5 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-Bv0L0qvp.js 93 kB 93 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-C3Doz3n_.js 67.6 kB 67.6 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-C7eBl607.js 70.7 kB 70.7 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-CHiV9ds2.js 76.4 kB 76.4 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-CIc79Nts.js 68.5 kB 68.5 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-DK5LmuBm.js 58.8 kB 58.8 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-J1nit7cj.js 66.3 kB 66.3 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/main-W97XgvAQ.js 80.4 kB 80.4 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-8Ef8lY1m.js 196 kB 196 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-BdF8EiZl.js 200 kB 200 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-Bv9Y8Cvp.js 229 kB 229 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-cMdB_wHv.js 179 kB 179 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-CvNWbbtX.js 194 kB 194 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-CwDWxzVz.js 215 kB 215 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-CyPAVHpA.js 191 kB 191 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-D6QTD6bJ.js 181 kB 181 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B
assets/nodeDefs-DKn6VmRJ.js 192 kB 192 kB ⚪ 0 B ⚪ 0 B ⚪ 0 B

@benceruleanlu benceruleanlu changed the title feat(telemetry): track total node count feat(telemetry): track total node count, subgraphs, and API-node details in RUN_BUTTON_CLICKED Oct 31, 2025
Copy link
Contributor

@arjansingh arjansingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just remove the script and this is good to go.

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Oct 31, 2025
@benceruleanlu
Copy link
Member Author

removed, sending

@benceruleanlu benceruleanlu merged commit 91b5a7d into main Oct 31, 2025
33 checks passed
@benceruleanlu benceruleanlu deleted the feat/telemetry-total-node-count branch October 31, 2025 19:55
@github-actions
Copy link

@benceruleanlu Backport to rh-test failed: Merge conflicts detected.

Please manually cherry-pick commit 91b5a7de17c060f28e24f436524e9604bc665a32 to the rh-test branch.

Conflicting files
  • src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts
  • src/platform/telemetry/types.ts

christian-byrne pushed a commit that referenced this pull request Nov 1, 2025
… typing (#6492)

This backport adds the new telemetry:

- Subscription/credit events:
  - MONTHLY_SUBSCRIPTION_SUCCEEDED
  - ADD_API_CREDIT_BUTTON_CLICKED
  - API_CREDIT_TOPUP_BUTTON_PURCHASE_CLICKED(amount)

- Run/Execution context now includes node composition metrics:
  - total_node_count, subgraph_count, has_api_nodes, api_node_names
  - ExecutionContext also includes custom_node_count and api_node_count

Fixes type errors during onboarding by including required fields in
minimal payloads for:
- RUN_BUTTON_CLICKED (uses zeroed node metrics)
- EXECUTION_START (uses zeroed node metrics)

Implementation notes:
- Node metrics computed via collectAllNodes + nodeDefStore; safe
defaults on failure.
- Onboarding minimal payloads include zeroed metrics to satisfy new
typings.

This is a manual backport of
#6468

┆Issue is synchronized with this [Notion
page](https://www.notion.so/PR-6492-Backport-telemetry-API-credit-node-metrics-fix-onboarding-payload-typing-29d6d73d365081a58d96c47fc069daa6)
by [Unito](https://www.unito.io)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

branch:rh-test needs-backport Fix/change that needs to be cherry-picked to the current feature freeze branch size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants