Skip to content

Conversation

KSDaemon
Copy link
Member

@KSDaemon KSDaemon commented Aug 20, 2025

This PR attempts to improve python parser compilation flow speed by avoiding high-cost parsing/transpiling AST for obvious cases when it's possible to construct the resulting expression by hands.

Here is original measurement of 100+ yaml data model compilation time:

Compiling 110 files...

🚀 PERFORMANCE REPORT 🚀

══════════════════════════════════════════════════════════════════════════════════════════
⏱️  TOTAL TIME: 6877.32ms

  doCompile                                │  6877.32ms │ 6877.32ms avg │ 100.0% │    1 calls
  transpileYaml                            │  6553.95ms │   19.00ms avg │  95.3% │  345 calls
  compileYamlFile                          │  6355.80ms │   19.62ms avg │  92.4% │  324 calls
  PythonParser->transpileToJs()            │  6186.45ms │    0.19ms avg │  90.0% │ 31740 calls
  yaml-transpileAndPrepareJsFile           │  5275.99ms │   16.59ms avg │  76.7% │  318 calls
  parsePythonAndTranspileToJs call 225     │  4031.55ms │    0.15ms avg │  58.6% │ 26349 calls
  compilation-phase-0                      │  2215.13ms │ 2215.13ms avg │  32.2% │    1 calls
  parsePythonAndTranspileToJs call 301     │  2180.16ms │    0.37ms avg │  31.7% │ 5877 calls
  compilation-phase-1                      │  2080.09ms │ 2080.09ms avg │  30.2% │    1 calls
  compilation-phase-3                      │  2064.41ms │ 2064.41ms avg │  30.0% │    1 calls
  asyncModules.reduce (jinja)              │   411.26ms │  137.09ms avg │   6.0% │    3 calls
  astIntoArrowFunction                     │   202.59ms │    0.03ms avg │   2.9% │ 5940 calls
  parsePythonAndTranspileToJs call 201     │    25.23ms │    0.17ms avg │   0.4% │  147 calls
  parsePythonAndTranspileToJs call 214     │     3.57ms │    0.20ms avg │   0.1% │   18 calls
  transpilation-stage-0                    │     0.84ms │    0.84ms avg │   0.0% │    1 calls
  transpilation-stage-1                    │     0.29ms │    0.29ms avg │   0.0% │    1 calls
  transpilation-stage-3                    │     0.27ms │    0.27ms avg │   0.0% │    1 calls
══════════════════════════════════════════════════════════════════════════════════════════

And here is the same measurement with the updated flow:

Compiling 110 files...

🚀 PERFORMANCE REPORT 🚀

══════════════════════════════════════════════════════════════════════════════════════════
⏱️  TOTAL TIME: 6543.44ms

  doCompile                                │  6543.44ms │ 6543.44ms avg │ 100.0% │    1 calls
  transpileYaml                            │  6221.67ms │   18.03ms avg │  95.1% │  345 calls
  compileYamlFile                          │  6030.43ms │   18.61ms avg │  92.2% │  324 calls
  PythonParser->transpileToJs()            │  5923.99ms │    0.20ms avg │  90.5% │ 30336 calls
  yaml-transpileAndPrepareJsFile           │  5018.65ms │   15.78ms avg │  76.7% │  318 calls
  compilation-phase-0                      │  2160.93ms │ 2160.93ms avg │  33.0% │    1 calls
  compilation-phase-1                      │  1947.60ms │ 1947.60ms avg │  29.8% │    1 calls
  compilation-phase-3                      │  1926.70ms │ 1926.70ms avg │  29.4% │    1 calls
  parsePythonAndTranspileToJs call 301     │  1923.06ms │    0.43ms avg │  29.4% │ 4473 calls
  asyncModules.reduce (jinja)              │   398.69ms │  132.90ms avg │   6.1% │    3 calls
  astIntoArrowFunction                     │   180.96ms │    0.04ms avg │   2.8% │ 4665 calls
  parsePythonAndTranspileToJs call 201     │    24.40ms │    0.17ms avg │   0.4% │  147 calls
  parsePythonAndTranspileToJs call 214     │     3.00ms │    0.17ms avg │   0.0% │   18 calls
  transpilation-stage-0                    │     1.84ms │    1.84ms avg │   0.0% │    1 calls
  transpilation-stage-1                    │     0.28ms │    0.28ms avg │   0.0% │    1 calls
  transpilation-stage-3                    │     0.25ms │    0.25ms avg │   0.0% │    1 calls
══════════════════════════════════════════════════════════════════════════════════════════
🎯 End of Performance Report

Not as much as wanted, but 10% - is still smth...

Check List

  • Tests have been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet

@KSDaemon KSDaemon requested a review from a team as a code owner August 20, 2025 14:39
@KSDaemon KSDaemon marked this pull request as draft August 20, 2025 14:41
@KSDaemon KSDaemon changed the title perf(schema-compiler): Improve yaml compilation speed WIP: perf(schema-compiler): Improve yaml compilation speed Aug 20, 2025
@KSDaemon KSDaemon force-pushed the chore/antlr-update branch from 0258955 to 8dc8f33 Compare August 20, 2025 16:18
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch from 33a9955 to d37d9eb Compare August 20, 2025 16:18
@KSDaemon KSDaemon force-pushed the chore/antlr-update branch from 8dc8f33 to 9a84e08 Compare August 21, 2025 08:54
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch from d37d9eb to 8860148 Compare August 21, 2025 08:55
@KSDaemon KSDaemon force-pushed the chore/antlr-update branch from 9a84e08 to af9247b Compare August 21, 2025 10:51
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch 3 times, most recently from c579089 to f98fdb9 Compare August 21, 2025 15:40
Base automatically changed from chore/antlr-update to master August 21, 2025 16:00
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch 3 times, most recently from 539bac4 to bdc16bf Compare August 26, 2025 16:05
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch from bdc16bf to 3487bba Compare August 26, 2025 16:06
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch from 3487bba to cf10a91 Compare August 26, 2025 16:06
@KSDaemon KSDaemon force-pushed the perf/py-parser-improve branch from dab20bc to cf9d35d Compare August 26, 2025 19:08
@KSDaemon KSDaemon marked this pull request as ready for review August 26, 2025 19:12
@KSDaemon KSDaemon changed the title WIP: perf(schema-compiler): Improve yaml compilation speed perf(schema-compiler): Improve yaml compilation speed Aug 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant