perf[alp]: compare |- alp pushdown #5443

joseph-isaacs · 2025-11-21T15:46:29Z

No description provided.

Signed-off-by: Joe Isaacs <[email protected]>

codspeed-hq · 2025-11-21T15:51:50Z

CodSpeed Performance Report

Merging #5443 will degrade performances by 38.86%

_{Comparing ji/alp-pushddown (81eaec2) with develop (f52dbf0)}

Summary

⚡ 5 improvements
❌ 6 regressions
✅ 1504 untouched
🆕 7 new
⏩ 91 skipped¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
🆕	`decompress[("alp_for_bp_f64", 0x45b9120)]`	N/A	29.8 ms	N/A
🆕	`decompress[("datetime_for_bp", 0x45bbbf0)]`	N/A	41.7 ms	N/A
🆕	`decompress[("dict_fsst_varbin_bp_string", 0x45bb070)]`	N/A	76.3 ms	N/A
🆕	`decompress[("dict_fsst_varbin_string", 0x45bacf0)]`	N/A	76.3 ms	N/A
🆕	`decompress[("dict_varbinview_string", 0x45b9b40)]`	N/A	75.6 ms	N/A
🆕	`decompress[("for_bp_u64", 0x45b89d0)]`	N/A	2.5 ms	N/A
🆕	`decompress[("runend_for_bp_u32", 0x45b9ed0)]`	N/A	2.1 ms	N/A
⚡	`batch[86016]`	662.8 µs	586.8 µs	+12.95%
⚡	`in_place_batch[1024]`	5.6 µs	4.6 µs	+20.19%
❌	`in_place_batch[73728]`	234.2 µs	280.4 µs	-16.47%
⚡	`in_place_pipeline[100352]`	153.6 µs	128.8 µs	+19.27%
❌	`in_place_pipeline[73728]`	95.5 µs	114.4 µs	-16.49%
❌	`in_place_pipeline[86016]`	110.9 µs	133 µs	-16.58%
❌	`pipeline[86016]`	114.9 µs	136 µs	-15.51%
❌	`pipeline_extra_copy[100352]`	132.8 µs	158.6 µs	-16.24%
⚡	`pipeline_extra_copy[262144]`	402.4 µs	335.2 µs	+20.06%
⚡	`pipeline_extra_copy[65536]`	106.1 µs	89.3 µs	+18.85%
❌	`verify_all_methods[16384]`	23.1 ms	37.8 ms	-38.86%

91 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

codecov · 2025-11-21T15:57:41Z

Codecov Report

❌ Patch coverage is 95.20295% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.73%. Comparing base (f52dbf0) to head (81eaec2).

Files with missing lines	Patch %	Lines
encodings/alp/src/alp/compute/expr_pushdown.rs	94.22%	13 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Joe Isaacs <[email protected]>

gatesn

This is... kinda nice??

I guess there's a lot more stuff to pull apart than just the eager compute function.
I wonder if after writing a few more of these there are certain patterns we have to extract sub-expressions out of an ExprArray and push them through another array?

gatesn · 2025-11-21T18:13:51Z

vortex-file/src/lib.rs

        EncodingRef::new_ref(vortex_zstd::ZstdEncoding.as_ref()),
    ]);
+
+    register_alp_rules(&session.arrays())


I think we should just have vortex_alp::initialize(session)

So there's a single entry-point for the crate to register all its things (including its encodings!)

gatesn · 2025-11-21T18:14:25Z

encodings/alp/src/alp/compute/mod.rs

 mod nan_count;
 mod take;

+pub use expr_pushdown::ALPExprPushdownRule;


vortex_alp::initialize(session)

pub(crate) at most ?

gatesn · 2025-11-21T18:14:29Z

encodings/alp/src/alp/mod.rs

 mod array;
 mod compress;
-mod compute;
+pub mod compute;


gatesn · 2025-11-21T18:17:25Z

encodings/alp/src/alp/compute/expr_pushdown.rs

+        }
+
+        // Convert to compute operator
+        let compute_op = match operator {


Feels like this does the same logic as the matches! macro on line 62?

gatesn · 2025-11-21T18:19:32Z

encodings/alp/src/alp/compute/expr_pushdown.rs

+        let literal_value = literal_expr.as_::<Literal>().data().clone();
+
+        // Don't optimize nullable comparisons
+        if literal_value.dtype().is_nullable() {


I'm 90% sure this is covered by the null checks above

me too but also only 90

encodings/alp/src/alp/compute/expr_pushdown.rs

cloudflare-workers-and-pages · 2025-11-24T11:12:29Z

Deploying vortex-bench with Cloudflare Pages

Latest commit:	`81eaec2`
Status:	✅ Deploy successful!
Preview URL:	https://bfb9335d.vortex-93b.pages.dev
Branch Preview URL:	https://ji-alp-pushddown.vortex-93b.pages.dev

View logs

joseph-isaacs · 2025-11-24T11:13:08Z

Yep, I agree I just wanted to see what it would be like.

joseph-isaacs · 2025-11-24T11:14:43Z

The other approach for scalar arrays. is to convert this into an expression and create a reduce_parent expr rule for alp($) > x

Signed-off-by: Joe Isaacs <[email protected]>

wip

c2cd0ab

Signed-off-by: Joe Isaacs <[email protected]>

joseph-isaacs changed the title ~~wip~~ wip: alp decode pushdown Nov 21, 2025

wip

f7025c4

Signed-off-by: Joe Isaacs <[email protected]>

joseph-isaacs changed the title ~~wip: alp decode pushdown~~ perf[alp]: alp decode pushdown Nov 21, 2025

joseph-isaacs changed the title ~~perf[alp]: alp decode pushdown~~ perf[alp]: compare |- alp pushdown Nov 21, 2025

joseph-isaacs added the feature Release label indicating a new feature or request label Nov 21, 2025

joseph-isaacs marked this pull request as ready for review November 21, 2025 16:34

joseph-isaacs added 2 commits November 21, 2025 16:49

wip

30f0907

Signed-off-by: Joe Isaacs <[email protected]>

wip

e3bea37

Signed-off-by: Joe Isaacs <[email protected]>

joseph-isaacs enabled auto-merge (squash) November 21, 2025 16:55

gatesn reviewed Nov 21, 2025

View reviewed changes

joseph-isaacs marked this pull request as draft November 24, 2025 11:16

auto-merge was automatically disabled November 24, 2025 11:16
Pull request was converted to draft

wip

f6f04e7

Signed-off-by: Joe Isaacs <[email protected]>

joseph-isaacs force-pushed the ji/alp-pushddown branch from e103f21 to f6f04e7 Compare November 24, 2025 11:33

joseph-isaacs added 2 commits November 24, 2025 11:38

Merge remote-tracking branch 'origin/develop' into ji/alp-pushddown

7fae39a

wip

81eaec2

Signed-off-by: Joe Isaacs <[email protected]>

perf[alp]: compare |- alp pushdown #5443

Are you sure you want to change the base?

perf[alp]: compare |- alp pushdown #5443

Uh oh!

Conversation

joseph-isaacs commented Nov 21, 2025

Uh oh!

codspeed-hq bot commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #5443 will degrade performances by 38.86%

Summary

Benchmarks breakdown

Footnotes

Uh oh!

codecov bot commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

gatesn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cloudflare-workers-and-pages bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying vortex-bench with Cloudflare Pages

Uh oh!

joseph-isaacs commented Nov 24, 2025

Uh oh!

joseph-isaacs commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codspeed-hq bot commented Nov 21, 2025 •

edited

Loading

codecov bot commented Nov 21, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Nov 24, 2025 •

edited

Loading

joseph-isaacs commented Nov 24, 2025 •

edited

Loading