Add DiskANN benchmark pipeline to GitHub Actions#857

Open

YuanyuanTian-hh wants to merge 22 commits intomainfrom

user/tianyuanyuan/add-benchmark-pipeline

Commits on Mar 19, 2026

Migrate DiskANN benchmark pipeline from ADO to GitHub Actions
Yuanyuan Tian (from Dev Box)
committed

Commits on Mar 20, 2026

Fix openai-100K distance metric and add gitignore patterns
Yuanyuan Tian (from Dev Box)
committed
Add push trigger to benchmarks workflow for pre-merge testing
Yuanyuan Tian (from Dev Box)
committed
Fix baseline run: use input config from current branch checkout
Yuanyuan Tian (from Dev Box)
committed
Fix markdown conversion: replace broken csvtomd with inline Python
Yuanyuan Tian (from Dev Box)
committed
Update benchmark configs: align build/search params to fix low recall
Yuanyuan Tian (from Dev Box)
committed

Commits on Mar 23, 2026

Fix recall_at: set to 100 to match groundtruth file K=100
Yuanyuan Tian (from Dev Box)
committed
Remove stale absolute contracts (qps/recall/total_time): calibrated for ADO mimir-enron, not applicable to public datasets on GitHub runners. Threshold calibration tracked in PBI.
Yuanyuan Tian (from Dev Box)
committed
remove comments
Yuanyuan Tian (from Dev Box)
committed
Fix build_ram_limit_gb: reduce 10->4 to fit GitHub runner RAM (7GB limit)
Yuanyuan Tian (from Dev Box)
committed
Fix wikipedia distance: cosine_normalized->cosine (vectors are not L2-normalized, metric is inner product)
Yuanyuan Tian (from Dev Box)
committed
Fix wikipedia distance: cosine->inner_product (groundtruth uses raw ip, not cosine similarity)
Yuanyuan Tian (from Dev Box)
committed

Commits on Mar 24, 2026

Align build threads
Yuanyuan Tian (from Dev Box)
committed
Speed up benchmarks: build threads 1->4, openai pq_chunks 384->192
Yuanyuan Tian (from Dev Box)
committed
Temp: disable concurrency cancellation for A/A batch testing
Yuanyuan Tian (from Dev Box)
committed

Commits on Mar 26, 2026

revert A/A test settings, update OpenAI config to SQ_1_2.0
Yuanyuan Tian (from Dev Box)
committed

Commits on Mar 27, 2026

Remove micro-benchmark-iai comments
Yuanyuan Tian (from Dev Box)
committed
use GitHub Release assets for benchmark datasets
Yuanyuan Tian (from Dev Box)
committed
extract csv-to-markdown into reusable script
Yuanyuan Tian (from Dev Box)
committed
calibrate contract thresholds from GitHub runner data
Yuanyuan Tian (from Dev Box)
committed
add daily A/A benchmark stability test with failure notification
Yuanyuan Tian (from Dev Box)
committed
widen mean_cpus threshold to 15% for shared-runner CPU noise
Yuanyuan Tian (from Dev Box)
committed