Skip to content

Conversation

fonsp
Copy link
Member

@fonsp fonsp commented Jun 8, 2023

Sort-of reverting #52 because our samples are not normally distributed, the calculated stddev is super high.

This PR tests the ratio between medians with a fixed rate (20%) again.

But! In #52 I added some nice things:

  • Tuning the benchmark before running it (this reduces spread)
  • Using the tuned parameters from one benchmark for the other, for a fair comparison.
  • Learned that median is more stable against outliers than mean

So this should still be more stable than before #52!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant