Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 31, 2025

The Predict_TieBreaker_ReturnsConsistentLabel test fails intermittently because the KNN tie-breaker uses GetHashCode(), which is non-deterministic for strings across program executions.

Changes

  • Replace hash-based tie-breaking with minimum distance tie-breaking in KNearestNeighbors.Predict()
  • When vote counts are equal, select the label whose nearest neighbor is closest to the prediction point

Before

.Select(g => new { Label = g.Key, Count = g.Count() })
.OrderByDescending(x => x.Count)
.ThenBy(x => x.Label?.GetHashCode() ?? 0)  // Non-deterministic

After

.Select(g => new { Label = g.Key, Count = g.Count(), MinDistance = g.Min(item => item.Distance) })
.OrderByDescending(x => x.Count)
.ThenBy(x => x.MinDistance)  // Deterministic and semantically meaningful
Original prompt

This section details on the original issue you should resolve

<issue_title>Tests in CI fail from time to time</issue_title>
<issue_description>Tests in CI fail from time to time due to randomness in tests.

Logs:
Run dotnet test --no-restore --collect "XPlat Code Coverage"
Utilities -> /home/runner/work/C-Sharp/C-Sharp/Utilities/bin/Debug/net8.0/Utilities.dll
Utilities.Tests -> /home/runner/work/C-Sharp/C-Sharp/Utilities.Tests/bin/Debug/net8.0/Utilities.Tests.dll
Test run for /home/runner/work/C-Sharp/C-Sharp/Utilities.Tests/bin/Debug/net8.0/Utilities.Tests.dll (.NETCoreApp,Version=v8.0)
DataStructures -> /home/runner/work/C-Sharp/C-Sharp/DataStructures/bin/Debug/net8.0/DataStructures.dll
DataStructures.Tests -> /home/runner/work/C-Sharp/C-Sharp/DataStructures.Tests/bin/Debug/net8.0/DataStructures.Tests.dll
Test run for /home/runner/work/C-Sharp/C-Sharp/DataStructures.Tests/bin/Debug/net8.0/DataStructures.Tests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.14.1 (x64)

VSTest version 17.14.1 (x64)

Starting test execution, please wait...
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
A total of 1 test files matched the specified pattern.

Passed! - Failed: 0, Passed: 35, Skipped: 0, Total: 35, Duration: 227 ms - Utilities.Tests.dll (net8.0)
ArgumentException

ArgumentNullException

Algorithms -> /home/runner/work/C-Sharp/C-Sharp/Algorithms/bin/Debug/net8.0/Algorithms.dll
Algorithms.Tests -> /home/runner/work/C-Sharp/C-Sharp/Algorithms.Tests/bin/Debug/net8.0/Algorithms.Tests.dll
Test run for /home/runner/work/C-Sharp/C-Sharp/Algorithms.Tests/bin/Debug/net8.0/Algorithms.Tests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.14.1 (x64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed! - Failed: 0, Passed: 825, Skipped: 0, Total: 825, Duration: 2 s - DataStructures.Tests.dll (net8.0)
Failed Predict_TieBreaker_ReturnsConsistentLabel [20 ms]
Error Message:
Assert.That(label, Is.EqualTo("A"))
String lengths are both 1. Strings differ at index 0.
Expected: "A"
But was: "B"
-----------^

Stack Trace:
at Algorithms.Tests.MachineLearning.KNearestNeighborsTests.Predict_TieBreaker_ReturnsConsistentLabel() in /home/runner/work/C-Sharp/C-Sharp/Algorithms.Tests/MachineLearning/KNearestNeighborsTests.cs:line 84

  1. at Algorithms.Tests.MachineLearning.KNearestNeighborsTests.Predict_TieBreaker_ReturnsConsistentLabel() in /home/runner/work/C-Sharp/C-Sharp/Algorithms.Tests/MachineLearning/KNearestNeighborsTests.cs:line 84

Failed! - Failed: 1, Passed: 12208, Skipped: 0, Total: 12209, Duration: 2 m 48 s - Algorithms.Tests.dll (net8.0)

Attachments:
/home/runner/work/C-Sharp/C-Sharp/Utilities.Tests/TestResults/992dcb76-e67d-4cd2-92a5-39ce380708f8/coverage.cobertura.xml
/home/runner/work/C-Sharp/C-Sharp/DataStructures.Tests/TestResults/725d98da-661b-4869-8e7d-a4de9466374d/coverage.cobertura.xml
/home/runner/work/C-Sharp/C-Sharp/Algorithms.Tests/TestResults/10535369-7336-4238-b2a9-401511937587/coverage.cobertura.xml
Error: Process completed with exit code 1.
</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Fix intermittent CI test failures due to randomness Fix non-deterministic tie-breaking in KNN causing flaky tests Oct 31, 2025
Copilot AI requested a review from siriak October 31, 2025 09:08
Copilot finished work on behalf of siriak October 31, 2025 09:08
@codecov
Copy link

codecov bot commented Oct 31, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.70%. Comparing base (c6c3aa2) to head (5e3b845).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #550      +/-   ##
==========================================
+ Coverage   96.68%   96.70%   +0.02%     
==========================================
  Files         280      281       +1     
  Lines       11059    11123      +64     
  Branches     1577     1590      +13     
==========================================
+ Hits        10692    10757      +65     
  Misses        232      232              
+ Partials      135      134       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@siriak siriak marked this pull request as ready for review October 31, 2025 10:14
Copilot AI review requested due to automatic review settings October 31, 2025 10:14
@siriak siriak merged commit 8bf3d7b into master Oct 31, 2025
12 checks passed
@siriak siriak deleted the copilot/fix-ci-test-randomness-issues branch October 31, 2025 10:15
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the K-Nearest Neighbors tie-breaking mechanism by using minimum distance instead of hash codes. When multiple labels have the same count among k-nearest neighbors, the algorithm now selects the label with the smallest distance to the test point, which is a more semantically meaningful and deterministic tie-breaking strategy.

Key changes:

  • Added MinDistance calculation in the label aggregation step
  • Changed tie-breaking from hash-based to distance-based ordering
Comments suppressed due to low confidence (1)

Algorithms/MachineLearning/KNearestNeighbors.cs:1

  • The tie-breaking test expects label 'A', but with the new distance-based tie-breaking logic, the result depends on which label has the minimum distance. For the test point (0.5, 0.5), both labels 'A' at (0.0, 0.0) and 'B' at (1.0, 1.0) are equidistant (both at distance ~0.707). The test should be updated to verify the actual behavior or use a scenario where distances are clearly different to properly validate the new tie-breaking mechanism.
using System;

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tests in CI fail from time to time

2 participants