CI Debugging Improvements

Problem

When CI builds fail, it's difficult to debug the root cause because:

1. Missing error details: Failed builds often show generic messages like "cmake configure failed" or "cmake build failed" without the actual error output
2. Information buried in logs: When error details are present, they're often buried in extremely long CI logs (thousands of lines) making them hard to find
3. Time-consuming debugging: Developers have to re-run builds locally or dig through massive log files to understand what went wrong

This leads to slower development cycles and frustration when trying to fix CI issues.

Solution Proposals

Option 1: Enhanced Error Reporting (Simple)

• Capture and display stderr/stdout from failed cmake commands
• Show the actual error message prominently in CI output
• Add clear separators around error sections for easy identification

Pros: Easy to implement, immediate improvement
Cons: Still requires scrolling through logs for complex issues

Option 2: Error Summary Section (Moderate)

• Collect all errors during the build process
• Display a summary section at the end with just the critical errors
• Include line numbers and file context where possible

Pros: Centralizes all errors in one place, easier to scan
Cons: Requires more complex error parsing logic

Option 3: CI Artifacts + Structured Logging (Advanced)

• Save detailed error logs as downloadable CI artifacts
• Use structured logging to categorize different types of failures
• Create a CI status page that highlights key failure points

Pros: Professional solution, great developer experience
Cons: Significant implementation effort, requires infrastructure changes

Recommendation

Start with Option 1 as it provides immediate value with minimal complexity. A good implementation would:

• Capture stderr/stdout from failed subprocess calls
• Format error output more clearly with visual separators
• Truncate extremely verbose output while preserving key details
• Highlight the most critical error messages at the end of CI runs

This approach balances quick wins with maintainability and can be implemented by modifying the cmake helper functions in tests/cmake.py.