[CI Failure Doctor] CI Failure Investigation - Run #38284

[github-actions]

Summary

TestAwInfoStepsFirewall in the Integration: Workflow Misc Part 2 job now fails because it still searches for the inline steps: { ... } block while the compiler moved AW info generation into generate_aw_info.cjs, so the compiled lock no longer contains the literal text even though the metadata still exists.

Failure Details

• Run: 22529742871
• Commit: 700150b
• Trigger: push

Root Cause Analysis

Integration: Workflow Misc Part 2 executed TestAwInfoStepsFirewall/firewall_enabled_with_copilot, which calls stringutil.MarkdownToLockFile and asserts that the generated lock file contains 'steps: {' and firewall: "squid" (lines 107‑128 of pkg/workflow/aw_info_steps_test.go). pkg/workflow/compiler_yaml.go lines 569‑603 now render the Generate agentic run info step by invoking /opt/gh-aw/actions/generate_aw_info.cjs and passing GH_AW_INFO_FIREWALL_TYPE, so the lock file no longer embeds the literal AW info object (the steps block was part of the inline script in the previous compiler output). As a result, the substring search fails even though the firewall metadata continues to flow through the env vars handled by generate_aw_info.cjs.

Failed Jobs and Errors

• Integration: Workflow Misc Part 2 – TestAwInfoStepsFirewall/firewall_enabled_with_copilot (aw_info_steps_test.go:112 and aw_info_steps_test.go:118 reported that 'steps: {' and the firewall line were missing)

Investigation Findings

• pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/basic-copilot.golden now shows the generate_aw_info step loading generate_aw_info.cjs via require and only emitting the GH_AW_INFO_* env vars, so the literal steps: { ... } snippet no longer appears in the lock file output.
• Attempting to reproduce locally with go test ./pkg/workflow -run TestAwInfoStepsFirewall failed because the Go 1.25 toolchain download from proxy.golang.org is forbidden in this environment, so I relied on the cached CI log and golden files instead.

Recommended Actions

• Update TestAwInfoStepsFirewall to validate the firewall configuration by reading the environment vars that are now passed to generate_aw_info.cjs (e.g., GH_AW_INFO_FIREWALL_TYPE/GH_AW_INFO_FIREWALL_ENABLED) or by executing the script and parsing the resulting aw_info.json, rather than searching for the inline steps: { snippet.
• Add regression coverage to the new script-based flow (e.g., a test that runs generate_aw_info.cjs with the relevant env vars) to ensure steps.firewall remains part of the JSON output even if the lock file no longer contains the literal text.

Prevention Strategies

Avoid brittle tests that rely on specific generated script text in the lock file—verify structured metadata (env vars or JSON) so refactors that move logic out of inline scripts do not break the suite.

AI Team Self-Improvement

When validating AW info metadata, inspect the structured env vars or generated JSON instead of looking for raw steps: { strings so the tests keep working after we refactor script packaging.

Historical Context

• The failure emerged because AW info generation was refactored to use generate_aw_info.cjs (as evidenced by compiler_yaml.go lines 569‑603), but the integration test still expects the old inline representation.
• Similar future refactors should trigger a warning: update tests to consume the structured metadata instead of matching literal code snippets.

🩺 Diagnosis provided by CI Failure Doctor

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d

• expires on Mar 1, 2026, 9:59 PM UTC