Skip to content

Conversation

@kemko
Copy link
Contributor

@kemko kemko commented Oct 8, 2025

Description

Fix a regression introduced in Nomad 1.9.0-beta.1 where CNI bridge networking with IPv6-only interfaces would fail with the error failed to configure network: no interface with an address.

The issue was that while the code correctly populated the AddressIPv6 field for IPv6 addresses, several validation checks only examined the Address field (IPv4), causing IPv6-only configurations to be incorrectly rejected. This affected users running IPv6-only infrastructure who were unable to upgrade from Nomad 1.8.x to 1.9.x.

This PR updates the validation logic in four locations to check both IPv4 and IPv6 address fields:

  • Interface selection logic in cniToAllocNet() - now accepts interfaces with either IPv4 or IPv6 addresses
  • Fallback logic - checks both Address and AddressIPv6 before falling back
  • Error check - only fails when both IPv4 and IPv6 are missing
  • AllocNetworkStatus.IsZero() - includes AddressIPv6 in the zero check

Testing & Reproduction steps

Reproduction (without fix):

  1. Set up a CNI bridge interface with only IPv6 addresses (no IPv4)
  2. Configure Nomad client to use bridge networking
  3. Run any job with network { mode = "bridge" }
  4. Observe the error: failed to configure network: no interface with an address

Testing:

  • Added new test TestCNI_cniToAllocNet_IPv6Only that verifies CNI results with only IPv6 addresses are handled correctly
  • The test follows the same pattern as the existing TestCNI_cniToAllocNet_Dualstack test
  • Existing dual-stack test continues to pass, ensuring no regression for IPv4+IPv6 configurations

Links

Fixes #26905

Related:

Contributor Checklist

  • Changelog Entry If this PR changes user-facing behavior, please generate and add a
    changelog entry using the make cl command.
  • Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
    ensure regressions will be caught.
  • Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
    and job configuration, please update the Nomad website documentation to reflect this. Refer to
    the website README for docs guidelines. Please also consider whether the
    change requires notes within the upgrade guide.

Reviewer Checklist

  • Backport Labels Please add the correct backport labels as described by the internal backporting document.
  • Commit Type Ensure the correct merge method is selected which should be "squash and merge" in the majority of situations. The main exceptions are long-lived feature branches or merges where history should be preserved.
  • Enterprise PRs If this is an enterprise only PR, please add any required changelog entry within the public repository.
  • If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

No changes to security controls. This is a bug fix that restores correct validation logic for network interface address checking.

kemko added 2 commits October 8, 2025 18:12
Add a test case to verify that CNI results containing only IPv6 addresses
are handled correctly. This is a regression test for a bug introduced in
hashicorpGH-23882 where IPv6-only interfaces would fail with 'no interface with an
address' error.

The test verifies that when a CNI plugin returns only an IPv6 address
(without IPv4), the allocation network status should be properly populated
with the IPv6 address in the AddressIPv6 field.
Fix a regression introduced in Nomad 1.9.0 where CNI bridge networking with
IPv6-only interfaces would fail with 'no interface with an address' error.

The issue was that while the code correctly populated the AddressIPv6 field
for IPv6 addresses, several validation checks only examined the Address field
(IPv4), causing IPv6-only configurations to be rejected.

Changes:
- Update interface selection logic in cniToAllocNet to accept interfaces with
  either IPv4 or IPv6 addresses (not just IPv4)
- Update fallback logic to check both Address and AddressIPv6 fields
- Update error check to only fail when both IPv4 and IPv6 are missing
- Update AllocNetworkStatus.IsZero() to check AddressIPv6 field

This allows CNI configurations with IPv6-only interfaces to work correctly,
restoring functionality from Nomad 1.8.x.

Fixes hashicorp#26905
@kemko kemko requested review from a team as code owners October 8, 2025 15:24
@hashicorp-cla-app
Copy link

hashicorp-cla-app bot commented Oct 8, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM to me for the specific problem described, but I think we're still going to end up running into the problem described in #25997 by @gabivlj, which we've never landed. Have you verified this end-to-end that the address is getting picked up by the driver?

(cc @gulducat to see if he has more thoughts on this too)

@tgross tgross requested a review from gulducat October 8, 2025 18:38
kemko added 3 commits October 9, 2025 12:56
Add multiple test cases to validate the behavior of CNI allocation when handling IPv6-only and dualstack configurations. The new tests ensure that the first address is selected correctly from multiple addresses across interfaces, maintaining consistent behavior with IPv4. This improves coverage for edge cases in CNI result processing and reinforces the recent fixes for IPv6 support.
Add fallback logic in cniToAllocNet to populate the Address field with the IPv6 address when no IPv4 address is available. This change ensures compatibility with existing code that relies on the Address field for service registration, particularly in scenarios where only IPv6 addresses are present.
@kemko
Copy link
Contributor Author

kemko commented Oct 9, 2025

Good point. I looked into the service registration code and it only checks the Address field, so IPv6-only would fail there.

Added the fallback from #25997 - copying IPv6 to Address when there's no IPv4. Tests updated to match.

Note that I tested the code correctness in an IPv6-only environment. I don't have access to a place where I could verify it works in dual-stack and IPv4-only setups.

@kemko
Copy link
Contributor Author

kemko commented Oct 13, 2025

Hi @tgross @gulducat,
I’ve updated the PR with the IPv6 fallback and better tests. Could you please take another look when you get a chance? Happy to tweak anything if it’s still not quite right.

Thanks!

Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice work on the tests.

@tgross tgross merged commit e79e979 into hashicorp:main Oct 13, 2025
31 of 33 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Nomad - Community Issues Triage Oct 13, 2025
@tgross tgross added backport/ent/1.9.x+ent Changes are backported to 1.9.x+ent backport/1.10.x backport to 1.10.x release line labels Oct 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/ent/1.9.x+ent Changes are backported to 1.9.x+ent backport/1.10.x backport to 1.10.x release line theme/ipv6 theme/networking type/bug

Projects

Development

Successfully merging this pull request may close these issues.

CNI networking fails with IPv6-only interfaces since 1.9.0

2 participants