[image-spec]: fix transitive nix flake path inputs not copied to Docker build context

[Vervious]

Why are the changes needed?

Training CI tests (e.g. training-pretrain-aws-test) fail with:

error: path '/build/local_packages/rust/shared/exautils/flake.nix' does not exist

The existing discover_nix_flake_path_inputs() only finds direct path inputs declared in flake.nix via regex. It does not discover transitive path dependencies recorded in flake.lock — i.e., dependencies of dependencies.

Concrete example in the exa_ml flake:

1. exa_ml/flake.nix → minos2_rust (direct, copied ✓)
2. minos2_rust/flake.nix → rust-exautils (transitive via flake.lock, not copied ✗)
3. rust-exautils/flake.nix → rust-base (transitive via flake.lock, not copied ✗)

When Nix evaluates the root flake inside the Docker build, it reads the root flake.lock which references these transitive paths — but the directories were never copied into local_packages/, so the build fails.

What changes were proposed in this pull request?

flytekit/image_spec/image_spec.py — New function discover_transitive_nix_flake_path_inputs():

• Parses flake.lock JSON to find all nodes with locked.type == "path" and a non-empty parent chain (transitive deps)
• Resolves each node's absolute source path by recursively walking the parent chain (parent[-1] = declaring parent; path is relative to that parent's directory)
• Uses a cache to avoid redundant resolution
• Skips nodes already handled as direct inputs (via already_resolved set)

flytekit/image_spec/default_builder.py — In _copy_local_packages_and_update_lock():

• After copying direct flake path inputs, calls discover_transitive_nix_flake_path_inputs() and copies the discovered transitive deps into local_packages/
• No flake.lock path rewriting is needed for transitive deps because local_packages/ mirrors the monorepo directory structure, preserving relative paths between packages

How was this patch tested?

Verified with an inline integration test that simulates the exa_ml dependency chain (root → minos2_rust → rust-exautils → rust-base) and confirms both transitive deps are discovered with correct absolute paths.

No committed unit tests were added — the function is exercised only through CI's Nix image builds.

Human review checklist

• Verify parent[-1] interpretation: The code assumes the last element of the parent chain in flake.lock v7 is the declaring parent whose directory the path is relative to. Confirm this matches Nix's flake.lock spec.
• Silent skip on unresolvable transitive deps: When _find_git_root() returns None for a transitive dep, it's silently skipped (continue). Direct inputs raise ValueError instead. Consider whether a warning/error would be more appropriate to surface issues early.
• Large directory copies: Transitive deps like rust-base could point to the entire rust/ directory. Confirm that copying large subtrees with gitignore filtering doesn't cause excessive build context size/time.
• Bare except Exception: JSON parsing failure returns empty list silently. Could mask real issues if flake.lock is corrupted.

---

Link to Devin Session: https://app.devin.ai/sessions/46b404cede304b2b8ce0d143523914d8
Requested by: @Vervious (benchan@exa.ai)

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring