What to do if my `bazel_dep` uses `toolchains_llvm`?

[filmil]

My dep uses toolchains_llvm in its MODULE.bazel. Its

Depending on it hits the error message at:

toolchains_llvm/toolchain/extensions/llvm.bzl

Line 54 in 96b08ff

fail("Only the root module can use the 'llvm' extension")

Is there a canonical way to fix this?

[filmil]

Example MODULE in an integration test: filmil/bazel_rules_nvc@14de5a6

[filmil]

It seems that if I locally modify the dependency so as to not use the extension from @toolchains_llvm, compilation then passes.

Would it be possible to ignore the toolchains setup in a non-root module instead of failing with an error?

[fmeum]

The standard way would be to add dev_dependency = True to the use_extension call. Does that work in your case?

[filmil]

No, this is what happens:

╰─>$ bazel build //...
ERROR: Analysis of target '//sim:sim' failed; build aborted: No repository visible as '@llvm_toolchain' from repository '@@bazel_rules_nvc+'

[filmil]

Ended up patching my bazel_dep: filmil/bazel_rules_nvc@d718db6

This approach doesn't seem right, so if there's an obvious thing I missed, let me know.

[fmeum]

Then the question is why the LLVM toolchain is not a dev dependency of bazel_rules_nvc. Does that ruleset have to reference any LLVM binaries explicitly?

[filmil]

It compiles NVC from source, and uses the LLVM shared libraries. Not sure if that counts.

[fmeum]

It does count, but using llvm_toolchains via a module extension for that purpose is pretty brittle (whence the check): The end user may register a different LLVM version, they may use a different name, ...

Happy to talk about ways to extend the module to support this use case, but the module extension is not how that can be done sustainably. Could you describe exactly what you need in more detail?

[filmil]

My intention was to define a module at https://github.com/filmil/bazel_rules_nvc. This module compiles the program https://github.com/nickg/nvc from source, and defines a toolchain that uses it.

I would then like to use the module bazel_rules_nvc in another module. I expect that when I do so, the nvc binary is made available for use.

In turn, in bazel_rules_nvc, I use toolchains_llvm to ensure that nvc is compiled with a hermetic toolchain. The module toolchains_llvm requires the use of a module extension to configure it, and hence I use the extension to configure it.

With this setup, I can not seem to find a way to have module resolution not fail, without patching the MODULE.bazel of bazel_rules_nvc when using it. I'm not quite clear what it is that I'm doing that I shouldn't be doing.

[fmeum]

In turn, in bazel_rules_nvc, I use toolchains_llvm to ensure that nvc is compiled with a hermetic toolchain.

Could you make this more precise? What kind of BUILD API do you need from toolchains_llvm to make this work? Would the build still pass with the default non-hermetic C++ toolchain shipped with Bazel?

[filmil]

Could you make this more precise? What kind of BUILD API do you need from toolchains_llvm to make this work? Would the build still pass with the default non-hermetic C++ toolchain shipped with Bazel?

Hm, I'm not quite sure what sort of thing you're after, so I apologize in advance if I don't tell you exactly what you wanted to know. The main issue here is that nvc is essentially a LLVM frontend, so I do want to compile it with clang specifically and then without unexpected compiler variations.

Maybe it is possible to do otherwise, but not using clang led to bizarre errors, which I hope not to have to reexamine here.

I think this means that using a nonhermetic C++ toolchain won't work, since if you don't specifically use clang you won't be able to compile it.

Maybe my understanding here is too simplistic, but my expectation was: "I compile a thing from source with this specific compiler; I then use the thing. Build system doesn't prevent me from using the thing.".

[fmeum]

Maybe my understanding here is too simplistic, but my expectation was: "I compile a thing from source with this specific compiler; I then use the thing. Build system doesn't prevent me from using the thing.".

The problem is who "I" is in this scenario. From the perspective of the end user, it would be a pretty bad experience if a (possibly transitive) bazel_dep registers a custom C++ toolchain - that's the kind of spooky action at a distance that we would like to avoid. This is particularly relevant for C++ toolchains as there is no way to perform meaningful version resolution or even just select between multiple. This is ultimately why we decided to lock down the toolchain extension in this way.

I see a number of ways forward:

1. If bazel_rules_nvc should generally be expected to work with a wide range of LLVM versions and perhaps even with a user-provided system clang, you could make the dep on toolchains_llvm a dev dependency and document that active users of your ruleset may want to use toolchains_llvm themselves to bring in a suitable toolchain.
2. If you need specific targets from an LLVM toolchain that may not be present in a general C++ toolchain (think clang-format), we could look into providing "resolved toolchain" targets that provide them.
3. If bazel_rules_nvc absolutely has to use a hermetic LLVM (perhaps even a particular version), you would need to apply transitions to enforce that. At that point we could look into how toolchains_llvm could support that, but it's by far the most complex solution and has a number of potentially annoying implications.

[filmil]

Thanks for explaining. In this case, I wear two hats - both a module provider and a module user. My intention is that the brains under both hats are pleased with the outcome, and that setup and use complexities are hidden.

All the options you mentioned seem onerous to at least someone. Last night I was thinking maybe the correct thing to do is to try and avoid compiling the thing from source to begin with. Sadly just by the virtue of how tightly coupled the source is with LLVM, it probably limits the feasible options. Patching MODULE.bazel on the user end, and providing an LLVM toolchain in the root module kinda worked, in the sense that analysis passes. Unfortunately, compilation fails for a somewhat comical reason - the compiler in the root module gets the option -Werror from somewhere, and errors out on -fuse-ld-path=... flag being deprecated. The compilation command line proved impervious to attempting to pass the appropriate -Wno-error=... flag, which is where I decided to stop for the day.

But reflecting on the experience thus far, perhaps a better approach would be to avoid compiling in the first place, as noted above. I'll try that next, possibly using bazel_linux_packages to get the binary installation from debian or ubuntu. That is probably its own can of worms, but presumably it only needs to be opened once.