`report_package("GAP")` error -- top level `@ccall` unsupported?

[fingolfin]

Running report_package in the current version of GAP.jl seems to not like some @ccall at the top level. Specifically the code it seems to dislike is this:

const GapObj = @ccall libgap.GAP_DeclareGapObj(:GapObj::Symbol, GAP::Module, Any::Any)::Any

const SmallBag = @ccall libgap.GAP_DeclareBag(:SmallBag::Symbol, GAP::Module, Any::Any, 0::Cint)::Any

const LargeBag = @ccall libgap.GAP_DeclareBag(:LargeBag::Symbol, GAP::Module, Any::Any, 1::Cint)::Any

Note that actually 238 errors are reported; the first three are about those three declarations of const variables. Most of the other 235 revolved around UndefVarError: GapObjnot defined inGAP`, so let's ignore those.

I can solve the problem by moving those @ccall invocations into helper functions:

_declare_gap_obj() = @ccall libgap.GAP_DeclareGapObj(:GapObj::Symbol, GAP::Module, Any::Any)::Any

_declare_bag(sym::Symbol, large::Bool) = @ccall libgap.GAP_DeclareBag(sym::Symbol, GAP::Module, Any::Any, large::Cint)::Any

const GapObj = _declare_gap_obj()::Type

const SmallBag = _declare_bag(:SmallBag::Symbol, false)

const LargeBag = _declare_bag(:LargeBag::Symbol, true)

So I'll do that but I wonder if either supporting @ccall on the top level, or issuing a more helpful error, would be a possibility?

Here is the first error:

julia> using JET

julia> report_package("GAP")
[toplevel-info] virtualized the context of Main (took 0.047 sec)
[toplevel-info] entered into /Users/mhorn/Projekte/OSCAR/GAP.jl/src/GAP.jl
...
[toplevel-info]  exited from /Users/mhorn/Projekte/OSCAR/GAP.jl/src/GAP.jl (took 3.25 sec)
═════ 238 toplevel errors found ═════
┌ @ /Users/mhorn/Projekte/OSCAR/GAP.jl/src/types.jl:12
│ unexpected ccall to Core.tuple(:GAP_DeclareGapObj, GAP.libgap)
│ Stacktrace:
│   [1] error(::String, ::Expr)
│     @ Base ./error.jl:54
│   [2] resolvefc(frame::JuliaInterpreter.Frame, expr::Any)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:134
│   [3] collect_args(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, call_expr::Expr; isfc::Bool)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:143
│   [4] collect_args
│     @ ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:140 [inlined]
│   [5] evaluate_foreigncall(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, call_expr::Expr)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:161
│   [6] eval_rhs(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, node::Expr)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:401
│   [7] step_expr!(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, node::Any, istoplevel::Bool)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:557
│   [8] step_expr!(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, node::Any, istoplevel::Bool)
│     @ JET ~/.julia/packages/JET/z9poX/src/toplevel/virtualprocess.jl:1421
│   [9] step_expr!(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, istoplevel::Bool)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/interpret.jl:620
│  [10] finish!(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, frame::JuliaInterpreter.Frame, istoplevel::Bool)
│     @ JuliaInterpreter ~/.julia/packages/JuliaInterpreter/41lEl/src/commands.jl:14
│  [11] _virtual_process!(res::JET.VirtualProcessResult, toplevelex::Expr, filename::String, analyzer::JET.JETAnalyzer{JET.BasicPass}, config::JET.ToplevelConfig, context::Module, pkg_mod_depth::Int64; force_concretize::Bool, lnnref::Base.RefValue{LineNumberNode})
│     @ JET ~/.julia/packages/JET/z9poX/src/toplevel/virtualprocess.jl:917
│  [12] _virtual_process!(res::JET.VirtualProcessResult, toplevelex::Expr, filename::String, analyzer::JET.JETAnalyzer{JET.BasicPass}, config::JET.ToplevelConfig, context::Module, pkg_mod_depth::Int64)
│     @ JET ~/.julia/packages/JET/z9poX/src/toplevel/virtualprocess.jl:683
│  [13] _virtual_process!(res::JET.VirtualProcessResult, s::String, filename::String, analyzer::JET.JETAnalyzer{JET.BasicPass}, config::JET.ToplevelConfig, context::Module, pkg_mod_depth::Int64)
│     @ JET ~/.julia/packages/JET/z9poX/src/toplevel/virtualprocess.jl:666
│  [14] handle_include(interp::JET.ConcreteInterpreter{JET.var"#usemodule_with_err_handling#64"{Base.RefValue{LineNumberNode}, String, JET.ToplevelConfig, Int64, Set{Symbol}, JET.var"#general_err_handler#54"{Base.RefValue{LineNumberNode}, String}}, JET.JETAnalyzer{JET.BasicPass}}, include_func::Any, args::Vector{Any})
│     @ JET ~/.julia/packages/JET/z9poX/src/toplevel/virtualprocess.jl:1574
└────────────────────────
...

julia> versioninfo()
Julia Version 1.12.0-rc1
Commit 228edd6610b (2025-07-12 20:11 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 10 × Apple M1 Max
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 8 virtual cores)

julia> pkgversion(JET)
v"0.10.6"

[aviatesk]

This probably relates to JuliaLang/julia#59165 and hopefully fixed by JuliaDebug/JuliaInterpreter.jl#700.

[lgoettgens]

JuliaDebug/JuliaInterpreter.jl#700 isn't released yet, right? I'll probably wait for a release then to test it out.

[aviatesk]

Yeah, I tagged a new release just now.

[lgoettgens]

I locally reverted @fingolfin 's workaround for this (basically just revert the changes to types.jl from https://github.com/oscar-system/GAP.jl/pull/1214/files#diff-525588e68b2421901be164965272940effa093de733a3fd36c4b9a4344b8c20c) and still get the same unexpected ccall to Core.tuple(:GAP_DeclareGapObj, GAP.libgap) as in the initial message here.

So, unfortunately, JuliaDebug/JuliaInterpreter.jl#700 does not fix it (at least not on julia 1.12)

[aviatesk]

Ah, the complete fix would probably require Julia 1.13. Thanks for testing this..

[fingolfin]

Thank you for investigating, @aviatesk .

Luckily our workaround can stay in permanently, so if we can verify this is indeed fixed in Julia 1.13, that'd be good enough, at least from my POV :-)

[lgoettgens]

I just tested it again, this time with julia 1.13.0-DEV.1325 and JET v0.10.10, and the unexpected ccall is still there.

[aviatesk]

Ok, thanks for testing. It's a shame the problem isn't fixed. It seems JET's current report_package approach, loading the whole package with JuliaInterpreter, is just prone to these kinds of issues. Using Revise to collect signature info from the package seems like the most reliable way. I'll fix this in v0.11.

[aviatesk]

Will be fixed on JET v0.11. Please use report_package(GAP::Module) instead.