Add JLJITLinkMemoryManager (ports memory manager to JITLink) #60105
Conversation
xal-0
force-pushed
the
jitlink-cgmemmgr
branch
3 times, most recently
from
5d5362e to
0b04319
Compare
Ports our RTDyLD memory manager to JITLink in order to avoid memory use regressions after switching to JITLink everywhere (JuliaLang#60031). This is a direct port: finalization must happen all at once, because it invalidates all allocation `wr_ptr`s. I decided it wasn't worth it to associate `OnFinalizedFunction` callbacks with each block, since they are large enough to make it extremely likely that all in-flight allocations land in the same block; everything must be relocated before finalization can happen. I plan to add support for DualMapAllocator on ARM64 macOS, as well as an alternative for executable memory later. For now, we fall back to the old MapperJITLinkMemoryManager. Release JLJITLinkMemoryManager lock when calling FinalizedCallbacks
xal-0
force-pushed
the
jitlink-cgmemmgr
branch
from
0b04319 to
c5c9964
Compare
|
About jitlink everywhere, are you planning to address llvm/llvm-project#63236? That caused us some pain when we switched aarch64-linux. |
|
That's what this change addresses, though the fallback to I thought I'd defer that work both because it's separable and because there is a better option on macOS that I am working on now: we can use the new APRR (aka fast permission restrictions). That lets us create special RWX mappings and toggle whether each thread sees it as R-X or RW- independently and without any system calls like mprotect. |
|
Cool, it wasn't to clear to me what were the memory use regressions mentioned in the first post, thanks for explaining it! |
2 participants
Ports our RTDyLD memory manager to JITLink in order to avoid memory use regressions after switching to JITLink everywhere (#60031). This is a direct port: finalization must happen all at once, because it invalidates all allocation
wr_ptrs. I decided it wasn't worth it to associateOnFinalizedFunctioncallbacks with each block, since they are large enough to make it extremely likely that all in-flight allocations land in the same block; everything must be relocated before finalization can happen.I plan to add support for DualMapAllocator on ARM64 macOS, as well as an alternative for executable memory later. For now, we fall back to the old MapperJITLinkMemoryManager.