Implement VRAM reclaim like CUDA.jl do?

[XLin0mu]

My situation is estimating different model architectures in a continuous REPL Main context to see their performance on tasks.

While using Reactant+Enzyme, GC.gc() won't work for reducing VRAM usage, and I have to restart whole julia REPL to release used VRAM. But LuxCUDA+Zygote allowing me to execute GC.gc(); CUDA.reclaim() for releasing VRAM instantly.

Is that possible to implement or not? Or else, is there any other methods to solve this, like explicitly marking object cleanable manually?

[wsmoses]

What is the error you're actually hitting?

Reactant/XLA by default makes a memory pool, which pre allocates memory chunks for use within an allocator. So even if you see memory usage as high, it should still be usable.

There are some variables you can use to change this, but it will make allocation more expensive of course:

Reactant.jl/docs/src/api/config.md

Line 49 in d635c44

- `XLA_REACTANT_GPU_PREALLOCATE`: Whether to preallocate GPU memory. Defaults to `true`.

[XLin0mu]

No directly concerning error message showed except CUDA-Out-Of-Memory, and then REPL crashed.

But I couldn't reproduce it again now unexpectedly even throughout my history code. I'll close this issue till it occurred clearly next time, thank you.