Relion 5 (SBGrid) + Topaz CUDNN_STATUS_MAPPING_ERROR

[duetduet]

Hi,

I am using Topaz via Relion 5 (launch with SBGrid) and get a "CUDNN_STATUS_MAPPING_ERROR".

I think this may be because Topaz was compiled with Python 3.8, all its native extensions (including cuDNN bindings) are built against Python 3.8's ABI. But PyTorch 2.3.1 expects Python 3.10+ ABI.

"Latest Stable PyTorch requires Python 3.10 or later." [https://pytorch.org/get-started/locally/]

What could be happening:

• Topaz (Python 3.8) creates tensors with one memory layout
• PyTorch 2.3.1 (Python 3.10+ ABI) interprets them incorrectly
• Corrupted descriptors are passed to cuDNN
• cuDNN rejects the malformed tensor mapping

For my current GPU architecture with a CUDA 12.4 the best PyTorch means PyTorch 2.4+ with cu124. I guess my next best bet is a conda environment with python 3.11 and proceed from there?

Has anyone else had this issue with Topaz via SBGrid?