MRRC Prisma Phantom QC

Build stats.json w/ peak values in background, phase encoding, and alias noise masks. Also runs SNR and tSNR within the phantom-containing voxels.

helpers/plots.R uses the flywheel DB to create a csv and plot. helpers/readme.md has more. MRRC Prisma 1 to Prisma 3 outputs on https://wiki.mrrc.pitt.edu/doku.php?id=data:qc.

Some care was taken to work on Flywheel but not depend on it. manifest.json details the flywheel configuration.

Other QC includes:

• Total coil FWHM in FID/ can be run as a flywheel file-curator gear or as a stand alone script. FID/fwhm.py handles both.
  • Works for qa_fid (default, adds fwhm to session info) and svs_se_40_wat via ACQ_LABEL (adds fwhm_svs to session info). Run by cron. See helpers/01_flywheel_hpc_mrrcqa.bash.
• z-shim value alerting is handled by shim_notify.py in the NPAC fork of file-curator's examples

Matlab

An MRC ("compiled", no license required) container exists (20260221).

See make test-docker-mlpy for smoke test, building, and running (second test) the container.

docker run --rm \
  -v $PWD/input:/input:ro \
  -v /tmp/out:/out  \
  npac/mrrcqa-ml:1.5.1.20260105 \
    input/QA_PRISMA3QA_20240809_180204_160000/EP2D_BOLD_P2_S2_5MIN_0003/ 0 /out/

Interactive with flywheel

See run-fwid.py with flywheel session id from web UI. Will run maltab code in temporary directory.

Dockerfile

Dockerfile.matlab-python uses the container created by maltab (R2024+) in mlbin/build_mrc_container.m to add flywheel specific run.py and it's dependencies. But entrypoint remains the matlab dostat program.

Octave

Program/QC.m input/QA_PRISMA3QA_20240809_180204_160000/EP2D_BOLD_P2_S2_5MIN_0003/ outputs/

jq .snrpk < outputs/stats.json # 242.8687622000182
ls outputs/bars.png

speed

import flywheel; fw=flywheel.client()
b = fw.jobs.find('gear_info.name=~mrrcqa,transitions.complete>2026-02-20,state=complete')
xx =  [(fw.get(x['parents']['project']).label, fw.get(x['parents']['session']).created.strftime("%F %H:%M"), x['created'].strftime("%F %H:%M"), x['gear_info']['version'], (x['transitions']['complete'] - x['transitions']['pending']).total_seconds()) for x in b];
print(pd.DataFrame(xx, columns=['proj','sesdate','queued','version','totaldur']).sort_values(['sesdate','proj','queued']))

     proj          sesdate           queued         version  totaldur
Prisma1QA 2026-02-20 11:40 2026-02-20 11:42 1.4.20250409.01  6390.041
Prisma2QA 2026-02-20 11:41 2026-02-20 11:42 1.4.20250409.01  6396.647
Prisma2QA 2026-02-20 20:45 2026-02-20 20:46 1.4.20250409.01  6990.388
Prisma1QA 2026-02-21 11:30 2026-02-21 11:32 1.4.20250409.01  6483.734

Prisma1QA 2026-02-23 11:34 2026-02-23 11:36  1.5.2.20260222   166.410
Prisma2QA 2026-02-23 11:35 2026-02-23 11:37  1.5.2.20260222   125.270
Prisma1QA 2026-02-24 11:44 2026-02-24 11:46  1.5.2.20260222   113.757
Prisma2QA 2026-02-24 12:05 2026-02-24 12:07  1.5.2.20260222   139.759

TODO

• break up Program/dostat.m and add tests using input/trunc
  • optimize/vectorize esp. std command? It's surprisingly slow
  • profile against matlab runtime - switch to ML compiled version if octave is much slow
• slim docker container: build octave without Xorg or java (likely to be useful for other containers later)

Testing

For Octave (slower) version of the pipeline

make test
make test-docker

Using octave %!test in-file tests. See bottom of Program/readshimvalues.m.

Editing

• Makefile guides through steps

  • see .docker then .gear for packaging
  • mess of other files for .gear-run.txt with various input files setup (download input zip and setup config.json)
    • fw-beta is used for gear setup
    • this started from hello-world gear
    • Program/run.py still used for flywheel entry (sets up input/ and output, eventually handles DB)

• bump version in manifest.json: "version": and "custom": { "gear-builder": { "image": "npac/mrrcqa:1.0.20240822" } } current match.

Noteworth files

file	desc
Program/QC.m	octave script, docker entrypoint into @chms matlab QC code
Program/run.py	flywheel-aware entrypoint for gear. unzips and dispatches to QC.m
Makefile	make interface to building and testing recipes
Dockerfile	describes software dependencies, recipe for container
config.json	fw-beta gear run input. built with Makefile

Interfacing with Flywheel

• repo init with copy from https://gitlab.com/flywheel-io/scientific-solutions/tutorials/Gear-Building-Tutorial/-/tree/hello-world

• for archive as file type in manifest.json, see https://docs.flywheel.io/User_Guides/user_file_types_in_flywheel/ (c.f nifti)

• clasification qa picked form list on https://flywheel-io.gitlab.io/tools/app/cli/fw-beta/gear/upload/

Phantom

Uploading

as in Makefile, the final upload uses fw-beta gear upload .. But the F5 managed firewall might block requests. And flywheel will error if using podman instead of docker.

Error: writing blob: determining upload URL: http: no Location header in response

Flywheel bug

v20.1 (2024-03) cannot receive docker push, fw gear upload is broken until v20.2

Podman vs Docker

fw-beta gear install wont work with podman and docker is rate limited. But can copy from podman to docker using save/load.

via SO, can docker load. General pipeline is like, but see flatten-docker.sh (below) for specifics.

docker save localhost/npac/mrrcqa:1.1.20250312.01 | bzip2 | pv | ssh r docker load

Load/Push to avoid firewall

We can use VM on the same host as fw-core (hosting docker registry) to work around enterprise firewall blocking gear upload/docker push

See flatten-docker.sh for removing layers and transferring. Needed to resolve unsupported NFS /var/lib/docker mount on testing VM with xattr on files (from matlab base image).

mitmproxy: debug firewall

SSLKEYLOGFILE="/tmp/ssl.log" mitmproxy
# trust anchor --store mitmproxy.crt # crt from http://mitm.it/
HTTPS_PROXY=http://localhost:8080 docker push fw.mrrc.upmc.edu/mrrcqa:1.1.20250312.01