Electrophysiology LORIS-MRI API
Description
This document is a proposal to add an optional server API to LORIS-MRI, which would be used to call Python code from the server endpoint and in turn enable the implementation of new interactive imaging features like MEG sensor positions and electrophysiology topology map visualization.
Planned features
More precisely, the planned features that would use this LORIS-MRI API are:
- Topology map: Based on some user input in the LORIS electrophysiology viewer, the server uses the MNE Python library to read the electrophysiology data and generate an image sent back and displayed to the user. It is important to note that the image is generated based on user input, so it is not possible to just generate it at ingestion time.
- MEG sensor positions: The MNE library is used to read the MEG data and get its sensor positions, which is then displayed in the 3D map of the electrophysiology viewer.
- Other future works (see future prospects section).
While these features could theoretically be implemented through a cross-language architecture that uses PHP endpoint and Python script subprocess. Doing so complexifies the code and the workflow, as well as limits the integration of the endpoint with its pipeline (more on it in the future prospects section). Moreover, there is a working proof-of-concept as the two discussed features have already been independelty implemented using a LORIS-MRI API-like architecture.
Main points
What this proposal is:
- An imaging-specific API used to implement high-interaction imaging-related features.
- A fully optional/opt-in API for specific these specific imaging-related routes.
What this proposal is not:
- An alternative or replacement to the existing LORIS API.
- A new dependency for existing code or projects.
Current state
Before delving into the architecture of the proposed LORIS-MRI API, a few words about the current state of the LORIS-MRI Python codebase, which has evolved significantly over the last two years:
- Foundations:
- Good database abstraction (dedicated models, query modules, decorators... even some database documentation !).
- Good utilities package.
- CI:
- Linter (Ruff)
- Strict typing for modern code (Pyright)
- Gradual typing for legacy code
- Daily integration tests for installation and all major pipelines
- Decent modularity:
- The codebase is divided in several packages with explicit dependencies (
loris-bids-reader, loris-eeg-chunker, loris-utils...).
- Packages can be individually installed or uninstalled.
- New independent packages can easily be added.
- The codebase could be further modularized (
loris-dicom-importer, loris-bids-importer...) but not a blocker for adding new packages.
- Access to scientific libraries (PyDICOM, PyBIDS, MNE...)
- Access to pipeline internals
Architecture
This proposal would add the following to LORIS-MRI:
- The generic server package: Contains the core server code, and provides hooks for other modules to add their own endpoints.
- The dedicated electrophysiology package: Hooks into the generic server package to add the electrophysiology-specific endpoints.
Two important points:
- The proposed packages are new packages: this proposal does not aim to change the existing code, and most importantly, keeps the new server separate from the current code (the server may depend on the current code, but not the opposite).
- The architecture is extensible: while the current proposal focuses on electrophysiology, other imaging modules could also hook into the server to add their own routes if needed.
Libraries
This proposal would use the following libraries:
- FastAPI: Modern standard to define server endpoints in Python, used in production by companies like Microsoft, Netflix, Uber or Cisco. This library is also used by the current topology map code.
- Uvicorn: The recommended web server for FastAPI, although there are other options.
These libraries are very popular Python libraries and do not cause conflicts with the current libraries used in LORIS-MRI.
Installation
The new LORIS-MRI API would be fully opt-in and not present in the default installation of LORIS-MRI, the optional installation of the electrophysiology API would be done using the following command:
pip install ./python/loris_ephys_api
This command simply installs the electrophysilogy API (and the generic LORIS-MRI server dependency) as a standard Python package.
Deployment
The LORIS-MRI API can be deployed as service on any LORIS VM or container, this can be done by adding a standard Linux service file such as /etc/systemd/system/loris-mri-server.service, with a content such as the following:
[Unit]
Description=LORIS electrophysiology API
After=network.target
[Service]
User=lorisadmin
Group=lorisadmin
WorkingDirectory=/opt/loris/bin/mri
ExecStart=/bin/bash -c 'source /opt/loris/bin/mri/environment && exec uvicorn loris_server:app --port 8000 --host 127.0.0.1'
Restart=always
RestartSec=5
Environment="PYTHONUNBUFFERED=1"
[Journal]
Storage=persistent
StandardOutput=append:/var/log/loris/loris-mri-server.log
StandardError=append:/var/log/loris/loris-mri-server.log
[Install]
WantedBy=multi-user.target
The LORIS-MRI API can then be used as any Linux service with commands like the following:
sudo systemctl start/stop/restart loris-mri-server to start/stop/restart the server (note that the server is always started by defaultt thanks to Restart=always).sudo journalctl -u loris-mri-server to view the server logs.sudo journalctl -u loris-mri-server -f to view the server logs in real-time.sudo journalctl -u loris-mri-server -p err to view only the server error logs.
Exposure
Since this proposal adds a web server to LORIS-MRI, the question of exposure and security is critical. Although the core server code should be rather simple and the number of endpoints limited, I believe the safest and easiest configuration would be to use the main LORIS server (PHP) as a proxy to the LORIS-MRI API. As such, the client only communicates with the main LORIS PHP server that forwards requests to the LORIS-MRI API, while the latter is not exposed outside of the machine it is running on.
The schema would look like the following:
External network │ Internal network
│
│ localhost
┌──────────┐ HTTPS │ ┌──────────────┐ (or VM/container) ┌─────────────┐
│ Client │ ──────────────▶│ Main LORIS │ ────────────────────▶│ LORIS-MRI │
│ │ ◀──────────────│ server (PHP) │ ◀────────────────────│ API │
└──────────┘ Response │ └──────────────┘ └─────────────┘
│ │
│ │ Authenticates 🔒 Not
│ │ & Validates Exposed
│ ▼
│ ✅ Logged-in?
│ ❌ Reject if not
Moreover, the main LORIS server could ensure that the user is logged in, and refuse to forward any request for unauthenticated users, adding yet another layer of security. Permissions to read the requested imaging files would still be checked on the LORIS-MRI API using the standard LORIS permission model.
From a performance standpoint, I do not expect any significant slowdown from the request forwarding as it all happens on the same server, especially given that LORIS is not a high-traffic or particularly optimized application.
Maintenance
The code for the new LORIS-MRI API would initially be maintained by the projects that use it, C-BIG and EegNet, and more specifically me and Jefferson. However, other projects that want to use this LORIS-MRI API are also welcome to contribute to the maintenance of the code. Nevertheless, the code would aim to be simple and thoroughly documented both in its architecture and mode of operation such that anyone can use it or modify it without requiring external help.
Future prospects
As said before, the LORIS-MRI API could also be used to implement future extensions of LORIS-MRI.
Personally, I would like to use it to provide the following imaging endpoints:
- DICOM importer endpoint
- BIDS importer endpoint
Because the LORIS-MRI API is in the same programming language as the imaging processing pipelines, this API can integrate more deeply with said pipelines. This notably allows the API to get more precise information about the state of a pipeline, which in turn allows to create more informative and interactive user experiences. Moreover, large file upload has historically proved hard to implement in the PHP API, so the LORIS-MRI Python API could provide an alternative for these specific edge cases.
More precisely, I would ultimately like to provide CLI clients that allow to upload DICOM and BIDS datasets to LORIS instances from any remote machine, using standard LORIS accounts and permissions, and that would provide secure feedback to the user using these tightly integrated endpoints.
Electrophysiology LORIS-MRI API
Description
This document is a proposal to add an optional server API to LORIS-MRI, which would be used to call Python code from the server endpoint and in turn enable the implementation of new interactive imaging features like MEG sensor positions and electrophysiology topology map visualization.
Planned features
More precisely, the planned features that would use this LORIS-MRI API are:
While these features could theoretically be implemented through a cross-language architecture that uses PHP endpoint and Python script subprocess. Doing so complexifies the code and the workflow, as well as limits the integration of the endpoint with its pipeline (more on it in the future prospects section). Moreover, there is a working proof-of-concept as the two discussed features have already been independelty implemented using a LORIS-MRI API-like architecture.
Main points
What this proposal is:
What this proposal is not:
Current state
Before delving into the architecture of the proposed LORIS-MRI API, a few words about the current state of the LORIS-MRI Python codebase, which has evolved significantly over the last two years:
loris-bids-reader,loris-eeg-chunker,loris-utils...).loris-dicom-importer,loris-bids-importer...) but not a blocker for adding new packages.Architecture
This proposal would add the following to LORIS-MRI:
Two important points:
Libraries
This proposal would use the following libraries:
These libraries are very popular Python libraries and do not cause conflicts with the current libraries used in LORIS-MRI.
Installation
The new LORIS-MRI API would be fully opt-in and not present in the default installation of LORIS-MRI, the optional installation of the electrophysiology API would be done using the following command:
This command simply installs the electrophysilogy API (and the generic LORIS-MRI server dependency) as a standard Python package.
Deployment
The LORIS-MRI API can be deployed as service on any LORIS VM or container, this can be done by adding a standard Linux service file such as
/etc/systemd/system/loris-mri-server.service, with a content such as the following:The LORIS-MRI API can then be used as any Linux service with commands like the following:
sudo systemctl start/stop/restart loris-mri-serverto start/stop/restart the server (note that the server is always started by defaultt thanks toRestart=always).sudo journalctl -u loris-mri-serverto view the server logs.sudo journalctl -u loris-mri-server -fto view the server logs in real-time.sudo journalctl -u loris-mri-server -p errto view only the server error logs.Exposure
Since this proposal adds a web server to LORIS-MRI, the question of exposure and security is critical. Although the core server code should be rather simple and the number of endpoints limited, I believe the safest and easiest configuration would be to use the main LORIS server (PHP) as a proxy to the LORIS-MRI API. As such, the client only communicates with the main LORIS PHP server that forwards requests to the LORIS-MRI API, while the latter is not exposed outside of the machine it is running on.
The schema would look like the following:
Moreover, the main LORIS server could ensure that the user is logged in, and refuse to forward any request for unauthenticated users, adding yet another layer of security. Permissions to read the requested imaging files would still be checked on the LORIS-MRI API using the standard LORIS permission model.
From a performance standpoint, I do not expect any significant slowdown from the request forwarding as it all happens on the same server, especially given that LORIS is not a high-traffic or particularly optimized application.
Maintenance
The code for the new LORIS-MRI API would initially be maintained by the projects that use it, C-BIG and EegNet, and more specifically me and Jefferson. However, other projects that want to use this LORIS-MRI API are also welcome to contribute to the maintenance of the code. Nevertheless, the code would aim to be simple and thoroughly documented both in its architecture and mode of operation such that anyone can use it or modify it without requiring external help.
Future prospects
As said before, the LORIS-MRI API could also be used to implement future extensions of LORIS-MRI.
Personally, I would like to use it to provide the following imaging endpoints:
Because the LORIS-MRI API is in the same programming language as the imaging processing pipelines, this API can integrate more deeply with said pipelines. This notably allows the API to get more precise information about the state of a pipeline, which in turn allows to create more informative and interactive user experiences. Moreover, large file upload has historically proved hard to implement in the PHP API, so the LORIS-MRI Python API could provide an alternative for these specific edge cases.
More precisely, I would ultimately like to provide CLI clients that allow to upload DICOM and BIDS datasets to LORIS instances from any remote machine, using standard LORIS accounts and permissions, and that would provide secure feedback to the user using these tightly integrated endpoints.