napari reader for multiscale-spatial-image datasets

[GenevieveBuckley]

I came across the multiscale-spatial-image package recently, and like it a lot. I thought it would be handy for me to have a napari plugin that can read these and return a pyramid/multiscale structure napari understands.

It's a very rough draft, but you can find it here:

pip install git+https://github.com/GenevieveBuckley/napari-multiscale-spatial-image.git

I thought I'd share in case some part of this could be helpful to the wider spatial-image community. Possibly it could be developed into something more general, too.

Most of the curent napari reader plugins for zarr crash if there are multiple datasets contained within the top .zarr root node. Of course you can write a 1-2 line script to open a specific level dataset in zarr and then pass it to napari, but it's less convenient than being able to drag and drop the file.

[thewtex]

@GenevieveBuckley beautiful! 🥇 💯

It looks like it has not been published yet?

Would you like to move the repository to the spatial-image organization?

@giovp @LucaMarconato any thoughts on how we can make this work best with https://github.com/scverse/napari-spatialdata ?

[LucaMarconato]

Great work! 🔥

In spatialdata the MultiscaleSpatialImage objects need to pass a validation step which ensures that they contain some extra information, such as an in-memory representation of the new transforms specs. For example, some of these validation constraints are described here: scverse/spatialdata#115. To facilitate this we provide parsers that takes a MultiscaleSpatialImage object as input and return the same object with the needed extra elements in .attrs.

In napari_spatialdata we don't load the data from disk as I see it is done in the plugin above, but we use the in-memory representation. Also, we have the constraint that we can have only one variable per scale. If we have multiple images per scale we create multiple MultiscaleSpatialImage objects.

So I think that the two plugins cover different use cases. But they have also a lot of logic in common, so feel free to take code from us/happy to contribute.

Notes:

• the code for transformations is still under review improviements to transforms: robustness and ergonomics scverse/spatialdata#100 and the io is still not 100% compatible with the NGFF incoming transforms spec Conversion between "new transforms" and "ngff transforms" scverse/spatialdata#114, Currently considering only scale in the transformations for multiscales scverse/spatialdata#125. We will finalize this when the transformation spec gets approved Coordinate systems and new coordinate transformations proposal ome/ngff#138 (rendered specs here).
• the napari code needs heavy refactoring, we will do this probably in a few weeks. For the moment I can point you to this file, which contains most of the logic.

[thewtex]

@LucaMarconato thanks for the updates -- looks very synergistic!

To facilitate this we provide parsers that takes a MultiscaleSpatialImage object as input and return the same object with the needed extra elements in .attrs

If there are more additions or constraints to be compatible with ngff in multiscale-spatial-image, happy to work together to get these integrated directly. 🤜 🤛

parsers that takes a MultiscaleSpatialImage

@astropenguin I am wondering if there is a path to elevate xarray-dataclasses awesome support for dataclasses to pydantic classes so we have built-in validation on the data structures?

@LucaMarconato @GenevieveBuckley it sounds like a good path forward may be to publish napari-spatialdata and napari-multiscale-spatial-image, creating functionality that serves end-user needs in napari. Then, look for ways to share common type conversion, especially MultiscaleSpatialImage -> napari Image layer in one package or in a separate common package.

[LucaMarconato]

I agree, probably the easiest is to sync again after polishing the napari-spatialdata code and publishing the plugins, and then see if we can extract some code logic into a common place.

[giovp]

late reply but I agree with @LucaMarconato re targeting different use case yet with potential of reusing implementation. I think it'd make sense to have a stand-alone multiscale-spatial-image viewer in napari. I could also see how some part of the readers could then be re-used by napari-spatialdata. In this case, napari-spatialdata would depend on napari multiscale-spatial-image.

@astropenguin I am wondering if there is a path to elevate xarray-dataclasses awesome support for dataclasses to pydantic classes so we have built-in validation on the data structures?

This would be really cool. Wonder if some synergies could also be explored with https://github.com/xarray-contrib/xarray-schema . I guess an ideal scenario would be to have one tool to both define the schema and use it for parsing and validation.

[GenevieveBuckley]

It looks like it has not been published yet?

No, it's a super rough draft right now, so I haven't published anything.

I think I'd like to get
(1) the translate metadata actually used by the plugin EDIT: seems like I already did this here and
(2) someone else familiar with these data structures to look it over and suggest edge cases where we might run into trouble (eg: are label datasets likely to be stored alongside the image datasets?).

Would you like to move the repository to the spatial-image organization?

Sure, I can do that. EDIT: Done, it's at https://github.com/spatial-image/napari-multiscale-spatial-image

... thoughts on how we can make this work best with https://github.com/scverse/napari-spatialdata ?

I didn't know about napari-spatialdata! Whatever we can do to help or join forces, I'm all for 😄