Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Olympus cellSens VSI support? #248

Closed
andreped opened this issue Feb 16, 2023 · 22 comments
Closed

Olympus cellSens VSI support? #248

andreped opened this issue Feb 16, 2023 · 22 comments

Comments

@andreped
Copy link
Contributor

Hello! Just came across this fantastic project, and found it extremely relevant for a variety of my projects.

We wish to use DSA to be able to visualize the WSIs that lie on a server, but I was surprised to find that I was unable to visualize a WSI stored in the cellSens VSI format, and it is resulted in the UI to hang and not displaying the image at all. HistomicsUI works fine with other openslide formats such as Aperio SVS, but the Olympus' format is the most common format at my department. Hence, it would be great if DSA supported it.

I just tried to add a WSI from the OpenSlide test data. Pick any of these WSIs and try for yourself to reproduce the issue.

From what I can see here, DSA depends on large_files which supports Bioformats which you can use to read these images.
@andreped
Copy link
Contributor Author

I think I see what the problem is. I tried to simply upload a single WSI (.vsi + corresponding folder) through the upload feature, and it would not let me.

So I guess it is likely that DSA does not yet support the cellSens VSI format, which requires you to have a .vsi file and a corresponding folder which contains the image data. Any comments, @manthey?

@dgutman
Copy link
Contributor

dgutman commented Feb 16, 2023 via email

@andreped
Copy link
Contributor Author

andreped commented Feb 17, 2023 •
edited
Loading

Download any of these zips from the openslide test data (see here), uncompress, and try to import and visualize with DSA.

Note that initially I was not uploading but importing, that is linking the data to a local drive, which in that case I was able to load both the ".vsi" and corresponding folder within the collection.

@dgutman
Copy link
Contributor

dgutman commented Feb 17, 2023 via email

@andreped
Copy link
Contributor Author

No rush. Enjoy your vacation!

Might be that I'm just missing some dependency, like Java or something, as I believe javabridge is used for bioformats, but as I believe all this is running within docker, I would think this should just work, at least prompt an error if not. But idk.

@andreped
Copy link
Contributor Author

I believe this is the cause why I'm unable to read the .vsi format:

dsa-girder-1     | Traceback (most recent call last):
dsa-girder-1     |   File "/opt/venv/lib/python3.9/site-packages/javabridge/jutil.py", line 290, in start_thread
dsa-girder-1     |     env = vm.create(args)
dsa-girder-1     |   File "_javabridge.pyx", line 654, in _javabridge.JB_VM.create
dsa-girder-1     | RuntimeError: Failed to create Java VM. Return code = -5
dsa-girder-1     | Failed to create Java VM
dsa-girder-1     | SLF4J: A number (4) of logging calls during the initialization phase have been intercepted and are
dsa-girder-1     | SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
dsa-girder-1     | SLF4J: See also http://www.slf4j.org/codes.html#replay

I would think all dependencies required were inside the docker image. Do I need anything locally? Right now I'm testing it on a macOS 12 Monterey laptop.

@manthey
Copy link
Contributor

manthey commented Feb 17, 2023

VSI files are read by the bioformats plugin, which runs in Java. The default docker images include all dependencies, including an appropriate version of java. I don't know why it if failing to start the jvm on your machine : -5 is usually the EIO error code, which is a cryptic IO error.

When bioformats reads the sample VSI files, it doesn't expose any information about lower resolution levels, so it ends up being very slow. They do eventually load and display.

@andreped
Copy link
Contributor Author

andreped commented Feb 17, 2023 •
edited
Loading

When bioformats reads the sample VSI files, it doesn't expose any information about lower resolution levels, so it ends up being very slow. They do eventually load and display.

I'm testing this on a laptop with 16 GB VRAM with macOS 12. Might be that it just takes an extreme amount of time to load/render, as I have limited amount of cores and memory. Can test this on a Ubuntu server next week. Have you tried if the WSI I mentioned works with HistomicsUI on your local setup?

@andreped
Copy link
Contributor Author

After a while, it seems like something is rendered from the OS-1.vsi (from the openslide test data here). However, it only seems like a single tile is rendered.

Are you able to reproduce the same on your end? As I have challenges getting DSA to work on my ubuntu 18.04 server currently (made separate issue about that here), I'm still stuck testing on my macbook with 16 GB RAM.

Skjermbilde 2023-02-19 172650
Skjermbilde 2023-02-19 172714

@andreped
Copy link
Contributor Author

After about 1h, I was surprised to see that it has gotten further. Nowhere near rendering the full WSI. In my case I'm often working with images of the size 200k x 160k, and I assume the clinicians that will be using this solution will not be that pleased...

So the VSI reader does indeed work, it is just extremely slow for this large images. Perhaps this is resolved with more memory or something I can setup differently in docker? If it is just the reader that is slow, is it possible to make it faster? FAST has great support for reading cellSens VSI as well as the other OpenSlide formats. It is also possible to access from Python (see pyFAST).

Skjermbilde 2023-02-19 175537

@dgutman
Copy link
Contributor

dgutman commented Feb 19, 2023 via email

@andreped
Copy link
Contributor Author

I normally use QuPath if I just want to visualize these WSIs, which uses the same Bioformats plugin as Fiji does. It reads and renders the WSI in real time, and you can zoom around and pan the view without any real issue, similarly as for other pyramidal TIFF formats, such as Generic TIFF or Aperio SVS, which OpenSlide supports (see here for formats).

I have never had any success reading these gigantic WSIs in Fiji. I don't Fiji is suitable for it. Just tried now and it did not produce any image - could be that I just need to play around with some import settings however.

I've also co-developed FastPathology, which uses FAST to read the image (removing the need for Bioformats and javabridge-annoyances). Can also use pyFAST to read from the VSI format into numpy arrays, but reading and rendering is happening in C++. Hence, it should be really fast. There is an ongoing open-source project called LearnPathology where they were able to stream tiles using FAST and render with OpenSeadragon (which I see DSA also supports), hence, that should be possible.

QuPath, FastPathology, and the LearnPathology web solution renders this image a lot faster than DSA. So there is definitely room for improvement. But since QuPath has extremely fast reading speeds with Bioformats, I guess it should work, but as QuPath is a Java software and DSA runs Bioformats through Javabridge, that could explain the poor read speeds.

@dgutman
Copy link
Contributor

dgutman commented Feb 19, 2023 via email

@dgutman
Copy link
Contributor

dgutman commented Feb 19, 2023 via email

@andreped
Copy link
Contributor Author

If there are python bindings we can probably add that library as a tile source/ reader.. I'll look into it.

Yes, there are. See here:
https://pypi.org/project/pyFAST/

Here is an example on how to read patches as numpy arrays with pyFAST:
https://fast.eriksmistad.no/generate_tissue_patches_from_wsi_8py-example.html

Using FAST to build up the tiles with OpenSeadragon worked wonders with OpenSeadragon. See here for how you could build up the tile server, at least thats how they did it in the LearnPathology project.

@manthey
Copy link
Contributor

manthey commented Feb 20, 2023

There was a straightforward way to speed up opening some files via bioformats (see girder/large_image#1063). With this, the VSI files from https://openslide.cs.cmu.edu/download/openslide-testdata/Olympus/ open pretty quickly (without that change, they open tediously slowly).

Before this, one of those files took ~20 minutes to serve tiles on my Ubuntu machine, now it is a few seconds.

@andreped
Copy link
Contributor Author

Before this, one of those files took ~20 minutes to serve tiles on my Ubuntu machine, now it is a few seconds.

Uuuuh! Sounds great! Will it work to clone the main branch of DSA now, or do I need to wait a little to get the latest release of large_image? I can run an experiment later today if it works :]

@dgutman
Copy link
Contributor

dgutman commented Feb 20, 2023 via email

@manthey
Copy link
Contributor

manthey commented Feb 20, 2023

pypi updated a bit ago.

@andreped
Copy link
Contributor Author

pypi updated a bit ago.

I assume the docker image would need to be rebuilt and updated to get the latest changes no, as per installation instructions? Alternatively I could try building the image myself and testing it (for debugging and testing purposes).

@manthey
Copy link
Contributor

manthey commented Feb 20, 2023

The various docker images are currently being built and tested by CI and will show up within an hour.

@andreped
Copy link
Contributor Author

andreped commented Feb 20, 2023 •
edited
Loading

Just tried the fix with the latest docker image. Works wonders and extremely rapid! Great work, @manthey!

Will do some more testing on much larger WSIs tomorrow on the production server, but I believe this should work fine for my application.

As this issue has now been resolved, you can close the issue.

Screenshot 2023-02-20 at 21 32 48

@manthey manthey closed this as completed Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dgutman @manthey @andreped