Skip to content

Commit 27f7de6

Browse files
committed
Add section on offset tables
1 parent 7521e26 commit 27f7de6

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

data/downloading-data/direct-loading.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,24 @@ volume = seg.get_volume(
157157
See [this][11] page for more information on highdicom's `Image` class, and
158158
[this][12] page for the `Segmentation` class.
159159

160+
### The Importance Offset Tables
161+
162+
Achieving good performance for these frame-level retrievals requires the
163+
presence of a "Basic Offset Table" or "Extended Offset Table" in the file.
164+
These tables specify the starting positions of each frame within the file.
165+
Without an offset table being present, libraries such as highdicom have to
166+
parse through the pixel data to find markers that tell it where frame
167+
boundaries are, which involves pulling down significantly more data and is
168+
therefore very slow. This mostly eliminates the potential speed benefits of
169+
frame-level retrieval. Unfortunately there is no simple way to know whether
170+
a file has an offset table without downloading the pixel data and checking it.
171+
If you find that an image takes a long time to load initially, it is
172+
probably because highdicom is constucting the offset table.
173+
174+
Most IDC images do include an offset table, but some of the older pathology
175+
slide images do not. [This page][14] contains some notes about whether
176+
individual collections include offset table.
177+
160178

161179
[1]: https://cloud.google.com/python/docs/reference/storage/latest/
162180
[2]: https://pydicom.github.io/pydicom/stable/index.html
@@ -171,3 +189,4 @@ See [this][11] page for more information on highdicom's `Image` class, and
171189
[11]: https://highdicom.readthedocs.io/en/latest/image.html
172190
[12]: https://highdicom.readthedocs.io/en/latest/seg.html
173191
[13]: https://highdicom.readthedocs.io/en/latest/image.html#lazy
192+
[14]: https://github.com/ImagingDataCommons/idc-wsi-conversion?tab=readme-ov-file#overview

0 commit comments

Comments
 (0)