Lost in the z‑axis
Can generative AI reconstruct true 3D tissues from serial sections?
Arshia HemmatWellcome Sanger Institute
1Why 3D
Tissues are 3D. Our gene maps are 2D.
Spatial transcriptomics reads gene expression where it happens, but only in one thin section of a much larger volume.
- A vessel winding through the tissue becomes unrelated dots in one section
- 3D tissue niches are seen in a single plane
- Signalling between layers goes unseen
In one section, a single winding vessel looks like five unrelated dots.
Five vessels? Or one very committed vessel?
1Why 3D
Getting 3D gene maps today is costly
Native 3D methods
Thick tissue or whole-volume assays exist, but depth and area are limited, protocols are long and the instruments are specialised.
Sequence every section
Cut the tissue into serial sections and profile each one. Building a full volume this way multiplies the cost and the work.
Neither is practical for routine studies.
2The gap
Sequence a few sections. Predict the rest.
H&E is cheap, so every section can be stained and imaged. Spatial transcriptomics is costly, so only some sections are profiled.
Can we fill in gene expression on the rest from H&E alone?
2The gap
Where current methods fall short
One section at a time
Most H&E-to-gene models predict each section on its own, so the same structure drifts from one section to the next.
They don’t scale to stacks
3D-aware methods exist, but memory grows with the stack. On our 52-section embryo, the 3D baseline ran out of memory on an 80 GB GPU.
3HoloTea
Three ideas behind HoloTea
Look at the neighbours
For every spot, find tissue that looks the same on the sections above and below, and use it as context.
Realistic starting noise
Gene counts are mostly zeros with rare bursts. Generation starts from noise with that same shape, predicted from the H&E.
Built for whole stacks
Cost grows in step with the number of spots, so all 327,532 spots of the embryo stack fit on one GPU.
4Does it work?
The test: hide sections, then predict them
Whole sections are held out. HoloTea predicts their expression from H&E alone, and we compare it with what was measured.
Breast cancer (HER2)Public, several sections per patient
Breast cancer (ST-Data)Public cohort
EmbryoIn-house: 52 serial sections, Visium HD, 327,532 spots
4Does it work?
Marker genes land in the right placeQuick quiz: which one did we measure?
CNMDbasal plate
CEMIPcardinal vein
A held-out embryo section the model never saw in training. The segmented pattern along the spinal cord comes back. Predictions are smoother than the measurement: spot-level noise is suppressed.
The giveaway: the measured one is the noisy one. Biology is messy; our model is tidier than it should be. This section was never seen in training, and the segmented pattern along the spinal cord comes back.
4Does it work?
Most accurate of the methods we compared
- Image only (UNI)
- 2D model (STFlow)
- 3D model (ASIGN)
- HoloTea
Correlation between predicted and measured expression of each gene on held-out sections. Higher is better.
Top highly variable genes. For the embryo, the 3D model had to be shrunk to fit in memory.
5What’s next
What it can’t do yet
Not yet
- Looks only one section up and down
- Predicts a panel of a few hundred genes, not the whole transcriptome
- Smooths low counts, so rare signals can fade
- Gives no confidence score for each prediction
Next
- Longer-range 3D context through the stack
- Calibrated uncertainty for every spot
- More genes and more tissue types
- Adding other imaging, such as immunofluorescence
Sequence a few sections. Predict the rest.
HoloTea turns routine serial histology into 3D gene expression maps.
Thank you
To my collaborators and the whole lab.
Arshia HemmatWellcome Sanger Institute