The gene layout carries more signal than the transformer does

Most single-cell foundation models were built by taking the language-model playbook and swapping in genes for words: a cell becomes a sequence of gene tokens, fed through a transformer trained the way GPT is trained. It works, but it throws away two things biology cares about — how strongly a gene is expressed, and which genes tend to move together. A new preprint, scVision: A Vision Foundation Model for Single-Cell Biology (Yesiloglu, Mostafa, Zou, Alizadeh, Wu, Xing, Adeli, and Islam), argues that if you fix that representation problem, you can drop the language-model framing entirely and treat a cell as what it actually is once you lay it out right: an image.

I read this paper as a physicist first and an AI practitioner second, and the thing that made me stop was not the accuracy numbers. It was an ablation, buried in the results, that says the coordinate system you choose matters more than the network you train on top of it.

From token sequence to spatial image

The construction is straightforward once you see it. Take the 10,816 most informative genes across a cross-tissue atlas and build a gene-by-gene co-expression matrix — how often each pair of genes lights up together across cells. Then use Gromov-Wasserstein optimal transport to place those genes on a fixed 104×104 lattice so that genes with correlated expression end up as spatial neighbors. This layout is computed once, pan-tissue, and reused for every cell afterward.

Given that fixed grid, any cell's transcriptome becomes what the authors call a scImage: pixel position encodes which gene, pixel intensity encodes expression level. A gene program — a coordinated module of genes turning on together, the kind of thing a biologist would call a pathway or a cell-state signature — shows up as a local texture patch instead of a scattered set of token positions in a sequence. The representation does the work that architecture would otherwise have to learn from scratch.

From there, the modeling is close to vanilla computer vision. A ViT-base encoder (about 86 million parameters) plus a lightweight decoder is pretrained with masked image modeling, hiding 75% of scImage patches at a time, across 72 million human cells drawn from a roughly 94-million-cell corpus. The encoder is then frozen — no fine-tuning — and used directly for downstream tasks.

What frozen embeddings buy you

The evaluation is zero-shot cell-type classification across six held-out studies the model never trained on: kidney cortex, ovary, focal cortical dysplasia cortex, Crohn's-disease ileum, human retina, and a 53-cell-type multi-organ single-nucleus reference. Against token-based baselines — scGPT, scFoundation, Geneformer — scVision comes out ahead on balanced accuracy in every comparison reported:

• Kidney cortex: 0.47 vs. 0.40 (scFoundation), 0.38 (scGPT), 0.15 (Geneformer)
• Ovary: 0.83 vs. 0.79, 0.78, 0.59
• Focal cortical dysplasia: 0.65 vs. 0.64 (scGPT), 0.57 (Geneformer), 0.46 (scFoundation)
• Multi-organ reference: 0.52 vs. 0.43, 0.36, 0.32

On multi-study batch integration, scVision matches the strongest token-based model while preserving more biological structure — and it does this without ever seeing a batch label during training, which is notable because batch-correction supervision is usually where these models cheat a little.

The ablation that actually matters

Here is the finding I keep coming back to. The authors ran two ablations on the multi-organ benchmark, holding everything else fixed:

1. Shuffle the gene layout — keep the exact same ViT, same weights, same masked-image-modeling pretraining — and accuracy falls from 0.52 to 0.29.
2. Remove the ViT entirely, replace it with a plain perceptron applied to the flattened scImage, and accuracy falls from 0.52 to only 0.46.

Destroying the geometry hurts more than deleting the network. Put differently: most of what makes scVision work isn't the transformer at all — it's the optimal-transport step that decided where each gene lives relative to its neighbors. The vision transformer is doing real but secondary work; the coordinate system is doing the heavy lifting.

Why this generalizes past biology

I spend most of my time building production systems around generative and agentic models, not wet-lab data, but this result is a clean instance of something I see constantly: teams reach for a bigger model or a different architecture before they've asked whether the representation handed to that model already threw away the signal they need. Token sequences are a convenient default, not a law of nature — they're the right choice for language because language actually has a dominant sequential dependency structure. Single-cell expression doesn't have that structure; it has a relational, roughly permutation-invariant structure, and forcing it into token order was always going to cost something. scVision's contribution isn't a cleverer transformer — ViT-base is off-the-shelf — it's noticing that the correct inductive bias was a coordinate system, not a sequence, and building that coordinate system with optimal transport rather than hand-designed heuristics.

The practical lesson for anyone building on top of foundation models: before you scale the network, check whether your input representation already encodes the relationships you're asking the model to learn. Sometimes the cheapest accuracy gain is upstream of the model entirely.

A caveat worth naming: this is a preprint (submitted July 15, 2026, not yet peer reviewed), the layout is built from a fixed 10,816-gene pan-tissue vocabulary, and the paper doesn't yet detail how well it extends to genes outside that set or to non-human data. Treat the numbers as strong first-round evidence, not a settled benchmark.

Full paper: arxiv.org/abs/2607.14163.