Image generators fabricate what they don't know, and search alone doesn't fix it
A new benchmark shows frontier visual generators collapse to 21-28/100 on prompts that require world knowledge outside their training data, and naive retrieval-augmented generation makes the problem worse, not better.
The failure mode nobody was measuring
Ask a text model about something that happened after its training cutoff, or about a niche fact it never saw enough times to memorize, and you get a hedge, a wrong guess, or — if it has tool access — a search query. Ask an image generator to draw a specific post-cutoff product launch, a trending character, or an obscure real entity, and you get something worse: a confident, fully-rendered, wrong answer. No hedge. No uncertainty signal. Just a fabrication rendered with the same polish as a correct answer.
That gap is the subject of Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation, by Haozhe Wang and eleven coauthors. The paper's framing is simple and, once stated, obviously correct: visual generators are trained on fixed corpora, but the requests users make are unbounded, evolving, and deeply long-tailed — new characters, trending entities, events that happened after the model froze. Rendering quality has kept improving. World-knowledge coverage hasn't, and until now nobody had a benchmark precise enough to show the size of the gap.
Quantifying a 40-point collapse
The authors built two things to make the failure mode measurable rather than anecdotal: SearchGen-20K, a set of 20,839 prompts spanning twelve failure categories and twenty-two domains, and SearchGen-Bench, the benchmark built on top of it. Alongside these they released SearchGen-Corpus-1M, a pre-executed multimodal retrieval corpus, so the benchmark is reproducible offline without live search infrastructure — a detail that matters because it means the eval isn't hostage to whatever a search API returns on a given day.
The headline number: frontier open generators score only 21 to 28 out of 100 on SearchGen-Bench. That's not a minor regression on a hard subset — it's a collapse invisible to existing benchmarks, because existing benchmarks don't test long-tail world knowledge in the first place. Most image-generation evals measure rendering fidelity, prompt adherence, and aesthetic quality on prompts a model was already likely to have absorbed during training. SearchGen-Bench is deliberately built to probe outside that comfort zone, and the score drop shows how much of a model's apparent competence was riding on the training distribution matching the eval distribution.
Why the obvious fix doesn't work
The natural response to a knowledge-boundary problem is retrieval: bolt search onto the generator, let it pull in reference images and facts for entities it doesn't know, and let agentic tool use close the gap the way it has for text models. The paper tests this directly and finds that naive search fails. Not fails to fully close the gap — fails outright, in a specific and instructive way: it retrieves indiscriminately, injecting noise into prompts the generator already handles correctly. A model that could render a well-known character from its own weights gets handed retrieved reference material it didn't need, and that unwanted context measurably degrades the output.
This is the paper's most useful finding, because it explains why the RAG playbook that works for text doesn't transfer cleanly to generation. Text retrieval augments a token stream that already has strong internal error-correction — extra context that's slightly off gets diluted by everything else the model knows. Image generation conditions much more tightly on the prompt and its context; irrelevant or noisy retrieved material doesn't get diluted, it gets rendered. Search has to be selective, not additive.
A generator-specific, moving boundary
The paper's diagnosis is that every generator has a knowledge boundary: the line between what it can internalize through training and what must stay in external context and get retrieved at generation time. Two things make this boundary hard to work with. First, it's generator-specific — a bigger or differently-trained model has internalized a different slice of the long tail, so the boundary isn't a fixed property of the task, it's a property of the specific model being asked. Second, it's evolving — new entities and events keep appearing after any training cutoff, so the boundary keeps moving even for a fixed model as the world changes around it.
The authors' proposed remedy is a teach-then-search co-training framework: instead of hand-specifying which prompts need retrieval, the training process discovers the boundary itself, learning to answer directly from internal knowledge when possible and to invoke search only when the generator is actually operating outside what it was taught. They report that even a minimal version of this recipe produces monotonic improvement — each round of co-training pushes the boundary further and the benchmark score up, rather than plateauing or oscillating.
That result is the seed of something bigger: a generator that can tell the difference between what it knows and what it needs to look up is a generator that can, in principle, keep improving as the world changes, without a full retrain every time a new entity becomes relevant. The paper is explicit that this could lay the foundation for recursive self-improvement in visual generation grounded in world knowledge that no training corpus will ever fully contain.
Where the evidence is solid and where it's promissory
It's worth being precise about what's demonstrated versus proposed. The benchmark, the corpus, and the naive-search-fails finding are concrete, quantified, and reproducible — the released dataset and pre-executed corpus mean anyone can rerun the eval and check the 21-28/100 number themselves. The teach-then-search co-training framework, by contrast, is described qualitatively in the paper; the "minimal version" that shows monotonic improvement is a proof of concept, not a fully specified, ablated training recipe. If you're evaluating this paper for what to build on, the benchmark is the immediately actionable artifact. The co-training method is a promising direction that still needs the kind of scaling and ablation work that turns a qualitative result into an engineering recipe.
For anyone building agentic image-generation systems with retrieval in the loop, the practical takeaway is narrower than "add search": you need a mechanism — learned or heuristic — that gates when to retrieve, not just a retriever bolted onto every prompt. Skipping that gate is exactly the failure mode this paper measured and named.