From given to gathered evidence: training a compact agent to investigate a patient over time
CASE trains an 8B vision-language agent to choose which visits, organs, MRI slices, and tools to inspect across longitudinal UK Biobank records, and introduces a 50,401-question benchmark built around that task, which the authors plan to make available to approved researchers. The recipe is familiar agentic RL; the real contributions are the environment, the reward design that keeps synthesis from collapsing, and a benchmark that measures acquisition rather than recall.
Most medical AI benchmarks hand the model the evidence and ask for an answer. A chest film, a report excerpt, a vignette: someone has already decided what matters. In real longitudinal care, that decision is most of the work. Evidence is spread across visits, organs, sequences, and reports, and a clinician has to choose what to pull up before reasoning about any of it.
From Given to Gathered Evidence: Agentic Learning for Longitudinal Medical Reasoning (Shao et al., Durham University, DAMO Academy and collaborators) takes that seriously. It recasts longitudinal reasoning as evidence seeking, builds an environment where the agent controls the evidence path, and trains a Qwen3-VL-8B policy that, under the authors' matched harness, outscores the frontier models it was compared against. I think the framing is the most useful part of the paper; the training recipe is a careful, domain-specific assembly of techniques we already know.
The benchmark makes acquisition the task
The CASE benchmark is built on UK Biobank: 50,401 multiple-choice questions from 4,739 participants, with 4,850 questions from 455 held-out participants for test. Questions come from doctor-diagnosed conditions and ICD-10 records, with cohort-sampled distractors that exclude the participant's own diagnoses.
Each question links to a patient-specific environment holding baseline and follow-up MRI, a median of 2.26 years apart, across five organ regions and seven sub-modalities. The agent gets only the clinical narrative and the question. It then decides which tool, organ, sequence, visit, and frame to retrieve, and whether to compare across visits. Tools expose MRI views, Vista3D-derived measurements and overlays, and MedGemma-27B reports.
The metric is strict first-pass exact answer-set accuracy; a malformed answer scores zero. Because train and test participants are disjoint and the answer depends on that participant's evidence, the policy cannot memorize question-answer pairs. It has to go and look.
One policy, three roles
CASE defines three roles sharing one set of weights: a Radiologist that selects and compares acquisitions, an Anatomist that works through segmentation measurements, and a Consultant that adds generated reports and a reusable skill memory. They differ only in role prompt and tool registry. I prefer this to a committee of specialists: one policy that learns acquisition and synthesis jointly is easier to train, serve, and audit. Every episode ends with a five-phase synthesis (plan, multi-source review, cross-visit comparison, key insight, answer) that structures the reasoning without prescribing a tool order.
The training recipe
Stage one is rejection-sampling fine-tuning. Qwen3.8-Max records role-conditioned trajectories, a Kimi K2.6 judge scores each trajectory's insight, and trajectories with at least one tool call and a score of 0.4 or higher are kept, with no exact-answer gate. The paper reports roughly 140,000 retained trajectories while the abstract cites 14,734; I would like to see that reconciled. The authors are clear that this stage teaches the form of acting, not the reasoning.
Stage two is GRPO with eight rollouts per prompt. One engineering choice deserves attention: segmentation masks and reports are precomputed per participant, and RL replays observations keyed by (participant, tool, arguments). Tool choice, order, and stopping stay policy-generated, but nobody waits on a GPU-bound segmentation model inside the rollout loop. Anyone running agentic RL against expensive tools should copy this.
The return combines an answer reward (+1 correct, -1 wrong, -2 unparsed), a format term, a weighted rubric score, and a linkage term that rewards the top two insights in each group only when the answer is also correct. That linkage targets fluent reasoning that would satisfy a text-only judge while pointing to the wrong answer.
The interesting part: a brake on restating findings
In mid-to-late RL the policy began restating findings instead of committing to a conclusion, and correctness reward collapsed. The authors' explanation is plausible: a compact model learns the surface form of an explanation before it reliably resolves the answer.
Their fix is privileged on-policy self-distillation (OPSD). Each rollout is scored twice by a frozen snapshot of the current policy, once as-is and once with the reference answer inserted as a hint. The truncated reverse KL between those views over the synthesis region becomes a detached penalty subtracted from the return before group normalization. It is not a differentiable distillation loss; it reorders whole trajectories by synthesis quality, so the shared advantage also shapes acquisition. Trajectories that ramble pay for it.
The ablation supports the design. From the same RSFT start (65.7 micro accuracy), plain GRPO reaches 69.3, adding rubric feedback 70.6, adding OPSD 73.8, and both together 76.1. Without OPSD, the rubric-only run hits the late collapse.
Results and how I read them
Averaged over roles, CASE-8B reaches 76.1 micro / 75.1 macro accuracy, against 65.3 for GPT-5.4, 68.6 for Claude Opus 4.8, 70.2 for Kimi K3, 59.7 for the Qwen3.8-Max teacher, and 46.0 for the untrained base. Beating the teacher by about 16 points is the strongest evidence that RL on the agent's own interactions adds something imitation cannot. Accuracy also drops when acquisition is withheld or follow-up imaging is removed, which argues against answer priors doing the work.
I would hold the frontier comparison loosely. Every method gets the same tools and budgets, but the harness and synthesis contract were built around this pipeline, and only one model was trained inside it. The fair reading is that a compact policy trained in a well-specified environment beats general models dropped into it cold. Other caveats: answers are multiple-choice with cohort-derived distractors; the multi-select disease-systems column stays near ten percent for every model; reports are model-generated and the judges text-only. The authors acknowledge some labels may not be observable from the images at all.
What I take from it
The lasting contribution is the problem definition. If we want clinical agents that investigate rather than recite, we need benchmarks where the evidence path is the model's decision, and this one is a credible start. Code and specifications are planned for Hugging Face and GitHub, and the benchmark itself is not public yet: the authors say it will be made available through UK Biobank to approved researchers. The two pieces I would take into production are the replayable tool environment and using divergence from an answer-aware copy of the policy as a trajectory-level brake on degenerate synthesis.