Notes on AI, research, and the work.
234 pieces on generative and agentic systems, the research underneath, and what it takes to ship them.
- Oct 2026 · 5 min
DynaFlow: teaching one agent to write the workflow, not the answer
DynaFlow fine-tunes only a main agent to write executable JavaScript workflows that coordinate frozen sub-agents, and reports 88.08% across six benchmarks. The idea is sound and the training signal is interesting, but the headline 10.9-point margin over baselines comes mostly from one benchmark where the baselines had no search access.
multi-agent-systemsagent-orchestrationagentic-aifine-tuningRead - Oct 2026 · 5 min
Trial by trace: grading the harness, not just the agent
TraceTrial-Bench scores agent harnesses separately from task outcomes and finds that 31.1% of runs succeed on the task while failing at least one harness-reliability criterion. The framing is the right one for agent builders; the evidence is only as strong as its LLM-judged scoring.
agentic-aiagent-harnessesllm-evaluationbenchmarksRead - Oct 2026 · 5 min
Which agent should you improve first? IF-MAP measures leverage, not blame
IF-MAP ranks the agents in an LLM multi-agent workflow by how strongly end-to-end loss responds to small prompt perturbations, then spends the improvement budget on the top one. Across six workflows it roughly doubles the gain of picking an agent at random, at a fraction of the cost of improving every agent.
multi-agent-systemsllm-agentsprompt-optimizationinfluence-functionsRead - Oct 2026 · 4 min
What OpenAI learned scaling ChatGPT's storage layer on Cosmos DB
OpenAI's Part II on Habitat describes a circular journey from shared to isolated and back to multi-tenant Cosmos DB containers, plus the caching and replication policy behind a reported 12x read speedup. The lasting lesson is that tenancy has to be changeable.
infrastructuredistributed-systemsmulti-tenancydatabasesRead - Oct 2026 · 5 min
IQuest-Q1: a training recipe for agentic CLI models
IQuest-Q1 is an open-weight 320B/15B-active MoE trained for agentic coding across multiple CLI harnesses. Its technical report is most valuable as a recipe: measurable stability headroom, harness-aware RL with failure attribution, serious reward-hacking defenses, and multi-teacher distillation plus merging.
agentic-aicoding-agentsmixture-of-expertsreinforcement-learningRead - Oct 2026 · 3 min
Voxtral 3: Mistral packages the full voice agent loop
Mistral's Voxtral 3 bundles new transcription, realtime and TTS models with a voice agent SDK that runs listen, reason, act and speak in one streaming pipeline. Strong diarization and sovereign deployment, but an incremental platform step rather than a research shift.
voice-agentsspeech-recognitiontext-to-speechmistralRead - Oct 2026 · 6 min
Is complex agentic memory always needed? A naive RAG baseline says often not
A new paper shows that dense retrieval plus reranking over raw conversation history ranks first among compared memory systems in six of seven benchmark-model settings, at a fraction of the ingestion cost. Before building an agent memory stack, check this baseline.
agentic-aiagent-memoryragretrievalRead - Oct 2026 · 4 min
Agent Skills moves to the Agentic AI Foundation
Anthropic is donating the Agent Skills standard to the Linux Foundation's Agentic AI Foundation, next to MCP. This is a governance change rather than a capability change, and its most important item is the roadmap for versioning and skill security.
agent-skillsagentic-aiopen-standardsai-governanceRead - Oct 2026 · 5 min
RAG techniques don't add up, and the winner changes with corpus size
A controlled study of 450 RAG configurations shows that retrieval gains measured one component at a time largely fail to stack, that retrieval quality and answer faithfulness reward different design choices, and that the best pipeline at 5K documents is not the best at 200K.
retrieval-augmented-generationrag-evaluationinformation-retrievalllm-systemsRead - Oct 2026 · 6 min
Cache engineering: let agents drop KV entries instead of rewriting history
A Tsinghua paper proposes letting agents drop and re-position entries in their own KV cache rather than trimming or summarizing the transcript, which avoids the re-prefill that context editing forces. Paired with the AKV serving engine, a simple rolling-drop policy cuts average cost by 27.7% while slightly improving benchmark scores.
agentskv-cachellm-servingcontext-engineeringRead - Oct 2026 · 3 min
Thousands of agents, one perturbation each: distributed reasoning over a brain CRISPR atlas
OpenAI, PerturbAI and Scripps Research ran thousands of parallel agents over a whole-brain CRISPR atlas and produced a reasoning atlas of candidate leads in 32 minutes. The most useful result is the single-agent ablation: 11 citations versus more than 700.
agentic-aimulti-agent-systemsai-for-sciencecomputational-biologyRead - Oct 2026 · 6 min
MAS-Guard: one guardrail layer from single agents to multi-agent systems
MAS-Guard pairs local per-agent guardrails with developer-owned charters and a control plane of deterministic and semantic checks, cutting attack success by 71.5-86.3% across six benchmarks. I walk through the design, the ablation, and why it reads as careful systems integration.
agent-securitymulti-agent-systemsguardrailsprompt-injectionRead - Oct 2026 · 5 min
Anthropic's new defenses against distillation attacks
Anthropic reports that layered safeguards cut the reasoning data distillers can extract from Claude by 10 to 100 times. I walk through the three layers, the numbers, and the trade-offs.
distillationmodel-securityanthropicapi-abuseRead - Oct 2026 · 6 min
Into the unknown: what happens when agents write papers nobody has written
A new benchmark has nine frontier agents write follow-up papers on unpublished directions from DOE lab scientists. None of 135 submissions meets every condition, and 114 fail an integrity check, mostly for claiming work that was never done.
ai-agentsbenchmarksscientific-researchagent-integrityRead - Oct 2026 · 5 min
PROBE and the third axis of agent evaluation: auditing policy compliance
PROBE trains a small RL auditor to talk to a frozen tool-using agent like a customer and surface trace-level policy violations. The idea is incremental, but the paper's most useful findings are about reward design, verifier honesty, and how shallow these audits still are.
agentic-aillm-evaluationreinforcement-learningred-teamingRead - Oct 2026 · 5 min
Mosaic and the multi-session gap in agentic RL
Agents that compact their context or fork subagents run as several sessions, but most RL training assumes one. Mosaic gives each session its own critic-free credit signal, targeting a real problem with evidence so far from one benchmark and one small model.
agentic-rlreinforcement-learningllm-agentscontext-managementRead - Oct 2026 · 5 min
Grounding lab agents in physical state, not just good reasoning
Google Research's new work on autonomous wet labs argues that the bottleneck for scientific agents is no longer ideation but grounded execution and fast experimental feedback. A stateful digital twin of the lab, plus closed design-test loops, is what lets an agent correct its own mistakes.
agentic-ailab-automationdigital-twinsai-for-scienceRead - Oct 2026 · 5 min
Librarian agents: training a model to organize the corpus before anyone asks
A new Stanford and CMU paper trains an LLM with reinforcement learning to turn a raw document pile into a navigable filesystem for a downstream agent. It reframes long-context retrieval as a learned, offline build step, and the reported gains are large enough to take seriously.
agentic-airetrieval-augmented-generationreinforcement-learninglong-contextRead - Oct 2026 · 4 min
Muse Spark Open: Meta's 816B multimodal mixture-of-experts, with open weights
Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts model with about 816B total parameters, about 42B active per token, and a 1M-token context. Here is what shipped, what the architecture says about running it, and where I think it fits.
open-weightsmixture-of-expertsmultimodalllm-inferenceRead - Oct 2026 · 5 min
ActionOPD: distilling an agent's decisions, not its tokens
ActionOPD distills a frontier agent into a small model by rewarding the student when its sampled actions semantically agree with a black-box teacher's sampled actions on the student's own states. It needs no teacher logprobs and no shared tokenizer, which makes distilling from API-only models a practical option.
agent-distillationon-policy-distillationagentic-aitool-useRead - Oct 2026 · 6 min
More methods, better agents? A reality check on self-distillation in agentic RL
A controlled benchmark of ten self-distillation methods against plain GRPO, covering 132 runs and about 12,000 H800 GPU-hours, finds no method that wins consistently. It also shows that final scores can hide unstable training and that extra compute often buys nothing. Here is what that means when you pick an agent-RL recipe.
agentic-aireinforcement-learningself-distillationgrpoRead - Oct 2026 · 5 min
PixelMoE: the LLM mixture-of-experts recipe works in pixel space
PixelMoE takes the standard LLM mixture-of-experts recipe (token-choice routing, a shared expert, bias-based load balancing) and applies it unchanged to pixel-space diffusion. At matched active parameters it beats dense models at every scale tested, and it reaches ImageNet FID 1.47 at 256 and 1.53 at 512 without representation alignment or perceptual losses.
mixture-of-expertsdiffusion-modelspixel-space-generationflow-matchingRead - Oct 2026 · 4 min
Google DeepMind and Singapore put a scam detector on the phone
Google DeepMind and GovTech Singapore fine-tuned lightweight Gemma models to detect conversational scams on-device, so private chats never leave the phone. It is a small applied project, but a clean template for privacy-preserving safety.
on-device-aigemmascam-detectionprivacyRead - Oct 2026 · 5 min
Agent memory needs a backward pass
ReHarness adds a backward revision step to text-based agent memory: after each task, the trajectory is used to rewrite the reflections that were retrieved for it. It is a simple fix for stale and overgeneralized memory in online continual learning, and it needs no replay, no validation set, and no gradients.
agentic-aiagent-memorycontinual-learningllm-agentsRead - Oct 2026 · 6 min
Long-horizon agent data fails when the environment does the work
TerminalHorizon builds 1,535 terminal-agent environments by checking, with an independent agent, that the hard work was not solved in advance. The data-engineering idea is worth copying, but the benchmark gains need more careful accounting than the paper gives them.
agentic-aisynthetic-dataterminal-agentsllm-evaluationRead - Oct 2026 · 5 min
VisionWeave: teaching a multimodal model where detail is worth paying for
VisionWeave trains an MLLM to choose, block by block, between four fine visual tokens and one coarse one. On Qwen3.8-27B it drops 43% of visual tokens at 98.9% of native quality and more than doubles SGLang throughput on a long-video workload. Here is how it works, and where I would and would not trust it.
multimodal-llmsinference-efficiencyvisual-tokensllm-servingRead - Oct 2026 · 5 min
From given to gathered evidence: training a compact agent to investigate a patient over time
CASE trains an 8B vision-language agent to choose which visits, organs, MRI slices, and tools to inspect across longitudinal UK Biobank records, and introduces a 50,401-question benchmark built around that task, which the authors plan to make available to approved researchers. The recipe is familiar agentic RL; the real contributions are the environment, the reward design that keeps synthesis from collapsing, and a benchmark that measures acquisition rather than recall.
agentic-aimedical-aireinforcement-learningvision-language-modelsRead - Oct 2026 · 7 min
Seeing is not predicting: what SpaceCast-Bench says about spatial reasoning in VLMs
SpaceCast-Bench tests whether vision-language models can build a scene from views, apply an unseen intervention, and reason about the result. The best model scores 58.0% against 87.2% for humans, and explicit 3D evidence helps far more than generated images or video.
vision-language-modelsspatial-reasoningbenchmarksembodied-aiRead - Oct 2026 · 5 min
TokenRouter: what it takes to serve token-level model routing
Token-level routing lets a small and a large model share a single response, but single-model serving engines can't run it efficiently. TokenRouter gives each model its own subserver and adds a delayed-batching scheduler, and the more useful lesson in it is about where serving overhead really lives.
llm-servingmodel-routinginference-optimizationbatchingRead - Oct 2026 · 3 min
The Open TTS Leaderboard trades votes for instruments
Hugging Face's Open TTS Leaderboard ranks open-source, multilingual text-to-speech and voice cloning models on objective metrics: ASR-based error rate, speaker similarity and latency. Results take hours instead of weeks of voting. It's an evaluation resource, not a new capability, and it says plainly what it can't measure.
text-to-speechevaluationvoice-cloningmultilingualRead