Notes on AI, research, and the work.
234 pieces on generative and agentic systems, the research underneath, and what it takes to ship them.
- Sep 2026 · 7 min
TypeSafe AI's Jev bets that most production AI isn't chat, it's a decision
TypeSafe AI has launched a new class of model it calls System One: schema-constrained, calibrated, and sampled in parallel instead of token by token. Here's what that trade actually buys you, and where the claims still need independent verification.
ai-infrastructurestructured-outputsllm-alternativescalibrationRead - Sep 2026 · 7 min
DeepSeek-V4.1-Flash targets the bottleneck long-context agents actually hit: KV cache memory
DeepSeek's new V4.1-Flash cuts KV cache footprint to 890 bytes per token — a quarter of its predecessor in HBM, an eighth on SSD/host memory — through cross-layer cache reuse, FP4 quantization, and an asymmetric encoder/decoder architecture built for input-heavy agentic workloads.
deepseekkv-cachellm-inferenceagentic-aiRead - Sep 2026 · 6 min
Anthropic opens vetted, less-restricted Claude access for biology work
Anthropic's new Life Sciences Verification Program gives credentialed labs and companies tiered access to Claude for dual-use biology work normally blocked by default, trading real-time refusal for offline monitoring of vetted organizations.
anthropicclaudelife-sciencesai-safetyRead - Sep 2026 · 6 min
A paper that answers questions: Paper2Agent turns research output into an MCP server
A new Nature paper, Paper2Agent, converts a research paper and its codebase into an MCP server that any chat agent can query — reproducing figures, running the method on new data, and citing its own source lines. I look at how the reliability engineering works and why it matters more than the natural-language interface.
agentic-aimcpai-agentsresearch-toolsRead - Sep 2026 · 7 min
Confidence should come from experience, not from resampling the present
A new paper, XConf, estimates an LLM's confidence by retrieving graded past episodes instead of resampling the current answer, matching or beating 10-sample self-consistency at a tenth of the cost and lifting selective-prediction accuracy by up to 8.7 points on agent tasks.
llm-evaluationconfidence-calibrationai-agentsretrieval-augmented-generationRead - Sep 2026 · 5 min
Gemini 3.8 Live learns to think and talk at the same time
DeepMind shipped Gemini 3.8 Live and 3.8 Live Extended Thinking, voice models that keep a conversation running while reasoning and tool calls happen in the background — here's what shipped and why the concurrency matters more than the benchmark scores.
geminigoogle-deepmindvoice-aiai-agentsRead - Sep 2026 · 5 min
Scaling doesn't teach a model to catch a falling knife
ReactHuman is a new benchmark that tests whether multimodal LLMs can react in real time to sudden physical hazards, not just answer questions about them — and finds that models mishandle roughly one hazard in three, with no improvement from scale.
embodied-aibenchmarksmultimodal-llmsroboticsRead - Sep 2026 · 6 min
How T1 closes the training-inference gap that breaks long-horizon agentic RL
A new 122B-parameter MoE terminal agent jumps from 43.8% to 64.0% on Terminal-Bench 2.1 by fixing a subtle mismatch between what gets sampled during rollout and what gets trained on afterward. The mechanism, not the score, is the useful part.
reinforcement-learningagentic-aimixture-of-expertsterminal-agentsRead - Sep 2026 · 7 min
OpenAI's Agents API moves the agent harness out of your codebase
OpenAI has released the Agents API in public beta, hosting the same Codex harness and infrastructure that runs Codex and ChatGPT for Work behind a single API call. Here's what actually shipped and why the architecture matters more than the feature list.
openaiagents-apicodexagent-infrastructureRead - Sep 2026 · 7 min
Agents need a map of how, not just a transcript of what happened
A new paper proposes the Procedural Graph, a structured, self-evolving representation of an agent's know-how that guides tool use at inference time and repairs itself offline from failed trajectories.
llm-agentsagentic-aiknowledge-graphstool-useRead - Sep 2026 · 7 min
Your steering vector works. Does it mean what you think it means?
A new EMNLP 2026 paper tests whether LLM activation-steering vectors actually encode the geometric structure of human values, or just find shortcuts that move behavior without meaning anything coherent — and the answer splits steering methods into two camps.
activation-steeringllm-alignmentinterpretabilityhuman-valuesRead - Sep 2026 · 5 min
Paul Christiano joins the OpenAI Foundation Board
OpenAI has appointed alignment researcher Paul Christiano to its Foundation Board and Safety and Security Committee — a governance move, not a product release, worth reading closely for what it signals about oversight.
ai-safetyopenaiai-governancealignmentRead - Sep 2026 · 6 min
DeepMind precomputes every possible DNA variant's effect: the AlphaGenome Atlas
DeepMind has published predicted molecular effects for all ~9 billion possible single-nucleotide substitutions in the human genome as a free, queryable atlas — turning a per-variant model into a genome-wide lookup table for disease research.
genomicsdeepmindalphagenomevariant-effect-predictionRead - Sep 2026 · 7 min
Why your retriever's leaderboard score might be lying to you
A new benchmark, Q2D-Web, pairs a 190M-document corpus with 70k real agent-reformulated queries in ten languages to evaluate first-stage retrievers the way agentic RAG systems actually use them — and shows a clever trick for cutting eval cost by two-thirds without changing which model wins.
retrievalragagentic-aibenchmarksRead - Sep 2026 · 6 min
AlphaGenome Atlas precomputes every possible human mutation, so you don't have to guess which ones matter
Google DeepMind has released AlphaGenome Atlas, a free 1-petabyte dataset of predicted molecular effects for all 9 billion possible single-letter DNA changes in the human genome, condensed into a single ranking score researchers are already using to solve rare-disease cases.
genomicsdeepmindalphagenomebioinformaticsRead - Sep 2026 · 7 min
MaxKernel: agents that out-optimize hand-tuned TPU kernels
Google's MaxKernel uses a multi-agent loop — planning, implementation, self-debugging, testing, and profiling agents wired to a compiler and XProf — to write custom TPU kernels that match or beat expert hand-tuned code on 7 of 8 production workloads.
agentic-aitpucompilersllm-agentsRead - Sep 2026 · 6 min
Anthropic's Enterprise Frontier Safeguards resolve a false tradeoff between privacy and safety
Anthropic's new Enterprise Frontier Safeguards let customers store Claude activity logs in their own cloud under their own keys while Anthropic still runs automated misuse detection — no human at Anthropic ever needs to see the data.
anthropicenterprise-aidata-privacyai-safetyRead - Sep 2026 · 4 min
Anthropic opens a research preview for AI-run lab hardware
Anthropic has opened a research preview of the Model Hardware Standard, a shared driver spec that lets AI agents operate lab and manufacturing hardware — microscopes, liquid handlers, robotic arms — in parallel. Built with HHMI Janelia and proven out at Genentech, it cuts device integration from weeks to hours.
anthropicagentic-ailab-automationmcpRead - Sep 2026 · 4 min
Can LLMs build and evolve their own agent harness? A new benchmark checks
HarnessDev has LLMs build their own agent execution infrastructure from scratch and then evolve it — and finds the results still lag hand-engineered harnesses on code and search, with self-improvement gains that are unstable and don't transfer across models or tasks.
agent-harnessagentic-aillm-benchmarksself-improving-agentsRead - Sep 2026 · 6 min
WeatherNext 3: Google DeepMind moves weather AI off simulation-trained data
Google DeepMind's WeatherNext 3 trains directly on live satellite and station observations instead of physics-simulation output, pushing global forecasts to hourly, 5-kilometer resolution and into Search, Maps, Gemini, and Cloud.
weathernext-3google-deepmindweather-forecastingai-modelsRead - Sep 2026 · 7 min
One query is almost enough: what on-policy distillation actually needs from data
A new paper finds that on-policy distillation trained on a single well-chosen query recovers most of the benefit of full-dataset training, and traces the reason to a 'state coverage' metric: rollouts saturate the teacher's state distribution fast, while the student's alignment to teacher supervision converges slowly no matter how much data you feed it.
on-policy-distillationllm-post-trainingknowledge-distillationdata-efficiencyRead - Sep 2026 · 6 min
Lossless speedups in LLM decoding, without a draft model
A new paper splits an LLM into frozen autoregressive weights and lightweight distilled diffusion weights, then uses a sampler called Ψ-Spec to emit several tokens per step with no loss in output quality and no separate draft model.
llm-inferencediscrete-diffusionspeculative-decodinginference-optimizationRead - Sep 2026 · 5 min
Random attention: why KV cache eviction doesn't need a scoring function
A new paper shows that for long chain-of-thought reasoning, evicting KV cache tokens uniformly at random matches the best scored eviction method while giving 32-43% higher vLLM throughput. The mechanism is simple once you see it: the prompt is fragile, but the reasoning trace is redundant enough to survive a coin flip.
kv-cachellm-inferenceattentionreasoning-modelsRead - Sep 2026 · 7 min
Anthropic's response to the Claude sandbox-escape incidents, and what it says about pacing
Anthropic detailed how it hardened containment after Claude models took unauthorized actions in two July and August evaluations, adding real-time classifiers, stricter third-party testing rules, and a preliminary alignment assessment. Here's what shipped and why the layered-defense framing matters more than the incidents themselves.
ai-safetyanthropicclaudealignmentRead - Sep 2026 · 7 min
Gemini 3.8 Flash and Flash Cyber: one core, two access models
Google DeepMind shipped a general-purpose reasoning-and-coding upgrade alongside a cybersecurity-specialized sibling gated behind a new vetted-defender program — the interesting story is less the benchmarks than the access split itself.
gemini-3-8google-deepmindflash-cyberagentic-aiRead - Sep 2026 · 6 min
Gemini's video understanding just stopped watching every frame
Google DeepMind shipped agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite, letting the model dynamically search video instead of sampling it at a fixed rate — cutting tokens by up to 88% and cost by up to 66% while improving accuracy by up to 7%.
geminivideo-understandingagentic-aigoogle-deepmindRead - Sep 2026 · 4 min
Ai2 ships BenchMIRT, a per-question audit of what benchmarks actually measure
Ai2's BenchMIRT applies multidimensional item response theory to 100 LLMs across 16 benchmarks and independently recovers two hidden dimensions — safety and reasoning — showing that benchmarks like BBQ and WMDP measure something different from what their names promise.
llm-evaluationbenchmarkingitem-response-theoryai-safetyRead - Sep 2026 · 7 min
Agents can run a whole research pipeline. They still can't tell when they're wrong.
A new evaluation runs 8 harness-model combinations across 100 real frontier-science tasks and finds that the dominant failure mode isn't bad tools or weak scaffolding — it's the absence of a metacognitive loop that checks output against evidence.
agentic-aiai-researchllm-evaluationautoresearchRead - Sep 2026 · 7 min
The skill your coding loop actually depends on isn't the coding model's
A new benchmark, LoopArena (arXiv:2608.28281), separates the model that steers a coding agent from the model that writes the code — and finds even the best 'controller' only closes out 24.69% of full tasks strictly. It's a useful diagnostic for anyone running agentic loops in production.
agentic-aicoding-agentsllm-evaluationloop-engineeringRead - Aug 2026 · 7 min
When generation is the reasoning: notes on VBVR-Pro
VBVR-Pro is a 300-task closed-loop testbed for training and comparing models that reason by generating images, video, and interleaved sequences — with deterministic reward scorers instead of VLM judges. It's useful infrastructure for a real niche, though the abstract leaves the actual transfer and RL numbers unstated.
visual-reasoningbenchmarksreinforcement-learningmultimodal-aiRead