Sakana AI’s Agentic Scientist Automates Research, but Human Review Remains Essential

On October 15, 2026, Sakana AI released The Agentic Scientist, the third generation of The AI Scientist, which coordinates coding agents from hypothesis testing through paper writing. It also open-sourced AutoIllustrate, a tool that creates research figures with editable source files.

Key Points#

• The Agentic Scientist coordinates coding agents such as Claude Code and Codex to propose hypotheses, run experiments, analyze results, and draft papers.
• Parallel workers test ideas against predefined criteria, while writing and auditing agents link the paper’s claims to experimental evidence.
• Sakana AI evaluated the system across 70 end-to-end runs on five research tasks, but automated paper reviews alone cannot establish scientific correctness.
• The code is available under the Apache License 2.0, with a CPU-only example for trying the workflow.
• AutoIllustrate generates draw.io source files and optionally uses an image model for finishing, allowing researchers to revise figures later.

What the System Does#

The Agentic Scientist is a framework for managing research agents. It controls the information each agent receives, how work is evaluated, and how results pass between stages. Researchers can use it for autonomous exploration or specify research directions and validation requirements themselves.

A project requires a starting implementation and an evaluator that checks correctness and calculates scores. Agents can also attempt to create these from a research brief, but Sakana AI describes this capability as experimental. For an initial trial, use a provided example whose evaluator you can inspect.

From Experiments to a Paper#

Multiple workers explore ideas in parallel. Within each worker, a proposal agent defines a hypothesis, an experimental plan, and a rubric—the conditions an implementation must meet to pass. An execution agent modifies the code and runs evaluations. Workers periodically share findings, drawing on past successes and failures in subsequent attempts.

After exploration, the system selects the highest-scoring candidate among those that passed the checks. It compares that candidate with the initial implementation and runs ablation experiments, removing individual changes to determine which ones produced improvements.

A writing agent then drafts a paper based on the collected results. It also creates a table linking claims to measurements and saved artifacts. An auditing agent checks for inconsistent numbers and claims that exceed the evidence, prompting revisions before the draft is finalized.

What the Results Show#

In one collaboration, researchers used a development version to extend Error Diffusion, a neural network training method that does not use backpropagation. Humans set the direction and evaluation criteria. They also identified an implementation that scored highly despite violating the intended learning rule, and strengthened the checks. The resulting paper, “Diffusing Blame,” underwent external peer review and was accepted for an oral presentation at ALIFE 2026.

In a separate autonomous experiment, exact-match accuracy on a document information retrieval test rose from 52% to 98%. The system adjusted how per-document LoRA parameters were selected and combined. These results were obtained on synthetic documents; performance on real documents and more complex questions still needs verification.

For a broader evaluation, Sakana AI ran the research workflow 70 times across five tasks, using nine language models and four agent execution tools. Papers received scores from 1 to 10 from the company’s AI Reviewer tool. Newer models generally scored higher, while differences between execution tools using the same model were small. These were automated assessments, not independent validations of the research.

Where Human Review Is Needed#

The Error Diffusion example illustrates a central risk: a program can score highly while solving the wrong problem. Evaluators and rubrics can miss errors they were not designed to detect. Passing checks should trigger closer scrutiny, rather than automatic acceptance.

Literature coverage also remains a challenge. Coding agents may search the web during exploration, but dedicated literature searches occur during paper writing, rather than at the start of idea generation. The framework is also better suited to testing changes to a selected implementation than to reproducing strong methods from existing papers for comparison.

The reported evaluations cover computational machine learning research. They do not demonstrate suitability for physical experiments or research involving human participants. Humans remain responsible for research direction, implementation review, and conclusions.

How to Try the Tools#

The Agentic Scientist is available in Sakana AI’s AI-Scientist-v3 GitHub repository under the Apache License 2.0. The agsci CLI uses Claude Code by default and also supports Codex, Gemini CLI, and other tools. The README recommends running each stage of the workflow inside Docker.

Start with the CPU-only circle-packing example. The task is to place 26 non-overlapping circles inside a square while maximizing the sum of their radii. It lets you examine the process from experiments to a paper without choosing a research task that requires a GPU.

AutoIllustrate creates explanatory figures separately. It generates a draw.io file, checks for overlapping shapes and arrow placement, and optionally produces a PNG finished with an image generation model. Keep the draw.io file so you can edit shapes, arrows, and labels. The PNG is a separate output. The default setup requires Claude Code and an OpenAI API key; the image finishing step can be skipped.

Next Steps#

Run the CPU-only circle-packing example in the AI-Scientist-v3 README, choose one claim in the generated paper, and trace it back to the saved experimental evidence.

Source: https://gihyo.jp/article/2026/10/the-agentic-scientist-v3