Claude Code 7-Hour Autonomous Test: Rakuten Delivery Time Cut by 79%

Rakuten's official June 2025 case study describes Claude Code completing a complex activation-vector extraction task in vLLM during a single seven-hour run with occasional guidance. The same case reports a reduction in average time to market for new features from 24 working days to 5 days. It also reports 99.9% numerical accuracy on the code modification compared with a reference method.

Those figures are meaningful because the source identifies the project, the metric, and the comparison. They are not evidence that every repository can be changed safely in seven hours or that numerical accuracy is the same as software correctness. A team adapting the case needs tests, review, permission controls, and rollback procedures.

What You Will Learn

• What Rakuten and Anthropic actually reported
• Why a seven-hour run is different from unattended production work
• How to interpret 99.9% numerical accuracy and 79% time-to-market reduction
• How to design a safer evaluation for your own repository

What Rakuten Reported

Rakuten's official blog lists three headline results. Claude Code achieved seven hours of sustained autonomous coding on a complex open-source refactoring project. Rakuten reduced average time to market for new features by 79%, from 24 days to 5 days. The implementation also delivered 99.9% accuracy on complex code modifications.

The detailed account identifies the seven-hour task more precisely. Machine Learning Engineer Kenta Naruse asked Claude Code to implement an activation-vector extraction method in vLLM. Rakuten describes vLLM as a large open-source library with 12.5 million lines of code across multiple programming languages. Claude Code completed the job in one run, while Naruse provided occasional guidance and said he did not write code during those seven hours.

The source is strongest when read as a defined customer case. It describes a task, a codebase, a result, and a business metric. It does not provide a controlled comparison across all repositories, models, developers, or security environments.

Read the official Rakuten case study for the original wording. The coding benchmark guide explains why a case result should not be confused with a benchmark score.

What the Seven-Hour Run Did Not Prove

A sustained coding run is not the same as a fully unattended deployment. Rakuten's account says the engineer provided occasional guidance. That means the session had human oversight even if the engineer did not write code during the run. The distinction matters for permissions, review, and incident response.

The case also does not prove that vLLM alone caused the result. The outcome depended on Claude Code, the task design, the repository, the reference method, the environment, the tests, and the engineer's guidance. A different project may have different dependencies or acceptance criteria.

Nor does the result show that a model understands a 12.5-million-line codebase in one undivided context. Large projects are usually explored through search, file reads, symbol references, tests, and tool calls. The article should not turn a repository-size fact into a claim about context capacity.

The long-running AI agents guide adds a useful view of checkpoints, state, and recovery. Those controls are needed when a coding task continues for hours.

Understanding the vLLM Task

Rakuten describes the task as implementing a specific activation-vector extraction method in vLLM. The result was compared with a reference method and reached 99.9% numerical accuracy. This gives the reader a clear example of a task with a measurable output.

Numerical accuracy is suitable for a function whose output can be compared against a known result. It is not a complete measure for documentation, security, API compatibility, maintainability, or behavior under unexpected inputs. A team should define several acceptance checks when the work is broader than one mathematical or numerical method.

For a similar evaluation, keep the reference method independent from the generated implementation. Run the same input set through both versions. Record tolerances, failed cases, runtime, memory use, and test coverage. Then inspect the diff and the surrounding interfaces.

Use the AI coding-agent cost analysis to connect the evaluation with token use, context size, retries, and reviewer time.

How to Read the 99.9% Accuracy Figure

The phrase 99.9% accuracy needs a denominator and a test method. Rakuten's case frames it as numerical accuracy compared with the reference method. The public case does not turn that number into a general code-correctness rate. A reader should therefore preserve the qualifier whenever the result is quoted.

When reproducing the case, publish the test set, tolerance, reference implementation, environment, and failed examples. If the test set is private, state that limitation. A precise metric is valuable only when its boundary remains visible.

What the 79% Time-to-Market Result Means

Rakuten reports that average time to market for new features fell from 24 working days to 5 days, a 79% reduction. This is a business workflow metric. It is different from the time Claude Code spent editing files or the time a developer spent reviewing the result.

Time to market can include planning, implementation, testing, review, release preparation, and coordination. The case says Rakuten redesigned development workflows around Claude Code and used it across unit tests, API mocks, components, bug fixes, documentation, and code review. The observed result therefore belongs to a broader organizational process, not only to one model response.

Do not present 79% as a forecast. A team should record its own baseline, task mix, review policy, release process, and defect rate. If the process changes during the pilot, report that change with the result.

For a related discussion of sustained agent sessions, see the parallel coding comparison.

Autonomy and Human Guidance

Autonomy describes how much work the agent performs without direct step-by-step typing. It does not mean the agent has unlimited authority. Rakuten's account says the engineer provided occasional guidance. A safe implementation should preserve that distinction in its permission model.

Give the agent a task boundary, repository scope, test command, and stop condition. Keep deployment, deletion, credential use, billing changes, and external publishing behind a human approval step. If the agent reaches an unexpected state, it should stop and return evidence rather than guessing.

A long run should also produce checkpoints. Save the plan, commands, files changed, tests run, failures, and unresolved questions. These records make a later review possible even when the session spans several hours.

Parallel Sessions and Workflow Design

Rakuten's case says engineers can run multiple Claude Code sessions in parallel by delegating four tasks while focusing on a fifth. The case also describes a later ambient-agent project using 24 parallel sessions for different aspects of a large monorepo update. These are reported workflow examples, not a recommendation to use a fixed number of sessions.

Parallel work is useful when tasks have clear boundaries. It can create conflicts when workers edit the same files or depend on a shared decision. Use separate branches or worktrees, assign task owners, and require a synthesis review before merging.

Read the parallel agent comparison before treating a session count as a performance metric.

Security and Repository Permissions

A seven-hour run can touch many files and call many tools, so permission boundaries should be explicit. Start with read access and a disposable branch. Allow writes only inside the task scope. Block production credentials, destructive commands, and unrestricted network access unless a human approves the action.

Protect secrets from prompts, logs, and generated files. Review dependency changes and generated scripts. Run security checks separately from functional tests. A numerical result can be accurate while the surrounding change introduces a permission or supply-chain risk.

Use staged deployment and rollback controls. The feature-flag guide explains how a controlled release can limit the impact of a change.

Reproducing the Case Safely

Do not start by giving an agent access to a complete production repository. Select a representative task with a clear reference output. Create a sanitized snapshot, define the tools, set a time limit, and specify the review gate.

1. Define: describe the task, files in scope, reference output, and forbidden changes.
2. Prepare: create a branch or isolated environment with the required dependencies.
3. Run: record prompts, tool calls, checkpoints, guidance, and time spent.
4. Verify: run unit, integration, numerical, security, and interface checks as relevant.
5. Review: inspect the diff, generated files, dependency changes, and unresolved warnings.
6. Release: deploy only after approval, monitoring, and rollback are ready.

Use the AI agent implementation guide for a broader view of tool, workflow, and deployment boundaries.

What to Measure in Your Own Pilot

A useful pilot measures more than coding speed. Record time to accepted change, review time, test failures, rework, security findings, token use, tool failures, and developer guidance. Keep the task set stable enough to compare runs.

Report a range and the number of tasks. One successful run is a case observation. It is not a general performance guarantee.

Production Readiness Checklist

The Rakuten case shows that long-running coding work can produce a measurable result in a demanding codebase. Moving from a case study to production requires additional controls:

• Repository access is scoped and logged.
• Every generated change has an owner and review gate.
• Tests cover the behavior that matters, including failure cases.
• Security scanning and dependency review run before release.
• Monitoring detects regressions after deployment.
• Rollback can restore the previous version without agent access.
• Usage and cost are tracked by task and user.

These controls are not a rejection of autonomy. They make the result auditable and keep a long session from becoming an unreviewed production change.

Why Case Studies Need Careful Comparisons

Customer case studies are useful because they show a real workflow and a real outcome. They are not always controlled experiments. The repository, staff, task selection, tooling, baseline, and reporting method may differ from another organization.

Compare like with like. If one result measures time to market and another measures lines changed, do not place them in one ranking. If one accuracy figure compares numerical output with a reference method and another counts passing tests, preserve the difference.

The case is strongest as a source for questions. Which tasks were selected? What guidance was provided? How was the reference output produced? What tests failed? What did review cost? Which controls prevented unsafe changes?

Conclusion: Treat the Case as Evidence, Not a Promise

Rakuten's official case reports seven hours of sustained autonomous coding, a 12.5-million-line vLLM task, 99.9% numerical accuracy against a reference method, and a 79% reduction in average feature time to market from 24 working days to 5 days. Those results are specific and useful. They should not be widened into a universal claim about unattended production coding or guaranteed productivity. Reproduce the workflow with bounded permissions, independent tests, human review, cost tracking, and rollback.