Agent JIT Compilation

What You'll Learn

• What the Agent JIT Compilation paper actually proposes for web-agent planning and scheduling.
• How JIT-Planner, JIT-Scheduler and invariant-enforcing tool protocols work together.
• What the paper measured against Browser-Use and OpenAI CUA across its benchmark tasks.
• Which limitations matter before adapting the research idea to production browser automation.

Browser agents usually turn a natural-language request into a sequence of actions such as clicking, typing and scrolling. A common implementation follows a fetch, screenshot, reason and execute loop. The agent observes a new browser state, asks a language model what to do next, performs the action and repeats the cycle. This is flexible, but repeated model calls can increase latency, cost and the opportunity for an incorrect tool action.

The paper Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling proposes a different execution model. Instead of treating every interaction as a fresh decision, it compiles a task description into executable code. That code may combine language-model evaluation, reusable tools and parallel work. The aim is to move more reasoning into plan synthesis and make the execution path easier to validate and optimise.

The paper was submitted to arXiv on 20 May 2026, revised on 29 May 2026 and marked as accepted at ICML 2026 on the abstract page. Its results are reported for five web applications and specific baselines. The primary source is the Agent JIT Compilation paper on arXiv. Read it before treating the benchmark results as a product claim.

For practical context on the systems that browser agents connect to, see our computer-use and browser-control guide. A research compiler, an agent framework and a production automation service are different layers, even when they use similar words.

What problem does Agent JIT Compilation address?

The baseline problem is not simply that a browser action takes time. It is that a general agent can spend a model call deciding an action that could have been represented as a reusable deterministic operation. If the task is known to require a sequence of valid operations, repeatedly asking the model to rediscover the sequence adds overhead and creates more opportunities for tool-ordering errors.

The paper describes three related weaknesses in a standard agent loop. First, a static primitive tool set can be general but expensive to use because the model repeatedly chooses low-level actions. Second, a sequential loop may fail to explore useful schedules such as task parallelism or hedging. Third, a model call can remain in the execution path after the plan is already sufficiently known, adding nondeterminism without improving the next tool action.

Agent JIT Compilation treats the task as a plan-selection problem. The system can generate several candidate code plans, check whether they respect tool specifications, estimate their cost and select a valid low-cost candidate. It can then schedule parts of the plan according to predicted latency. This does not remove the need for model calls. It changes where they occur and how the system decides when they are necessary.

That distinction is important for implementation. The approach works best when the application exposes reusable, typed operations with predictable state transitions. A tool such as `list_items` or `open_message` gives the planner more structure than a primitive action such as “click near these coordinates.” A compiler cannot validate an operation that has no clear input, output or state contract.

The approach also introduces a trade-off. Planning before execution can reduce repeated reasoning, but plan generation itself costs time and may be wrong. A system needs a way to reject invalid plans, fall back when the environment changes and avoid executing a cached action against a different page state. The research contribution is therefore a combination of planning, scheduling and state validation rather than a single prompt trick.

How the JIT-Planner works

JIT-Planner generates multiple code candidates for a task. Each candidate can contain reusable tool calls, language-model evaluations and control flow. The planner validates the candidate against the tool manifest, estimates the execution cost through its control-flow graph and keeps the minimum-cost valid plan.

Candidate generation can happen in parallel. The paper describes workers that independently sample plans until a required number of valid candidates is collected or an early stopping condition is reached. This creates a practical question for deployment: the system spends additional compute to search for a better plan, so the expected saving during execution must justify the planning overhead.

Static validation examines the sequence of states implied by the plan. If a tool requires a page or object state that the preceding tools do not establish, the plan is rejected before execution. The planner can then use the validation failure as feedback and generate another candidate. This is different from discovering the error only after a browser action has already caused a bad side effect.

Cost estimation uses the plan’s structure. A language-model evaluation inside a loop can be more expensive than a single evaluation before the loop. The paper describes a cost model that assigns costs to tool and evaluation calls and applies a nested-loop penalty. The minimum-cost plan is selected only from candidates that pass the state-flow checks.

The plan cache is another important component. The paper describes offline synthesis of reusable code tools from execution traces. In a production system, a cache must have a version, scope and invalidation policy. A tool that was safe for one page state or account permission may become unsafe after an interface change, an authorisation change or a new workflow.

Our MCP security checklist covers a related control problem. Reusable tools need more than a name. They need input constraints, output expectations, access boundaries, logging and a policy for handling stale or incomplete state.

What is the invariant-enforcing tool protocol?

The paper’s tool protocol gives each reusable operation a contract. The contract can specify a precondition, an optional runtime pre-check, a postcondition, an optional runtime post-check, input and output schemas, and the implementation that performs the operation.

Preconditions and postconditions turn a sequence of actions into a state-flow problem. A tool that opens a message can promise that the message detail is active. A tool that sends a message can require a composed draft and a confirmed recipient. If the plan does not establish the required state, the compiler can reject it before the action reaches the browser.

The protocol is not a formal proof that the world is safe. A postcondition can be incorrectly written, a browser can behave unexpectedly and a tool implementation can violate its own contract. Runtime checks remain important. The benefit is that the system makes assumptions explicit and gives the planner a structure to test.

The paper reports that incorrect action sequences accounted for a large share of web-automation errors in its analysis. The exact rate is benchmark-specific. It should not be copied as a universal industry statistic. The general lesson is safer: state contracts are valuable when the cost of a wrong action is high or when a later step depends on a precise earlier state.

How the JIT-Scheduler chooses execution strategies

JIT-Scheduler addresses the time after a plan has been generated. The paper describes three strategies: serial execution, parallel execution and hedged execution. Serial execution handles work one step after another. Parallel execution distributes independent work across workers. Hedging starts redundant attempts and returns the first valid result under the configured policy.

The scheduler estimates which browser elements or tools a plan will use and samples from latency distributions learned from prior interactions. It then estimates the expected cost of each strategy and chooses the strategy with the lowest mean latency. The calculation is a prediction, not a guarantee. If the learned distributions are stale or the task differs from the training examples, the selected strategy may be wrong.

Parallelism is not always faster. Independent subtasks can benefit from multiple workers, but coordination and setup add overhead. A simple task may finish sooner in serial execution. Hedging can help when latency has high variance or when one of several attempts is likely to find a fast path, but it can increase resource use and duplicate side effects if the operations are not safely idempotent.

Safety rules must therefore constrain scheduling. A system can parallelise read-only lookups more easily than account changes, purchases, deletions or messages. A scheduler should know which operations can be repeated, which require exclusive access and which need a human approval step. It should also define what happens when two workers return conflicting results.

The research contribution is not “always use parallel browser actions.” It is to choose a strategy based on estimated task structure and latency rather than hard-code one strategy for every task. Production teams should measure the prediction error and the resource cost before enabling aggressive parallelism.

What did the ICML 2026 paper measure?

The arXiv abstract reports results across 5 web applications. JIT-Planner achieved 10.4× speedup and 28% higher accuracy over Browser-Use, while JIT-Scheduler achieved 2.4× speedup and 9% higher accuracy over OpenAI CUA. These values are the paper’s aggregate comparisons under its experimental setup. They do not mean that any browser agent will automatically become 10.4× faster after adding a compiler.

The full paper describes 2 benchmarks and 5 applications. The REAL benchmark includes Dashdish, Gomail and Omnizon, while WebArena includes GitLab and Reddit. Together, the evaluation covers 37 tasks involving e-commerce, communication, collaboration and social interactions. The task harness checks completion through application state differences or predefined evaluation functions.

The baseline matters. Browser-Use represents a step-by-step approach that selects an action, executes it, observes the result and repeats. Browser-Use plus cache adds synthesised tools but does not include the full cost-optimising planner and scheduler. OpenAI CUA is used as a comparison for the scheduler experiments. These baselines are not interchangeable, so the comparisons should be read in their respective sections.

The paper reports that JIT-Planner’s best-cost plan had mean latency of 11.7 seconds versus 61.7 seconds for the worst-cost plan, a 5.3× difference. It also reports an end-to-end comparison in which JIT-Planner achieved 10.4× speedup over Browser-Use with a 122.1-second baseline and 6.8× over Browser-Use with cached tools at an 80.1-second baseline. These are experiment-specific measurements, not current service-level objectives.

For the scheduler, the paper evaluates under a constraint of 4 available vCPUs. It reports that strategy performance varies by task, with serial, parallel and hedge options winning in different situations. A fixed strategy can therefore be inferior to an adaptive scheduler, but an adaptive scheduler adds prediction and orchestration complexity.

Accuracy, latency and the meaning of a speedup

A speedup is meaningful only when the denominator, task set, hardware, model, success definition and measurement window are visible. The paper compares latency and accuracy rather than reporting a single “faster” number without context. Even then, the results come from a controlled evaluation with particular applications and task distributions.

Accuracy can also have several meanings. The paper measures whether a plan is valid or whether a task reaches its objective under the benchmark harness. That is not the same as factual accuracy, user satisfaction, safety or successful completion in an arbitrary public website. A plan can be fast and still choose the wrong product, wrong account or wrong recipient if the task specification is incomplete.

Latency has layers. Planning latency includes candidate generation and validation. Inference latency comes from model calls during execution. Tool latency includes browser actuation, DOM operations and other environment work. A design can reduce one layer while increasing another. Report the complete end-to-end measurement rather than celebrating a faster component in isolation.

Cost is another dimension. Generating candidates in parallel can use more model calls. Hedging can duplicate work. Cached tools require storage, testing and invalidation. A lower wall-clock time may not reduce total compute cost. Product teams should track latency, success, model tokens, tool calls, vCPU time, retries and side effects together.

Use confidence intervals or repeated trials when moving beyond a demonstration. If a result changes substantially across applications, report the range. The paper itself describes variation across applications, including different speedups and accuracy improvements. That variation is evidence against a universal performance promise.

For a broader model-and-tool discussion, see our AI coding agents guide. The engineering rule is the same: benchmark the actual workload and disclose the baseline.

How to adapt the idea to a production browser agent

Begin with a narrow application and a small set of reusable tools. Define the states that matter, the allowed transitions and the operations that can be safely repeated. Avoid compiling arbitrary clicks into a broad permission to act. A production plan should have an explicit scope, time limit, account context and stopping condition.

Create a tool manifest. Each tool should declare its inputs, outputs, preconditions, postconditions, permissions, side effects and observability. Add runtime checks for important state claims. If the browser changes unexpectedly, invalidate the plan or fall back to a controlled recovery path instead of silently continuing.

Separate read-only and mutating actions. Searching, extracting and comparing information can often be parallelised. Sending, purchasing, deleting, publishing or changing account settings may require serial execution and a human confirmation. If an operation is not idempotent, hedging needs special safeguards or should be disabled.

Instrument the system before optimising it. Record planning time, number of candidates, validation failures, model calls, tool calls, execution time, retries, task outcome, human interventions and resource use. Keep a trace that allows an engineer to explain why the planner selected a plan and why the scheduler selected a strategy.

Use staged rollout. Replay known tasks in a sandbox, compare the compiled path with the baseline, test stale pages and permission changes, and run adversarial cases. Start with read-only tasks and introduce side effects only after the control and recovery story is clear.

Our MCP and cloud-operations guide is useful for thinking about tool schemas and permission boundaries. The JIT idea can reduce unnecessary model calls, but it does not remove the need for authentication, authorisation and audit logging.

Failure modes and safety controls

Safety should be evaluated at the action boundary. A model response that looks harmless can still trigger a high-impact tool. Review the plan, tool permissions, data exposure, approval gates and audit trail. Do not treat compilation as a security control unless the compiler actually enforces the relevant policy.

Browser automation can also encounter prompt injection or malicious page content. Treat page text as untrusted input. A compiled plan should not allow a page to rewrite its own permissions or bypass a required approval. Keep trusted instructions, user data and web content in separate channels where the architecture permits.

For an enterprise threat perspective, read our AI cybersecurity threats guide. Faster execution can increase the speed of harm if the action boundary is not controlled.

What the paper does not prove

The paper does not prove that Agent JIT Compilation is a drop-in replacement for every browser agent. Its evaluation uses 5 applications and 37 tasks across 2 benchmarks. Real websites can have authentication challenges, dynamic layouts, rate limits, hidden state, payment flows, anti-bot systems and unpredictable content that are not represented equally in a controlled benchmark.

The reported 10.4× and 2.4× speedups are comparisons against named baselines under the paper’s experiment. They are not promises for public websites, all models, all browsers or all task lengths. The reported accuracy improvements are also tied to the benchmark’s task-completion definitions. A production team must reproduce the comparison on its own workload.

The approach does not eliminate language-model uncertainty. JIT-Planner still generates candidate code plans and JIT-Scheduler still predicts usage and latency. Tool contracts reduce certain errors, but incomplete contracts, wrong assumptions or unexpected browser states can still produce failure.

Parallelism can create side effects and higher resource use. Caching can preserve stale behaviour. A minimum-cost plan can be the wrong choice if cost estimates omit safety, reversibility, data sensitivity or user preference. Production objectives should therefore use a constrained cost function, not latency alone.

Finally, the paper is a research result, not an independent certification of any commercial framework. The arXiv page records the paper’s version history and ICML acceptance status, while the full HTML contains the experimental details. Use the primary source and inspect the code, benchmark harness and assumptions before making a comparative claim.

Practical checklist for evaluating a JIT browser-agent design

1. Define the task boundary: state the permitted website, account, data and side effects.
2. Specify tool contracts: document inputs, outputs, preconditions, postconditions and permissions.
3. Build a baseline: measure the existing sequential agent on the same tasks and hardware.
4. Measure end to end: report planning, inference, tool, retry, resource and human-review time.
5. Separate read and write paths: allow parallelism where operations are independent and reversible.
6. Test failure cases: include stale pages, missing elements, ambiguous requests, injection and partial completion.
7. Monitor drift: compare predicted and actual latency, success and state transitions after deployment.
8. Keep a fallback: stop or return to a controlled baseline when contracts or environment checks fail.

A useful evaluation report should include the task inventory, application versions, model versions, browser environment, worker budget, success definition, latency distribution, resource cost and failure taxonomy. Without these details, a speedup is difficult to reproduce and easy to overstate.

Teams should also decide whether compilation is performed online, offline or in a hybrid mode. Online compilation can adapt to new tasks but adds latency and risk at request time. Offline synthesis can be reviewed and tested before release but may become stale. A hybrid design can use reviewed tools with online plan selection under strict policy and permission limits.

Final assessment of Agent JIT Compilation

Agent JIT Compilation is a well-defined research approach for changing how computer-use agents plan and execute web tasks. The paper’s JIT-Planner generates and validates multiple code plans before selecting a low-cost candidate. JIT-Scheduler chooses among serial, parallel and hedge strategies using learned latency estimates. The invariant-enforcing protocol makes tool state assumptions explicit.

The headline results are useful when stated precisely: across 5 web applications, the paper reports 10.4× speedup and 28% higher accuracy for JIT-Planner over Browser-Use, and 2.4× speedup and 9% higher accuracy for JIT-Scheduler over OpenAI CUA. Those are benchmark measurements, not a universal claim that every browser agent becomes faster or more accurate.

The most transferable lesson is architectural. Reduce unnecessary model calls, expose reusable tools, validate state transitions, choose execution strategies according to task structure and measure the full system. The most important limitation is equally clear: compiled plans, caches and parallel workers need permission controls, runtime checks, recovery paths and workload-specific evaluation.

Use the primary arXiv paper for the exact experiment, then reproduce the comparison on your own browser workflows. If the task can create a financial, legal, privacy or safety consequence, speed should remain subordinate to correctness, reversibility and human control.