WebArena: A Realistic Web Environment for Building Autonomous Agents

Web agents need to operate across realistic pages, preserve context, and sometimes change application state. Simplified environments omit important interactions, while live websites are difficult to hold constant across experiments. Matching one reference action sequence also misses alternative valid ways to fulfil the same instruction.

A self-hosted environment assembles functional shopping, discussion, software-development, and content-management websites, together with tools and reference resources.

A task collection expresses realistic information-seeking and state-changing user goals, including objectives that require multiple pages or websites.

Task-specific evaluators check answers and application state, providing a functional success criterion rather than requiring one prescribed navigation path.

The websites contain application data and functionality modelled on common web services. Running controlled copies allows their state to be initialized and reset. Auxiliary sites supply resources such as maps and reference information that an agent may need while solving a task.

A task provides a natural-language objective and a starting configuration. The agent repeatedly receives the current page's URL and a configured representation of its content, such as a screenshot, HTML, or accessibility information. It issues browser operations through the available action interface and observes the resulting page before choosing another action.

Information-seeking tasks are checked against reference answers. Depending on the task, the evaluator may require an exact value, the presence of specified content, or semantic equivalence. State-changing tasks inspect the relevant page or underlying application data; for example, a content-editing task can be checked by reading the resulting content and its location.

These validators are implemented separately from the policy. An agent can therefore take a different route through the websites while still receiving credit for the required outcome. The released environment also includes baseline-agent and evaluation code for reproducing the experimental setup.

The paper evaluates agents on multi-step information retrieval and application operations across the controlled websites. The primary task-completion measure comes from the functional validators. Human evaluation checks how the same task collection can be completed by people.

Baseline comparisons vary language models, prompts, and the available interaction configuration. Analyses examine task categories and failed trajectories. The benchmark's controlled state supports repeatability, while its different answer and state checks remain explicit parts of the evaluation protocol.