AI Browser Agents Explained: Browser Use, Computer Use and MCP

An AI browser agent is a program in which a large language model operates a web browser: it reads the page, decides on the next action, then clicks, types or scrolls, and repeats until the task is done. Today you build an AI browser agent with an agent library such as Browser Use, a model's computer use or browser tools from Anthropic or OpenAI, or a browser MCP server such as Playwright MCP; when the goal is web data rather than clicking, an MCP server that runs ready-made scrapers, like Scrape.ly's, is the faster and more reliable path.

This guide explains how the agent loop works, with a small script that shows what an agent actually "sees" on a page, compares the main tools side by side, lists the limits their own documentation states, and shows where Scrape.ly fits: a hosted MCP server with 570+ Actors for data, and a hosted anti-bot browser for the automations you write yourself.

How an AI browser agent works

Every browser agent runs the same loop:

1. Observe. Capture the current state of the page.
2. Decide. Send that state, the task and the history to the model, which picks one action or a short batch of them.
3. Act. Execute the action in the browser: navigate, click, type, scroll, select.
4. Check. Observe again and confirm the step worked before moving on.

The tools differ mostly in step 1, what the model gets to look at. There are three approaches:

• Screenshots and coordinates. The model gets an image of the screen and answers with pixel positions to click. This works on anything a human can see, including desktop apps, but every step costs an image and the model has to locate elements visually. Claude's computer use tool and OpenAI's computer use work this way.
• The accessibility tree. The model gets a text outline of the page's roles, names and text, and refers to elements by ID. Playwright MCP is built on this; Anthropic's browser use tool combines it with screenshots.
• Selectors and schemas. A framework turns natural-language instructions into real page selectors and validates extracted data against a schema. Stagehand's act, observe and extract work this way.

Here is the accessibility-tree view, made visible. This Playwright script opens the practice store books.toscrape.com and prints the snapshot of the first product card, which is close to what an accessibility-tree agent receives:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://books.toscrape.com/")

    # What an accessibility-tree agent reads: roles, names and text, no pixels
    snapshot = page.locator("article.product_pod").first.aria_snapshot()
    print(snapshot)

    # What a screenshot agent reads: an image it has to interpret
    page.screenshot(path="page.png")
    browser.close()

Save it as what_agent_sees.py and run it with uv run --with playwright python what_agent_sees.py, after a one-time uv run --with playwright playwright install chromium. The output:

- article:
  - link "A Light in the Attic":
    - /url: catalogue/a-light-in-the-attic_1000/index.html
    - img "A Light in the Attic"
  - paragraph:
  - heading "A Light in the ..." [level=3]:
    - link "A Light in the ...":
      - /url: catalogue/a-light-in-the-attic_1000/index.html
  - paragraph: £51.77
  - paragraph:  In stock
  - button "Add to basket"

The title, price, stock and the button are all there as text the model can act on. The empty paragraph is the book's star rating, which the page draws with a CSS class, so the tree has nothing to say about it, while a screenshot shows three stars. Each way of looking misses something, and that's why some tools send both.

AI browser agents compared

Consumer AI browsers such as Perplexity's Comet, which its maker calls "the AI browser that acts as a personal assistant", put the same idea inside the browser you browse with. They're built for one person's daily tasks rather than for agents you deploy.

Browser Use: the open-source agent library

Browser Use describes itself as "the open-source browser agent". You give its Agent a task in plain language and a model, call run(), and it drives the browser step by step until it can answer. Its README offers three ways in: a fully hosted cloud agent, a CLI that plugs into coding agents such as Claude Code, and the Python library (Python 3.11 or newer). Its driver can control a local browser, a Browser Use Cloud browser, or "an existing remote browser over CDP", which matters later in this guide: a CDP endpoint can be a hosted browser.

Browser Use is a good choice when the task is genuinely interactive and you want to own the agent code. You bring the model key, and the model's tokens are part of every step's cost.

Computer use: Claude and OpenAI drive the screen

Claude. Anthropic's computer use tool gives Claude "screenshot capabilities and mouse/keyboard control for autonomous desktop interaction". Your application runs every action in an environment you control, and Anthropic recommends "a dedicated virtual machine or container with minimal privileges". For work that stays inside web pages, Anthropic points to a separate browser use tool that reads the page structure as well as screenshots. Its docs also say that for "reading pages you can point Claude to, or finding sources on the web, the web fetch tool and web search tool are lighter."

OpenAI. OpenAI's computer use guide runs the loop through the Responses API. For GPT-6 Astra it recommends code execution, where the model writes Playwright or PyAutoGUI code that your runtime executes, with the structured computer tool as the alternative. It gives the same advice as Anthropic: "For public information that needs no interaction, use web search."

Both vendors are clear that computer use is for interaction. Neither recommends it for reading public information.

MCP: browser tools and data tools

The Model Context Protocol lets any compatible client, such as Claude, Cursor, ChatGPT or VS Code, call tools on a server. For the web there are two very different kinds of server.

Browser servers hand the model the controls. Playwright MCP exposes navigation, clicks, typing and snapshots, and lets the model work "through structured accessibility snapshots, bypassing the need for screenshots or visually-tuned models". Its README is candid about the cost: coding agents increasingly favor Playwright's CLI because it avoids "loading large tool schemas and verbose accessibility trees into the model context". Our Playwright MCP guide covers setup and options.

Data servers hand the model the result. With the Scrape.ly MCP server, the agent never drives a page. It searches the Scrape.ly Store with search-actors, reads an Actor's input schema with fetch-actor-details, starts it with call-actor, and reads structured rows with get-actor-output. The Actor already knows the site: its pages, its pagination, its anti-bot defenses and its data fields.

The difference shows on a real task. Asked to compare five mechanical keyboards on Amazon, an agent on the Scrape.ly server made one call-actor request: the Amazon Product Scraper opened the search and all five product pages and returned price, list price, rating, rating count and monthly sales for each in 15.7 seconds, for $0.0057. A browser agent doing the same job needs a model turn for every page it opens and every field it reads. The whole run is in our Amazon MCP guide.

Where browser agents struggle

The vendors' own documentation lists the limits. Plan around them:

• Speed. Anthropic: computer use latency "might be too slow compared to regular human-directed computer actions", so it suggests tasks "where speed isn't critical".
• Precision. Claude "might make mistakes or hallucinate when outputting specific coordinates", and dropdowns and scrollbars can be "tricky" with mouse movements.
• Prompt injection. "Instructions on webpages or contained in images might override your instructions," Anthropic warns. OpenAI says "page content can influence the model" and advises a fresh browser profile or isolated desktop.
• Token cost. Declaring Claude's computer use toolset adds about 4,500 input tokens to a request, and every screenshot is billed as image input.
• Consent. Anthropic recommends asking a human to confirm anything with real-world consequences, "such as accepting cookies, completing financial transactions, or agreeing to terms of service".
• Blocking. A browser agent is still an automated browser. Sites check fingerprints, IP reputation and behavior, and show CAPTCHAs or block outright. See anti-blocking on Scrape.ly for how our Actors and browser handle it.

When to use a browser agent, and when to use a scraper

The rule of thumb matches what Anthropic and OpenAI write: if nothing on the page needs a click, don't pay for a browser agent to click.

How Scrape.ly fits

For data: the MCP server and the Store. Connect any MCP client to https://mcp.scrape.ly with one URL and OAuth, and the agent can run any of the Store's Actors, from Google Maps to Amazon to Indeed. Results come back as rows the agent reads in pages, prices are per result, and callOptions.maxTotalChargeUsd caps what a run can spend. Scoped API keys with daily charge limits keep unattended agents in budget.

For reading the open web: the RAG Web Browser. The RAG Web Browser Actor searches Google, opens the top results and returns clean Markdown or text for the model, or fetches a single URL. It costs $0.004 per page of Google results, with the pages that search finds included, or $0.002 per fetched page for a direct URL, plus $0.02 when a direct URL has to be loaded in a browser, platform usage included.

For your own browser automation: the Scrape.ly browser. When you do need a real browser, run it as an Actor and switch on the Scrape.ly browser: a hosted Chromium with anti-bot fingerprints, Cloudflare Turnstile solving and its own residential proxy. With the setting on, Playwright and Puppeteer launches in the Actor connect to it without code changes. Any framework that attaches over CDP can also connect directly from inside a run:

import { chromium } from 'playwright';

const browser = await chromium.connectOverCDP(process.env.CDP_URL);

Browser time is billed in browser credits at $0.45 per browser hour list price and counts toward the run's maxTotalChargeUsd. Our explainers on cloud browsers and headless browsers go deeper, and the best MCP servers for web scraping compares the data-server side.

FAQ

What is an AI browser agent?

It's software in which a language model operates a web browser. The model observes the page as a screenshot, an accessibility tree or the DOM, chooses an action such as a click or a keystroke, and repeats until the task is finished.

What is the best AI browser agent?

For web data, the best setup is an MCP server that runs ready-made scrapers, like Scrape.ly's, because the agent gets structured results without driving a page. For interactive tasks you write yourself, Browser Use is a strong open-source library, and Claude's and OpenAI's computer use tools cover desktop apps as well as browsers.

Is there a free AI browser agent?

Browser Use (MIT license) and Playwright MCP (Apache 2.0) are free and open source. You still pay for the model's tokens, and for a cloud browser if you use one. Scrape.ly's MCP server is free to connect, and the free plan includes $5 of usage every month.

Can AI agents browse the web?

Yes. With a browser tool, computer use or a browser MCP server, an agent can open pages, click and fill in forms. For reading public pages, both Anthropic and OpenAI point to lighter tools such as web search and web fetch.

What is the difference between Browser Use and computer use?

Browser Use is an open-source library that drives a browser for any model you plug in. Computer use is a model capability from Anthropic and OpenAI: the model sees screenshots and controls the mouse and keyboard of a browser or a whole desktop that your application runs.

Is Playwright MCP a browser agent?

Not by itself. It's an MCP server that gives an agent browser tools based on Playwright's accessibility snapshots. Combined with a model in Claude, Cursor or VS Code, it becomes the browser half of an agent.