What Is Screen Scraping (and How It Differs From Web Scraping)

Screen scraping is the extraction of data from what a program shows on screen, rather than from its database, files or API. The scraper reads the text of a terminal session, the fields of a desktop window, the rendered content of a web page, or even the pixels of a screenshot, and turns it into data another program can use.

The term is older than the web. It started with software that read mainframe terminal screens, and today it covers robotic process automation (RPA), OCR, and a specific practice in banking that regulators have spent years phasing out. Below are the three kinds, where each is used, how it differs from web scraping, and why reading structured data beats reading the screen whenever you have the choice.

Screen scraping vs web scraping

The two overlap: scraping a rendered web page is a form of it. The difference is what you read.

Rule of thumb: reading the screen is the last resort when there is no structured source. On the web there almost always is one, which is why web scraping has largely replaced it there. Our guide to web scraping covers that side in depth.

The three kinds of screen scraping

Terminal scraping

The original kind. Old mainframe and midrange applications talk to users through fixed-size text screens (the IBM 3270 and 5250 terminals are the classic examples). The scraper connects as if it were a terminal, sends keystrokes, and reads values from known rows and columns. Modern command-line tools are scraped the same way: run the program, capture its text output, and parse it.

This Python snippet scrapes the output of df -kP, which prints disk usage in a fixed POSIX format:

import subprocess

# df -kP prints a POSIX-format table: one header line, then one line per filesystem
output = subprocess.run(["df", "-kP"], capture_output=True, text=True, check=True).stdout

for line in output.splitlines()[1:]:
    fields = line.split(None, 5)  # the mount point (last column) may contain spaces
    if len(fields) != 6 or not fields[1].isdigit():
        print(f"skipped a line we can't parse: {line!r}")
        continue
    filesystem, size_kb, used_kb, available_kb, capacity, mounted_on = fields
    if int(size_kb) > 1_000_000:  # ignore tiny pseudo-filesystems
        print(f"{mounted_on}: {capacity} used, {int(available_kb) // 1_048_576} GB free")

Our first version of this script, run on macOS on 23 October 2026, crashed. One line of the output read map auto_home 0 0 0 100% /System/Volumes/Data/home: the filesystem name itself contains a space, so every column shifted by one. The guard on line 8 fixes it, and the bug is the whole lesson in miniature. Text laid out for human eyes has no reliable structure, so your parser has to anticipate every way the display can vary.

GUI scraping and RPA

Desktop applications without an API can be read through the operating system's accessibility and UI automation interfaces, which expose each window's buttons, labels and text fields. RPA platforms build on this: a software robot logs into an old claims system, copies values from one screen, pastes them into another application, and repeats the routine thousands of times a day.

Common uses:

• Legacy modernisation. Moving data out of a system whose vendor is gone or whose database nobody can query.
• Back-office automation. Copying data between insurance, ERP, healthcare or government systems that don't integrate.
• Testing and monitoring. Checking that an application shows the right values after a change.

It works, but it's slow (one session per robot), and it breaks whenever the interface changes: a renamed button or a new pop-up stops the robot.

OCR and visual scraping

When there isn't even a text layer, a scraper captures pixels and runs optical character recognition: scanned invoices, remote-desktop sessions, video frames, images of tables. Vision-capable AI models have made this far more accurate than classic OCR engines, but it is still the most error-prone kind. A "5" read as an "S" in a price column is easy to miss.

Screen scraping in banking and open banking

In finance, "screen scraping" has a narrower meaning. A budgeting app or payment service asks for your online-banking username and password, logs in as you, and reads your balance and transactions from the bank's website or app. For years this was how most account-aggregation services worked. It also meant handing your banking credentials to a third party, and banks couldn't tell the aggregator's logins from yours.

Regulators have moved the market toward dedicated, permissioned interfaces:

• European Union (PSD2). The EU's technical standards on strong customer authentication and secure communication, Commission Delegated Regulation (EU) 2018/389, applied from 14 September 2019. Banks must give licensed providers access either through a dedicated interface or through an adapted customer interface in which the provider identifies itself. The European Banking Authority stated in its final report on the fallback exemption that "the existing practice of third-party providers accessing the PSU data via the customer interface and without identification (commonly referred to as 'screen scraping') will no longer be allowed once the RTS apply." If a bank's dedicated interface fails, Article 33(4) lets providers fall back to the customer interface, but only while identifying themselves (unless the national authority has exempted the bank).
• United States (CFPB section 1033). The Consumer Financial Protection Bureau's Personal Financial Data Rights rule, published in November 2024, requires banks, credit unions and other financial providers to make consumers' data available to them and to authorised third parties. According to the CFPB's own summary in the Federal Register, the rule "prohibited data providers from relying on a third party's use of screen scraping to access the developer interface required by the rule and discourag[ed] the use of screen scraping by third parties when more secure methods of data access were available." The rule is under reconsideration: in August 2025 the CFPB asked for comments, including on "the costs and benefits of the PFDR Rule's provisions designed to reduce the use of screen scraping." Check the current status before relying on its dates.

This is general information, not legal advice. Note what these rules are about: logging in with someone else's credentials to reach private account data. Collecting public web pages is a different activity with different rules, covered in our guide is web scraping legal?

Reading the screen vs reading the data: one page, three ways

On the web, the same choice between screen and structure shows up on every JavaScript-heavy page. quotes.toscrape.com/js, a practice site, draws its quotes with JavaScript. A plain HTML parser finds nothing, but the data the script draws from is sitting in the page source:

import json
import re

import requests
from bs4 import BeautifulSoup

url = "https://quotes.toscrape.com/js/"
html = requests.get(url, timeout=15).text

# What a plain HTML parser sees: no quotes, because JavaScript draws them later
print(len(BeautifulSoup(html, "html.parser").select("div.quote")), "quote elements in the raw HTML")

# What the page's script draws them from: a JSON array in the source
data = json.loads(re.search(r"var data = (\[.*?\]);", html, re.S).group(1))
print(len(data), "quotes in the embedded data")
print(data[0]["author"]["name"], "|", data[0]["text"][:60])

0 quote elements in the raw HTML
10 quotes in the embedded data
Albert Einstein | “The world as we have created it is a process of our thinkin

The read-the-screen route renders the page in a real browser and reads what appears. With Playwright (pip install playwright, then playwright install chromium):

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://quotes.toscrape.com/js/")
    page.wait_for_selector("div.quote")
    quotes = page.locator("div.quote span.text").all_inner_texts()
    print(len(quotes), "quotes after rendering")
    browser.close()

Both printed 10 quotes when we ran them on 23 October 2026 (Python 3.12, Playwright 1.64). The embedded-JSON version needs no browser, runs in a fraction of the time, and gets structured fields (author, tags) without parsing display text. The browser version is the fallback for pages whose data only exists after scripts run or a user clicks. A third route, OCR on a screenshot of the page, would work too, and would be the slowest and least accurate of the three.

How Scrape.ly fits

Scrape.ly is built for the structured end of that spectrum: web scraping that reads HTML, embedded data and the page's own data requests, with a real browser only where one is needed.

• Ready-made scrapers return structured records. The Scrape.ly Store has 568 Actors for sites such as Google Maps, Amazon, LinkedIn Jobs and Reddit. Each returns typed JSON fields (prices as numbers, ratings, IDs, timestamps), not text lifted from a screen, and exports to CSV or Excel.
• A generic scraper for any public site. The Web Scraper fetches pages over HTTP by default and runs your extraction function on them. Set useChrome and the same function runs on the page after its scripts have run, with optional scrolling and cookie-banner dismissal.
• A hosted browser for your own browser automations. When you write a Playwright or Puppeteer Actor, the Scrape.ly browser runs it on a hosted Chromium with anti-bot fingerprints, Cloudflare Turnstile solving and its own residential proxy, without changes to your code. List price: $0.45 per browser hour. A Playwright template gets you started.
• APIs and agents. Every Actor runs behind one REST API, and the MCP server lets AI agents run them. See web scraping vs API for when to use a platform's official API instead.

What to watch out for

• Credentials. Never ask users for passwords to scrape their private accounts. In banking, use the regulated interfaces; elsewhere, use the service's official API with OAuth.
• Brittleness. Screen positions, fonts and window layouts change. Validate every value you read and alert on empty or malformed fields.
• Misreads. OCR confuses similar characters. Cross-check totals and use number formats to catch errors.
• Personal data. Screens in back-office systems are full of customer data. Read only the fields you need and keep them only as long as you need them.
• A structured source you missed. Before you build one, check for an export, an API, a database replica or, on the web, the JSON behind the page.

FAQ

Is screen scraping the same as web scraping?

No, but they overlap. The first reads data from any program's display, including terminals, desktop apps and images. Web scraping reads data from websites, usually from the HTML or the data behind it rather than from the rendered screen.

What is screen scraping in banking?

It's when a third-party app logs into your online banking with your username and password and reads your balances and transactions from the bank's site. Rules such as the EU's PSD2 standards and the US CFPB's section 1033 rule push this access onto dedicated interfaces where the provider identifies itself.

Is screen scraping legal?

It depends on what you read and how. Automating screens of systems you're authorised to use is everyday RPA practice, while logging into other people's accounts is regulated in banking and restricted by terms elsewhere. Personal data rules apply either way. This is general information, not legal advice.

What is screen scraping used for today?

RPA in back offices, moving data out of legacy systems, reading documents with OCR, test automation, and parsing command-line output. On the web it has mostly been replaced by web scraping of HTML and structured data.

Is screen scraping still used in open banking?

Less and less. In the EU, providers must identify themselves to banks under PSD2's standards, which ended anonymous credential-based scraping of payment accounts. In the US, the CFPB's 2024 rule discouraged it but is being reconsidered, so practice varies.