--- title: "How to Scrape Google Scholar (2026 Guide) | Scrape.ly" description: "Scrape Google Scholar papers, citation counts, cited-by lists and author metrics without a proxy. No-code steps, API code, a real run and its exact cost." url: https://scrape.ly/blog/scrape-google-scholar markdown: https://scrape.ly/blog/scrape-google-scholar.md --- [Tutorials](https://scrape.ly/blog/category/tutorials.md) # How to Scrape Google Scholar: Papers, Citations and Author Profiles Scrape Google Scholar searches, cited-by lists and author profiles into clean rows: title, authors, venue, year, citation count, versions and the free PDF link. No-code steps, curl, Python, Node.js and MCP, plus a real 10-paper run that cost about a cent. [Querystone Actor publisher on the Scrape.ly Store](https://scrape.ly/blog/author/querystone.md) - Published Oct 26, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head pulls a paper card with a blue title and two columns of text off a results page headed by the blue Google Scholar cap, and drops it into a crate](https://scrape.ly/media/scrape-google-scholar.png?w=1080) To scrape Google Scholar, send keywords, a "Cited by" ID or an author profile to a scraper that reads Scholar's public results and saves each paper with its title, authors, venue, year, citation count and free full-text link. On Scrape.ly the Google Scholar Scraper does this from a search box or one API call, at $1.05 per 1,000 papers, with no Google account, API key or proxy to manage. Below you'll find the fields it returns, the no-code steps, code for curl, Python and Node.js, the AI-agent route over MCP, and a real run from 26 October 2026: the top 10 papers for "large language models", with the log and the exact cost. More Google how-tos: [Google Shopping](https://scrape.ly/blog/scrape-google-shopping.md) and [Google Flights](https://scrape.ly/blog/scrape-google-flights.md). ## What data you can get from Google Scholar The [Google Scholar Scraper](https://scrape.ly/store/querystone/google-scholar-scraper.md) has three modes you can mix in one run: keyword searches, "cited by" lists and author profiles. Paper rows (`type: "paper"`) hold: | Field | What it holds | | ----------------------------------------------- | --------------------------------------------------------------------- | | `position`, `title`, `url` | Rank in the search, paper title and link | | `authors`, `authorsTruncated` | Author names as Scholar prints them, and whether Scholar cut the list | | `publication`, `year`, `source` | Venue, publication year and host domain | | `snippet` | Scholar's abstract snippet as plain text | | `citedByCount`, `citedByUrl` | Times cited and the "Cited by" link | | `clusterId` | Scholar's paper ID; feed it back in as a cited-by input | | `versionsCount`, `versionsUrl`, `relatedUrl` | All versions and related articles | | `fullTextUrl`, `fullTextType`, `fullTextSource` | The free `[PDF]` or `[HTML]` link Scholar shows, and its host | | `documentType` | `article`, `book`, `citation`, `pdf` or `html` | Author rows (`type: "author"`) add a researcher's affiliation, interests, all-time and recent citations, h-index, i10-index, citations per year, co-authors and up to 5,000 articles. They're charged separately, so a keyword search never pays for profiles you didn't ask for. ## How to scrape Google Scholar without code 1. Open the Actor's page in the Scrape.ly Store. 2. Pick the preset [Large language model papers, 10 results](https://scrape.ly/store/querystone/google-scholar-scraper/presets/search-light.md). "Try this input" loads it in the console. This is the exact input of our run: ```json { "queries": ["large language models"], "maxResults": 10, "sortBy": "relevance", "includeCitations": false, "includePatents": false, "reviewArticlesOnly": false, "maxArticlesPerAuthor": 5, "articleSort": "citations", "language": "en" } ``` 3. Type your own topic and click **Start**. Ten papers took under 6 seconds in our run. 4. Open the **Papers** view and export JSON, CSV, Excel or HTML. The fields that matter most: - `queries`: keywords, one per line. Scholar operators work: `author:"hinton"`, `source:nature`, `intitle:` and "exact phrases". Pasted Scholar search links work too. - `citedBy`: a `clusterId` from any paper row, or a full "Cited by" link, to list the papers citing it. The [Papers citing an article](https://scrape.ly/store/querystone/google-scholar-scraper/presets/cited-by.md) preset shows the shape. - `authors`: Scholar author IDs or profile links, one profile row each, as in the [author profile preset](https://scrape.ly/store/querystone/google-scholar-scraper/presets/search-filters.md). - `maxResults`: papers per search or cited-by list, up to 1,000. - `yearFrom`, `yearTo`, `sortBy`, `resultLanguages`: filters. `sortBy: "date"` returns papers added in the last year, newest first. Scrape.ly Store Google Scholar Scraper Papers from keyword searches and cited-by lists, plus author profiles with h-index and citations per year. Clean JSON at $1.05 per 1,000 papers, no proxy needed. [Try it free](https://scrape.ly/store/querystone/google-scholar-scraper.md) [Browse the Store](https://scrape.ly/store.md) ## Scrape Google Scholar with the API Create a token under **Settings → API & Integrations** in the dashboard and export it as `SCRAPELY_TOKEN`. Searches run in fast mode and finish in seconds, so the synchronous endpoint returns the papers in one call: ```shell curl -X POST "https://api.scrape.ly/v2/actors/querystone~google-scholar-scraper/run-sync-get-dataset-items" -H "Authorization: Bearer $SCRAPELY_TOKEN" -H "Content-Type: application/json" -d '{"queries": ["large language models"], "maxResults": 10, "includeCitations": false}' ``` The call waits up to 45 seconds and returns a JSON array; add `?format=csv` for CSV. Author profiles and `authorProfileLinks` read Scholar's own pages and take longer. If a run passes 45 seconds, the call answers `408` while the run keeps going, so don't call again: start it with `POST /v2/actors/querystone~google-scholar-scraper/runs`, poll `/v2/actor-runs/RUN_ID?waitForFinish=60` and read `/v2/actor-runs/RUN_ID/dataset/items`. The [Actors API docs](https://scrape.ly/docs/api-actors.md) show each call. Python (`pip install scrapely-client-python`). This script finds the most-cited paper for a topic, then lists the newest papers that cite it, which is the core loop of a citation tracker: ```python import os from scrapely_client import ScrapelyClient client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly") scholar = client.actor("querystone/google-scholar-scraper") search = scholar.call(run_input={"queries": ["large language models"], "maxResults": 10, "includeCitations": False}) papers = list(client.dataset(search.default_dataset_id).iterate_items()) top = max(papers, key=lambda p: p["citedByCount"] or 0) print(f'{top["citedByCount"]:>6} citations {top["title"]} ({top["year"]})') citing = scholar.call(run_input={"citedBy": [top["clusterId"]], "maxResults": 20, "yearFrom": 2026}) for p in client.dataset(citing.default_dataset_id).iterate_items(): print(f'{p["year"]} {p["citedByCount"] or 0:>4} {p["title"][:70]} {p.get("fullTextUrl") or ""}') ``` Node.js (`npm install scrapely-client`): ```javascript import { ScrapelyClient } from 'scrapely-client'; const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' }); const run = await client.actor('querystone/google-scholar-scraper').call({ queries: ['large language models'], maxResults: 10, includeCitations: false, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items.map((p) => [p.position, p.citedByCount, p.year, p.title])); ``` The [Python client](https://scrape.ly/docs/client-python.md) docs cover paging through large datasets. If you think of this data as an endpoint, the [Google Scholar API](https://scrape.ly/api/google-scholar-api.md) page is where we cover the Google Scholar scraper as an API product. ### With an AI agent (MCP) Scrape.ly's hosted MCP server is at `https://mcp.scrape.ly`. In Cursor, add it to `~/.cursor/mcp.json`: ```json { "mcpServers": { "scrapely": { "url": "https://mcp.scrape.ly", "headers": { "Authorization": "Bearer YOUR_TOKEN" } } } } ``` In Claude Code, run `claude mcp add --transport http scrapely https://mcp.scrape.ly` and sign in with `/mcp`. Use `https://mcp.scrape.ly?actors=querystone/google-scholar-scraper` to give the agent only this tool. Then ask: "Use scrapely to find the 10 most relevant Google Scholar papers on retrieval-augmented generation since 2024, and give me title, year, citations and the free PDF link." Every row is flat JSON, which suits literature-review agents and RAG pipelines. The run in this article was started over MCP too. The [MCP docs](https://scrape.ly/docs/mcp.md) list the other clients. ## A real run: the top 10 papers on large language models We ran the preset on 26 October 2026. It saved 10 papers in 5.7 seconds, and its list price was $0.0105. The first five rows (authors are shown as Scholar prints them; they appear in their published capacity, and we list the first author only): | # | Title | First author | Venue | Year | Cited by | Full text | | - | -------------------------------------------------------------------------------------- | ------------------- | ---------------------- | ---- | -------- | --------- | | 1 | A survey on evaluation of large language models | Y Chang et al. | ACM transactions on … | 2024 | 7,002 | PDF | | 2 | Large language models: A survey | S Minaee et al. | arXiv preprint arXiv … | 2024 | 3,639 | PDF | | 3 | A comprehensive overview of large language models | H Naveed et al. | ACM Transactions on … | 2025 | 3,864 | PDF | | 4 | Challenges and applications of large language models | J Kaddour et al. | arXiv preprint arXiv … | 2023 | 1,100 | PDF | | 5 | A review of current trends, techniques, and challenges in large language models (llms) | R Patil, V Gudivada | Applied Sciences | 2024 | 551 | HTML | ```plaintext 20261026T14:25:23.694Z [Scrapely] Container started (setup 0.9 s) 20261026T14:25:25.440Z [scrapely.actor] INFO Using fast search mode. A search switches to direct mode (Scholar pages) only if fast search mode fails or the search needs it. 20261026T14:25:25.440Z [scrapely.actor] INFO Reading 1 search(es) and 0 author profile(s), up to 10 results per search. 20261026T14:25:28.180Z [scrapely.actor] INFO Searches in fast search mode: 1, in direct mode: 0. Fast search mode: 3 API pages, 3 calls, 0 retried, 2 short pages asked again, slowest 1082 ms. 20261026T14:25:28.180Z [scrapely.actor] INFO Loaded 0 pages (0 through the browser, 0 incomplete). DATACENTER: 0 requests, 0.00 MB, 0 throttled or blocked; RESIDENTIAL: 0 requests, 0.00 MB, 0 throttled or blocked. 20261026T14:25:28.180Z [scrapely.actor] INFO Saved 10 papers. ``` What the ten papers show: - **Relevance isn't citation order.** The survey ranked third held 3,864 citations, more than the one ranked second (3,639). The paper with the second-highest count overall, "Large language models in medicine" in Nature Medicine (6,334), sat at position 10. Sort by `citedByCount` yourself if impact is what you're after. - **27,412 citations across ten papers.** The most-cited was the evaluation survey at 7,002; the least, a paper in Critical AI at 331. - **Free full text for all ten.** Eight had a PDF link and two an HTML one, hosted on the ACM Digital Library, arXiv, MDPI, Springer, Elsevier, Duke University Press and nature.com. - **Recent work dominates.** Five papers were from 2024, three from 2023, one from 2025 and one from 2026. - **Scholar shortens things.** Seven author lists were cut (`authorsTruncated: true`) and five venue names ended in "…". That's how Scholar prints them; the Actor flags it instead of guessing. - **No throttling.** The log shows fast search mode, 3 calls, zero retries and zero Scholar pages loaded, which is why no proxy was needed. ## What it costs Two charge events, platform usage (compute, proxy, browser) included: **$0.00105 per paper** ($1.05 per 1,000), from searches and cited-by lists, and **$0.01049 per author profile** ($10.49 per 1,000), including the articles you ask for. Retries, throttled pages and failed inputs aren't charged. Prices checked on 26 October 2026. - **Our run:** 10 papers × $0.00105 = **$0.0105** at list price. The account was billed $0.0084 after its plan's 20% Store discount. - **A literature review:** 20 queries × 50 papers = 1,000 papers, about $1.05. - **A hiring shortlist:** 30 author profiles × $0.01049 = $0.31, each with citations per year and top articles. The Free plan's $5 of monthly credit covers about 4,760 papers. See [pricing](https://scrape.ly/pricing.md) for plans. ## What to watch out for - **1,000 results per search is Scholar's ceiling.** Deep pages also throttle: on the publisher's test runs, fast mode saved 41 to 50 papers when 50 were asked for and 75 of 100. For more, split a topic into narrower searches, for example by year range. The status message says when a search stopped early. - **Snippets, not abstracts.** `snippet` is the text Scholar shows, not the full abstract. For full abstracts by DOI, pair this Actor with the [OpenAlex Research Papers Scraper](https://scrape.ly/store/querystone/openalex-scraper.md). - **Author profile IDs cost time.** In fast mode `authorProfiles` is `null`. Turn on `authorProfileLinks` to fill it; the Actor then reads Scholar's own pages, which is slower and subject to throttling. Author profile rows themselves always work. - **Citation-only records are a choice.** `includeCitations: false` (as in the preset) drops `[CITATION]` entries Scholar knows only from reference lists. Keep them for exhaustive citation counts. - **No export formats.** BibTeX, APA and MLA "Cite" formats aren't exported; build them from the fields. - **Profiles are personal data.** Author rows hold a person's name, affiliation and verified email domain. Collect them only when you need them, as in the legal section below. ## Is it legal to scrape Google Scholar? The Actor reads the public pages and results anyone can open without an account, and paper metadata (titles, venues, years, citation counts) is bibliographic fact. Google is direct about automated access, though. Scholar's [help page](https://scholar.google.com/intl/en/scholar/help.html) answers a researcher whose script got blocked: "please respect our robots.txt when you access Google Scholar using automated software", and it states "we're unable to provide bulk access". [Scholar's robots.txt](https://scholar.google.com/robots.txt) disallows `/scholar` for all crawlers, and Google's [Terms of Service](https://policies.google.com/terms) (effective July 30, 2026) list automated access that violates robots.txt instructions as abuse. Read them before you build a product on this data. Author profiles add privacy law. A researcher's name, affiliation and email domain are personal data under the GDPR and the CCPA, even when published, so collect profiles for a clear purpose such as citation analysis, keep only the fields you need, and don't use them for unsolicited marketing. Listing authors of published papers in their published capacity, as our table does, is ordinary bibliographic practice. Our guide on [whether web scraping is legal](https://scrape.ly/blog/is-web-scraping-legal.md) covers the general rules. This is not legal advice: for commercial use, ask a lawyer about your case. ## Why Scrape.ly for Google Scholar data - **No official API, and no bulk access.** Google offers no Scholar API and says on its help page it can't provide bulk access. The Actor turns the public results into rows. - **No blocks to fight.** The open-source `scholarly` library warns that `search_pubs` and `citedby` queries "will lead to Google Scholar blocking your requests and may eventually block your IP address" ([its README](https://github.com/scholarly-python-package/scholarly)). Fast search mode served our run with zero Scholar page loads, and direct-mode pages are retried on fresh IPs automatically. - **Fields, not selectors.** [Crawlbase's tutorial](https://crawlbase.com/blog/scrape-google-scholar-results/) parses `div.gs_r` and `h3.gs_rt` and splits the byline by hand. Here authors come as a list, year and venue split out, citation counts as numbers. - **Cheap at research scale.** $1.05 per 1,000 papers, platform usage included, with a hard cap per run. For the wider family of Google results, see our guides to [scraping Google search results](https://scrape.ly/blog/scrape-google-search-results.md) and the [best SERP APIs](https://scrape.ly/blog/best-serp-apis.md). ## FAQ ### Is there an official Google Scholar API? No. Google doesn't publish one, and Scholar's help page says it can't provide bulk access to records. Third-party scrapers like this one read the public results and return them as JSON over a normal REST API. ### Can you scrape Google Scholar with Python? Yes. Install `scrapely-client-python`, call `querystone/google-scholar-scraper` with your queries, and iterate over the dataset, as in the citation-tracker example above. The scraping runs in the cloud, so your script needs no proxy, browser or parser. ### How do I scrape the papers that cite an article? Copy the `clusterId` from a paper row (or the paper's "Cited by" link) into `citedBy`. The Actor saves the citing papers as normal paper rows, with the cited paper's ID in `citedById`. ### Can I scrape a researcher's h-index and citations? Yes. Put the Scholar author ID or profile link in `authors`. Each profile row has all-time and recent citations, h-index, i10-index, citations per year, co-authors and up to 5,000 articles, at $0.01049 per profile. ### Why does Google Scholar block my scraper? Scholar throttles signed-out automated traffic, especially on deep result pages, and asks scripts to respect its robots.txt. The Actor's fast search mode avoids Scholar's pages for searches, and pages it must read are retried on fresh IPs. Retries are never charged. ### How much does it cost to scrape Google Scholar? $0.00105 per paper and $0.01049 per author profile, platform usage included. Our 10-paper run cost $0.0105 at list price. Scrape.ly Store Scrape Google Scholar Run the large-language-models preset as-is, or type your own topic, a cited-by ID or an author profile. $1.05 per 1,000 papers, platform usage included. [Try it free](https://scrape.ly/store/querystone/google-scholar-scraper.md) [Browse the Store](https://scrape.ly/store.md) About the author [Querystone](https://scrape.ly/blog/author/querystone.md) Actor publisher on the Scrape.ly Store Search results, AI answers, web pages and market feeds as structured data. For SEO teams, developers and AI builders, plus finance, crypto and sports data for analysts. A Scrapely studio. - [Website](https://scrape.ly/users/querystone.md) Keep reading ## More in Tutorials - [![Pixel-art robot with an orange >\_ on its terminal head pulls a review card with five green stars off a cork board topped with the green Trustpilot star, where another such card is pinned](https://scrape.ly/media/scrape-trustpilot-reviews.png?w=750)](https://scrape.ly/blog/scrape-trustpilot-reviews.md) ### [How to Scrape Trustpilot Reviews: Ratings, Text and Company Replies](https://scrape.ly/blog/scrape-trustpilot-reviews.md) [Scrape Trustpilot reviews for any company by domain or brand name: stars, title, text, country, verification, dates and the company's reply. No-code steps, curl, Python, Node.js and MCP, plus a real run from 26 October 2026 and its exact cost.](https://scrape.ly/blog/scrape-trustpilot-reviews.md) - [![Pixel-art robot with an orange >\_ on its terminal head pulls a chart slip with a purple rising line out of a monitor showing the purple Yahoo Finance icon beside ticker rows, above a rising candlestick chart, and drops it into a crate](https://scrape.ly/media/scrape-yahoo-finance.png?w=750)](https://scrape.ly/blog/scrape-yahoo-finance.md) ### [How to Scrape Yahoo Finance: Quotes, Financials, History and Options](https://scrape.ly/blog/scrape-yahoo-finance.md) [Scrape Yahoo Finance for any stock, ETF, crypto pair or index and get one JSON record per ticker: live quote, profile, price history, financial statements, dividends, analyst ratings, holders, options and earnings dates. A real three-ticker run, its cost and code.](https://scrape.ly/blog/scrape-yahoo-finance.md) - [![Pixel-art robot with an orange >\_ on its terminal head pulls a company review card with a briefcase and five gold stars out of an office building with lit windows, two glass doors and a potted plant, and drops it into a crate of job cards](https://scrape.ly/media/scrape-glassdoor.png?w=750)](https://scrape.ly/blog/scrape-glassdoor.md) ### [How to Scrape Glassdoor Jobs and Reviews](https://scrape.ly/blog/scrape-glassdoor.md) [Scrape Glassdoor job listings with pay ranges and company ratings, and employee reviews with every sub-rating, without logging in. No-code steps, curl, Python, Node.js and MCP, plus two real runs from 26 October 2026 and their exact cost.](https://scrape.ly/blog/scrape-glassdoor.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:hello@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.