PubMed Scraper

🧬 PubMed Scraper

Search PubMed's more than 38 million biomedical citations with the same queries you would type on pubmed.ncbi.nlm.nih.gov, or look up PMIDs, PMCIDs and DOIs, and get clean JSON in seconds. Every article comes with its full abstract, authors with affiliations and ORCID iDs, MeSH terms, keywords, publication types, DOI and a formatted citation. Switch on full text to also get open-access PubMed Central articles as Markdown for RAG pipelines and AI agents. No login, no API key.

✨ Why use PubMed Scraper?

• ⚡ Fast — official NCBI E-utilities, 200 records per request; a 20-article search finishes in about 10 seconds and 1,000 articles in about 30.
• 📄 Complete records — full structured abstracts (not the truncated snippets of the search page), all authors and affiliations, MeSH terms with major topics, grants and chemicals.
• 🤖 Built for AI and RAG — flat, predictable fields and optional PMC full text as Markdown with sections, tables and figure captions.
• 🔎 Real PubMed search — field tags, Boolean logic, date ranges, article types, journals, languages, free full text and human-study filters; PubMed search URLs work too.
• 🤝 Polite by design — stays under NCBI's limit of 3 requests per second without an API key.
• 💸 Cheap — pay per article, nothing for empty searches.

🎯 Use cases

• 🧠 RAG corpora — collect open-access papers on a disease or drug with their full text for a vector store.
• 🕵️ Agent tools — let an AI agent search PubMed, read abstracts and pull the full article when available.
• 📚 Systematic reviews — export every randomized controlled trial or meta-analysis on a topic with abstracts and MeSH terms.
• 📈 Research monitoring — schedule a "most recent" search to catch new papers in your field every week.
• 🏥 Medical affairs and pharma intelligence — track publications on a compound, a target or a competitor's trials.
• 🧑‍🔬 KOL and affiliation mapping — find who publishes on a topic and where they work.

🚀 How to use

1. Enter one or more search queries (keywords or PubMed syntax such as asthma[mh] AND children[tiab]), and/or paste PMIDs, PubMed URLs, PMCIDs or DOIs.
2. Optionally add filters: date range, article types, journals, languages, free full text, with abstract, human studies.
3. Set Max articles per query and the sort order.
4. Turn on Include PMC full text if you need whole articles.
5. Run it and export the results as JSON, CSV, Excel or HTML, or read them through the API.

📥 Input

{
  "searchQueries": ["crispr gene therapy"],
  "articleTypes": ["Review"],
  "dateFrom": "2023-01-01",
  "maxResults": 50,
  "sortBy": "date",
  "includeFullText": true
}

📤 Output

Example from a real run (lists, abstract and full text shortened):

{
  "pmid": "38786024",
  "title": "CRISPR-Based Gene Therapies: From Preclinical to Clinical Treatments.",
  "abstract": "In recent years, clustered regularly interspaced short palindromic repeats (CRISPRs) and CRISPR-associated (Cas) protein have emerged as a revolutionary gene editing tool to treat inherited disorders ...",
  "authors": [
    "Marine Laurent",
    "Marine Geoffroy",
    "Giulia Pavani",
    "..."
  ],
  "authorCount": 4,
  "firstAuthor": "Marine Laurent",
  "authorDetails": [
    {
      "name": "Marine Laurent",
      "lastName": "Laurent",
      "foreName": "Marine",
      "initials": "M",
      "collective": false,
      "orcid": null,
      "affiliations": [
        "INTEGRARE, UMR_S951, Genethon, Inserm, Univ Evry, Université Paris-Saclay, 91190 Evry, France."
      ]
    }
  ],
  "affiliations": [
    "INTEGRARE, UMR_S951, Genethon, Inserm, Univ Evry, Université Paris-Saclay, 91190 Evry, France.",
    "..."
  ],
  "journal": "Cells",
  "journalAbbrev": "Cells",
  "volume": "13",
  "issue": "10",
  "pages": null,
  "pubDate": "2024-05-08",
  "pubYear": 2024,
  "publicationTypes": [
    "Journal Article",
    "Review",
    "Research Support, Non-U.S. Gov't"
  ],
  "meshTerms": [
    "Humans",
    "Genetic Therapy",
    "CRISPR-Cas Systems",
    "Animals",
    "Gene Editing",
    "..."
  ],
  "meshDetails": [
    {
      "descriptor": "Humans",
      "descriptorId": "D006801",
      "qualifiers": [],
      "majorTopic": false
    }
  ],
  "keywords": [
    "CRISPR/Cas9",
    "Duchenne muscular dystrophy",
    "blood disorders",
    "gene editing",
    "..."
  ],
  "doi": "10.3390/cells13100800",
  "pmcid": "PMC11119143",
  "isFreeInPmc": true,
  "isRetracted": false,
  "citation": "Cells. 2024 May 08;13(10). doi: 10.3390/cells13100800.",
  "pubmedUrl": "https://pubmed.ncbi.nlm.nih.gov/38786024/",
  "doiUrl": "https://doi.org/10.3390/cells13100800",
  "pmcUrl": "https://pmc.ncbi.nlm.nih.gov/articles/PMC11119143/",
  "input": "38786024",
  "inputType": "id",
  "scrapedAt": "2026-10-15T23:03:05Z",
  "fullText": "# CRISPR-Based Gene Therapies: From Preclinical to Clinical Treatments\n\n## Abstract\n\nIn recent years, clustered regularly interspaced short palindromic repeats (CRISPRs) and CRISPR-associated (Cas) protein have emerged as a revolutionary gene editing...",
  "fullTextSource": "pmc",
  "fullTextWords": 8897,
  "fullTextLicense": "https://creativecommons.org/licenses/by/4.0/",
  "fullTextError": null
}

💰 Pricing

• $3.00 per 1,000 articles with the full record (abstract, authors, affiliations, MeSH terms, identifiers).
• $5.00 per 1,000 articles when PMC full text is switched on and found; articles without an open full text are billed at the normal rate.
• Example: 1,000 articles ≈ $3.00, plus a few cents of platform usage (compute and proxy traffic), which is billed separately.
• Empty searches and invalid inputs cost nothing.

❓ FAQ

Is it legal to scrape PubMed? The actor uses NCBI's official public E-utilities and respects their usage limits. PubMed citation data is freely reusable; full texts keep the license shown in fullTextLicense.

Do I need an NCBI account, API key or proxy? No. The default datacenter proxy is enough.

How many articles can I get? Up to 10,000 per query, which is NCBI's limit for search paging. Split bigger jobs by date range.

Why do some articles have no full text? Only articles in PubMed Central's open-access and author-manuscript collections can be downloaded; fullTextError says why one is missing (not in PMC, or the publisher does not allow it).

Is "Best match" the same order as the website? It uses NCBI's relevance sort, which can differ slightly from the website's machine-learned Best Match order. Most recent, publication date, author and journal sorts are identical.

Can I use it from an AI agent or on a schedule? Yes. Call it through the API or MCP with a query and read the dataset, or schedule a "most recent" search.

⚠️ Limits

• At most 10,000 articles per query (NCBI limit).
• Citation counts and "similar articles" are not included.
• Full text only for PubMed Central open-access and author-manuscript articles; figures are reduced to their captions, and complex tables with merged cells can come out simplified.
• Books and documents (NCBI Bookshelf records) are skipped.