A scraper that gets data from one web page with Cheerio. The simplest way to start scraping.
Create it locally
$scrapely create my-actor --template js-start
Main file
.JS
src/main.jsimport { Actor, log } from 'scrapely';
// Axios - Promise based HTTP client for the browser and node.js (Read more at https://axios-http.com/docs/intro).
import axios from 'axios';
// Cheerio - The fast, flexible & elegant library for parsing and manipulating HTML and XML (Read more at https://cheerio.js.org/).
import * as cheerio from 'cheerio';
// this is ESM project, and as such, it requires you to specify extensions in your relative imports
// read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions
// import { router } from './routes.js';
// The init() call configures the Actor to correctly work with the platform-provided environment - mainly the storage infrastructure. It is necessary that every Actor performs an init() call.
await Actor.init();
// Structure of input is defined in input_schema.json
const input = await Actor.getInput();
const { url } = input;
// Fetch the HTML content of the page.
const response = await axios.get(url);
// Parse the downloaded HTML with Cheerio to enable data extraction.
const $ = cheerio.load(response.data);
// Extract all headings from the page (tag name and text).
const headings = [];
$('h1, h2, h3, h4, h5, h6').each((i, element) => {
const headingObject = {
level: $(element).prop('tagName').toLowerCase(),
text: $(element).text(),
};
log.info('Extracted heading', headingObject);
headings.push(headingObject);
});
// Save headings to Dataset - a table-like storage.
await Actor.pushData(headings);
// Gracefully exit the Actor process. It's recommended to quit all Actors with an exit().
await Actor.exit();
JavaScript one-page scraper (Cheerio)
A scraper that gets data from one web page with Cheerio. The simplest way to start scraping.
Included features
- Scrapely SDK — Actor lifecycle, input, storage and proxies
- Cheerio — fast HTML parsing
- Input schema — define and validate the Actor's input (
.actor/input_schema.json) - Dataset schema — structured results with a preview view (
.actor/dataset_schema.json)
Getting started locally
npm install
npm run start # run the Actor
Deploying to Scrapely
Create an Actor from this template in the Scrapely Console (Develop new → template → Git). Scrapely creates a repository in your account, pushes this code, and builds the Actor. Every push to your repository triggers a new build.