Skip to content

Bulk File & Image Downloader

Try for free

Download every file from a list of links, a dataset field or JSON items: images, PDFs, documents, audio, video and archives. Each file gets a download link and one link gets all files as a ZIP, or the files go to your own S3 bucket. Filters by type, size and image dimensions.

📥 Bulk File & Image Downloader

Download every file from a list of links, from a field of your dataset or from JSON items: images, PDFs, Word and Excel files, audio, video, archives or anything else. Each file gets its own download link, one link downloads all of them as a ZIP, or the files go straight to your own S3 bucket. No login or API key, and you pay $0.22 per 1,000 files on the Free plan (down to $0.15 on Business), with platform usage included.

✨ Why use Bulk File & Image Downloader?

  • 📦 Any file, in bulk: hundreds or tens of thousands of links in one run, downloaded in parallel, each file checked by its real content (type, size, image dimensions, MD5 checksum).
  • 🔗 Straight from your scraper results: pick a dataset and name the field with the links (images, details.photos, gallery[0].src ...). Lists of links in one field work too.
  • 🗂️ Your choice of destination: a download link per file plus one ZIP of everything, your own AWS S3 or S3-compatible bucket (R2, B2, Wasabi, MinIO ...), or check-only mode that saves nothing.
  • 🏷️ Files named your way: {name}, {index}, {hash} or any field of the dataset item, for example {item.sku}-{name} for product photos.
  • 🧹 Clean results: duplicate links downloaded once, filters for file type, size and image dimensions, and web pages that are not files reported instead of saved.
  • 💸 Pay per file: failed and skipped links are free, and every file includes its first 10 MB.

🎯 Use cases

  • 🛒 Product images: download every photo from an online-shop scrape, named by SKU, for your own catalogue or a marketplace upload.
  • 🏠 Listings archives: keep the photos of property, car or job listings before the listing goes offline.
  • 📄 Document collections: fetch hundreds of PDF reports, filings, manuals or price lists from a list of links into one ZIP.
  • 🤖 Training data: build image or document datasets for machine learning, with minimum sizes and dimensions so icons and thumbnails stay out.
  • ☁️ Straight to cloud storage: move scraped media into your S3 bucket on a schedule, skipping files that are already there.
  • ✅ Link checking: find broken file links (404, refused, web pages instead of files) and read the size and type of each file without storing anything.

🚀 How to use

  1. Paste your links into File links, or pick a Dataset with file links (and name the Field with the file links when it is not one of the usual ones).
  2. Optionally keep only some File types, sizes or image dimensions under Filters.
  3. Choose where to Save files to: the key-value store (default), your S3 bucket, or check only.
  4. Click Start. Each link becomes a row with the file's download link, type, size and checksum.
  5. Download single files from the table, all files at once from the ZIP link (zipUrl in every row and allFilesZipUrl in the run's summary), or export the table as JSON, CSV or Excel.

📥 Input

FieldWhat it does
fileUrlsDirect file links, one per line. Google Drive, Dropbox and GitHub share links are turned into download links.
datasetIdDownload the links found in one of your datasets.
urlFieldField with the links: images, details.images, gallery[0].src, photos.url; several fields separated by commas. Empty: the usual link fields are found automatically.
itemsJSON items for API calls: objects that hold links, or plain link strings.
maxFilesStop after this many links (empty: every link).
datasetOffsetSkip this many dataset items first, to split a very large dataset over several runs.
fileTypesKeep only images, documents, audio, video and/or archives.
extensionsKeep files with these extensions too (pdf, epub ...); add html to save web pages.
minSizeKb, maxSizeMbSkip files smaller or larger than this.
minWidth, minHeightSkip images smaller than this many pixels.
saveTostore (default: link per file + ZIP), s3 (your bucket) or none (check only).
fileNameTemplateFile names with {name}, {index}, {md5}, {hash}, {host}, {item}, {item.<field>}.
keyValueStoreNameSave into a named store of yours that is kept after the run's data retention.
convertWebpToPngSave WebP images as PNG.
includeItemCopy the dataset item each link came from into the row.
s3Bucket, s3AccessKeyId, s3SecretAccessKey, s3Region, s3Endpoint, s3Prefix, s3SkipExistingYour S3 or S3-compatible bucket. The keys are secret inputs, used for this run only.
maxConcurrency, downloadTimeoutSecs, maxRetriesDownloads at once (16), timeout per file (300 s) and retries (3).
skipDuplicatesDownload a repeated link only once (on).
requestHeadersExtra headers, for example a Referer for sites that only serve images to their own pages.
proxyConfigurationDatacenter proxy by default; Residential for sites that refuse it.

From a dataset, with product photos named by SKU:

{
  "datasetId": "<your dataset ID>",
  "urlField": "images",
  "fileNameTemplate": "{item.sku}-{name}",
  "fileTypes": ["images"],
  "minWidth": 300
}

From a list of links:

{
  "fileUrls": [
    "https://upload.wikimedia.org/wikipedia/commons/9/9a/Gull_portrait_ca_usa.jpg",
    "https://raw.githubusercontent.com/mozilla/pdf.js/v4.0.379/test/pdfs/tracemonkey.pdf"
  ]
}

📤 Output

One row per link. Failed and skipped links have error and errorCode set and are free.

FieldDescription
urlThe link you gave.
fileUrlDownload link of the saved file (key-value store: valid 7 days; S3: the object's address).
fileNameName of the saved file (store key or S3 key).
contentType, extensionType read from the file's content, e.g. image/jpeg, jpg.
bytesFile size in bytes.
width, heightImage dimensions in pixels (images only).
md5MD5 checksum of the downloaded file, for de-duplication and integrity checks.
zipUrlOne link that downloads every file of the run as a ZIP (key-value store mode).
s3Uris3://bucket/key of the uploaded file (S3 mode).
savedTostore, s3 or none.
finalUrl, httpStatusWhere the file was finally served from after redirects, and the HTTP status.
index, itemIndexPosition of the link, and of the dataset or JSON item it came from.
itemThe dataset item the link came from (with includeItem).
error, errorCodeWhy a link was not saved: NOT_FOUND, FORBIDDEN, TIMEOUT, NOT_A_FILE, TOO_LARGE, TYPE_FILTERED, TOO_SMALL, IMAGE_TOO_SMALL, BLOCKED_URL ...

Example row from a test run:

{
  "url": "https://upload.wikimedia.org/wikipedia/commons/9/9a/Gull_portrait_ca_usa.jpg",
  "fileUrl": "https://api.scrape.ly/v2/key-value-stores/8a0ac9c0-0e53-494a-9e5a-c1afa788c59f/records/Gull_portrait_ca_usa.jpg?signature=...&attachment=1",
  "fileName": "Gull_portrait_ca_usa.jpg",
  "contentType": "image/jpeg",
  "extension": "jpg",
  "bytes": 718902,
  "width": 2272,
  "height": 1704,
  "md5": "1277fe6872879509a52832c49977b611",
  "savedTo": "store",
  "finalUrl": "https://upload.wikimedia.org/wikipedia/commons/9/9a/Gull_portrait_ca_usa.jpg",
  "httpStatus": 200,
  "index": 1,
  "itemIndex": null,
  "error": null,
  "errorCode": null,
  "zipUrl": "https://api.scrape.ly/v2/key-value-stores/8a0ac9c0-0e53-494a-9e5a-c1afa788c59f/records?signature=..."
}

The run's summary (key-value store record OUTPUT) has the totals, allFilesZipUrl, the store ID and the errors by code.

💰 Pricing

Pay per event, with platform usage (compute, proxy, storage) included:

EventFreeStarterScaleBusiness
File downloaded (first 10 MB included)$0.00022$0.00022$0.00018$0.00015
Large file, each further 10 MB$0.0006$0.0006$0.0005$0.0004
Residential proxy, per MB (instead of the 10 MB fee)$0.0004$0.0004$0.00035$0.0003
Run start$0.0037$0.0037$0.003$0.0025

Examples on the Free plan: 1,000 product photos of about 300 KB each cost about $0.22 plus $0.0037 for the run start. A 45 MB video costs $0.00022 + 4 x $0.0006 = $0.0026. Failed links, skipped files and duplicate links cost nothing.

❓ FAQ

Is it legal to download these files? The actor downloads files from links you give, the way a browser would. You are responsible for having the right to download and use them (copyright, terms of the site). Do not use it for files you may not copy.

Do I need a login, an API key or a proxy? No login or key. A datacenter proxy is used by default; switch to Residential if a site refuses downloads. Files behind a login cannot be downloaded, but you can add headers (for example a Referer) under Request headers.

How many files can I download? As many as you give: there is no limit on the number of links. In our tests 1,000 images took about 1.5 minutes and 10,000 small files about 18 minutes, with flat memory. datasetOffset and maxFiles let you split a very large dataset over several runs if you prefer.

How long are the download links valid? Links to the run's own store are signed for 7 days and the files are kept for your plan's data retention. Use Key-value store name to keep files in a named store, or save to your S3 bucket.

What does check-only mode cost? The same as a download: each file is still downloaded once to read its real type, size, dimensions and checksum, it is just not stored.

What if a link fails? The row says why (error, errorCode), and the link is free. Timeouts, refusals and server errors are retried up to 3 times from new IP addresses; missing files (404) are not retried.

Can I schedule it or call it from the API? Yes. Run it on a schedule, after a scraper with an automation, or from the API with fileUrls, items or datasetId in the input.

⚠️ Limits

  • Files saved to the key-value store can be up to 100 MB each, the most one stored file may have; bigger files get a TOO_LARGE row.
  • In S3 mode one file can be as large as the run's disk allows: about 1.3 GB at the default 512 MB memory; a run with more memory gets more disk. Check-only mode has no file size limit.
  • Store download links are signed for 7 days.
  • Pages that need a login, a cookie or JavaScript to reveal the file cannot be downloaded; give the direct file link.
  • Web pages (HTML) are reported as NOT_A_FILE unless you add html to File extensions.
  • Image dimensions are read for raster images (JPEG, PNG, WebP, GIF, AVIF, TIFF, BMP, ICO), not for SVG or HEIC.
  • Links that point to private or internal network addresses are refused.

Related actors

//From the blog\\

Latest articles

All articles
G2
4.7/5
Excellent
Trustpilot

The internet is full of data.
Start owning yours.

Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes.

Get Started FreeChat with Support
No credit card. No setup calls.