---
title: "SQL Server · Scrape.ly Docs"
description: "Add or upsert the results of each successful run as rows in your own SQL Server table."
url: https://scrape.ly/docs/sqlserver
markdown: https://scrape.ly/docs/sqlserver.md
---

# SQL Server

> Add or upsert the results of each successful run as rows in your own SQL Server table.

Adds the dataset items of each successful run as rows to a table in your own SQL Server database: Azure SQL Database, Amazon RDS for SQL Server, or any other SQL Server reachable from the internet. Scrape.ly connects with a connection string you paste. It is stored encrypted and never shown again.

## Prepare the table

Scrape.ly writes into a table that already exists; it never creates or changes tables. Create one with a column for each result field you want, and give Scrape.ly a database user that may only write to it:

```sql
CREATE TABLE dbo.results (
  url        nvarchar(450) PRIMARY KEY,
  title      nvarchar(max),
  price      decimal(10,2),
  tags       nvarchar(max),
  scraped_at datetime2 DEFAULT sysdatetime()
);

CREATE LOGIN scrapely WITH PASSWORD = 'a-long-random-password';
CREATE USER scrapely FOR LOGIN scrapely;
GRANT SELECT, INSERT, UPDATE ON dbo.results TO scrapely;
```

On Azure SQL Database, create a contained user in your database instead of a login: `CREATE USER scrapely WITH PASSWORD = 'a-long-random-password';`, then run the same `GRANT`. `UPDATE` is only needed for **Update existing rows**.

## Send results to a table

1. Click **New automation** (in **Automations**, or on an Actor's **Automations** tab) and pick **Send results to SQL Server**, or click **Or create manually** and, under **Send**, click **SQL Server**.
2. Paste the **Connection string**, such as `sqlserver://scrapely:password@db.example.com:1433/app`.
3. Enter the **Table**: `results`, or `schema.results` for a schema other than `dbo`.
4. Choose the **Write mode**: **Add new rows**, or **Update existing rows (upsert)** with a **Key column**.
5. Optionally set **Max rows per run** (1 to 100,000, default 1,000).
6. Use **Save and send test**. The test connects, checks that the table exists and that the user may write to it, and writes nothing.

**Connection**: Scrape.ly connects from the internet, so the server must accept outside connections. On Azure SQL Database, the server's firewall must allow connections from Scrape.ly, which has no fixed IP address: on the server, open **Networking**, set **Public network access** to **Selected networks**, click **Add a firewall rule** for the range `0.0.0.0` to `255.255.255.255`, and click **Save**. The login and password still protect the database. **Allow Azure services and resources to access this server** is not enough on its own: it only lets in services running inside Azure. Changes can take up to 5 minutes. The connection is always encrypted. The server's certificate is checked only when the connection string has `trustServerCertificate=false`, because many self-hosted servers use certificates that only their own tools trust. Only SQL logins (user and password) work, not Windows authentication. Connection strings with a host name are accepted; IP addresses, `localhost`, and private network addresses are not.

**Columns**: a result field is written to the column with the same name (upper or lower case doesn't matter). A nested field whose parent has no column of its own goes into the column named parent and child joined with `_`, so `address.city` goes into `address_city`. Objects and lists go into text columns as JSON. Fields without a column are skipped, and the delivery history lists them. Columns without a field get their default value. Computed, identity, and `rowversion` columns are never written. A row SQL Server refuses, such as `abc` for an `int` column, is skipped.

**Upsert**: with **Update existing rows**, a row whose key column already holds the item's value is updated instead of added (`MERGE`). The key column needs no unique index, but a primary key or unique index on it keeps upserts fast and stops duplicate rows. Only the columns the item has a field for are updated; the others keep their values. Items without a value for the key column are skipped and counted. When one run has several items with the same key, the last one wins.

**When it writes**: only for succeeded runs, and only when the run returned items. Rows go in batches that stay under SQL Server's limit of 2,100 values per statement, at most 1,000 rows at a time, each batch in its own transaction, so a failure partway leaves the earlier batches written and the delivery history says how many rows are in the table. A row SQL Server refuses is skipped and counted; the other rows are still written.

**Limits**: each SQL Server automation of this kind writes at most 30 times an hour, and an account at most 500 exports a day (Google Sheets, S3, GitHub, Airtable, HubSpot, Salesforce, Notion, Dropbox, OneDrive, Postgres, MySQL, MongoDB, SQL Server, ClickHouse, Azure Blob, BigQuery, Snowflake, and OpenAI Vector Store together). A statement that takes longer than 60 seconds is stopped, and an export stops after 8 minutes. If the server can't be reached, the login is refused, the table doesn't exist, the user may not write to it, or none of the fields match a column, the delivery history on the automation's page says so.

## API

A webhook with a `sqlserver` notification. Only `ACTOR.RUN.SUCCEEDED` is allowed.

```bash
curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Results to SQL Server",
    "notification": {
      "channel": "sqlserver",
      "connectionString": "sqlserver://scrapely:password@db.example.com:1433/app",
      "table": "dbo.results",
      "mode": "upsert",
      "keyColumn": "url",
      "maxRows": 1000
    }
  }'
```

`mode` is `insert` (default) or `upsert`; `keyColumn` is only used with `upsert`. Responses carry `notification.sqlserverSettings` with `table`, `mode`, `keyColumn`, `maxRows`, and a `connectionHint` (`user@host:port/database`); the connection string is never returned. On `PUT`, leaving out `connectionString` keeps the saved one. In an automation, use a delivery with `"channel": "sqlserver"` and the same fields.

---

- Previous: [MongoDB](https://scrape.ly/docs/mongodb.md)
- Next: [ClickHouse](https://scrape.ly/docs/clickhouse.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)