---
title: "OpenAI Vector Store · Scrape.ly Docs"
description: "Add the results of each successful run to your OpenAI vector store for file search."
url: https://scrape.ly/docs/openai-vector-store
markdown: https://scrape.ly/docs/openai-vector-store.md
---

# OpenAI Vector Store

> Add the results of each successful run to your OpenAI vector store for file search.

Adds the dataset of each successful run to one of your OpenAI vector stores, so assistants and the Responses API can find it with file search. Scrape.ly uploads the results as JSON files with your own OpenAI API key and adds them to the vector store; OpenAI splits, embeds and indexes them. The key is stored encrypted, only ever sent to `api.openai.com`, and never shown again.

Uploading, storing and indexing the files is billed to your OpenAI account, not to Scrape.ly.

## Get a key and a vector store

In the [OpenAI dashboard](https://platform.openai.com):

1. Open **Storage** → **Vector stores**, create a vector store (or open an existing one), and copy its ID. It looks like `vs_abc123…`.
2. Open **API keys** and create a key in the same project as the vector store. A restricted key works if it has **Write** access to **Files** and **Vector stores**.

## Add results after each run

Set it up with the **Send results to an OpenAI vector store** recipe in **New automation**, or add **OpenAI Vector Store** as a delivery in the [automation editor](https://scrape.ly/docs/automations.md#5-send) to add the results of several Actors, after filters and an AI step.

1. Click **New automation** (in **Automations**, or on an Actor's **Automations** tab) and pick **Send results to an OpenAI vector store**, or click **Or create manually** and, under **Send**, click **OpenAI Vector Store**.
2. Paste the **OpenAI API key** and the **Vector store ID**.
3. Optionally set **Max items per run** (empty adds every item).
4. Use **Save and send test**. The test only reads the vector store to check the key and the ID; it adds nothing.

**Files**: each run adds JSON files named `<actor-name>-<run-id>-part-001.json`, each a JSON array of whole items. Large runs are split into parts of about 10 MB, so every file stays within what file search indexes. The files are tagged with the attributes `scrapely_run_id` and `scrapely_actor`, which you can use as file search filters.

**When it uploads**: only for succeeded runs, and only when the run returned items. OpenAI indexes the files in the background after they are added; the vector store shows them as in progress until then. If adding the files fails, the uploaded files are deleted again. An export that takes longer than 8 minutes is stopped; for very large runs, set **Max items per run**.

**Older runs**: files from earlier runs stay in the vector store. To keep it small, delete old files in the OpenAI dashboard, or set an expiration policy on the vector store.

**Limits**: each OpenAI Vector Store automation of this kind uploads at most 30 times an hour, and an account at most 500 exports a day (all result deliveries together). If the key is refused, the account has no credit left, or the vector store doesn't exist, the delivery history on the automation's page says so.

## API

A webhook with an `openai` notification. Only `ACTOR.RUN.SUCCEEDED` is allowed.

```bash
curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Results to OpenAI",
    "notification": {
      "channel": "openai",
      "openaiApiKey": "sk-proj-…",
      "vectorStoreId": "vs_abc123"
    }
  }'
```

`maxItems` is optional; leave it out or set it to `null` to add every item. Responses carry `notification.openaiSettings` with the `vectorStoreId`, `maxItems` and a `keyHint` (`sk-…` and the last four characters); the key is never returned. On `PUT`, leaving out `openaiApiKey` keeps the saved one. In an automation, use a delivery with `"channel": "openai"` and the same fields.

---

- Previous: [Azure Blob Storage](https://scrape.ly/docs/azure-blob.md)
- Next: [Postgres](https://scrape.ly/docs/postgres.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)