An AI agent for social media research, with built-in observability and tracing.
Create it locally
Main file
import { Actor, log } from 'scrapely';
import { OpenAIChatModel } from 'beeai-framework/adapters/openai/backend/chat';
import { ReActAgent } from 'beeai-framework/agents/react/agent';
import { UnconstrainedMemory } from 'beeai-framework/memory/unconstrainedMemory';
import { z } from 'zod';
import { StructuredOutputGenerator } from './structured_response_generator.js';
import { CalculatorSumTool } from './tools/calculator.js';
import { ScrapePageTool } from './tools/scrape_page.js';
// Scrapely LLM gateway: OpenAI-compatible endpoint (OpenRouter model IDs) billed against the user's
// Scrapely account, so no provider API key is needed. The run's SCRAPELY_TOKEN is used as the API key.
// The platform injects SCRAPELY_LLM_BASE_URL into every run.
const LLM_BASE_URL = process.env.SCRAPELY_LLM_BASE_URL || 'https://api.scrape.ly/v2/llm/api/v1';
// This is an ESM project, and as such, it requires you to specify extensions in your relative imports.
// Read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions
// Note that we need to use `.js` even when inside TS files
// import { router } from './routes.js';
// Actor input schema
interface Input {
query: string;
modelName: string;
debug?: boolean;
}
// The init() call configures the Actor to correctly work with the platform-provided environment - mainly the storage infrastructure. It is necessary that every Actor performs an init() call.
await Actor.init();
// Charge for Actor start
await Actor.charge({ eventName: 'actor-start' });
// Handle input
const {
// The query default value is provided only for template testing purposes.
// You can remove it.
query,
modelName,
debug,
} = (await Actor.getInput()) as Input;
if (debug) {
log.setLevel(log.LEVELS.DEBUG);
}
if (!query) {
throw new Error('An agent query is required.');
}
/**
* Actor code
*/
// Create an agent that can use tools.
// See https://framework.beeai.dev/modules/agents
log.debug(`Using model: ${modelName}`);
const llm = new OpenAIChatModel(
modelName,
{},
{
baseURL: LLM_BASE_URL,
apiKey: process.env.SCRAPELY_TOKEN,
},
);
const agent = new ReActAgent({
llm,
memory: new UnconstrainedMemory(),
tools: [new CalculatorSumTool(), new ScrapePageTool()],
});
// Store tool messages for later structured output generation.
// This can be removed if you don't need structured output.
const structuredOutputGenerator = new StructuredOutputGenerator(llm);
// Prompt the agent with the query.
// Debug log agent status updates, e.g., thoughts, tool calls, etc.
const response = await agent.run({ prompt: query }).observe((emitter) => {
emitter.on('update', async ({ update }) => {
log.debug(`Agent (${update.key}) 🤖 : ${update.value}`);
// Save tool messages for later structured output generation.
// This can be removed if you don't need structured output.
if (['tool_name', 'tool_output', 'tool_input'].includes(update.key as string)) {
structuredOutputGenerator.processToolMessage(
update.key as 'tool_name' | 'tool_output' | 'tool_input',
update.value,
);
}
// End of tool message saving.
});
});
log.info(`Agent 🤖 : ${response.result.text}`);
// Hacky way to get the structured output.
// Using the stored tool messages and the user query to create a structured output.
const structuredResponse = await structuredOutputGenerator.generateStructuredOutput(
query,
z.object({
summary: z.string(),
keyPoints: z.array(z.string()),
sources: z.array(
z.object({
url: z.string(),
title: z.string().nullable().optional(),
textLength: z.number().nullable().optional(),
}),
),
}),
);
log.debug(`Structured response: ${JSON.stringify(structuredResponse)}`);
// Charge for task completion
await Actor.charge({ eventName: 'task-completed' });
// Push results to the dataset.
await Actor.pushData({
query,
response: response.result.text,
// This can be removed if you don't need structured output.
structuredResponse: structuredResponse.object,
});
log.info('Pushed the data into the dataset!');
// Gracefully exit the Actor process. It's recommended to quit all Actors with an exit().
await Actor.exit();
TypeScript web research agent (BeeAI)
A BeeAI ReAct agent that researches websites with a page-scraping tool and returns a structured summary.
Included features
- Scrapely SDK — Actor lifecycle, input, storage and proxies
- BeeAI — agent framework
- Scrapely LLM gateway — call OpenRouter models with the run token; usage is billed to your Scrapely account, no provider API key needed
- Input schema — define and validate the Actor's input (
.actor/input_schema.json) - Dataset schema — structured results with a preview view (
.actor/dataset_schema.json)
Getting started locally
npm install
npm run start # run the Actor
npm run build # compile to dist/
LLM calls go to SCRAPELY_LLM_BASE_URL (set automatically on the platform) with the run's
SCRAPELY_TOKEN, so the agent works inside a Scrapely run without any provider API key.
Deploying to Scrapely
Create an Actor from this template in the Scrapely Console (Develop new → template → Git). Scrapely creates a repository in your account, pushes this code, and builds the Actor. Every push to your repository triggers a new build.