MCP & AI agents

LangGraph

Use Scrape.ly tools in LangGraph agents and workflows.

LangGraph agents use the Scrape.ly tools from the langchain-scrapely package. There's no separate package. See LangChain for every tool and its options.

.bash

pip install langchain langchain-scrapely langchain-openai
export SCRAPELY_TOKEN=your-api-key
export OPENAI_API_KEY=your-openai-key

Quick start

create_agent builds a LangGraph agent:

.python

from langchain.agents import create_agent
from langchain_scrapely import ScrapelyActorsTool

agent = create_agent('openai:gpt-4o', tools=[ScrapelyActorsTool('username/actor-name')])
result = agent.invoke({'messages': [{'role': 'user', 'content': 'Find the top coffee shops in Berlin'}]})
print(result['messages'][-1].content)

Let the agent choose the Actor

The general tools let the agent run any Actor or saved task by name and read results in pages:

.python

from langchain_scrapely import SCRAPELY_CORE_TOOLS

agent = create_agent('openai:gpt-4o', tools=[cls() for cls in SCRAPELY_CORE_TOOLS])

for step in agent.stream(
    {'messages': [{'role': 'user', 'content': 'Run username/actor-name for "coffee Berlin" and summarize the results'}]},
    stream_mode='values',
):
    step['messages'][-1].pretty_print()

stream_mode='values' prints each step as it happens: the tool calls, their results, and the final answer.

Combine tools

Pass several Actor tools together, and the agent picks the one it needs for each step:

.python

tools = [
    ScrapelyActorsTool('username/search-actor'),
    ScrapelyActorsTool('username/profile-actor'),
]
agent = create_agent('openai:gpt-4o', tools=tools)