ScraperAPI MCP server

The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.

This is the self-hosted (local) server. A hosted (remote) version is also available.

Features

Full implementation of the Model Context Protocol specification
Seamless integration with ScraperAPI for web scraping
Simple setup with Python or Docker

Architecture

          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
          └───────────────┘     └───────────────────────┘     └───────────────┘
                                            │
                                            ▼
                                  ┌──────────────────┐
                                  │  ScraperAPI API  │
                                  └──────────────────┘

Installation

The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.

Prerequisites

Python 3.11+
Docker (optional)

Using Python

Install the package:

pip install scraperapi-mcp-server

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "python",
      "args": ["-m", "scraperapi_mcp_server"],
      "env": {
        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
      }
    }
  }
}

Using Docker

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "-e",
        "API_KEY=${API_KEY}",
        "--rm",
        "scraperapi-mcp-server"]
    }
  }
}

Tip

If your command is not working (for example, you see a package not found error when trying to start the server), double-check the path you are using. To find the correct path, activate your virtual environment first, then run:

which <YOUR_COMMAND>

Available tools

API

scrape — Scrape any web page or image, bypassing anti-bot protection

Retrieve the content of a web page, or download an image, from a URL.

Parameter	Type	Description	Required
`url`	string	Target URL to scrape	Yes
`render`	boolean	Enable JavaScript rendering for dynamic pages (default: `false`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No
`premium`	boolean	Use premium residential/mobile proxies (default: `false`)	No
`ultra_premium`	boolean	Advanced anti-bot bypass; incompatible with `premium` (default: `false`)	No
`device_type`	string	`mobile` or `desktop` user agent	No
`output_format`	string	`markdown` (default), `text`, `csv`, or `json`	No
`autoparse`	boolean	Auto-parse supported sites into structured data (default: `false`)	No

Returns: the scraped content as a string, or image data for image URLs.

SDEs

Structured Data Endpoints (SDEs) return pre-parsed JSON (or CSV) instead of raw HTML. Every SDE tool also accepts output_format (json by default, or csv), tld, and country_code, shown in each tool's table.

Google

google_search — Parsed Google Search (SERP) results

Parameter	Type	Description	Required
`query`	string	Search query, as typed into Google	Yes
`num`	integer	Number of results to return on the page	No
`start`	integer	Zero-based result offset for pagination	No
`hl`	string	Interface/host language code (e.g. `en`, `es`)	No
`gl`	string	Country edition to search (2-letter code)	No
`uule`	string	Google `uule` geolocation string (advanced)	No
`date_range_start`	string	Start of a custom date range, `MM/DD/YYYY`	No
`date_range_end`	string	End of a custom date range, `MM/DD/YYYY`	No
`time_period`	string	Recent window: `1H`, `1D`, `1W`, `1M`, or `1Y`	No
`include_html`	boolean	Include raw HTML alongside parsed data (default: `false`)	No
`tbs`	string	Raw Google `tbs` filter parameter (advanced)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Top-level domain (e.g. `com`, `co.uk`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

google_news — Parsed Google News results

Parameter	Type	Description	Required
`query`	string	Search query	Yes
`num`	integer	Number of results to return on the page	No
`start`	integer	Zero-based result offset for pagination	No
`hl`	string	Interface/host language code	No
`gl`	string	Country edition to search (2-letter code)	No
`uule`	string	Google `uule` geolocation string (advanced)	No
`date_range_start`	string	Start of a custom date range, `MM/DD/YYYY`	No
`date_range_end`	string	End of a custom date range, `MM/DD/YYYY`	No
`time_period`	string	Recent window: `1H`, `1D`, `1W`, `1M`, or `1Y`	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Top-level domain (e.g. `com`, `co.uk`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

google_jobs — Parsed Google Jobs listings

Parameter	Type	Description	Required
`query`	string	Search query	Yes
`num`	integer	Number of results to return on the page	No
`start`	integer	Zero-based result offset for pagination	No
`hl`	string	Interface/host language code	No
`gl`	string	Country edition to search (2-letter code)	No
`uule`	string	Google `uule` geolocation string (advanced)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Top-level domain (e.g. `com`, `co.uk`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

google_shopping — Parsed Google Shopping product results

Parameter	Type	Description	Required
`query`	string	Search query	Yes
`num`	integer	Number of results to return on the page	No
`start`	integer	Zero-based result offset for pagination	No
`hl`	string	Interface/host language code	No
`gl`	string	Country edition to search (2-letter code)	No
`uule`	string	Google `uule` geolocation string (advanced)	No
`include_html`	boolean	Include raw HTML alongside parsed data (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Top-level domain (e.g. `com`, `co.uk`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

google_maps_search — Parsed Google Maps place/business results

Parameter	Type	Description	Required
`query`	string	Search query (e.g. `coffee shops in Austin`)	Yes
`latitude`	number	Latitude of the map center, in decimal degrees	Yes
`longitude`	number	Longitude of the map center, in decimal degrees	Yes
`zoom`	integer	Map zoom level (roughly 3=country, 10=city, 15=street)	No
`include_html`	boolean	Include raw HTML alongside parsed data (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Top-level domain (e.g. `com`, `co.uk`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

Amazon

amazon_product — Parsed Amazon product details by ASIN

Parameter	Type	Description	Required
`asin`	string	10-character Amazon product identifier (e.g. `B08N5WRWNW`)	Yes
`language`	string	Language code for localized content (e.g. `en_US`)	No
`include_html`	boolean	Include raw HTML alongside parsed data (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Amazon TLD (e.g. `com`, `co.uk`, `de`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

amazon_search — Parsed Amazon product search results

Parameter	Type	Description	Required
`query`	string	Search query, as typed into Amazon	Yes
`page`	integer	1-based results page number	No
`sort_by`	string	Sort order (e.g. `price-asc-rank`, `review-rank`)	No
`department`	string	Restrict search to a department/category (e.g. `electronics`)	No
`ref`	string	Amazon `ref` referral/context token (advanced)	No
`language`	string	Language code for localized content	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Amazon TLD (e.g. `com`, `co.uk`, `de`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

amazon_offers — Parsed seller offers for an Amazon product

Parameter	Type	Description	Required
`asin`	string	10-character Amazon product identifier	Yes
`condition`	string	Comma-separated condition filters (e.g. `f_new,f_usedlikenew,f_usedverygood,f_usedgood,f_usedacceptable`)	No
`f_new`	boolean	Include only New-condition offers	No
`f_used_like_new`	boolean	Include Used - Like New offers	No
`f_used_very_good`	boolean	Include Used - Very Good offers	No
`f_used_good`	boolean	Include Used - Good offers	No
`f_used_acceptable`	boolean	Include Used - Acceptable offers	No
`language`	string	Language code for localized content	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Amazon TLD (e.g. `com`, `co.uk`, `de`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

Walmart

walmart_search — Parsed Walmart product search results

Parameter	Type	Description	Required
`query`	string	Product search query, as typed into Walmart	Yes
`page`	integer	1-based results page number	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Walmart TLD (e.g. `com`, `ca`, `com.mx`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

walmart_product — Parsed Walmart product details by product ID

Parameter	Type	Description	Required
`product_id`	string	Walmart product ID	Yes
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Walmart TLD (e.g. `com`, `ca`, `com.mx`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

walmart_category — Parsed products within a Walmart category

Parameter	Type	Description	Required
`category`	string	Walmart category ID	Yes
`page`	integer	1-based results page number	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Walmart TLD (e.g. `com`, `ca`, `com.mx`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

walmart_review — Parsed customer reviews for a Walmart product

Parameter	Type	Description	Required
`product_id`	string	Walmart product ID	Yes
`page`	integer	1-based results page number	No
`sort`	string	Sort order (e.g. `helpful`, `recent`)	No
`ratings`	string	Comma-separated star ratings to filter by (e.g. `4,5`)	No
`verified_purchase`	boolean	Include only reviews with a "Verified Purchase" badge (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Walmart TLD (e.g. `com`, `ca`, `com.mx`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

eBay

ebay_search — Parsed eBay product search results

Parameter	Type	Description	Required
`query`	string	Search keywords	Yes
`page`	integer	1-based results page number	No
`items_per_page`	integer	Number of items per page	No
`seller_id`	string	Restrict results to a specific seller	No
`condition`	string	Comma-separated: `new`, `used`, `open_box`, `refurbished`, `for_parts`, `not_working`	No
`buying_format`	string	`buy_it_now`, `auction`, or `accepts_offers`	No
`show_only`	string	Comma-separated: `returns_accepted`, `authorized_seller`, `completed_items`, `sold_items`, `sale_items`, `listed_as_lots`, `search_in_description`, `benefits_charity`, `authenticity_guarantee`	No
`sort_by`	string	`best_match`, `ending_soonest`, `newly_listed`, `price_lowest`, `price_highest`, or `distance_nearest`	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	eBay TLD (e.g. `com`, `co.uk`, `de`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

ebay_product — Parsed eBay listing details by item ID

Parameter	Type	Description	Required
`product_id`	string	eBay item ID	Yes
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	eBay TLD (e.g. `com`, `co.uk`, `de`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

Redfin

Every Redfin tool takes a full Redfin URL matching the tool (property, search, or agent page).

redfin_for_sale — Parsed Redfin listing data for a home for sale

Parameter	Type	Description	Required
`url`	string	Full Redfin for-sale property URL	Yes
`raw`	boolean	Return raw extracted JSON instead of parsed data (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Redfin TLD (`com`, `ca`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

redfin_for_rent — Parsed Redfin listing data for a rental property

Parameter	Type	Description	Required
`url`	string	Full Redfin rental property URL	Yes
`raw`	boolean	Return raw extracted JSON instead of parsed data (default: `false`)	No
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Redfin TLD (`com`, `ca`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

redfin_search — Parsed Redfin search results

Parameter	Type	Description	Required
`url`	string	Full Redfin search-results URL (with filters)	Yes
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Redfin TLD (`com`, `ca`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

redfin_agent — Parsed Redfin real-estate agent profile data

Parameter	Type	Description	Required
`url`	string	Full Redfin agent profile URL	Yes
`output_format`	string	`json` (default) or `csv`	No
`tld`	string	Redfin TLD (`com`, `ca`)	No
`country_code`	string	ISO 2-letter country code for geo-targeting	No

Crawler

The crawler is asynchronous: crawler_job_start returns a job id immediately, then you poll crawler_job_status until the crawl finishes. Per-page results can also be pushed to a callback_url webhook.

crawler_job_start — Start an async crawl job from a starting URL

Parameter	Type	Description	Required
`start_url`	string	The URL where crawling begins (depth 0)	Yes
`url_regexp_include`	string	Regex selecting which links to follow. Use `.*` to crawl all links. Advanced: named groups `(?<full_url>...)` / `(?<relative_url>...)` target absolute vs relative URLs	Yes
`max_depth`	integer	Maximum crawl depth (start URL is depth 0). Provide `max_depth` or `crawl_budget`	No*
`crawl_budget`	integer	Maximum ScraperAPI credits the crawl may consume. Provide `max_depth` or `crawl_budget`	No*
`url_regexp_exclude`	string	Regex for URLs to exclude from crawling	No
`api_params`	object	Per-scrape controls applied to each page (e.g. `render`, `country_code`, `premium`, `device_type`, `output_format`)	No
`callback_url`	string	Webhook URL to receive per-page results and the final summary	No
`additional_data`	object	Arbitrary metadata to attach to the job	No
`schedule`	object	Recurring schedule: `{ name, interval (once/hourly/daily/weekly/monthly), cron }`	No
`enabled`	boolean	For scheduled projects, whether the schedule is enabled (default: `true`)	No

* Provide either max_depth or crawl_budget.

Returns: JSON with the job id and initial status (e.g. {"status": "initiated", "jobId": "..."}).

crawler_job_status — Get the status of a crawl job

Parameter	Type	Description	Required
`job_id`	string	The job id returned by `crawler_job_start`	Yes

Returns: JSON with the job's page counts (done/failed/active).

crawler_job_delete — Cancel and delete a crawl job

Parameter	Type	Description	Required
`job_id`	string	The job id returned by `crawler_job_start`	Yes

AI Parser

Build a reusable parser from a few example URLs, then apply it to any similar page for structured extraction. Two-phase: ai_parser_create returns a parser id immediately and generation runs in the background — poll ai_parser_get_details until its status is FINISHED, then call ai_parser_parse_url.

ai_parser_create — Create a reusable AI parser from example URLs

Parameter	Type	Description	Required
`name`	string	A name for the parser	Yes
`urls`	array<string>	1–3 example URLs of the same page type (same structure)	Yes
`scraper_params`	object	ScraperAPI fetch options for the example pages: `render`, `country_code`, `premium`, `session_number`, `keep_headers`, `device_type`, `ultra_premium`, `follow_redirect`, `retry_404`	No
`fields`	array	Pre-declared fields to extract: `[{ name, description, type?, selector? }]` (`type` is `string`/`number`/`array`). If omitted, fields are inferred	No

Returns: JSON with the new parser's id and version. Generation is asynchronous — poll ai_parser_get_details until status is FINISHED.

ai_parser_get_details — Get a parser's details and generation status

Parameter	Type	Description	Required
`parser_id`	string	The parser id returned by `ai_parser_create`	Yes
`version`	integer	Specific parser version (default: latest)	No

Returns: JSON with the parser's status (GENERATING/FINISHED/FAILED), fields, and example results.

ai_parser_parse_url — Parse a URL with an existing parser

Parameter	Type	Description	Required
`parser_id`	string	The parser id to run	Yes
`url`	string	The URL to scrape and parse	Yes
`version`	integer	Specific parser version (default: latest)	No

Returns: JSON {"parser": ..., "version": ..., "result": {...}}. Costs 1 credit per call.

ai_parser_list — List the parsers on your account

No parameters. Returns a JSON array of parser summaries (id, name, status, version, generation time).

ai_parser_delete — Delete a parser

Parameter	Type	Description	Required
`parser_id`	string	The parser id to delete	Yes

ai_parser_update — Edit a parser's fields (creates a new version)

Parameter	Type	Description	Required
`parser_id`	string	The parser id to update	Yes
`version`	integer	Version to base the update on (default: latest)	No
`add_fields`	array	Fields to add: `[{ name, description, type?, selector? }]` (triggers async regeneration)	No
`modify_fields`	array	Fields to modify (triggers async regeneration)	No
`rename_fields`	array	Renames: `[{ name, new_name }]` (applied immediately)	No
`remove_fields`	array<string>	Field names to remove (applied immediately)	No

Prompt templates

Please scrape this URL <URL>. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
Can you scrape URL <URL> to extract <SPECIFIC_DATA>? If the request returns missing/incomplete <SPECIFIC_DATA>, set render=true to enable JS Rendering.

Configuration

Settings

Configure the server through environment variables. Only API_KEY is required.

Variable	Default	Description
`API_KEY`	—	Required. Your ScraperAPI API key.
`API_TIMEOUT_SECONDS`	`70`	Per-request timeout, in seconds.
`RATE_LIMIT_MAX_CALLS`	`10`	Maximum tool calls allowed per rate-limit window.
`RATE_LIMIT_WINDOW_SECONDS`	`60`	Length of the rate-limit window, in seconds.
`IMAGE_SIZE_LIMIT_BYTES`	`700000`	Maximum size of an image the `scrape` tool returns inline.

Client Setup

Use the JSON configuration file from the Installation section. Below are steps for common clients.

Claude Desktop & Claude Code

Claude Desktop:

Open Claude Desktop and click the settings icon
Select the "Developer" tab
Click "Edit Config" and paste the JSON configuration file

Claude Code:

Add the server manually to your .claude/settings.json with the JSON configuration file, or run:

claude mcp add scraperapi -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> -- python -m scraperapi_mcp_server

Cursor Editor

Open Cursor
Access the Settings Menu
Open Cursor Settings
Go to Tools & Integrations section
Click '+ Add MCP Server'
Choose Manual and paste the JSON configuration file

More here

Windsurf Editor

Open Windsurf
Access the Settings Menu
Click on the Cascade settings
Click on the MCP server section
Click on the gear icon, the mcp_config.json file will open
Paste the JSON configuration file

More here

Cline (VS Code extension)

Open VS Code and click the Cline icon in the activity bar to open the Cline panel
Click the MCP Servers icon in the top navigation bar of the Cline pane
Select the "Configure" tab
Click "Configure MCP Servers" at the bottom of the pane — this opens cline_mcp_settings.json
Paste the JSON configuration file

More here

Development

Local setup

Clone the repository:

git clone https://github.com/scraperapi/scraperapi-mcp
cd scraperapi-mcp

Install dependencies:

Using Poetry:
```
poetry install
```

Using pip:

# Create virtual environment and activate it
python -m venv .venv
source .venv/bin/activate # MacOS/Linux
# OR
.venv/Scripts/activate # Windows

# Install the local package in editable mode
pip install -e .

Using Docker:

# Build the Docker image locally
docker build -t scraperapi-mcp-server .

Run the server

Using Python:
```
python -m scraperapi_mcp_server
```

Using Docker:

# Run the Docker container with your API key
docker run -e API_KEY=<YOUR_SCRAPERAPI_API_KEY> scraperapi-mcp-server

Debug

python3 -m scraperapi_mcp_server --debug

Testing

This project uses pytest for testing.

Install Test Dependencies

Using Poetry:
```
poetry install --with dev
```

Using pip:

pip install -e .
pip install pytest pytest-mock pytest-asyncio

Running Tests

# Run All Tests
pytest

# Run Specific Test
pytest <TEST_FILE_PATH>

Name		Name	Last commit message	Last commit date
Latest commit History 26 Commits
src/scraperapi_mcp_server		src/scraperapi_mcp_server
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
.python-version		.python-version
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
glama.json		glama.json
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

ScraperAPI MCP server

Table of Contents

Features

Architecture

Installation

Prerequisites

Using Python

Using Docker

Available tools

API

SDEs

Google

Amazon

Walmart

eBay

Redfin

Crawler

AI Parser

Prompt templates

Configuration

Settings

Client Setup

Development

Local setup

Run the server

Debug

Testing

Install Test Dependencies

Running Tests

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 3

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages