Page Inspect

Scrapy

- Download
- Documentation
- Community
- Companies
- FAQs

- Download
- Documentation
- Community
- Companies
- FAQs

Maintained by Zyte with over 500 other contributors

Forks0

Stars0

# The world’s most-used open source data extraction framework

A collaborative, open source framework for extracting public web data.

$pipinstallscrapy

Read the Docs

Current version 2.13.3DownloadReleases

Web Scraping Copilot

Upgrade your coding agent with Scrapy skills

Generate selectors, unblock sites, deploy to Scrapy Cloud and more.

Scrape faster in VS Code

Free to use - No account required

## Fast & Powerful

Write the rules to extract the data you need, and let Scrapy do the rest.

## Customizable

Build spiders in Python and tailor them to any site or data model.

## Open Source

Maintained by a thriving community, used by millions of developers worldwide.

## Get started in minutes

Build spiders faster

Select a feature to see the corresponding code snippet.

Create a Scrapy Project

Initialize a new Scrapy project with a single command.

Run a Spider

Run a Scrapy spider to scrape data.

Export Data

Save extracted data to a file in your format of choice.

Use Scrapy Shell

Test and debug your scraping logic interactively.

$ scrapy startproject myproject

## Why Create a Scrapy Project?

Creating a Scrapy project sets up the necessary folder structure and files to start building your web scraper efficiently.

Learn more about Why Create a Scrapy Project?

## Join the community

“ If it wasn't for Scrapy, my freelancing career, and then the scraping business would have never taken off. The Scrapy framework, and especially its documentation, simplifies crawling and scraping for anyone with basic Python skills. I don't know, now there is this emotive bond with Scrapy that I've developed over the years.”

Nishant Choudhary

Founder of DataFlirt.com

“ Scrapy: the gift that revolutionised web scraping and inspired me to pay it forward with Scrapoxy!"
"Scrapy is the central engine of all my projects, since its 1st release.”

Fabien Vauchelles

Creator of Scrapoxy

“ Scrapy stands out as an exceptional tool for seamlessly managing open requests, making large-scale data collection efficient and reliable."
"With its remarkable scraping speed, Scrapy transforms complex web extraction tasks into swift, streamlined operations that save time and maximise output.”

Hyder Khan

Flipdish

“ Scrapy is the cornerstone of web scraping with Python. Without it, scraping would be much harder."
"During my career, I've built thousands of scrapers, mostly with Scrapy. It made my life a lot easier.”

Pierluigi Vinciguerra

Co Founder and CTO at Databoutique.com

“ With over 6 years in web scraping, Scrapy has been my go-to framework for building fast, reliable, and scalable data pipelines."
"In my 6+ years of scraping experience, nothing has matched Scrapy’s balance of power, flexibility, and community support.”

Ganesh Satheendran

Turbolab Technologies

“ When Google Search enforced JavaScript, our scraper broke. Struggling to build a working alternative with Puppeteer on a short notice, we fired up Scrapy, Splash, and whipped up a script. Lo and behold – it worked! Our Google benchmark was buzzing again.”

Adam

Proxyway

Join us on

Discord

Join us at the

Extract Summit 2025

## Join millions of developers using Scrapy.

55.1k Stars, 10.8k Forks.

Fast, free web scraping backed by a thriving community.

Get startedExplore docs →

- Download
- Documentation
- Community
- Companies
- FAQs

Forks0

Stars0