D4Vinci/Scrapling

▲ 15,971 stars today★ 99,536⑂ 10,417

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy

About D4Vinci/Scrapling

D4Vinci/Scrapling is an open-source project on GitHub, mainly written in Python. 🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy It currently holds 99,536 stars and 10,417 forks with 16 open issues, and was last pushed on 2026-11-05 (repository created 2024-10-13).

Project Overview

Git Homed tracks it on the Today's Trending board.

GitHub Repository Details

Repository D4Vinci/Scrapling · default branch main · size 16790 KB · watchers 346 · source: GitHub REST API and repository README

README

https://github.com/D4Vinci/Scrapling/blob/HEAD/Scrapling Poster
Effortless Web Scraping for the Modern Web

https://github.com/D4Vinci/Scrapling/blob/HEAD/D4Vinci%2FScrapling | Trendshift https://github.com/D4Vinci/Scrapling/blob/HEAD/Star History Rank
https://github.com/D4Vinci/Scrapling/blob/HEAD/README بالعربية https://github.com/D4Vinci/Scrapling/blob/HEAD/README en Español https://github.com/D4Vinci/Scrapling/blob/HEAD/README em Português (Brasil) https://github.com/D4Vinci/Scrapling/blob/HEAD/README en Français https://github.com/D4Vinci/Scrapling/blob/HEAD/README auf Deutsch https://github.com/D4Vinci/Scrapling/blob/HEAD/简体中文版自述文件 https://github.com/D4Vinci/Scrapling/blob/HEAD/日本語のREADME https://github.com/D4Vinci/Scrapling/blob/HEAD/Русская версия README https://github.com/D4Vinci/Scrapling/blob/HEAD/한국어 README
https://github.com/D4Vinci/Scrapling/blob/HEAD/Tests https://github.com/D4Vinci/Scrapling/blob/HEAD/PyPI version https://github.com/D4Vinci/Scrapling/blob/HEAD/Docker Pulls https://github.com/D4Vinci/Scrapling/blob/HEAD/PyPI package downloads https://github.com/D4Vinci/Scrapling/blob/HEAD/Static Badge https://github.com/D4Vinci/Scrapling/blob/HEAD/OpenClaw Skill
https://github.com/D4Vinci/Scrapling/blob/HEAD/Discord https://github.com/D4Vinci/Scrapling/blob/HEAD/X (formerly Twitter) Follow
https://github.com/D4Vinci/Scrapling/blob/HEAD/Supported Python versions

Selection methods · Fetchers · Spiders · Proxy Rotation · CLI · MCP

Scrapling is a Python Web Scraping framework that brings fast HTTP requests, stealth browsers, powerful parsing, and concurrent crawling into one library.

Scrape static pages, render JavaScript, interact with websites, and extract data from the API calls behind them. Build crawlers with reusable sessions, proxy rotation, automatic retries, pause/resume, and built-in data exports.

Use it from Python, the terminal, or your AI agent through its MCP server and Agent Skill. From extracting a few fields to collecting a whole website as clean Markdown, Scrapling covers the full scraping workflow.

What you can do with Scrapling

Start with a single page:
from scrapling.fetchers import Fetcher

page = Fetcher.get('https://quotes.toscrape.com/') quotes = page.css('.quote .text::text').getall() print(quotes)

Or scale up with a spider:

from scrapling.spiders import Spider, Response

class QuotesSpider(Spider): name = "quotes" start_urls = ["https://quotes.toscrape.com/"]

async def parse(self, response: Response): for quote in response.css('.quote'): yield { "text": quote.css('.text::text').get(), "author": quote.css('.author::text').get(), }

result = QuotesSpider().start() print(len(result.items)) result.items.to_json("quotes.json")

This prints 10 and saves the quotes and authors to quotes.json.

https://github.com/D4Vinci/Scrapling/blob/HEAD/At DataImpulse, we specialize in developing custom proxy services for your business. Make requests from anywhere, collect data, and enjoy fast connections with our premium proxies.

Platinum Sponsors

NodeMaven - The most efficient proxy provider for Web Scraping and Automation with the Highest Quality IP on the market. SCRAPLING2X - 2X traffic for Black Friday
Scrapling handles Cloudflare Turnstile. For enterprise-grade protection, Hyper Solutions provides API endpoints that generate valid antibot tokens for Akamai, DataDome, Kasada, and Incapsula. Simple API calls, no browser automation required.
Evomi : residential proxies from $0.49/GB. Scraping browser with fully spoofed Chromium, residential IPs, auto CAPTCHA solving, and anti-bot bypass.
Scraper API for hassle-free results. MCP and N8N integrations are available.
TikHub.io provides 900+ stable APIs across 16+ platforms including TikTok, X, YouTube & Instagram, with 40M+ datasets.
Also offers DISCOUNTED AI models - Claude, GPT, GEMINI & more up to 71% off.
Read a full review of Scrapling on The Web Scraping Club (Nov 2025), the #1 newsletter dedicated to Web Scraping.
Swiftproxy provides scalable residential proxies with 80M+ IPs across 195+ countries, delivering fast, reliable connections, automatic rotation, and strong anti-block performance. Free trial available.
Byteful supplies ethical residential, mobile, ISP, and datacenter proxies, plus scraping and AI automation tools. Use code SCRAPLING10: 10% off residential proxies.
ProxyLane: 90% fewer CAPTCHAs on your Scrapling runs. 28M real P2P residential devices, 98.7% clean IPs, 195 countries. City/ISP/ZIP targeting, sticky/rotating sessions, traffic never expires. Try 350 MB for $1.95. Code SCRAPEANDTAKE: 25% off.
LundProxy: Stop paying per GB. Unlimited residential traffic, 100M+ IPs, 190+ countries. Pay by bandwidth. Request a free 50 Mbps residential proxies trial.
Resolvr: Scraping at scale means CAPTCHAs at scale. Resolvr solves reCAPTCHA and Cloudflare challenges on real devices through one API. Free beta access is available now.
hCaptcha in your Scrapling runs? NoneCap returns valid hCaptcha tokens, Enterprise sitekeys included, through one API. 1,300 free credits on signup.
$1/GB at DataImpulse. Ethical first-party proxies. 90M+ IPs, 195+ countries. Residential, datacenter, mobile. Pay-as-you-go = no subscription. Traffic never expires.

Do you want to show your ad here? Click here

Sponsors


Do you want to show your ad here? Click here and choose the tier that suits you!

---

Key Features

Spiders - A Full Crawling Framework

Advanced Websites Fetching with Session Support

Parsing and Data Extraction

AI Features

GitHub Stars & Activity

99,536Stars
10,417Forks
16Open issues
PythonLanguage

GitHub Popularity

GitHub stars99,536
Forks10,417
Open issues16
Primary languagePython
LicenseBSD-3-Clause
Stars gained today15,971
Created2024-10-13
Last pushed2026-11-05

Trending History

Monthly boardrank #17 · ▲ 15,971 stars

Related GitHub Projects

1

Graphify-Labs / graphify

Python★ 149,279⑂ 14,354▲ 615 stars
→
2

ComposioHQ / awesome-claude-skills

Python★ 79,689⑂ 9,754▲ 88 stars
→
3

usestrix / strix

Python★ 76,182⑂ 8,643▲ 325 stars
→
4

mvanhorn / last30days-skill

Python★ 68,501⑂ 6,101▲ 135 stars
→
5

jingyaogong / minimind

Python★ 65,627⑂ 8,605▲ 188 stars
→
6

virgiliojr94 / book-to-skill

Python★ 38,208⑂ 4,063▲ 161 stars
→
7

elementalsouls / Claude-BugHunter

Python★ 7,963⑂ 1,237▲ 1,094 stars
→
8

kajisho5 / ffmpeg-skill

Python★ 7,450⑂ 681▲ 367 stars
→

More Trending Repositories