All projects
27 repositories · updated 2026-11-07
Loading…
1scrapy / scrapyScrapy, a fast high-level web crawling & scraping framework for Python.PythonBSD-3-Clause · PermissiveInsufficient rank history65,250
2soxoj / maigret🕵️♂️ Collect a dossier on a person by username from 6K websitesPythonMIT · PermissiveInsufficient rank history39,773
3ScrapeGraphAI / Scrapegraph-aiPython scraper based on AIPythonMIT · PermissiveInsufficient rank history32,587
4gocolly / collyElegant Scraper and Crawler Framework for GolangGoApache-2.0 · PermissiveInsufficient rank history25,578
5psf / requests-htmlPythonic HTML Parsing for HumansTMPythonMIT · PermissiveInactiveInsufficient rank history13,800
6daijro / camoufox🦊 Anti-detect browserPythonMPL-2.0 · Share derivative sourceInsufficient rank history13,090
7ultrafunkamsterdam / undetected-chromedriverCustom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)PythonGPL-3.0 · Share derivative sourceInactiveInsufficient rank history12,899
8jo-inc / camofox-browserStealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.JavaScriptMIT · PermissiveInsufficient rank history12,192
9code4craft / webmagicA scalable web crawler framework for Java.JavaApache-2.0 · PermissiveInactiveInsufficient rank history11,668
10apify / crawlee-pythonCrawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.PythonApache-2.0 · PermissiveInsufficient rank history9,657
11lorien / awesome-web-scrapingList of libraries, tools and APIs for web scraping and data processing.JavaScriptLicense not declaredInsufficient rank history8,213
12alirezamika / autoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for PythonPythonMIT · PermissiveInsufficient rank history8,074
13tabulapdf / tabulaTabula is a tool for liberating data tables trapped inside PDF filesCSSMIT · PermissiveInactiveInsufficient rank history7,492
14autoscrape-labs / pydollPydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.HTMLMIT · PermissiveInsufficient rank history7,133
15adbar / trafilaturaPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XMLPythonApache-2.0 · PermissiveInsufficient rank history7,082
16proxifly / free-proxy-list🚀 Free HTTP, SOCKS4, & SOCKS5 proxy list * Updated every 5 minutes * and rotating proxy API (100+ countries)GPL-3.0 · Share derivative sourceInsufficient rank history7,014
17yujiosaka / headless-chrome-crawlerDistributed crawler powered by Headless ChromeJavaScriptMIT · PermissiveInactiveInsufficient rank history5,631
18bjesus / pipetSwiss-army tool for scraping and extracting data from online assets, made for hackersGoMIT · PermissiveInactiveInsufficient rank history4,773
19d60 / twikitTwitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter BotPythonMIT · PermissiveInactiveInsufficient rank history4,763
20sparklemotion / mechanizeMechanize is a ruby library that makes automated web interaction easy.RubyMIT · PermissiveInsufficient rank history4,445
