firecrawl/firecrawl

🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

⭐ 85300 | 🍴 6145

unclecode/crawl4ai

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

⭐ 60928 | 🍴 6221

scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

⭐ 59872 | 🍴 11256

NanmiCoder/MediaCrawler

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫

⭐ 44232 | 🍴 9706