Crawling - Moozonian

GitHub Repo https://github.com/waditu/tushare

TuShare is a utility for crawling historical data of China stocks

GitHub Repo https://github.com/exa-labs/exa-mcp-server

Exa MCP for web search and web crawling!

GitHub Repo https://github.com/wzdnzd/aggregator

One-stop Proxies Crawling and Aggregation Platform

GitHub Repo https://github.com/projectdiscovery/katana

A next-generation crawling and spidering framework.

GitHub Repo https://github.com/crawl/crawl

Dungeon Crawl: Stone Soup official repository

GitHub Repo https://github.com/qinxuye/cola

A high-level distributed crawling framework.

GitHub Repo https://github.com/dotnetcore/DotnetSpider

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

GitHub Repo https://github.com/elliotgao2/gain

Web crawling framework based on asyncio.

GitHub Repo https://github.com/scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

GitHub Repo https://github.com/s0md3v/uro

declutters url lists for crawling/pentesting