crawler
28 progetti condividono questo topic GitHub
Le linee collegano membri che sono misurabilmente correlati tra loro. La dimensione dei punti riflette le stelle.
🧬 Membri
firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
★ 195.7k
Scrapling
🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't…
★ 89.7k
crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run…
★ 86.4k
EasySpider
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/…
★ 44.2k
Scrapegraph-ai
Python scraper based on AI
★ 32.3k
invisible_playwright_mcp
Playwright MCP server undetected by anti-bots and captchas: AI agent browses the web on anti-detect stealth…
★ 31.8k · GitHub ↗
Jobs_Applier_AI_Agent_AIHawk
Open source AI job application bot in Python: browser automation and web scraping to read job postings, then…
★ 30.3k
crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript…
★ 26.3k
Douyin_TikTok_Download_API
🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API，支持 AI agent 通过 MCP…
★ 20.9k · GitHub ↗
maxun
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn…
★ 17.3k
crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data…
★ 9.6k
xiaobei
为OPC/中小微企业量身打造的自媒体获客AI Agent
★ 8.3k
autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
★ 7.6k
trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as…
★ 7k
myGPTReader
A community-driven way to read and chat with AI bots - powered by chatGPT.
★ 4.4k
scylla
Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language…
★ 4k
AutoCrawler
Google, Naver multiprocess image web crawler (Selenium)
★ 1.7k
boss-zhipin-scraper
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP…
★ 1.7k · GitHub ↗
fess
Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud…
★ 1.1k
crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for…
★ 1.1k · GitHub ↗
crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for…
★ 1.1k
chatWeb
ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions…
★ 916
hacker-news-digest
:newspaper: Let ChatGPT Summarize Hacker News for You
★ 755
Craw4LLM
Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"
★ 666
reader
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions,…
★ 558
Fast-Powerful-Whisper-AI-Services-API
⚡ 一款用于自动语音识别 (ASR)、翻译的高性能异步 API。不需要购买Whisper…
★ 470
scraperai
ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.
★ 421
extractor
Use LLMs to robustly extract web data
★ 320
🔗 Famiglie affini
Misurato dai temi di GitHub condivisi da entrambi i progetti, ponderato in base a quanto è raro ciascun tema.