webscraping

14 projects share this GitHub topic

Lines connect members that are measurably related to each other. Dot size reflects stars.

🧬 Members

firecrawl

The context API to search, scrape, and interact with the web at scale. 🔥

★ 174.8k

Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale…

★ 77.6k

huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

★ 49.9k

CloakBrowser

Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level…

★ 31k

Scrapegraph-ai

Python scraper based on AI

★ 30.2k

gpt-researcher

An autonomous agent that conducts deep research on any data using any LLM providers

★ 29.2k

maxun

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn…

★ 17.3k

CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

★ 3.3k

scrapeghost

👻 Experimental library for scraping websites using OpenAI's GPT API.

★ 1.4k

parsera

Lightweight library for scraping web-sites with LLMs

★ 1.3k

ShardBrowser

Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ProxyShard team.…

★ 779

extractor

Use LLMs to robustly extract web data

★ 320

llm-reader

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web…

★ 301

Soup

Fine-tune and post-train LLMs with one command—no SSH, no config hell.

★ 64

Measured from GitHub topics shared by both projects, weighted by how rare each topic is.