Crawl a site to generate knowledge files to create your own custom GPT from a URL
Extract information from websites to build your own custom AI assistant.
git clone https://github.com/builderio/gpt-crawler

Excerpts from the project README on GitHub. Copyright and licensing remain with the respective authors.
Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.
Get an email alert on its next release or when it starts trending — never miss the moment.
Free · no card · unsubscribe anytimeCrawl a site to generate knowledge files to create your own custom GPT from a URL
gpt-crawler has 22.3k stars on GitHub. It has been forked 2.4k times. gpt-crawler is written mainly in TypeScript. It has been in active development since 2023. gpt-crawler is available under the ISC license. Its main topics are ai.
Crawl a site to generate knowledge files to create your own custom GPT from a URL
gpt-crawler is an open-source project. It is released under the ISC license.
Yes. gpt-crawler is free and open source — you can use, modify and self-host it.
gpt-crawler is available under the ISC license.
gpt-crawler is written mainly in TypeScript.
Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.
[](https://olud.ai/project/builderio-gpt-crawler.html)