Home Projects gpt-crawler
gpt-crawler
TypeScript

gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL

by BuilderIO · GitHub
Stars
Forks
ISC
License
Created
Last commit
Category
Language
aiISCTypeScript
View on GitHub
In plain words

Extract information from websites to build your own custom AI assistant.

From the README

Install
git clone https://github.com/builderio/gpt-crawler

Excerpts from the project README on GitHub. Copyright and licensing remain with the respective authors.

You maintain this project?

Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.

Claim this page →
gpt-crawler — GitHub preview card
📈 Star history
22.31k22.25k
2026-06-282026-08-31
📈 Track gpt-crawler

Get an email alert on its next release or when it starts trending — never miss the moment.

Free · no card · unsubscribe anytime
Get email alerts →
📄 About

Crawl a site to generate knowledge files to create your own custom GPT from a URL

gpt-crawler has 22.3k stars on GitHub. It has been forked 2.4k times. gpt-crawler is written mainly in TypeScript. It has been in active development since 2023. gpt-crawler is available under the ISC license. Its main topics are ai.

Frequently asked questions

What is gpt-crawler?

Crawl a site to generate knowledge files to create your own custom GPT from a URL

Is gpt-crawler open source?

gpt-crawler is an open-source project. It is released under the ISC license.

Is gpt-crawler free?

Yes. gpt-crawler is free and open source — you can use, modify and self-host it.

What license does gpt-crawler use?

gpt-crawler is available under the ISC license.

What language is gpt-crawler written in?

gpt-crawler is written mainly in TypeScript.

🏅 Maintainer of this project?
olud.ai badge — gpt-crawler

Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.

[![olud.ai](https://olud.ai/badge.php?tool=builderio-gpt-crawler)](https://olud.ai/project/builderio-gpt-crawler.html)
More badge options →
🧬 Related projects