Home› Projects› petastorm
petastorm
Python

petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

by uber · GitHub
Stars
Forks
License
Created
Last commit
Language
deep-learningmachine-learningparquetApache-2.0Python
View on GitHub
In plain words

Train and evaluate deep learning models using datasets stored in Apache Parquet format on your own machine or in a group.

You maintain this project?

Claim its page: indexed whatever its rank, translated into six languages, and enriched with what you write yourself.

Claim this page →
petastorm — GitHub preview card
📈 Star history
1 8891 888
2026-07-202026-08-31
📈 Track petastorm

Get an email alert on its next release or when it starts trending — never miss the moment.

Free · no card · unsubscribe anytime
Get email alerts →
📄 About

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

petastorm has 1.9k stars on GitHub. It has been forked 283 times. petastorm is written mainly in Python. It has been in active development since 2018. petastorm is available under the Apache-2.0 license. Its main topics are deep-learning, machine-learning, parquet, parquet-files.

❓ Frequently asked questions

What is petastorm?

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

Is petastorm open source?

petastorm is an open-source project. It is released under the Apache-2.0 license.

Is petastorm free?

Yes. petastorm is free and open source — you can use, modify and self-host it.

What license does petastorm use?

petastorm is available under the Apache-2.0 license.

What language is petastorm written in?

petastorm is written mainly in Python.

🏅 Maintainer of this project?
olud.ai badge — petastorm

Add this live badge to your README — your GitHub stars and directory rank, refreshed daily.

[![olud.ai](https://olud.ai/badge.php?tool=uber-petastorm)](https://olud.ai/project/uber-petastorm.html)
More badge options →
🧬 Related projects