⬢github Rust · 33 ★ +23 since we first saw it · pushed 2 d ago · Apache-2.0
aperiodic-io/tparquet
Parquet for time series. Trade and tick files at about half their ZSTD size (0.49× on 139M trades from four market data providers), lossless, and still Parquet. > Backed by aperiodic.io
tparquet is a Parquet extension that adds a new compression codec, PCODEC, wrapping the pcodec numeric codec. It shrinks time-series tick and trade files to roughly half of ZSTD-compressed Parquet size while staying lossless and fully readable by standard Parquet tools like DuckDB, pandas, Polars, Spark, and Trino.
Why now: It was featured on Hacker News for achieving ~50% compression versus ZSTD Parquet on market data, with benchmarks across 31 datasets and backed by a committed commercial data provider.
Who it is for: Engineers and quants handling large financial time-series data who want smaller Parquet files without changing their existing tooling.
compressioncompression-formatsmarket-dataparquetparquet-filesparquet-formattimeseriestimeseries-datatimeseries-database
Stars over our 188 snapshots: 10 to 33, since 1 d ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/aperiodic-io/tparquet