apache-arrow

Instant Kubernetes-Native Application Observability

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

❄️ Coolest database around 🧊 Embeddable column database written in Go.

Manipulate JSON-like data with NumPy-like idioms.

Loaders for big data visualization. Website:

A Python library for fast, interactive geospatial vector data visualization in Jupyter.

Geospatial extensions for Polars

Multi-Modal Database replacing MongoDB, Neo4J, and Elastic with 1 faster ACID solution, with NetworkX and Pandas interfaces, and bindings for C 99, C++ 17, Python 3, Java, GoLang 🗄️

Rust-based WebAssembly bindings to read and write Apache Parquet data

Specification for storing geospatial data in Apache Arrow

Infrastructures™ for Machine Learning Training/Inference in Production.

GeoArrow in Rust, Python, and JavaScript (WebAssembly) with vectorized geometry operations

Official Julia implementation of Apache Arrow

A Rust DataFrame implementation, built on Apache Arrow

A SQLite vtable extension to read Parquet files

Fletcher: A framework to integrate FPGA accelerators with Apache Arrow

Manipulate arrays of complex data structures as easily as Numpy.

ParquetSharp is a .NET library for reading and writing Apache Parquet files.

Unified storage framework for the entire machine learning lifecycle