data-quality

Learn how to design, develop, deploy and iterate on production-grade ML applications.

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Always know what to expect from your data.

Refine high-quality datasets and visual AI models

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

The Open Source Feature Store for AI/ML

lakeFS - Data version control for your data lake | Git for data

Learn how to design, develop, deploy and iterate on production-grade ML applications.

Compare tables within or across databases

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

⚡ Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

Feathr – A scalable, unified data and AI engineering platform for enterprise

The Virtual Feature Store. Turn your existing data infrastructure into a feature store.

re_data - fix data issues before your users & CEO would discover them 😊

First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

A curated, but incomplete, list of data-centric AI resources.

Automatically find issues in image datasets and practice data-centric computer vision.