data-integration

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Turns Data and AI algorithms into production-ready web applications in no time.

An orchestration platform for the development, production, and observation of data assets.

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Flink CDC is a streaming data integration tool

The open source ELT framework powered by Apache Arrow

Upserts, Deletes And Incremental Processing on Big Data.

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

Privacy and Security focused Segment-alternative, in Golang and React

A data integration framework

Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

BitSail is a distributed high-performance data integration engine which supports batch, streaming and incremental scenarios. BitSail is widely used to synchronize hundreds of trillions of data every day.

Hop Orchestration Platform

Insightful Tutorials and Papers about Knowledge Graphs