GitHub stars · StarDay

1pathwaycom / pathwayPython ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.PythonLicense not declared62,012

2google-ai-edge / mediapipeCross-platform, customizable ML solutions for live and streaming media.C++Apache-2.0 · Permissive37,584

3vectordotdev / vectorA high-performance observability data pipeline.RustMPL-2.0 · Share derivative source22,785

4questdb / questdbQuestDB is a high performance, open-source, time-series databaseJavaApache-2.0 · Permissive17,502

5zhisheng17 / flink-learningflink learning blog. http://www.54tianzhisheng.cn/ Contains Flink introduction, concepts, principles, actual combat, performance tuning, source code analysis, etc. Learning cases involving Flink Connector, Metrics, Library, DataStream API, Table API & SQL, etc., as well as large-scale project cases of Flink implementation (PVUV, log storage, real-time deduplication of tens of billions of data, monitoring and alarming) are shared. Welcome everyone to support my column "Big Data Real-time Computing Engine Flink Practical Combat and Performance Optimization"JavaApache-2.0 · Permissive15,101

6oxnr / awesome-bigdataA curated list of awesome big data frameworks, ressources and other awesomeness.MIT · Permissive14,714

7redpanda-data / redpandaRedpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!C++License not declared12,654

8madd86 / awesome-system-designA curated list of awesome System Design (A.K.A. Distributed Systems) resources.CC0-1.0 · Permissive12,649

9ThreeDotsLabs / watermillBuilding event-driven applications the easy way in Go.GoMIT · Permissive9,938

10risingwavelabs / risingwaveEvent streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.RustApache-2.0 · Permissive9,402

11redpanda-data / connectFancy stream processing made operationally mundaneGoLicense not declared8,793

12fluent / fluent-bitFast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and WindowsCApache-2.0 · Permissive8,183

13robinhood / faustPython Stream ProcessingPythonLicense not declared6,823

14hazelcast / hazelcastHazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.JavaLicense not declared6,634

15MaterializeInc / materializeThe live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQLRustLicense not declared6,392

16apache / hudiUpserts, Deletes And Incremental Processing on Big Data.JavaApache-2.0 · Permissive6,362

17online-ml / river🌊 Online machine learning in PythonPythonBSD-3-Clause · Permissive6,140

18ag2ai / faststreamAsynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.PythonApache-2.0 · Permissive5,391

19fluvio-community / fluvio🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.RustApache-2.0 · Permissive5,260

20ArroyoSystems / arroyoDistributed stream processing engine in RustRustApache-2.0 · Permissive5,077