Description

Apache Spark Streaming, an extension of the core Apache Spark application programming interface (API), is a scalable streaming processing system designed for data engineers and scientists, and application developers. It enables users to build streaming applications, ingest data in mini-batches, perform resilient distributed datasets (RDD) transformations, join streams against historical data, as well as run ad-hoc queries on stream state. Features include language-integrated API, stateful exactly-once semantics, streaming combinations with batch and interactive queries, batch and real-time processing integration, low latency and scalability, discretized stream (DStream), machine learning and graph processing algorithms support, Hadoop integration, dynamic load balancing, and native integration with advanced processing libraries.