Amazon Kinesis Data Streams Adds Support for Streaming Apache Iceberg Tables
Amazon Kinesis Data Streams now offers fully managed streaming tables, continuously delivering data as queryable Apache Iceberg tables on Amazon S3. This new capability reduces data delivery costs by up to 50% and downstream query costs by up to 30% through inline compaction, addressing challenges like operational complexity and the small file problem. It provides a serverless solution for unifying streaming data with Iceberg, integrating with services like Amazon Athena, Redshift, and Spark. The offering also includes an AI-assisted Agent Skill to simplify setup and configuration.
- →Fully Managed Streaming Tables for Apache Iceberg on Amazon S3
- →Cost Reduction and Improved Query Performance with Intelligent Compaction
- →Kinesis Data Streams Agent Skill for AI-Assisted Configuration
Features (1) ›
- Fully Managed Streaming Tables for Apache Iceberg on Amazon S3
Amazon Kinesis Data Streams now supports streaming tables, a fully managed capability that continuously delivers streaming data as queryable Apache Iceberg tables on Amazon S3. This eliminates the need for custom applications, self-managed compute, and operational overhead for integrating Kinesis with Iceberg.
Enhancements (1) ›
- Cost Reduction and Improved Query Performance with Intelligent Compaction
The new streaming tables feature reduces data delivery costs to S3 Tables by up to 50% and downstream query costs by up to 30% through intelligent inline compaction. This capability addresses the 'small file problem' common in streaming ingestion, producing query-optimized Parquet files while maintaining minute-level data freshness and providing exactly-once delivery semantics.
Notes (1) ›
- Kinesis Data Streams Agent Skill for AI-Assisted Configuration
AWS introduces the Kinesis Data Streams Agent Skill, providing AI-assisted guidance for setting up streaming tables integrations. This skill helps configure delivery to S3 Tables (Iceberg) or general S3, including schema registry and IAM role setup, making it easier to manage configurations at scale with agentic workflows and infrastructure-as-code patterns.
https://aws.amazon.com/blogs/big-data/deliver-real-time-data-to-streaming-tables-for-apache-iceberg-with-amazon-kinesis-data-streams/
Related releases
- Amazon Kinesis Data Streams now delivers to general purpose S3 buckets AWS What's New ·
- Amazon Kinesis Data Streams introduces streaming tables for Apache Iceberg on S3 AWS What's New ·
- AWS Lambda Recursive Loop Detection Extends to All Commercial Regions AWS What's New ·
- AWS details Razor Group's journey to a modern data lakehouse AWS Big Data Blog ·
- Mountpoint for Amazon S3 adds memory usage controls AWS What's New ·
- AWS Glue now supports catalog federation for remote Apache Iceberg in GovCloud (US) regions AWS What's New ·