Amazon S3 Tables: managed Apache Iceberg tables inside S3

2024-12-03 · Publisher : AWS · Domain : Cloud & DevOps

At re:Invent 2024, AWS launches S3 Tables, a new S3 bucket type that stores Apache Iceberg tables with automatic maintenance. AWS claims up to 3x faster queries and 10x more transactions per second.

### What changes On 3 December 2024, during AWS re:Invent (2-6 December in Las Vegas), AWS launches Amazon S3 Tables. It is a third S3 bucket type, alongside general-purpose and directory buckets: the "table bucket". It stores tabular data in Apache Iceberg format and behaves like an analytics warehouse inside object storage. AWS notes that Apache Iceberg is, in its words, the most popular way to manage Parquet files, with thousands of AWS customers already using it. ### Key facts - Up to 3x better query performance and up to 10x more transactions per second than self-managed Iceberg tables in S3 (AWS figures). - Automatic maintenance: compaction, snapshot management and removal of unreferenced files; compaction target configurable between 64 and 512 MiB. - Integrations: Amazon Athena, EMR, Redshift, QuickSight, Apache Spark, and the AWS Glue Data Catalog (in preview). - Available at launch in three regions: US East (Ohio, N. Virginia) and US West (Oregon). - Billing on storage, requests, object monitoring and compaction. - The service can query billions of files representing petabytes or even exabytes of data. ### Why it matters Apache Iceberg has become the reference open table format for lakehouse architectures. By building its management into S3, AWS removes part of the operations (compaction, cleanup) that data teams handled themselves. For an IT services firm, it is a simple entry point to a lakehouse, with greater dependence on the managed service. ### Takeaway AWS makes Iceberg a native S3 storage type: less operations, but one more layer at the provider.

Tags : AWS, Données, Cloud, Conférence