Snowflake introduces Polaris Catalog, an open catalog for Apache Iceberg

2024-06-03 · Publisher : Snowflake · Domain : Cloud & DevOps

At its Data Cloud Summit, Snowflake introduces Polaris Catalog, a vendor-neutral catalog based on Apache Iceberg's REST API, hostable at Snowflake or self-hosted. The code will be open sourced within 90 days.

### Background On 3 June 2024, **Snowflake** introduced **Polaris Catalog**, a catalog designed to strengthen interoperability of **Apache Iceberg** tables across several processing engines. It implements Iceberg's open **REST** specification. Snowflake announced that the code will be **open sourced within 90 days** and available "soon" in public preview on its infrastructure. ### Key facts - Neutrality: the catalog can be hosted on Snowflake's managed infrastructure or self-hosted (Docker, Kubernetes), and the infrastructure can be changed without vendor constraints. - Supported engines and tools: Apache Doris, Flink, Spark, PyIceberg, StarRocks, Trino; partners named: Confluent, Dremio, Salesforce; clouds: AWS, Google Cloud, Microsoft Azure. - Integration with Snowflake Horizon for governance: column masking, row access policies, object tagging and sharing of Iceberg tables. - Follow-up: on 30 July 2024, Snowflake released the code under the Apache 2.0 licence on GitHub; the Snowflake-based service entered public preview; Dremio committed to contributing Project Nessie's capabilities (Git-like versioning, multi-table transactions). ### Why it matters The table catalog is the control point that decides which engines read which data. By offering an open catalog, Snowflake answers Databricks' offensive on Iceberg (the Tabular acquisition the next day) and positions itself as a player in the open format rather than a closed warehouse. For architects, the benefit is the option of keeping a catalog independent of the compute engine. ### Takeaway Polaris Catalog makes the Iceberg catalog a ground for openness, just as Databricks buys Tabular.

Tags : Données, Open Source, Licence