A vendor-neutral, open catalog implementation for Apache Iceberg, Polaris Catalog will be open sourced in the next 90 days.
Snowflake has announced at its annual user conference, Snowflake Summit 2024, that Polaris Catalog, a vendor-neutral, open catalog implementation for Apache Iceberg will be open sourced in the next 90 days to provide enterprises and the entire Iceberg community with new levels of choice, flexibility and control over their data.
The implementation offers full enterprise security and Apache Iceberg interoperability with Amazon Web Services (AWS), Confluent, Dremio, Google Cloud, Microsoft Azure, Salesforce and more. “Organizations want open storage and interoperable query engines without lock-in. Now, with the support of industry leaders, we are further simplifying how any organization can easily access their data across diverse systems with increased flexibility and control,” said Christian Kleinerman, EVP of Product, Snowflake.
“Polaris Catalog extends Snowflake’s commitment to Apache Iceberg as the open standard of choice and signals the intent from industry leaders in enabling customers and the wider Iceberg community to harness their data through an open and neutral approach, empowering cross-engine interoperability on that data.”
The announcement comes on the heels of Snowflake and Microsoft’s recent partnership expansion – pitched as creating more seamless interoperability between Snowflake and Fabric.
Apache Iceberg emerged from incubation to a top-level Apache Software Foundation project in May 2020 and has since surged in popularity to become a leading open-source data table format.
A part of what makes Apache Iceberg so powerful is its vibrant community of diverse adopters, contributors and commercial offerings. To ensure Polaris Catalog can meet the evolving needs of the wider community and landscape, Snowflake is collaborating with the Iceberg ecosystem to drive the project forward.
“From day one at Microsoft, we’ve been focused on empowering every user on the planet to achieve more, and this starts with a strong data foundation. Through our support and contributions to open data standards, including Delta Parquet, Apache Iceberg, and Apache XTable, we’re furthering this mission by enabling organizations with a new level of open data interoperability, so they can do more with their data,” said Arun Ulagaratchagan, Corporate Vice President, Azure Data, Microsoft.
“Snowflake continues to serve as a strategic partner of ours, and we’re excited by their willingness to work with the Iceberg community on an open catalog to empower our joint customers and the wider open-source community with more flexibility and control over their open Iceberg data.”
Chris Grusz, Managing Director, Technology Partnerships, Amazon Web Services, said: “AWS is committed to working with partners, such as Snowflake, on open-source solutions that can accelerate choice for customers.
“We’re pleased to work with Snowflake to continue to make Apache Iceberg stay interoperable across our engines.”
Shaun Clowes, Chief Product Officer, Confluent, said: “With Tableflow on Confluent Cloud, organizations will be able to turn data streams from across the business into Apache Iceberg tables with one click. Together, Snowflake’s Polaris Catalog and Tableflow enable data teams to easily access these tables for critical application development and downstream analytics.”
Tomer Shiran, Founder, Dremio, said: ”Customers want thriving open ecosystems and to own their storage, data and metadata. They don’t want to be locked-in,” said Tomer Shiran, Founder, Dremio. “We’re committed to supporting open standards, such as Apache Iceberg and the open catalogs Project Nessie and Polaris Catalog. These open technologies will provide the ecosystem interoperability and choice that customers deserve.” Neema Raphael, Chief Data Officer and Head of Data Engineering, Goldman Sachs, said: “We open sourced our data platform, Legend, which enables us to work with open source table formats like Iceberg that will provide more interoperability across query engines like Snowflake. The launch of an open source Iceberg Catalog like Polaris is an exciting next step in furthering that commitment to interoperability.”
Raveendrnathan Loganathan, Executive Vice President of Software Engineering, Salesforce, said: “Our Salesforce Data Cloud has been built from the ground up with Open Standards Apache Parquet for files & Apache Iceberg for tables, fostering zero copy innovations to unlock trapped data, derive insights, and orchestrate actions across the Customer 360.
“We’re thrilled to have Snowflake as a member of our Zero Copy Partner Network and we’re excited to see how this new open catalog standard will further zero copy access in the enterprise.”
