Skip to main content
Skip to main content
Edit this page

REST Catalog

Experimental feature. Learn more.
Note

Integration with the REST Catalog works with Iceberg tables only. This integration supports both AWS S3 and other cloud storage providers.

ClickHouse supports integration with multiple catalogs (Unity, Glue, REST, Polaris, etc.). This guide will walk you through the steps to query your data using ClickHouse and the REST Catalog specification.

The REST Catalog is a standardized API specification for Iceberg catalogs, supported by various platforms including:

  • Local development environments (using docker-compose setups)
  • Managed services like Tabular.io
  • Self-hosted REST catalog implementations
Note

As this feature is experimental, you will need to enable it using: SET allow_experimental_database_rest_catalog = 1;

Local Development Setup​

For local development and testing, you can use a containerized REST catalog setup. This approach is ideal for learning, prototyping, and development environments.

Prerequisites​

  1. Docker and Docker Compose: Ensure Docker is installed and running
  2. Sample Setup: You can use various docker-compose setups (see Alternative Docker Images below)

Setting up Local REST Catalog​

You can use various containerized REST catalog implementations such as Databricks docker-spark-iceberg which provides a complete Spark + Iceberg + REST catalog environment with docker-compose, making it ideal for testing Iceberg integrations.

You'll need to add ClickHouse as a dependency in your docker-compose setup:

Connecting to Local REST Catalog​

Connect to your ClickHouse container:

Then create the database connection to the REST catalog:

Querying REST catalog tables using ClickHouse​

Now that the connection is in place, you can start querying via the REST catalog. For example:

To query a table:

Backticks required

Backticks are required because ClickHouse doesn't support more than one namespace.

To inspect the table DDL:

Loading data from your Data Lake into ClickHouse​

If you need to load data from the REST catalog into ClickHouse, start by creating a local ClickHouse table:

Then load the data from your REST catalog table via an INSERT INTO SELECT: