Skip to content

Conversation

@jasonhernandez
Copy link

Description of changes

Adds a Materialize backend for Ibis

This PR introduces a new backend for Materialize, a live data layer with native support for generalized incremental view maintenance. Materialize strives to maintain PostgreSQL wire compatibility, but it offers a number of unique capabilities that require expanding beyond the existing Postgres backend for Materialize users. In particular, Materialize provides:

  • Indexes on views, enabling fast, incremental computation over complex SQL queries.
  • Robust materialized view support
  • Clusters for providing physical workload isolation between multiple concurrent use cases within the same database.

The Materialize team is excited to collaborate with the Ibis community and is committed to maintaining and improving this backend going forward.

@github-actions github-actions bot added docs Documentation related issues or PRs tests Issues or PRs related to tests ci Continuous Integration issues or PRs dependencies Issues or PRs related to dependencies sql Backends that generate SQL labels Oct 17, 2025
@jasonhernandez jasonhernandez force-pushed the feat/add-Materialize-backend branch 4 times, most recently from 35b0a11 to 349400c Compare October 20, 2025 17:24
@cpcloud
Copy link
Member

cpcloud commented Oct 24, 2025

@jasonhernandez Thanks for the PR! I will try to give it a review this weekend or later this evening.

It looks like the materialize test suite is timing out at 60 minutes. Any idea what's going on there?

Copy link
Member

@cpcloud cpcloud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good! This is one of the most thorough initial backend PRs I've ever seen, great work.

A few comments, mostly around exception handling and too-broad-try clauses.

@jasonhernandez jasonhernandez force-pushed the feat/add-Materialize-backend branch from d7a9d8a to ef1e1d8 Compare November 14, 2025 22:16
This commit adds comprehensive support for Materialize, including:
- Core backend implementation with cluster and connection management
- Support for materialized views, indexes, sources, sinks, and secrets
- Idiomatic Materialize SQL patterns (mz_now(), SUBSCRIBE, etc.)
- Test coverage for Materialize-specific features
- Integration with existing Ibis test suite
@jasonhernandez jasonhernandez force-pushed the feat/add-Materialize-backend branch 2 times, most recently from 39004fc to 46a9f19 Compare November 14, 2025 22:42
@jasonhernandez
Copy link
Author

@jasonhernandez Thanks for the PR! I will try to give it a review this weekend or later this evening.

It looks like the materialize test suite is timing out at 60 minutes. Any idea what's going on there?

I was hoping you might have known of something obvious that might cause that timeout. I'm going to investigate it further.
I was out over the past few weeks - back at things now for a good while.

@jasonhernandez jasonhernandez force-pushed the feat/add-Materialize-backend branch from 46a9f19 to 7e62c90 Compare November 15, 2025 01:38
@jasonhernandez jasonhernandez force-pushed the feat/add-Materialize-backend branch from 7e62c90 to bf29fa1 Compare November 15, 2025 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Continuous Integration issues or PRs dependencies Issues or PRs related to dependencies docs Documentation related issues or PRs sql Backends that generate SQL tests Issues or PRs related to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants