misc/dbt-materialize/README.md
dbt adapter for Materialize.
For a complete step-by-step guide on how to use dbt and Materialize, check the documentation.
dbt-materialize is available on PyPI. To install the latest version via
pip (optionally using a virtual environment), run:
python3 -m venv dbt-venv # create the virtual environment
source dbt-venv/bin/activate # activate the virtual environment
pip install dbt-core dbt-materialize # install dbt-core and the adapter
dbt-materialize requires Materialize v0.68.0+.
To connect to a Materialize instance, use the reference profile configuration in your connection profile:
dbt-materialize:
target: dev
outputs:
dev:
type: materialize
threads: 1
host: [host]
port: [port]
user: [[email protected]]
pass: [password]
dbname: [database]
schema: [dbt schema]
cluster: [cluster] # default 'quickstart'
sslmode: require
keepalives_idle: 0 # default 0
retries: 1 # default 1 retry on error/timeout when opening connections
search_path: [optional, override the default search_path]
Complete sample profiles can be found in sample_profiles.yml.
| Type | Supported? | Details |
|---|---|---|
source | YES | Creates a source. |
view | YES | Creates a view. |
materializedview | YES | (Deprecated) Creates a materialized view. |
materialized_view | YES | Creates a materialized view. The materializedview legacy materialization name is supported for backwards compatibility. |
table | YES | Creates a materialized view. (Actual table support pending #5266.) |
sink | YES | Creates a sink. |
ephemeral | YES | Executes queries using CTEs. |
incremental | NO | Use the materialized_view materialization instead! dbt's incremental models are valuable because they only spend your time and money transforming your new data as it arrives. Luckily, this is exactly what Materialize's materialized views were built to do! Better yet, our materialized views will always return up-to-date results without manual or configured refreshes. For more information, check out our documentation. |
Use the indexes option to define a list of indexes on
source, view, table or materialized view materializations. Each
Materialize index can have the following components:
| Component | Value | Description |
|---|---|---|
columns | list | One or more columns on which the index is defined. To create an index that uses all columns, use the default component instead. |
name | string | The name for the index. If unspecified, Materialize will use the materialization name and column names provided. |
cluster | string | The cluster to use to create the index. If unspecified, indexes will be created in the cluster used to create the materialization. |
default | bool | Default: False. If set to True, creates a default index that uses all columns. |
We provide a materialize-dbt-utils package with Materialize-specific
implementations of dispatched macros from dbt-utils. To use this package in
your dbt project, check the latest installation instructions in dbt Hub.
Supported.
Supported. Custom schemas in dbt might behave differently than you'd expect, so make sure to read the documentation!
You can instruct dbt to create a dbt source
in Materialize using the custom source materialization, which allows for
injecting the complete source statement into your .sql file.
source freshness is not supported because using Materialize, your sources will
always be fresh.
dbt docs is supported,
as well as --persist-docs.
dbt test is supported.
If you set the optional --store-failures flag or store-failures config,
dbt will save the results of a test query to a materialized_view. To use a
view instead, use the store_failures_as config.
These objects will be created in a schema suffixed or named dbt_test__audit by
default. Change this value by setting a schema config. If both
store_failures and
store_failures_as are specified, store_failures_as takes precedence.
Not supported. Support is not planned for the near term.
A huge thank you to Josh Wills, who created the original version of this adapter. 🤠