Back to Cube

Snowflake

docs/content/product/configuration/data-sources/snowflake.mdx

1.6.4313.8 KB
Original Source

Snowflake

Snowflake is a popular cloud-based data platform.

Prerequisites

<InfoBox>

In order to connect Cube to Snowflake, you need to grant certain permissions to the Snowflake role used by Cube. Cube requires the role to have USAGE on databases and schemas and SELECT on tables. An example configuration:

GRANT USAGE ON DATABASE ABC TO ROLE XYZ;
GRANT USAGE ON ALL SCHEMAS IN DATABASE ABC TO ROLE XYZ;
GRANT USAGE ON FUTURE SCHEMAS IN DATABASE ABC TO ROLE XYZ;
GRANT SELECT ON ALL TABLES IN DATABASE ABC TO ROLE XYZ;
GRANT SELECT ON FUTURE TABLES IN DATABASE ABC TO ROLE XYZ;
</InfoBox>
  • Account/Server URL for Snowflake.
  • User name and password or an RSA private key for the Snowflake account.
  • Optionally, the warehouse name, the user role, and the database name.

Setup

<WarningBox> If you're having Network error and Snowflake can't be reached please make sure you tried [format 2 for an account id][snowflake-format-2]. </WarningBox>

Manual

Add the following to a .env file in your Cube project:

dotenv
CUBEJS_DB_TYPE=snowflake
CUBEJS_DB_SNOWFLAKE_ACCOUNT=XXXXXXXXX.us-east-1
CUBEJS_DB_SNOWFLAKE_WAREHOUSE=MY_SNOWFLAKE_WAREHOUSE
CUBEJS_DB_NAME=my_snowflake_database
CUBEJS_DB_USER=snowflake_user
CUBEJS_DB_PASS=**********

Cube Cloud

<InfoBox heading="Allowing connections from Cube Cloud IP">

In some cases you'll need to allow connections from your Cube Cloud deployment IP address to your database. You can copy the IP address from either the Database Setup step in deployment creation, or from <Btn>Settings → Configuration</Btn> in your deployment.

</InfoBox>

The following fields are required when creating a Snowflake connection:

<Screenshot alt="Cube Cloud Snowflake Configuration Screen" src="https://ucarecdn.com/2a113d20-33ca-4634-b6fa-8886df4d215c/" />

Cube Cloud also supports connecting to data sources within private VPCs if dedicated infrastructure is used. Check out the VPC connectivity guide for details.

Environment Variables

Environment VariableDescriptionPossible ValuesRequired
<EnvVar>CUBEJS_DB_SNOWFLAKE_ACCOUNT</EnvVar>The Snowflake account identifier to use when connecting to the databaseA valid Snowflake account ID
<EnvVar>CUBEJS_DB_SNOWFLAKE_REGION</EnvVar>The Snowflake region to use when connecting to the databaseA valid Snowflake region
<EnvVar>CUBEJS_DB_SNOWFLAKE_WAREHOUSE</EnvVar>The Snowflake warehouse to use when connecting to the databaseA valid Snowflake warehouse in the account
<EnvVar>CUBEJS_DB_SNOWFLAKE_ROLE</EnvVar>The Snowflake role to use when connecting to the databaseA valid Snowflake role in the account
<EnvVar>CUBEJS_DB_SNOWFLAKE_CLIENT_SESSION_KEEP_ALIVE</EnvVar>If true, keep the Snowflake connection alive indefinitelytrue, false
<EnvVar>CUBEJS_DB_NAME</EnvVar>The name of the database to connect toA valid database name
<EnvVar>CUBEJS_DB_USER</EnvVar>The username used to connect to the databaseA valid database username
<EnvVar>CUBEJS_DB_PASS</EnvVar>The password used to connect to the databaseA valid database password
<EnvVar>CUBEJS_DB_SNOWFLAKE_AUTHENTICATOR</EnvVar>The type of authenticator to use with Snowflake. Use SNOWFLAKE with username/password, or SNOWFLAKE_JWT with key pairs. Defaults to SNOWFLAKESNOWFLAKE, SNOWFLAKE_JWT, OAUTH
<EnvVar>CUBEJS_DB_SNOWFLAKE_PRIVATE_KEY</EnvVar>The content of the private RSA keyContent of the private RSA key (encrypted or not)
<EnvVar>CUBEJS_DB_SNOWFLAKE_PRIVATE_KEY_PATH</EnvVar>The path to the private RSA keyA valid path to the private RSA key
<EnvVar>CUBEJS_DB_SNOWFLAKE_PRIVATE_KEY_PASS</EnvVar>The password for the private RSA key. Only required for encrypted keysA valid password for the encrypted private RSA key
<EnvVar>CUBEJS_DB_SNOWFLAKE_OAUTH_TOKEN</EnvVar>The OAuth tokenA valid OAuth token (string)
<EnvVar>CUBEJS_DB_SNOWFLAKE_OAUTH_TOKEN_PATH</EnvVar>The path to the valid oauth toket fileA valid path for the oauth token file
<EnvVar>CUBEJS_DB_SNOWFLAKE_HOST</EnvVar>Host address to which the driver should connectA valid hostname
<EnvVar>CUBEJS_DB_SNOWFLAKE_QUOTED_IDENTIFIERS_IGNORE_CASE</EnvVar>Whether or not quoted identifiers should be case insensitive. Default is falsetrue, false
<EnvVar>CUBEJS_DB_MAX_POOL</EnvVar>The maximum number of concurrent database connections to pool. Default is 20A valid number
<EnvVar>CUBEJS_CONCURRENCY</EnvVar>The number of concurrent queries to the data sourceA valid number

Pre-Aggregation Feature Support

count_distinct_approx

Measures of type count_distinct_approx can be used in pre-aggregations when using Snowflake as a source database. To learn more about Snowflake's support for approximate aggregate functions, click here.

Pre-Aggregation Build Strategies

<InfoBox>

To learn more about pre-aggregation build strategies, head here.

</InfoBox>
FeatureWorks with read-only mode?Is default?
Batching
Export Bucket

By default, Snowflake uses batching to build pre-aggregations.

Batching

No extra configuration is required to configure batching for Snowflake.

Export Bucket

Snowflake supports using both AWS S3 and Google Cloud Storage for export bucket functionality.

AWS S3

<InfoBox>

Ensure proper IAM privileges are configured for S3 bucket reads and writes, using either storage integration or user credentials for Snowflake and either IAM roles/IRSA or user credentials for Cube Store, with mixed configurations supported.

</InfoBox>

Using IAM user credentials for both:

dotenv
CUBEJS_DB_EXPORT_BUCKET_TYPE=s3
CUBEJS_DB_EXPORT_BUCKET=my.bucket.on.s3
CUBEJS_DB_EXPORT_BUCKET_AWS_KEY=<AWS_KEY>
CUBEJS_DB_EXPORT_BUCKET_AWS_SECRET=<AWS_SECRET>
CUBEJS_DB_EXPORT_BUCKET_AWS_REGION=<AWS_REGION>

Using a Storage Integration to write to export buckets and user credentials to read from Cube Store:

dotenv
CUBEJS_DB_EXPORT_BUCKET_TYPE=s3
CUBEJS_DB_EXPORT_BUCKET=my.bucket.on.s3
CUBEJS_DB_EXPORT_INTEGRATION=aws_int
CUBEJS_DB_EXPORT_BUCKET_AWS_KEY=<AWS_KEY>
CUBEJS_DB_EXPORT_BUCKET_AWS_SECRET=<AWS_SECRET>
CUBEJS_DB_EXPORT_BUCKET_AWS_REGION=<AWS_REGION>

Using a Storage Integration to write to export bucket and IAM role/IRSA to read from Cube Store:**

dotenv
CUBEJS_DB_EXPORT_BUCKET_TYPE=s3
CUBEJS_DB_EXPORT_BUCKET=my.bucket.on.s3
CUBEJS_DB_EXPORT_INTEGRATION=aws_int
CUBEJS_DB_EXPORT_BUCKET_AWS_REGION=<AWS_REGION>

Google Cloud Storage

<InfoBox>

When using an export bucket, remember to assign the Storage Object Admin role to your Google Cloud credentials (<EnvVar>CUBEJS_DB_EXPORT_GCS_CREDENTIALS</EnvVar>).

</InfoBox>

Before configuring Cube, an integration must be created and configured in Snowflake. Take note of the integration name (gcs_int from the example link) as you'll need it to configure Cube.

Once the Snowflake integration is set up, configure Cube using the following:

dotenv
CUBEJS_DB_EXPORT_BUCKET=snowflake-export-bucket
CUBEJS_DB_EXPORT_BUCKET_TYPE=gcs
CUBEJS_DB_EXPORT_GCS_CREDENTIALS=<BASE64_ENCODED_SERVICE_CREDENTIALS_JSON>
CUBEJS_DB_EXPORT_INTEGRATION=gcs_int

Azure

To use Azure Blob Storage as an export bucket, follow the guide on using a Snowflake storage integration (Option 1). Take note of the integration name (azure_int from the example link) as you'll need it to configure Cube.

Retrieve the storage account access key from your Azure account.

Once the Snowflake integration is set up, configure Cube using the following:

dotenv
CUBEJS_DB_EXPORT_BUCKET_TYPE=azure
CUBEJS_DB_EXPORT_BUCKET=wasbs://[email protected]
CUBEJS_DB_EXPORT_BUCKET_AZURE_KEY=<AZURE_STORAGE_ACCOUNT_ACCESS_KEY>
CUBEJS_DB_EXPORT_INTEGRATION=azure_int

SSL

Cube does not require any additional configuration to enable SSL as Snowflake connections are made over HTTPS.