Query Google BigQuery

To query Google BigQuery with Flux:

1. Import the sql package.

2. Use sql.from() and provide the following parameters:

   • driverName: bigquery
   • dataSourceName: See data source name
   • query: SQL query to execute

import "sql"

sql.from(
    driverName: "bigquery",
    dataSourceName: "bigquery://projectid/?apiKey=mySuP3r5ecR3tAP1K3y",
    query: "SELECT * FROM exampleTable",
)

BigQuery data source name

The bigquery driver uses the following DSN syntaxes (also known as a connection string):

bigquery://projectid/?param1=value&param2=value
bigquery://projectid/location?param1=value&param2=value

Common BigQuery URL parameters

• dataset - BigQuery dataset ID. When set, you can use unqualified table names in queries.

BigQuery authentication parameters

The Flux BigQuery implementation uses the Google Cloud Go SDK. Provide your authentication credentials using one of the following methods:

• Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to identify the location of your credential JSON file.

• Provide your base-64 encoded service account, refresh token, or JSON credentials using the credentials URL parameter in your BigQuery DSN.

  Example credentials URL parameter

  Copy

  bigquery://projectid/?credentials=eyJ0eXBlIjoiYXV0...
  

Data type conversion

sql.from() converts BigQuery data types to Flux data types.

{{% caption %}} All other BigQuery data types (including DATE, TIME and DATETIME) are converted to strings. {{% /caption %}}