feature_store.yaml

Overview

feature_store.yaml is a file that is placed at the root of the Feature Repository. This file contains configuration about how the feature store runs. An example feature_store.yaml is shown below:

{% code title="feature_store.yaml" %}

yaml

project: loyal_spider
registry: data/registry.db
provider: local
online_store:
    type: sqlite
    path: data/online_store.db

{% endcode %}

Fields in feature_store.yaml

provider ("local" or "gcp") — Defines the environment in which Feast will execute data flows.
registry (a local or GCS filepath) — Defines the location of the feature registry.
online_store — Configures the online store. This field will have various subfields depending on the type of online store:
- type ("sqlite" or "datastore") — Defines the type of online store.
- path (a local filepath) — Parameter for the sqlite online store. Defines the path to the SQLite database file.
- project_id — Optional parameter for the datastore online store. Sets the GCP project id used by Feast, if not set Feast will use the default GCP project id in the local environment.
project — Defines a namespace for the entire feature store. Can be used to isolate multiple deployments in a single installation of Feast.

feature_server

The feature_server block configures the Python Feature Server when it is used to serve online features and handle /push requests. This section is optional and only applies when running the Python feature server.

An example configuration:

yaml

feature_server:
  type: local
  metrics: # Prometheus metrics configuration. Also achievable via `feast serve --metrics`.
    enabled: true             # Enable Prometheus metrics server on port 8000
    resource: true            # CPU / memory gauges
    request: true             # endpoint latency histograms & request counters
    online_features: true     # online feature retrieval counters + store read & ODFV transform timing
    push: true                # push request counters
    materialization: true     # materialization counters & duration histograms
    freshness: true           # per-feature-view freshness gauges
  offline_push_batching_enabled: true # Enables batching of offline writes processed by /push. Online writes are unaffected.
  offline_push_batching_batch_size: 100 # Maximum number of buffered rows before writing to the offline store.
  offline_push_batching_batch_interval_seconds: 5 # Maximum time rows may remain buffered before a forced flush.

Providers

The provider field defines the environment in which Feast will execute data flows. As a result, it also determines the default values for other fields.

Local

When using the local provider:

Feast can read from local Parquet data sources.
Feast performs historical feature retrieval (point-in-time joins) using pandas.
Feast performs online feature serving from a SQLite database.

GCP

When using the GCP provider:

Feast can read data from BigQuery data sources.
Feast performs historical feature retrieval (point-in-time joins) in BigQuery.
Feast performs online feature serving from Google Cloud Datastore.

Permissions

<table> <thead> <tr> <th style="text-align:left">Command </th> <th style="text-align:left">Component</th> <th style="text-align:left">Permissions</th> <th style="text-align:left">Recommended Role</th> </tr> </thead> <tbody> <tr> <td style="text-align:left">Apply </td> <td style="text-align:left">BigQuery (source)</td> <td style="text-align:left"> bigquery.jobs.create bigquery.readsessions.create bigquery.readsessions.getData </td> <td style="text-align:left">roles/bigquery.user</td> </tr> <tr> <td style="text-align:left">Apply </td> <td style="text-align:left">Datastore (destination)</td> <td style="text-align:left"> datastore.entities.allocateIds datastore.entities.create datastore.entities.delete datastore.entities.get datastore.entities.list datastore.entities.update </td> <td style="text-align:left">roles/datastore.owner</td> </tr> <tr> <td style="text-align:left">Materialize </td> <td style="text-align:left">BigQuery (source)</td> <td style="text-align:left">bigquery.jobs.create</td> <td style="text-align:left">roles/bigquery.user</td> </tr> <tr> <td style="text-align:left">Materialize </td> <td style="text-align:left">Datastore (destination)</td> <td style="text-align:left"> datastore.entities.allocateIds datastore.entities.create datastore.entities.delete datastore.entities.get datastore.entities.list datastore.entities.update datastore.databases.get </td> <td style="text-align:left">roles/datastore.owner</td> </tr> <tr> <td style="text-align:left">Get Online Features </td> <td style="text-align:left">Datastore</td> <td style="text-align:left">datastore.entities.get</td> <td style="text-align:left">roles/datastore.user</td> </tr> <tr> <td style="text-align:left">Get Historical Features </td> <td style="text-align:left">BigQuery (source)</td> <td style="text-align:left"> bigquery.datasets.get bigquery.tables.get bigquery.tables.create bigquery.tables.updateData bigquery.tables.update bigquery.tables.delete bigquery.tables.getData </td> <td style="text-align:left">roles/bigquery.dataEditor</td> </tr> </tbody> </table>