python

# WE MUST ENSURE PYTHON CONSISTENCY BETWEEN NOTEBOOK AND FEAST SERVERS
# LAUNCH THIS NOTEBOOK FROM A CLEAN PYTHON ENVIRONMENT >3.9
%pip install feast==0.40.1

Install Feast on Kind

Objective

Provide a reference implementation of a runbook to deploy a Feast development environment on a Kubernets cluster using Kind.

Prerequisites

Kind cluster and a Docker runtime container
kubectl Kubernetes CLI tool.
Helm Kubernetes package manager.
yq YAML processor.

Install Prerequisites

The following commands install and configure all the prerequisites on MacOS environment. You can find the equivalent instructions on the offical documentation pages:

Install Kind and Docker runtime (e.g. Colima).
Create Kind cluster named feast.
Install and setup the kubectl context.
Helm.
yq.

bash

brew install colima
colima start
brew install kind
kind create cluster --name feast
kind start
brew install helm
brew install kubectl
kubectl config use-context kind-feast
brew install yq

Additionally, we create a feast namespace and use it as the default for the kubectl CLI:

python

!kubectl create ns feast
!kubectl config set-context --current --namespace feast

Validate the cluster setup:

python

!kubectl get ns

Deployment Architecture

The primary objective of this runbook is to guide the deployment of Feast services on a Kubernetes Kind cluster, using the default postgres template to set up a basic feature store.

🚀 We will also add instructions to repeat the example with a custom project, for a personalized experience.

In this notebook, we will deploy a distributed topology of Feast services, which includes:

Registry Server: Exposes endpoints at the default port 6570 and handles metadata storage for feature definitions.
Online Store Server: Exposes endpoints at the default port 6566. This service uses the Registry Server to query metadata and is responsible for low-latency serving of features.
Offline Store Server: Exposes endpoints at the default port 8815. It uses the Registry Server to query metadata and provides access to batch data for historical feature retrieval.

Each service is backed by a PostgreSQL database, which is also deployed within the same Kind cluster.

Finally, port forwarding will be configured to expose these Feast services locally. This will allow a local client, implemented in the accompanying client notebook, to interact with the deployed services.

Install PostgreSQL

Install the reference deployment to install and configure a simple PostgreSQL database.

python

!kubectl apply -f postgres/postgres.yaml
!kubectl wait --for=condition=available deployment/postgres --timeout=2m

python

!kubectl get pods
!kubectl get svc

Create the feature store project

Use the feast init command to create the default project.

We also start port forwarding for the postgres service to populate the tables with default data.

🚀 If you want to use a custom configuration, replace it under the sample/feature_repo folder and skip this section

python

from src.utils import port_forward
psql_process = port_forward("postgres", 5432, 5432)

We are going to emulate the feast init -t postgres sample command using Python code. This is needed to mock the request of additional parameters to configure the DB connection and also request the upload of example data to Postgres tables.

python

from feast.repo_operations import init_repo
from unittest import mock
from feast.templates.postgres.bootstrap import bootstrap

project_directory = "sample"
template = "postgres"

with mock.patch("click.prompt", side_effect=["localhost", "5432", "feast", "public", "feast", "feast"]):
  with mock.patch("click.confirm", side_effect=[True]):
    init_repo(project_directory, template)

Verify that the DB includes the expected tables with pre-populated data.

python

!PSQL_POD=$(kubectl get pods -l app=postgres -oname) && kubectl exec $PSQL_POD -- psql -h localhost -U feast feast -c '\dt'
!PSQL_POD=$(kubectl get pods -l app=postgres -oname) && kubectl exec $PSQL_POD -- psql -h localhost -U feast feast -c 'select count(*) from feast_driver_hourly_stats'

Finally, let's stop port forwarding.

python

psql_process.terminate()
!ps -ef | grep port-forward

Generate server configurations

Each server has its own configuration that we generate from the one initialized before.

We use yq to manipulate the original configuration and generate the server specifics.

Note: from now on, we assume that the Feast service names will be as follows:

For Registry Server: registry-server
For Online Store: online-server
For Offline Store: offline-server

🚀 If you used different service names, replace the host parameter in the following yq commands.

python

%env FEATURE_REPO_DIR=sample/feature_repo
# Adjust the database host to match the postgres service
!yq -i '.registry.path="postgresql://feast:feast@postgres:5432/feast"' $FEATURE_REPO_DIR/feature_store.yaml
!yq -i '.online_store.host="postgres"' $FEATURE_REPO_DIR/feature_store.yaml
!yq -i '.offline_store.host="postgres"' $FEATURE_REPO_DIR/feature_store.yaml
!cat $FEATURE_REPO_DIR/feature_store.yaml

python

# Registry server has only `registry` section
!cat $FEATURE_REPO_DIR/feature_store.yaml | yq '.project | {key: .}, .registry | {key: .}, .provider | {key: .}, .entity_key_serialization_version | {key: .}' > registry_feature_store.yaml
! cat registry_feature_store.yaml

python

# Online server has `online_store` section, a remote `registry` and a remote `offline_store`
!cat $FEATURE_REPO_DIR/feature_store.yaml | yq '.project | {key: .}, .provider | {key: .}, .online_store  | {key: .}, .entity_key_serialization_version | {key: .}' > online_feature_store.yaml
!yq -i '.registry.path="registry-server:80"' online_feature_store.yaml
!yq -i '.registry.registry_type="remote"' online_feature_store.yaml
!yq -i '.offline_store.type="remote"' online_feature_store.yaml
!yq -i '.offline_store.host="offline-server"' online_feature_store.yaml
!yq -i '.offline_store.port=80' online_feature_store.yaml

!cat online_feature_store.yaml

python

# Offline server has `offline_store` section and a remote `registry`
!cat $FEATURE_REPO_DIR/feature_store.yaml | yq '.project | {key: .}, .provider | {key: .}, .offline_store | {key: .}, .entity_key_serialization_version | {key: .}' > offline_feature_store.yaml
!yq -i '.registry.path="registry-server:80"' offline_feature_store.yaml
!yq -i '.registry.registry_type="remote"' offline_feature_store.yaml
!cat offline_feature_store.yaml

Encode configuration files

Next step is to encode in base64 the configuration files for each server. We'll store the output in environment variables.

python

import os
def base64_file(file):
  import base64

  with open(file, 'rb') as file:
      yaml_content = file.read()
  return base64.b64encode(yaml_content).decode('utf-8')

os.environ['REGISTRY_CONFIG_BASE64'] = base64_file('registry_feature_store.yaml')
os.environ['ONLINE_CONFIG_BASE64'] = base64_file('online_feature_store.yaml')
os.environ['OFFLINE_CONFIG_BASE64'] = base64_file('offline_feature_store.yaml')

python

!env | grep BASE64

Install servers

We'll use the charts defined in this local repository to install the servers.

The installation order reflects the dependency between the deployments:

Registry Server starts first because it has no dependencies
Then Offline Server as it depends only on the Registry Server
Last the Online Server that depends on both the other servers

python

%env FEAST_IMAGE_REPO=quay.io/feastdev/feature-server
%env FEAST_IMAGE_VERSION=latest

python

# Registry
!helm upgrade --install feast-registry ../../infra/charts/feast-feature-server \
--set fullnameOverride=registry-server --set feast_mode=registry \
--set image.repository=${FEAST_IMAGE_REPO} --set image.tag=${FEAST_IMAGE_VERSION} \
--set feature_store_yaml_base64=$REGISTRY_CONFIG_BASE64

!kubectl wait --for=condition=available deployment/registry-server --timeout=2m

python

# Offline
!helm upgrade --install feast-offline ../../infra/charts/feast-feature-server \
--set fullnameOverride=offline-server --set feast_mode=offline \
--set image.repository=${FEAST_IMAGE_REPO} --set image.tag=${FEAST_IMAGE_VERSION} \
--set feature_store_yaml_base64=$OFFLINE_CONFIG_BASE64

!kubectl wait --for=condition=available deployment/offline-server --timeout=2m

python

# Online
!helm upgrade --install feast-online ../../infra/charts/feast-feature-server \
--set fullnameOverride=online-server --set feast_mode=online \
--set image.repository=${FEAST_IMAGE_REPO} --set image.tag=${FEAST_IMAGE_VERSION} \
--set feature_store_yaml_base64=$ONLINE_CONFIG_BASE64

!kubectl wait --for=condition=available deployment/online-server --timeout=2m

Validate deployment

Fist validate application and service status:

python

!kubectl get svc
!kubectl get deployments
!kubectl get pods

Then verify the content of the local configuration file (it's stored in /tmp/ folder with random subfolder).

python

!kubectl exec deployment/registry-server -- find /tmp -name feature_store.yaml -exec cat {} \;

python

!kubectl exec deployment/offline-server -- find /tmp -name feature_store.yaml -exec cat {} \;

python

!kubectl exec deployment/online-server -- find /tmp -name feature_store.yaml -exec cat {} \;

Finally, let's verify the feast version in each server

python

!kubectl exec deployment/registry-server -- feast version
!kubectl exec deployment/offline-server -- feast version
!kubectl exec deployment/online-server -- feast version