python

# WE MUST ENSURE PYTHON CONSISTENCY BETWEEN NOTEBOOK AND FEAST SERVERS
# LAUNCH THIS NOTEBOOK FROM A CLEAN PYTHON ENVIRONMENT >3.9
%pip install feast==0.40.1

Run a test client

🚀 This test is developer to work only with the default feature store generated by feast init.

To test a custom feature store you need to run a custom test application, but still using the same client configuration that we've prepared.

Apply the feature store definitions

The feature store cannot be initialized using remote services.

We'll use the original feature_store.yaml from within a Kubernetes Job to run feast apply.

For the same reason, we also run an initial materialization from the Job, otherwise it would fail because of uninmplemented APIs in the remote servers, like online_write_batch.

First we create a ConfigMap holding the required code and configuration.

python

%env FEATURE_REPO_DIR=sample/feature_repo
!kubectl delete configmap sample-repo
!kubectl create configmap sample-repo --from-file=${FEATURE_REPO_DIR}/example_repo.py,${FEATURE_REPO_DIR}/feature_store.yaml
!echo
!echo "Inspect keys of sample-repo ConfigMap"
!kubectl get configmaps sample-repo -oyaml | yq '.data[] | key'

Then we create the Job to apply the definitions, according to the init-job.yaml manifest

python

!kubectl delete -f init-job.yaml
!kubectl apply -f init-job.yaml

Monitoring the log of the Job.

python

!INIT_JOB_POD=$(kubectl get pods -l job-name=feast-apply-job -oname) && kubectl wait --for=condition=podscheduled $INIT_JOB_POD --timeout=2m
!INIT_JOB_POD=$(kubectl get pods -l job-name=feast-apply-job -oname) && kubectl logs -f $INIT_JOB_POD

Forwarding the feast service ports

To run the test client from the notebook, we need to forward the service ports to ports on the current host.

python

!kubectl get svc

python

from src.utils import port_forward
registry_process = port_forward("registry-server", 8001)
offline_process = port_forward("offline-server", 8002)
online_process = port_forward("online-server", 8003)

python

!ps -ef | grep port-forward

Client configuration

The client configuration is using only remote clients connected to the forwarded ports, from 8001 to 8003.

python

!cat client/feature_store.yaml

Install test code

First we copy the test code from sample/feature_repo to client folder.

python

!cp sample/feature_repo/test_workflow.py client
!ls client/*.py

We update the original test to comment the apply, teardown and materialize-incremental commands.

python

!sed -i.bk 's/subprocess.run/# subprocess.run/' client/test_workflow.py
!sed -i.bk 's/print("\\n--- Run feast/# print("\\n--- Run feast/' client/test_workflow.py
!sed -i.bk 's/store.materialize_incremental/# store.materialize_incremental/' client/test_workflow.py
!sed -i.bk 's/print("\\n--- Load features/# print("\\n--- Load features/' client/test_workflow.py
!diff client/test_workflow.py sample/feature_repo/test_workflow.py

Finally, we run the full test suite from the client folder.

python

!cd client && python test_workflow.py

Note If you see the following error, it is likely due to the issue #4392: Remote registry client does not map application errors:

Feature view driver_hourly_stats_fresh does not exist in project sample

Terminate port forwarding

python

registry_process.terminate()
offline_process.terminate()
online_process.terminate()
!ps -ef | grep port-forward