ContextQMD
Back to Feast

Deploy PostgreSQL with Helm in TLS Mode

examples/operator-postgres-tls-demo/01-Install-postgres-tls-using-helm.ipynb

0.63.06.5 KB
Original Source

Deploy PostgreSQL with Helm in TLS Mode

<font color="red"> NOTE: This PostgreSQL setup guide is intended to demonstrate the capabilities of the Feast operator in configuring Feast with PostgreSQL in TLS mode. For ongoing assistance with Postgres setup, we recommend consulting the official Helm PostgreSQL documentation.</font>

Step 1: Install Prerequisites

Before starting, ensure you have the following installed:

  • kubectl (Kubernetes CLI)
  • helm (Helm CLI)
  • A Kubernetes cluster (e.g., Minikube, GKE, EKS, or AKS)

Note: When deploying PostgreSQL and Feast on a Kubernetes cluster, it's important to ensure that your cluster has sufficient resources to support both applications.

python
# Verify kubectl and helm are installed
!kubectl version --client
!helm version

Step 2: Add the Bitnami Helm Repository

python
# Add the Bitnami Helm repository
!helm repo add bitnami https://charts.bitnami.com/bitnami
!helm repo update

Step 3: create kubernetes feast namespace

python
!kubectl create ns feast
!kubectl config set-context --current --namespace feast

Step 4: Generate Self Signed TLS Certificates

Note:

  • Self signed certificates are used only for demo purpose, consider using a managed certificate service (e.g., Let's Encrypt) instead of self-signed certificates.
  • "Replace the CN values in the certificate generation step with your actual domain names.",

Delete the directory of existing certificates if you running this demo not first time.

python
# Delete certificates directory if you are running this example not first time.
!rm -rf postgres-tls-certs

Generate the certificates by executing below scripts.

python
# Create a directory for certificates
!mkdir -p postgres-tls-certs

# Generate a CA certificate
!openssl req -new -x509 -days 365 -nodes -out postgres-tls-certs/ca.crt -keyout postgres-tls-certs/ca.key -subj "/CN=PostgreSQL CA"

# Generate a server certificate
!openssl req -new -nodes -out postgres-tls-certs/server.csr -keyout postgres-tls-certs/server.key -subj "/CN=postgresql.feast.svc.cluster.local"
!openssl x509 -req -in postgres-tls-certs/server.csr -days 365 -CA postgres-tls-certs/ca.crt -CAkey postgres-tls-certs/ca.key -CAcreateserial -out postgres-tls-certs/server.crt

# Generate a client certificate
!openssl req -new -nodes -out postgres-tls-certs/client.csr -keyout postgres-tls-certs/client.key -subj "/CN=admin"
!openssl x509 -req -in postgres-tls-certs/client.csr -days 365 -CA postgres-tls-certs/ca.crt -CAkey postgres-tls-certs/ca.key -CAcreateserial -out postgres-tls-certs/client.crt

Step 5: Create Kubernetes Secrets for Certificates

In this step, we will create two Kubernetes secrets that reference the certificates generated earlier step:

  • postgresql-server-certs
    This secret contains the server certificates and will be used by the PostgreSQL server.

  • postgresql-client-certs
    This secret contains the client certificates and will be used by the PostgreSQL client. In our case it will be feast application.

python
# Create a secret for the server certificates
!kubectl create secret generic postgresql-server-certs   --from-file=ca.crt=./postgres-tls-certs/ca.crt   --from-file=tls.crt=./postgres-tls-certs/server.crt   --from-file=tls.key=./postgres-tls-certs/server.key

# Create a secret for the client certificates
!kubectl create secret generic postgresql-client-certs   --from-file=ca.crt=./postgres-tls-certs/ca.crt   --from-file=tls.crt=./postgres-tls-certs/client.crt   --from-file=tls.key=./postgres-tls-certs/client.key

Step 6: Deploy PostgreSQL with Helm

python
# Helm values for TLS configuration
helm_values = """
tls:
  enabled: true
  certificatesSecret: "postgresql-server-certs"
  certFilename: "tls.crt"
  certKeyFilename: "tls.key"
  certCAFilename: "ca.crt"

volumePermissions:
  enabled: true

# Set fixed PostgreSQL credentials

global:
  postgresql:
    auth:
      username: admin
      password: password
      database: feast
"""

# Write the values to a file
with open("values.yaml", "w") as f:
    f.write(helm_values)

# Install PostgreSQL with Helm
!helm install postgresql bitnami/postgresql --version 16.4.9 -f values.yaml -n feast

Step 7: Verify the postgres Deployment

python
# Wait for the status of the PostgreSQL pod to be in Ready status.
!kubectl wait --for=condition=Ready pod -l app.kubernetes.io/name=postgresql --timeout=60s

# Insert an empty line in the output for verbocity.
print()

# display the pod status.
!kubectl get pods -l app.kubernetes.io/name=postgresql

# Insert an empty line in the output for verbocity.
print()

# check if the ssl is on and the path to certificates is configured.
!kubectl exec postgresql-0 -- cat /opt/bitnami/postgresql/conf/postgresql.conf | grep ssl

# Insert an empty line in the output for verbocity.
print()

# Connect to PostgreSQL using TLS (non-interactive mode)
!kubectl exec postgresql-0 -- env PGPASSWORD=password psql -U admin -d feast -c '\l'

Step 8: Port forwarding in the terminal for the connection testing using python

Note: If you do not intend to test the PostgreSQL connection from outside the Kubernetes cluster, you can skip the remaining steps.

Note: To test a connection to a PostgreSQL database outside of your Kubernetes cluster, you'll need to execute the following command in your system's terminal window. This is necessary because Jupyter Notebook does not support running commands in a separate thread.

kubectl port-forward svc/postgresql 5432:5432

Step 9: Check the connection using Python sql alchemy

python
# Define database connection parameters
DB_USER = "admin"
DB_PASSWORD = "password"
DB_HOST = "localhost"
DB_PORT = "5432"
DB_NAME = "feast"

# TLS Certificate Paths
SSL_CERT = "postgres-tls-certs/client.crt"
SSL_KEY = "postgres-tls-certs/client.key"
SSL_ROOT_CERT = "postgres-tls-certs/ca.crt"

import os
os.environ["FEAST_CA_CERT_FILE_PATH"] = "postgres-tls-certs/ca.crt"

from sqlalchemy import create_engine
# Create SQLAlchemy connection string
DATABASE_URL = (
    f"postgresql+psycopg://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}?"
    f"sslmode=verify-ca&sslrootcert={SSL_ROOT_CERT}&sslcert={SSL_CERT}&sslkey={SSL_KEY}"
)

print(DATABASE_URL)

# Create SQLAlchemy engine
engine = create_engine(DATABASE_URL)

# Test connection
try:
    with engine.connect() as connection:
        print("Connected successfully!")
except Exception as e:
    print("Connection failed: Make sure that port forwarding step is done in the terminal.", e)