Back to Tuist

Tuist Platform Helm Chart

infra/helm/platform/README.md

4.198.14.8 KB
Original Source

Tuist Platform Helm Chart

Platform-level Helm umbrella chart installed once per Kubernetes cluster that hosts the Tuist-managed deployment. It bundles the infrastructure that our per-app chart (infra/helm/tuist/) assumes is already running.

What's in it

ComponentPurpose
cert-managerTLS certificate issuance via Let's Encrypt + Cloudflare DNS-01
ingress-nginxIngress controller backed by a cloud LoadBalancer
external-dnsSync Ingress / Service hostnames into Cloudflare DNS
external-secretsPull secrets from external stores (1Password, SOPS, etc.) into the cluster
metrics-serverResource metrics API (pods.metrics.k8s.io) consumed by HPAs and kubectl top
ClusterIssuerShared Let's Encrypt issuer wired to Cloudflare DNS-01
CiliumEgressGatewayPolicyOptional stable outbound source IP for hosted Tuist server traffic

Bootstrap

bash
# 1. Create the target namespace.
kubectl create namespace platform

# 2. Create the Cloudflare API token Secret out-of-band. The token must have
#    Zone.DNS:Edit scope on the managed zone(s). Never commit this value.
kubectl -n platform create secret generic cloudflare-api-token \
  --from-literal=api-token="$CLOUDFLARE_API_TOKEN"

# 3. Fetch chart dependencies.
helm dependency update infra/helm/platform

# 4. Install the platform with the right provider overlay.
helm upgrade --install platform infra/helm/platform \
  -n platform \
  -f infra/helm/platform/values-hetzner.yaml

Other clouds can plug in by adding a values-<provider>.yaml overlay that sets the provider-specific LoadBalancer annotations + any LB-specific ingress-nginx config.

k8s:install-platform also loads values-<cluster-name>.yaml when present. Use that cluster overlay for static environment configuration such as stable egress IPs.

Local validation

bash
helm dependency update infra/helm/platform
helm template platform infra/helm/platform | kubectl apply --dry-run=client -f -
helm lint infra/helm/platform

Stable Server Egress

Server pods use a Cilium egress gateway so customer-facing outbound traffic leaves from a stable environment-specific address. These addresses are Hetzner Floating IPs in the tuist-workloads project.

ClusterNamespaceEgress IPHCloud resourceHost configurerStatus
tuist-stagingtuist-staging78.47.186.71Floating IP tuist-staging-server-egressEnabledActive and verified
tuist-canarytuist-canary78.47.174.50Floating IP tuist-canary-server-egressEnabledActive and verified
tuisttuist116.202.0.10Floating IP tuist-production-server-egressEnabledActive and verified

When enabled, a values-<cluster-name>.yaml overlay renders:

  • CiliumEgressGatewayPolicy/tuist-server-stable-egress, which selects server pods in the configured namespace and SNATs public-internet traffic to the configured egress IP.
  • DaemonSet/kube-system/tuist-server-stable-egress-host-configurer, which runs only on the node labelled tuist.dev/stable-egress-gateway=server and keeps the Floating IP plus source route present on that node's eth0.

If the gateway node is replaced or you need to fail over manually:

bash
export KUBECONFIG=~/.kube/tuist-production.yaml
export FLOATING_IP_NAME=tuist-production-server-egress
export EGRESS_IP=116.202.0.10
export OLD_NODE=<old-gateway-node>
export NEW_NODE=<new-general-pool-node>

# Move the cloud route in Hetzner first. Use the tuist-workloads hcloud token.
hcloud floating-ip assign "$FLOATING_IP_NAME" "$NEW_NODE"

kubectl label node "$OLD_NODE" tuist.dev/stable-egress-gateway- --overwrite
kubectl label node "$NEW_NODE" tuist.dev/stable-egress-gateway=server --overwrite

# Force Cilium to re-select the gateway after the label move.
kubectl annotate ciliumegressgatewaypolicy tuist-server-stable-egress \
  "tuist.dev/reapplied-at=$(date -u +%s)" --overwrite

# Verify from any server pod.
kubectl -n tuist exec deploy/tuist-tuist-server -- curl -fsS https://api.ipify.org

The verification command should print the environment's configured egress IP. Existing outbound connections may reset when the gateway node changes; new connections should use the new gateway once the policy is re-applied.

Notes

  • The ingress-nginx LoadBalancer is annotated for Hetzner Cloud (Nuremberg region) by default. Adjust ingress-nginx.controller.service.annotations when the cluster lands on a different provider.
  • external-dns is scoped by txtOwnerId: tuist-platform — one cluster, one TXT prefix. Run it with policy: sync only if you're happy with it deleting DNS records that aren't tracked by any Ingress.
  • cert-manager CRDs are installed by the subchart (installCRDs: true). If another tool manages them, turn that off.