tekton/README.md
Why does Tekton pipelines have a folder called tekton? Cuz we think it would be cool
if the tekton folder were the place to look for CI/CD logic in most repos!
We dogfood our project by using Tekton Pipelines to build, test and release
Tekton Pipelines! This directory contains the
Tasks and
Pipelines
that we use.
To create an official release, follow the steps in the release-cheat-sheet.
Sometimes we'll find bugs that we want to backport fixes for into previous releases or discover things that were missing from a release that are required by upstream consumers of a project. In that case we'll make a patch release. To make one:
needs-cherry-pick.release-<version number>x, e.g. release-v0.13.0x
and push it to the repo https://github.com/tektoncd/pipeline (you may need help from
an OWNER with permission to push) if that release branch does not exist.-x to include
the original commit information).needs-cherry-pick.needs-cherry-pick
from all issues that have been cherry picked.The nightly release pipeline is triggered nightly by Tekton.
This Pipeline uses:
To start from scratch and use these Pipelines and Tasks:
# If this is your first time installing Tekton in the cluster you might need to give yourself permission to do so
kubectl create clusterrolebinding cluster-admin-binding-someusername \
--clusterrole=cluster-admin \
--user=$(gcloud config get-value core/account)
# Example, Tekton v0.9.1
export TEKTON_VERSION=0.9.1
kubectl apply --filename https://infra.tekton.dev/tekton-releases/pipeline/previous/v${TEKTON_VERSION}/release.yaml
Add all the Tasks to the cluster, including the
golang
Tasks from the
tektoncd/catalog, and the
release Tasks from
tektoncd/plumbing.
Use a version of the tektoncdcatalog
tasks that is compatible with version of Tekton being released, usually master.
Install Task from plumbing too:
# Apply the Tasks we are using from the catalog
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/golang-build/0.3/golang-build.yaml
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/golang-test/0.2/golang-test.yaml
# Apply Tasks and other resources from Plumbing.
#
# If you want to install everything, including tekton-nightly components,
# run this command from the root of the plumbing repo (this requires
# "tekton-nightly" namespace to already be created in your cluster):
kubectl kustomize ./tekton/resources/release | kubectl apply -f -
# If you don't want the tekton-nightly components then run the following
# command from the root of the plumbing repo:
kubectl kustomize ./tekton/resources/release/overlays/default | kubectl apply -f -
Apply the tasks from the pipeline repo:
# Apply the Tasks and Pipelines we use from this repo
kubectl apply -f tekton/publish.yaml
kubectl apply -f tekton/release-pipeline.yaml
# Apply the resources - note that when manually releasing you'll re-apply these
kubectl apply -f tekton/resources.yaml
Tasks and Pipelines from this repo are:
publish.yaml - This Task uses
kaniko to build and
publish base images, and uses
ko to build all of the container images we
release and generate the release.yamlrelease-pipeline.yaml - This Pipeline
uses the
golang
Tasks from the
tektoncd/catalog and
publish.yaml's Task.To connect to the cloud instance and OKE cluster we need the Oracle Cloud CLI client. Install Oracle Cloud CLI from https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm
The next step is to establish connection from the local client to the cloud instance. Login to the Oracle Cloud Console and create a new API key from the user profile.
Follow the steps here: https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm#two
Download a Private Key and Add a new API key as mentioned in the doc. Copy the config file to ~/.oci/config and update the path to the private key file in config.
With this the config is ready for usage by the CLI.
Test the connection by doing a get of the OKE cluster id. Refer here https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.70.0/oci_cli_docs/cmdref/ce.html for the CLI options. Command to create a kubeconfig in your local could be obtained from console navigating to the OKE > Actions > Access Cluster. Run the command pointing to the PUBLIC_ENDPOINT and we should be connected to the cluster.
NOTE: When executing release pipelines, some tasks require OCI CLI commands which need credentials. The OCI credentials secret is already deployed to the dogfooding cluster via terraform and is mounted as a workspace to tasks that require it (such as the precheck task). Release managers do not need to create this secret manually. This is stated here for troubleshooting purposes.
Post-processing services perform post release automated tasks. Today the only
service available collects the PipelineRun logs uploads them to the release
bucket. To use release post-processing services, the PipelineResource in
resources.yaml must be configured with a valid targetURL in the
cloud event PipelineResource named post-release-trigger:
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: post-release-trigger
spec:
type: cloudEvent
params:
- name: targetURI
value: http://el-pipeline-release-post-processing.default.svc.cluster.local:8080 # This has to be changed to a valid URL
The targetURL should point to the event listener configured in the cluster.
The example above is configured with the correct value for the dogfooding
cluster, using the event listener pipeline-release-post-processing.
Some supporting scripts have been written using Python3:
release.yaml files created
by koIn order to run ko, and to be able to use a cluster's default credentials, we
need an image which contains:
kogolang - Required by ko to buildgcloud - Required to auth with default namespace credentialsThe image which we use for this is built from tekton/ko/Dockerfile.
go-containerregistry#383
is about publishing a ko image, which hopefully we'll be able to move it.
The GitHub Actions workflow provides an alternative approach for automated nightly releases with enhanced CI/CD capabilities and better integration with GitHub infrastructure.
The nightly release workflow is triggered daily and uses:
Automated Scheduling:
Multi-mode Operation:
tektoncd/pipeline repository with full release capabilitiesScheduled Release: The workflow runs automatically every night and will create a release if:
Manual Release:
# Trigger via GitHub UI or CLI
gh workflow run nightly-release.yaml \
--field kubernetes_version=v1.33.0 \
--field force_release=true \
--field dry_run=false
Fork Testing: For testing in forks, the workflow automatically:
gs://tekton-releases-nightly-{repo-owner}ghcr.io/{owner}/pipeline/* instead of production registryThe workflow generates:
vYYYYMMDD-{sha7} formatThis approach provides better observability, easier debugging, and more flexible configuration compared to the traditional Tekton-only pipeline approach.