content/v2.0/operations/operation.md
An Operation runs a function pipeline once to completion to perform operational
tasks that don't fit the typical resource creation pattern. Unlike compositions
that continuously reconcile desired state, Operations focus on tasks like
backups, rolling upgrades, configuration validation, and scheduled maintenance.
Operations are like Kubernetes Jobs - they run once to completion rather than continuously reconciling. Like compositions, Operations use function pipelines to implement their logic, but they're designed for operational workflows instead of resource composition.
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: backup-database
spec:
mode: Pipeline
pipeline:
- step: create-backup
functionRef:
name: function-database-backup
input:
apiVersion: fn.crossplane.io/v1beta1
kind: DatabaseBackupInput
database: production-db
retentionDays: 30
When you create this Operation, Crossplane:
{{<hint "important">}}
Operations are an alpha feature. You must enable them by adding
--enable-operations to Crossplane's arguments.
{{</hint>}}
Operations and compositions both use function pipelines, but with important differences:
Composition Functions:
Operation Functions:
Functions can support both modes by declaring the appropriate capabilities in
their package metadata. Function authors declare this in the crossplane.yaml
file when building the function package:
apiVersion: meta.pkg.crossplane.io/v1
kind: Function
metadata:
name: my-function
spec:
capabilities:
- composition
- operation
This allows Crossplane to know which modes the function supports and avoid trying to use a composition-only function for operations.
{{<hint "note">}} The following examples use hypothetical functions for illustration. At launch, only function-python supports operations. {{</hint>}}
Use Operations for controlled rolling upgrades:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: cluster-upgrade
spec:
mode: Pipeline
pipeline:
- step: rolling-upgrade
functionRef:
name: function-cluster-upgrade
input:
apiVersion: fn.crossplane.io/v1beta1
kind: ClusterUpgradeInput
targetVersion: "1.28"
batches: [0.25, 0.5, 1.0] # 25%, 50%, then 100%
healthChecks: [Synced, Ready]
Use Operations for specific maintenance tasks:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: certificate-rotation
spec:
mode: Pipeline
pipeline:
- step: rotate-certificates
functionRef:
name: function-cert-rotation
input:
apiVersion: fn.crossplane.io/v1beta1
kind: CertRotationInput
targetCertificates:
matchLabels:
rotate: "true"
Operations automatically retry when they fail. Configure the retry limit to control how often attempts occur:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: resilient-operation
spec:
retryLimit: 10 # Try up to 10 times before giving up (default: 5)
mode: Pipeline
pipeline:
- step: flaky-task
functionRef:
name: function-flaky-task
input:
apiVersion: fn.crossplane.io/v1beta1
kind: FlakyTaskInput
# Task that might fail due to temporary issues
timeout: "30s"
Retry behavior:
status.failuresretryLimit, the Operation becomes
Succeeded=FalseOperations can provide credentials to functions through Secrets:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: secure-backup
spec:
mode: Pipeline
pipeline:
- step: backup-with-credentials
functionRef:
name: function-backup
credentials:
- name: backup-creds
source: Secret
secretRef:
namespace: crossplane-system
name: backup-credentials
key: api-key
- name: database-creds
source: Secret
secretRef:
namespace: crossplane-system
name: database-credentials
key: connection-string
input:
apiVersion: fn.crossplane.io/v1beta1
kind: BackupInput
destination: s3://my-backup-bucket
Complex operations can use multiple pipeline steps:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: multi-step-deployment
spec:
mode: Pipeline
pipeline:
- step: validate-config
functionRef:
name: function-validator
input:
apiVersion: fn.crossplane.io/v1beta1
kind: ValidatorInput
configName: app-config
- step: backup-current
functionRef:
name: function-backup
input:
apiVersion: fn.crossplane.io/v1beta1
kind: BackupInput
target: current-deployment
- step: deploy-new-version
functionRef:
name: function-deploy
input:
apiVersion: fn.crossplane.io/v1beta1
kind: DeployInput
image: myapp:v2.0.0
strategy: rollingUpdate
- step: verify-health
functionRef:
name: function-health-check
input:
apiVersion: fn.crossplane.io/v1beta1
kind: HealthCheckInput
timeout: 300s
healthEndpoint: /health
If your Operation needs to access resources that Crossplane doesn't have permissions for by default, create a ClusterRole that aggregates to Crossplane:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: operation-additional-permissions
labels:
rbac.crossplane.io/aggregate-to-crossplane: "true"
rules:
# Additional permissions for Operations
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "list", "patch", "update"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list"]
# Add other resources your Operations need to access
This ClusterRole is automatically aggregated to Crossplane's main ClusterRole, giving the Crossplane service account the permissions needed for your Operations.
{{<hint "note">}} The [RBAC manager]({{<ref "../guides/pods#rbac-manager-pod">}}) automatically grants Crossplane access to Crossplane resources (MRs, XRs, etc.). You only need to create more ClusterRoles for other Kubernetes resources that your Operations need to access.
For more details on RBAC configuration, see the [Compositions RBAC documentation]({{<ref "../composition/compositions#grant-access-to-composed-resources">}}). {{</hint>}}
{{<hint "note" >}}
Function response caching is an alpha feature. Enable it by setting the
--enable-function-response-cache feature flag.
{{< /hint >}}
Operations can use function response caching to improve performance for operations that:
The cache works the same way as for Compositions - function responses with time to live values cache and reuse identical requests until they expire.
Function response caching helps Operations that:
For cache configuration details, see the [Function response cache documentation]({{<ref "../composition/compositions#function-response-cache">}}).
Operations can preload resources for functions to access:
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: resource-aware-operation
spec:
mode: Pipeline
pipeline:
- step: process-deployment
functionRef:
name: function-processor
requirements:
requiredResources:
- requirementName: app-deployment
apiVersion: apps/v1
kind: Deployment
name: my-app
namespace: production
- requirementName: app-service
apiVersion: v1
kind: Service
name: my-app-service
namespace: production
input:
apiVersion: fn.crossplane.io/v1beta1
kind: ProcessorInput
action: upgrade
Functions access these resources through the standard request structure:
from crossplane.function import request, response
def operate(req, rsp):
# Access required resources
deployment = request.get_required_resource(req, "app-deployment")
service = request.get_required_resource(req, "app-service")
if not deployment or not service:
response.set_output(rsp, {"error": "Required resources not found"})
return
# Process the resources
new_replicas = deployment["spec"]["replicas"] * 2
# Return updated resources with full GVK and metadata for server-side apply
rsp.desired.resources["app-deployment"].resource.update({
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"name": deployment["metadata"]["name"],
"namespace": deployment["metadata"]["namespace"]
},
"spec": {"replicas": new_replicas}
})
Operations provide rich status information:
status:
conditions:
- type: Synced
status: "True"
reason: ReconcileSuccess
- type: Succeeded
status: "True"
reason: PipelineSuccess
- type: ValidPipeline
status: "True"
reason: ValidPipeline
failures: 1 # Number of retry attempts
pipeline:
- step: create-backup
output:
backupId: "backup-20240115-103000"
size: "2.3GB"
appliedResourceRefs:
- apiVersion: "v1"
kind: "Secret"
namespace: "production"
name: "backup-secret"
- apiVersion: "apps/v1"
kind: "Deployment"
name: "updated-deployment"
Key status fields:
conditions: Standard Crossplane conditions (Synced) and Operation-specific conditions:
Succeeded: True when the operation completed successfully, False when it failedValidPipeline: True when all functions have the required operation capabilityfailures: Number of times the operation has failed and retriedpipeline: Output from each function step for tracking progressappliedResourceRefs: References to all resources the Operation created or modifiedOperations emit Kubernetes events for important activities:
Check operation status:
kubectl get operation my-operation -o wide
View detailed information:
kubectl describe operation my-operation
Common failure scenarios:
Operations do nothing - Operations feature not enabled:
# Operation exists but has no status conditions and never progresses
status: {}
Solution: enable Operations by adding --enable-operations to Crossplane's startup arguments.
ValidPipeline condition is False - Function doesn't support operations:
conditions:
- type: ValidPipeline
status: "False"
reason: InvalidFunctionCapability
message: "Function function-name doesn't support operations"
Solution: use a function that declares operation capability.
Succeeded condition is False - Function run failed:
conditions:
- type: Succeeded
status: "False"
reason: PipelineFailure
message: "Function returned error: connection timeout"
Solution: view function logs and fix the underlying issue.
Resource apply failures - View events for details:
kubectl get events --field-selector involvedObject.name=my-operation
Debug function runs:
# View function logs
kubectl logs -n crossplane-system deployment/function-python
# Check operation events
kubectl get events --field-selector involvedObject.kind=Operation
# Inspect operation status in detail
kubectl get operation my-operation -o jsonpath='{.status.pipeline}' | jq '.'
Operations can create or change any Kubernetes resources using server-side apply with force ownership. This means:
What Operations can do:
What Operations can't do:
{{<hint "important">}} Use caution with Operations that change resources managed by other controllers. Operations force ownership when applying changes, which can cause conflicts. {{</hint>}}
You can preview the output of any Operation using the Crossplane CLI. You don't need a Crossplane control plane to do this. The Crossplane CLI uses Docker Engine to run functions.
{{<hint "tip">}} See the [Crossplane CLI docs]({{<ref "../cli">}}) to learn how to install and use the Crossplane CLI. {{< /hint >}}
{{<hint "important">}}
Running crossplane alpha render op requires Docker.
{{< /hint >}}
Provide an operation, composition functions, and any required resources to render the output locally.
crossplane alpha render op operation.yaml functions.yaml --required-resources=ingress.yaml
crossplane alpha render op prints the Operation status and any resources the
operation functions created or modified. It shows what would happen if you
applied the Operation to a cluster.
---
# Operation status showing function results
apiVersion: ops.crossplane.io/v1alpha1
kind: Operation
metadata:
name: ingress-cert-monitor
status:
conditions:
- type: Succeeded
status: "True"
reason: PipelineSuccess
pipeline:
- step: check-ingress-certificate
output:
certificateExpires: "Sep 29 08:34:02 2025 GMT"
daysUntilExpiry: 53
hostname: google.com
ingressName: example-app
status: ok
---
# Modified Ingress resource with certificate annotations
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-monitor.crossplane.io/expires: Sep 29 08:34:02 2025 GMT
cert-monitor.crossplane.io/days-until-expiry: "53"
cert-monitor.crossplane.io/status: ok
name: example-app
namespace: default
spec:
# ... ingress spec unchanged
Use --required-resources to provide resources that your operation functions
need access to. You can specify multiple files or use glob patterns:
# Multiple specific files
crossplane alpha render op operation.yaml functions.yaml \
--required-resources=deployment.yaml,service.yaml,configmap.yaml
# Glob pattern for all YAML files in a directory
crossplane alpha render op operation.yaml functions.yaml \
--required-resources="resources/*.yaml"
{{<hint "tip">}}
Use the crossplane alpha render op command to test your Operations locally
before deploying them to a cluster. The command helps validate function logic
and required resource access patterns.
{{</hint>}}
operation capability in
function metadata to enable Operations support