Certificate Request CRD

Table of Contents

• Summary
• Motivation
  • Goals
  • Non-Goals
• Proposal
  • API Changes
    • Approved and Denied Conditions
      • Behaviour
      • RBAC
        • Scope
        • API Group
        • Resource
        • Verbs
        • Resource Names
      • Webhook signer string building
      • cert-manager Approver
  • Controller Behaviour
    • ACME
    • Issuing Controller
  • Failure
  • Internal API Resource Behaviour
  • CertificateRequest Annotations
  • Test Plan
  • Risks and Mitigations
  • Graduation Criteria
    • Alpha
    • Alpha -> Beta Graduation
    • Beta -> GA Graduation
    • Removing a deprecated flag
  • Version Skew Strategy

:warning: Parts of this design are out of date with regards to the current implementation.

See also https://cert-manager.io/docs/concepts/certificaterequest/.

Summary

Currently, certificates issued via cert-manager rely on the Certificate resource being reconciled by the Certificate controller. This resource imposes limitations on what issuers are able to honour the Certificate resource as well as other opinionated implementation details.

This proposal adds a new custom resource CertificateRequest that contains an x509 certificate signing request, a target issuer, and other metadata about the request. Each issuer will have their own CertificateRequest controller to watch for resources that are referencing them. The Certificate controller will then rely on creating CertificateRequests to resolve its own Spec.

Motivation

Currently the required use of the Certificate resource means that users are forced to:

• expose the signed certificate's private key to the API server
• be limited to the finite set of issuers as implemented into the cert-manager project, or, "in-tree"
• adhere to the Certificate controller's opinionated implementation, limiting scope for integrations with other projects
• rely on developers of the cert-manager project for reviews and approval of new issuers

Due to these issues, cert-manager can be often unsuitable for some use cases/integrations or users are unsatisfied with some behaviour. Lack of exposure of options that a raw x509 certificate signing request provides can also be a source of frustration.

With cert-manger maintainers ensuring that all issuers are always fully supported and tested, it becomes difficult for new issuers to become accepted. Some developers of new issuers would be happy to maintain these issuers themselves however is not possible with all issuers belonging in the same code base and repository.

Goals

• Introduce the CertificateRequest resource.
• Create a CertificateRequest controller for each in-tree issuer to resolve CertificateRequest.
• Change the implementation of the Certificate controller to rely on the CertificateRequest resource to resolve the request.
• Update documentation detailing this new behaviour and how it can be used to develop out-of-tree implementations of an issuer CertificateRequest controller.
• Create a boilerplate/scaffolding example code to help quick start developers on creating a controller with best practices.

Non-Goals

• This proposal does not document or explore possible or planned integrations using this new functionality.
• This proposal will not investigate possible alignment or merging with the Kubernetes internal CertificateSigningRequest resource. Although it is of interest, the motivation is mostly in order to get a built-in approval workflow for CertificateRequests. The feasibility of being able to implement a solution using the built-in type in the near future however is small, so we'd rather 'trail-blaze' here and then try and fold our changes back upstream at a later date.

Proposal

API Changes

This proposal will create the following new API types in the cert-manager.io group;

// CertificateRequestSpec defines the desired state of CertificateRequest
type CertificateRequestSpec struct {
	// Requested certificate default Duration
	// +optional
	Duration *metav1.Duration `json:"duration,omitempty"`

	// IssuerRef is a reference to the issuer for this CertificateRequest.  If
	// the 'kind' field is not set, or set to 'Issuer', an Issuer resource with
	// the given name in the same namespace as the CertificateRequest will be
	// used.  If the 'kind' field is set to 'ClusterIssuer', a ClusterIssuer with
	// the provided name will be used. The 'name' field in this stanza is
	// required at all times. The group field refers to the API group of the
	// issuer which defaults to 'cert-manager.io' if empty.
	IssuerRef ObjectReference `json:"issuerRef"`

	// Byte slice containing the PEM encoded CertificateSigningRequest
	CSRPEM []byte `json:"csr"`

	// IsCA will mark the resulting certificate as valid for signing. This
	// implies that the 'signing' usage is set
	// +optional
	IsCA bool `json:"isCA,omitempty"`
}

// CertificateStatus defines the observed state of CertificateRequest and
// resulting signed certificate.
type CertificateRequestStatus struct {
	// +optional
	Conditions []CertificateRequestCondition `json:"conditions,omitempty"`

	// Byte slice containing a PEM encoded signed certificate resulting from the
	// given certificate signing request.
	// +optional
	Certificate []byte `json:"certificate,omitempty"`

	// Byte slice containing the PEM encoded certificate authority of the signed
	// certificate.
	// +optional
	CA []byte `json:"ca,omitempty"`

	// FailureTime stores the time that this CertificateRequest failed.
	// This is used to influence garbage collection and back-off.
	// +optional
	FailureTime *metav1.Time `json:"failureTime,omitempty"`
}

The CertificateRequestCondition resembles much the same of the CertificateRequestCondition.

The ObjectReference field type has had a new field Group added as follows:

// ObjectReference is a reference to an object with a given name, kind and group.
type ObjectReference struct {
	Name string `json:"name"`
	// +optional
	Kind string `json:"kind,omitempty"`
	// +optional
	Group string `json:"group,omitempty"`
}

The group refers to the API group that the target Issuer belongs to. This enables namespacing of references to different issuers of external API groups.

Approved and Denied Conditions

The CertificateRequest resource has Approved and Denied conditions. These conditions are based upon the certificates.k8s.io CertificateSigningRequest conditions of the same name. The purpose of these conditions is so that users, or "approvers", are able to mark a request as either Approved or Denied. This condition gates a signer from signing the request. A signer waits for a CertificateRequest to have an Approved condition before signing. A CertificateRequest with a Denied condition will never be signed.

Behaviour

The Approved and Denied conditions are two distinct condition types on the CertificateRequest. These conditions must only have the status of True, and are mutually exclusive (i.e. a CertificateRequest cannot have an Approved and Denied condition simultaneously). This behaviour is enforced in the cert-manager validating admission webhook.

An "approver" is an entity that is responsible for setting the Approved/Denied conditions. It is up to the approver's implementation as to what CertificateRequests are managed by that approver.

The Reason field of the Approved/Denied condition should be set to who set the condition. Who can be interpreted however makes sense to the approver implementation. For example, it may include the API group of an approving policy controller, or the client agent of a manual request.

The Message field of the Approved/Denied condition should be set to why the condition is set. Again, why can be interpreted however makes sense to the implementation of the approver. For example, the name of the resource that approves this request, the violations which caused the request to be denied, or the team to who manually approved the request.

A CertificateRequest that is Denied is considered to be in a final, failed state. If it was created for an issuance of a Certificate, the associated issuance will be failed.

RBAC

Approved and Denied conditions are set by requesting against the /status endpoint of the CertificateRequest resource. This is a divergence of the certificates.k8s.io CertificateSigningRequest resource that utilises approval, as CustomResourceDefinitions may only define /status or /scale sub-resources. Approvers must therefore have permissions to update the status sub-resource of CertificateRequests. Approvers missing this permissions will have their request rejected by the API server.

Setting the Approved or Denied conditions are restricted by the approver having sufficient RBAC permissions. These permissions are based upon the request itself - specifically the request's IssuerRef:

apiGroups: ["cert-manager.io"]
resources: ["signers"]
verbs: ["approve"]
resourceNames:
 # namespaced signers
 - "<signer-resource-types>.<signer-group>/<signer-namespace>.<signer-name>"
 # cluster scoped signers
 - "<signer-resource-types>.<signer-group>/<signer-name>"
 # all signers of this resource name
 - "<signer-resource-types>.<signer-group>/*"

An example ClusterRole that would grant the permissions to set the Approve and Denied conditions of CertificateRequests that reference the cluster scoped myissuers external issuer, in the group my-example.io, with the name myapp:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: my-example-io-my-issuer-myapp-approver
rules:
  - apiGroups: ["cert-manager.io"]
    resources: ["signers"]
    verbs: ["approve"]
    resourceNames: ["myissuers.my-example.io/myapp"]

These permissions are enforced using a SubjectAccessReview that is requested by the cert-manager Webhook component when an approver attempts to set the Approve or Denied condition. The approver UserInfo and IssuerRef fields on the CertificateRequest are used to build the SubjectAccessReview.

If the approver does not have sufficient permissions defined above to set the Approved or Denied conditions, the request will be rejected by the cert-manager validating admission webhook.

Scope

The RBAC permissions must be granted at the cluster scope (i.e. ClusterRoleBinding). This matches the same scoping behaviour as certificates.k8s.io CertificateSigningRequests.

Namespaced signers are represented by a namespaced resource using the syntax of <signer-resource-types>.<signer-group>/<signer-namespace>.<signer-name>

apiGroup

The apiGroup must always be cert-manager.io, as all CertificateRequests are in the cert-manager.io group. Even though some signers may themselves not belong to the cert-manager.io group, the approve operation does.

resource

The resource must always be signers. This matches the same resource as required by certificates.k8s.io CertificateSigningRequests.

verbs

The verbs must always be approve. This verb grants the approver the permissions to set both Approved and Denied conditions.

resourceNames

Each entry of resourceNames is made up of 3 parts: the signer resource type, signer group, and the signer name.

The signer resource type should be the same string of the resource type of the signer.

The signer group should be the same string of the group of signer.

The name may be either a wild card '*' denoting all signer names in all namespaces, or instead a string which must match optionally a namespace, followed by the signer name as it appears in the IssuerRef.

An example of signing all myissuer signers in all namespaces, and clustermyissuers with the name myapp, in the my-example.io group:

    resourceNames:
    # With the following, the user (i.e., the approver) can approve the CertificateRequests
    # that reference any namespaced-scoped issuer of the type MyIssuers, in the
    # group my-example.io, in any namespace with any name:
    - myissuers.my-example.io/*
    # Similar to the above, but any cluster-scoped issuer of the type
    # ClusterMyIssuer:
    - clustermyissuers.my-example.io/myapp

    resourceNames:
    # With the following, the user (i.e., the approver) can only approve the
    # CertificateRequests in the namespace "foo" and that reference the
    # namespace-scoped MyIssuer, in the group my-example.io, named "myapp":
    - myissuers.my-example.io/foo.myapp
    # Similar to the above, but in the namespace "bar":
    - myissuers.my-example.io/bar.myapp

Webhook signer string building

The validating admission webhook is responsible for making SubjectAccessReviews to evaluate whether the approver has sufficient permissions to add the Approved or Denied conditions. The webhook will attempt to first check whether the use can approve all signers in all namespaced (wildcard), and if this fails, whether it can approve for the exact signer (.<signer-name> or <signer-namespace>.<signer-name>).

The webhook will keep a cache of the Discovery API which will be used to determine whether a referenced signer is namespaced or not. If it is namespaced, the <signer-namespace> will be populated with the namespace that the CertificateRequest resides in. If the scope of the resource cannot be determined, the request will be rejected.

cert-manager Approver

The cert-manager controller is deployed with an internal approver controller. This controller will attempt to approve all CertificateRequests that are created, regardless of the contents of the request, or the issuer they reference.

By default, this approver is given the permission to approve all internal signers, that is Issuer and ClusterIssuer cert-manager.io signers. This means its ClusterRole is given the resourceNames: issuers.cert-manager.io/* and clusterissuers.cert-manager.io/*

External issuers may consider whether they wish to install further RBAC that would allow the cert-manager approver controller to also approve CertificateRequests that reference these external issuers. This would involve creating a ClusterRole that permits approving that signer type, and binding that role to the cert-manager-controller ServiceAccount.

The default cert-manager approver controller may be disabled by adding the --controllers=*,-certificaterequests-approver argument to the cert-manager-controller component. This allows for other approvers to make decisions about CertificateRequests, without racing against the internal approver controller.

Controller Behaviour

The philosophy for the CertificateRequest controllers are planned to be as minimal as possible in that the single goal of them is to enable its owning Issuer to create the resulting certificate. Once a sync on a CertificateRequest has been observed, the general flow is as follows:

• Check the group belongs to the owning Issuer, exit if not.

• Check if CertificateRequest is in a terminal failed state. A controller may choose to add additional conditions to a failed CertificateRequest, but must not attempt to issue a certificate. Currently terminal failed states are:

  • Ready condition with a Failed reason // usually set by the issuer
  • InvalidRequest condition with True status // usually set by the issuer
  • Denied condition with True status // usually set by approver

• Check the Issuer type is of the same type, exit if not.

• Verify the Spec of the CertificateRequest.

• If a certificate exits then update the status if needed and exit.

• Sign the certificate via the Issuer using the contents of Spec.

It is worth noting that whether the certificate is invalid, out-of-date or failed then the controller should take no further action on the resource. It is the responsibility of a higher level controller such as the Certificate controller to take further action to retry the certificate issuance through managing the life cycle of the CertificateRequest resources.

With all Issuers updated with CertificateRequest controllers, the Certificate controller will be migrated to begin to use and manage the life cycle CertificateRequests to resolve it's Spec. Further concrete implementation details TBD.

ACME

The CertificateRequest controller is responsible for creating Order resources to fulfil ACME certificate requests. If the Order fails due to a non-networking or other non-transient issue then the Order is marked as failed

• this means the CertificateRequest too shall be marked as failed and no further processing will take place by the CertificateRequest controller on this resource.

Issuing Controller

Issuing controller considers all Denied CertificateRequests to be in a final failed state. The issuance will be failed and will be repeatedly retried with an exponential backoff ../20220118.certificate-issuance-exponential-backoff.md. If the cause of the denial was a misconfigured Certificate spec, the issuance will be retried immediately once the spec is corrected. If the cause of the denial was misconfigured policy resources, a user who has fixed the resources and wants to retry immediately can do so using cmctl renew

The issuing controller does not check Approved condition. It is the issuer's responsibility not to issue certificates for CertificateRequests that have not been approved.

Failure

A CertificateRequest is considered in a final failed state if:

• it has a Ready condition with Failed reason
• it has a Denied condition with True status
• it has InvalidRequest reason with True status

Internal API Resource Behaviour

The group name of IssuerRef inside CertificateRequests is to be defaulted to "cert-manager.io" if the field is empty, using a mutating webhook. This means that if unspecified, CertificateRequest objects will be put into the ownership of the default pool of issuers in the cert-manager project.

Until the mutating webhook is fully implemented, we will handle defaulting internally in the controller.

CertificateRequest Annotations

In order for CertificateRequest controllers to resolve requests, extra information may be needed that is not present in the API Spec. To pass on this information, a set of one or more annotations should be defined, with reliable value pairs. These annotations should be considered optional. Any CertificateRequest controller that relies on these to function should fallback gracefully or be marked as failed in the event a required annotation is missing. The currently defined annotations are:

• cert-manager.io/private-key-secret-name: The name of the secret, in the same namespace as the CertificateRequest, that stores the private key which was used to sign the x509 certificate signing request. This is required by the SelfSigning issuer to sign its own certificate. If this annotation is missing or empty, the SelfSign CertificateRequest controller will mark the resource as failed and no further processing will take place on it. Currently the Certificate controller adds this annotation to all CertificateRequest resources it creates with the defined SecretName in the Spec of the Certificate.

Test Plan

Standard unit and end-to-end tests will be used to verify new behaviour, as used by cert-manager currently. Current end-to-end tests for Certificate resources will also give a good signal for CertificateRequests once the controller has migrated its implementation.

Risks and Mitigations

The introduction and consequently the reliance on this core resource for all cert-manager functions means it poses a high risk to bugs or unexpected behaviour appearing across the whole codebase. With this, it is key to ensure the change happens in incremental roll-outs and proper care is taken during testing.

The new resource could be potentially confusing for current cert-manager users. To mitigate this, proper documentation should be created to explain the changes. It should also be made clear that the resource is typically only to be consumed or managed by a more complex controller or system, not necessarily a human user.

Graduation Criteria

Alpha

• Creation of CertificateRequest resource.
• A CA issuer CertificateRequest controller.
• Exposing the single controller via a feature gated flag.

Alpha -> Beta Graduation

• All issuers have a CertificateRequest controller.
• All controllers are enabled by default.
• The Certificate controller optionally makes use of the CertificateRequest resource to resolve certificates when a feature flag is enabled.

Beta -> GA Graduation

• The CertificateRequest API resource should be considered stable.
• The Certificate controller makes use of the CertificateRequest resource to resolve certificates.

Removing a deprecated flag

Version Skew Strategy