ContextQMD
Back to Argo Cd

v3.3 to 3.4

docs/operator-manual/upgrading/3.3-3.4.md

3.4.07.1 KB
Original Source

v3.3 to 3.4

Breaking Changes

Cluster version format change

New behavior:

3.4 now stores the cluster version in a more detailed format, vMajor.Minor.Patch compared to the previous format Major.Minor.

This change is aligning how ArgoCD interprets K8s cluster version with how Helm 3.19.0 and above interprets it.

This change makes it easier to compare versions and to support future features. It also allows for more accurate version comparisons and better compatibility with future Kubernetes releases.

This change was also back ported to Argo CD 3.3.3.

Impact:

Application Sets with Cluster Generators, that fetch clusters based on their Kubernetes version and use argocd.argoproj.io/auto-label-cluster-info on the cluster secret, need to be updated to use argocd.argoproj.io/kubernetes-version with the vMajor.Minor.Patch format instead of the previous Major.Minor format. More details here.

Additionally, API, UI and CLI commands that retrieve cluster information now return the version in the vMajor.Minor.Patch format.

The env variable $KUBE_VERSION that is used with Argo CD CMP Plugins remains unchanged and returns the version in Major.Minor.Patch format, so CMP Plugins are not impacted.

Behavioral Improvements / Fixes

Applications with Missing health status

The behavior of Application health status has changed to be more consistent and informative. Previously, Applications would show Missing health status inconsistently depending on whether missing resources had built-in or custom health checks defined.

New behavior:

  • Applications now show Missing health only when ALL resources are missing (e.g., before the first sync)
  • Individual missing resources no longer affect the Application's overall health status
  • The health status now reflects the aggregated health of existing resources
  • The OutOfSync status already indicates when resources are missing, making the health status redundant for individual missing resources
  • If the defined resource health check is explicitly returning Missing for an existing resource, that will still be reflected in the overall Application health

Impact:

  • Applications with some missing resources will now show the health of their existing resources (e.g., Healthy, Progressing, Degraded) instead of Missing
  • Automation relying on the Application Health status to detect missing resources should now check the Sync status for OutOfSync instead, and optionally inspect individual resource health if needed.
  • Users can now distinguish between an Application that has never been synced (all resources missing = Missing health) vs. an Application with some resources deleted (shows health of remaining resources)

gRPC Service Config DNS Lookups Disabled by Default

ArgoCD components now disable gRPC service config lookups via DNS TXT records by default to prevent excessive DNS queries and timeouts in dual-stack (IPv4+IPv6) Kubernetes environments.

Background:

gRPC clients by default attempt to discover service configuration by querying DNS TXT records for _grpc_config.<hostname>. In dual-stack environments, these lookups can result in excessive DNS queries and timeouts, causing repo-server crashes and sync failures.

New behavior:

  • The environment variable GRPC_ENABLE_TXT_SERVICE_CONFIG is now set to false by default for all ArgoCD components
  • This prevents gRPC from attempting DNS TXT record lookups for service configuration
  • Most users do not use DNS TXT records for gRPC service configuration

Impact:

  • Positive: Eliminates excessive DNS queries in dual-stack environments, preventing timeouts and improving reliability
  • Minimal: The vast majority of users do not use DNS TXT records for gRPC service configuration and will see no functional change
  • Re-enablement: Users who do rely on gRPC service config via DNS TXT records can re-enable this feature by setting the parameter controller.grpc.enable.txt.service.config: "true" in the argocd-cmd-params-cm ConfigMap

Example to re-enable (if needed):

yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cmd-params-cm
  namespace: argocd
data:
  controller.grpc.enable.txt.service.config: "true"

Related Issue

https://github.com/argoproj/argo-cd/issues/24991

More detailed cluster version

3.4.0 now stores the cluster version in a more detailed format, Major.Minor.Patch compared to the previous format Major.Minor. This change is to make it easier to compare versions and to support future features. This change also allows for more accurate version comparisons and better compatibility with future Kubernetes releases.

Users will notice it in the UI and the CLI commands that retrieve cluster information.

API Changes

<!-- Changes in API surface: Any removal of fields or changed API defaults -->

Security Changes

<!-- Changes in security behavior: Any removal of sensitive fields or changed authentication defaults -->

Deprecated Items

<!-- Flags / behavior to be removed in future releases Any deprecated config API or CLI flags -->

Kustomize Upgraded

<!-- Bundled Kustomize version bump: Any breaking behavior Notes about specific upstream behavior changes (e.g., namespace propagation fixes). -->

Helm Upgraded

<!-- Bundled Helm version bump: Any breaking behavior Verify if your charts depend on any features tied to Helm versions. -->

Dex Upgraded

Argo CD v3.4 upgrades the bundled Dex version to 2.45.0. There are a few things to keep in mind, according to the release notes.

  • Gomplate is used internally, but users that have configured Dex for preprocessing may be impacted.
  • The ContinueOnConnectorFailure feature flag is now enabled by default. To disable it, set the following argocd-cmd-params-cm ConfigMap parameter dexserver.connector.failure.continue to false.

go-oidc Upgraded

go-oidc v3.17.0 changes the order of steps for token verification.

Previous order:

parse claims → signature

Updated order:

signature → parse claims

Possible User Impact and Mitigation

For a token that is both expired and signed by a key that has since removed from JWKS due to rotation, the signature verification failure will not redirect the user to SSO. Instead the user would be redirected to the login page and would need to manually initiate the SSO login.

This scenario can be avoided by implementing the provider best practice of serving previously used keys for a grace period which exceeds the token lifetime.

Open Telemetry Upgraded

Argo CD v3.4 upgrades go.opentelemetry.io/otel to 1.42.0, with semantic conventions v1.40.0 release. Please see semantic conventions release notes for a list of changed metric names and dropped attributes, which may require changes in your Argo CD Grafana dashboards/alerts based on OTEL.

Custom Healthchecks Added