Back to Dapr

Dapr 1.17.13

docs/release_notes/v1.17.13.md

1.18.35.1 KB
Original Source

Dapr 1.17.13

This update contains the following bug fixes:

Stalled workflows permanently stuck after the last workflow worker disconnects

Problem

A workflow that entered the STALLED state (version not available, patch mismatch, or payload size exceeded) could become permanently stuck if the last connected workflow worker disconnected while the workflow was stalled. Reconnecting workers, including workers registering the exact workflow version the stall was waiting for, did not resume it: status queries kept reporting STALLED and the only recovery was restarting daprd. When it occurred, the sidecar logged:

error while disconnecting work item stream: failed to deactivate workflow '<id>': actor is stalled

Impact

Stalling exists precisely so that a workflow survives its workers going away and resumes once capable workers return, most commonly a rolling upgrade where the old application version disconnects and the new version reconnects. You were affected if you use workflow versioning, patching, or a configured --max-body-size, and all workflow workers of an application disconnected while a workflow was stalled; for example during application restarts, rolling upgrades, scale-to-zero, or behind a load balancer with WorkflowsClusteredDeployment enabled.

Root Cause

A stalled workflow actor parks its execution in-process, holding the execution reminder in flight until its context is cancelled, and marks its lock as stalled. When the last workflow worker disconnects, daprd unregisters the workflow actor types and deactivates all workflow actors, but deactivation begins by acquiring the actor's lock, and the lock rejects acquisition while the actor is stalled. Whether the workflow could later recover came down to a race: if the scheduler's reminder stream teardown cancelled the parked execution before deactivation reached the actor, deactivation succeeded and the unacknowledged reminder was redelivered when workers reconnected; if deactivation won the race, it failed with actor is stalled, leaving the actor activated and holding the reminder in flight, so reconnecting workers had nothing to redeliver and the workflow never re-executed.

Solution

Deactivating a stalled workflow actor now wakes the parked execution instead of failing: the held execution returns immediately, leaving the execution reminder unacknowledged, and deactivation completes. When workflow workers reconnect, the reminder is redelivered and the workflow re-executes, resuming and completing once the connected workers satisfy the stall condition (for example the required workflow version is registered, or daprd was restarted with a larger --max-body-size). Recovery from a stall no longer depends on timing, and a daprd restart is no longer required.

Azure component authentication halting at the SPIFFE credential instead of falling back

Problem

Azure (Microsoft Entra ID) components authenticate by trying a chain of credentials in order until one succeeds. When azureClientId and azureTenantId were set, the chain included the SPIFFE workload identity credential, and if no SPIFFE JWT SVID source was available the chain stopped at that step:

ChainedTokenCredential: failed to acquire a token.
Attempted credentials:
	ClientAssertionCredential: failed to get JWT SVID source from context

Credentials later in the chain, such as managed identity or the Azure CLI, were never attempted, so the component failed to authenticate even though a working credential was available.

Impact

You were affected on Dapr 1.16.0 or later (where the SPIFFE credential joined the default chain) in either of these configurations:

  • An Azure component with azureClientId and azureTenantId set but no client secret or certificate, relying on a later credential in the default chain (for example managed identity or the Azure CLI).
  • An explicit azureAuthMethods list placing spiffeworkloadidentity before another method (for example spiffeworkloadidentity,managedidentity), expecting fallback when SPIFFE is not configured.

Root Cause

ChainedTokenCredential only continues past a credential that reports a credentialUnavailableError; any other error is treated as fatal and ends the chain. The SPIFFE credential returned a plain error when the context carried no JWT SVID source, so a missing prerequisite was treated as a fatal authentication failure rather than a signal to try the next credential.

Solution

The SPIFFE credential now reports itself as unavailable when no JWT SVID source is present, before any token request is made, which is exactly what ChainedTokenCredential requires to continue to the next credential in the chain. Both the default chain and explicitly ordered azureAuthMethods lists now fall through as expected, and behavior when a SPIFFE source is configured is unchanged.