Dynamic Retry-After Computation Design

Overview

This document describes the design for computing dynamic Retry-After values based on queue state, drain rate, and request processing stage. The goal is to provide clients with intelligent polling intervals that adapt to system load.

Queue Architecture

The relayer has different queue structures per request type:

Input Proof (Single Queue)

[HTTP] → [TX Throttler Queue] → [Gateway TX]
              ↑
         TPS-based drain (per_seconds)

User Decrypt / Public Decrypt (Dual Queue)

[HTTP] → [Readiness Queue] → [Readiness Check] → [TX Throttler Queue] → [Gateway TX]
              ↑                                          ↑
    Concurrency-based drain                       TPS-based drain
       (max_concurrency)                          (per_seconds)

Key differences:

• TX Throttler: Rate-limited via TPS (tokens per second) using governor
• Readiness Queue: Concurrency-limited via semaphore (max_concurrency parallel tasks)

Configuration

Core Parameters

Nominal Processing Times

All nominal times are required in configuration (no code defaults):

Copro/KMS Backoff Intervals

For ReceiptReceived state only (Copro/KMS response time is unpredictable):

Variables

ETA Computation Formulas

Input Proof

Decrypt (User & Public)

Key Points

1. Queued: Uses request's actual position p in the queue (not total queue size)
2. Processing for Decrypt: Check which queue the request is in:
   • readiness_throttler.get_position(id) returns None → removed from readiness, check TX queue
   • tx_throttler.get_position(id) returns Some(p) → use TX queue formula
   • Both return None → out of readiness, not yet in TX queue
3. TxInFlight: Uses processing time P (time to get response after TX sent)
4. ReceiptReceived: Uses backoff B(E) since Copro/KMS response time is unpredictable

Example Calculations

Using parameters: D=10, C=50, R=2s, P_input=2s, P_decrypt=4s, T=100ms, M=0.2, B(E)=3s

Input Proof (P = 2000ms)

Queued / Processing: ⌈(p/D × 1000 + P + T) × (1+M) / 1000⌉

TxInFlight: ⌈P × 1.2 / 1000⌉

= ⌈2000 × 1.2 / 1000⌉ = 3s (constant)

ReceiptReceived: B(E) = 3s (constant)

Decrypt (P = 4000ms)

Queued (in readiness): ⌈(p/C × 1000 + Q/D × 1000 + P + T) × (1+M) / 1000⌉

Assuming p = Q (same number of entries in both queues):

Processing (out of readiness, not in TX): ⌈(R + Q/D × 1000 + P + T) × (1+M) / 1000⌉

Processing (in TX queue): ⌈(p/D × 1000 + P + T) × (1+M) / 1000⌉

TxInFlight: ⌈P × 1.2 / 1000⌉

= ⌈4000 × 1.2 / 1000⌉ = 5s (constant)

ReceiptReceived: B(E) = 3s (constant)

Summary Table (p=100, Q=100)

Response Format

POST Response (202 Accepted)

HTTP/1.1 202 Accepted
Retry-After: 27

{"status": "queued", "job_id": "...", "eta_seconds": 27}

GET Response (202 In Progress)

HTTP/1.1 202 Accepted
Retry-After: 10

{"status": "queued", "state": "tx_in_flight", "eta_seconds": 10, "elapsed_seconds": 15}

Admin Configuration

All parameters are runtime-updatable via admin endpoints:

• Nominal processing times per request type
• TX throttler TPS (drain rate)
• Retry-after bounds (min/max)
• Safety margin
• Copro/KMS backoff intervals

Design Rationale

Why ReceiptReceived uses fixed backoff (no safety margin):

• Copro/KMS response time is fundamentally unpredictable
• Backoff intervals are already conservative by design
• Adding margin would just increase polling delay unnecessarily

Why milliseconds internally:

• All internal calculations use milliseconds to avoid rounding errors
• Conversion to seconds only happens when setting the Retry-After header

Why position-based instead of queue-size-based:

• For GET requests polling an existing Queued request, using total queue size is incorrect
• A request that's been waiting and is now at position 5 should get a shorter ETA than a new request at position 100
• Using get_position(id) provides accurate estimates as the request advances through the queue