This guide is for running Vereda in production: sizing its knobs, reading its runtime state, and shutting it down cleanly. For the API itself, see the README, the feature guides, and the API reference.
Vereda has two independent concurrency limits:
concurrency, default 50) — a semaphore shared across every partition. It bounds total in-flight executions (first attempts and retries alike) against your process, regardless of how many hosts you talk to. Up to maxQueueSize (default 100) more may wait for a permit; beyond that a request resolves with QueueFullError (scope: "global").partitions[name].concurrency, default 5) — how many retries may run at once for a single partition (host). partitions[name].maxQueueSize (default 100) bounds how many more retries may wait behind that limit before a request is rejected with QueueFullError.The per-partition limit applies only to retries: the first attempt for every request skips the partition bulkhead — this is deliberate (see Design philosophy in the README): a struggling downstream should throttle its own retry traffic, not new work. The global limit applies to every attempt, so a host that fails slowly can hold global permits and delay or reject requests to healthy hosts under enough load; keep attemptMs short for such hosts, or set limitFirstAttempts.
Ballpark sizing:
concurrency close to what the downstream service can actually sustain concurrently (its own connection pool, rate limit, or known capacity). Too high and a struggling host's retries pile pressure back onto it; too low and legitimate retry traffic queues behind slow ones unnecessarily.maxQueueSize based on how much retry backlog is acceptable before you'd rather fail fast. A small queue (10–50) surfaces backpressure quickly; a large one absorbs bursts but risks stale-by-the-time-they-run retries.concurrency as a process-wide ceiling — the sum of what you're willing to have in flight across all partitions at once, informed by your own outbound connection limits or memory budget, not by any single downstream's capacity.defaultPartition instead of listing each one. It applies to every partition not named in partitions, with the same fields (concurrency, maxQueueSize, retry, timeout, circuitBreaker, retryBudget, rateLimit, limitFirstAttempts). A listed partition doesn't inherit from it, so a listed entry is the whole config for that host on top of the client-level settings.ratio is the extra load you'll accept during an outage (0.1 = 10%). Keep windowMs well above your backoff delays so retries are counted against the same window as the traffic that funded them. Lower minRetriesPerSec for hosts that see only a few requests per second, or the floor will admit more retries than the ratio would. Watch vereda.retry_budget_exhausted: a steady rate of refusals means the host is degraded, not just having an occasional failure.429 stays the exception. Set maxWaitMs below timeout.totalMs when you'd rather fail fast with RateLimitedError than spend most of a request's deadline waiting for a token. A rate limit applies per process: with N replicas calling the same host, divide the quota by N.partition.limitFirstAttempts: true only if a partition's downstream is fragile enough that even first attempts need throttling (R6). Off by default because it changes latency for fresh traffic, not just retries.When a partition fills up, client.partitions() (below) is how you observe it before it becomes an incident.
attemptMs vs. totalMsTwo different timeouts. attemptMs is required on the client-level timeout (pass Infinity to opt out explicitly); totalMs is optional and disabled by default:
timeout.attemptMs — a per-attempt deadline. Each individual attempt (the first one and every retry) is aborted if it runs longer than this, and the result folds into the normal retry loop as a TimeoutError (retried like any other transient failure).timeout.totalMs — a whole-ticket deadline, starting when request()/get()/etc. is called. It covers time spent queued, sleeping in backoff, and executing — everywhere. On expiry the ticket resolves with DeadlineExceededError, a terminal error; no further retries happen even if attempts remain. A retry whose delay would run to or past the deadline isn't slept on: the ticket fails right away with the same error, with the last attempt's error as cause. Only the delay is compared, not delay + attemptMs, because attemptMs is an upper bound and a retry that answers quickly can still fit.Use attemptMs to bound a single slow call (protects you from a hung connection). Use totalMs to bound how long a caller is willing to wait for an eventual answer at all, independent of how many retries that took. They compose: a request can time out on an individual attempt several times and still be retried, right up until totalMs cuts it off entirely.
A reasonable starting point: set attemptMs to your p99 expected latency for a healthy call, and totalMs to the longest you're willing to make the caller wait end-to-end (factoring in maxRetries × backoff).
Both deadlines also bound reading the body of a Response Vereda hands back unread — a success with neither parse nor responseType, or an HttpError's .response — capped at whichever is sooner, measured from that attempt's start: the remaining attemptMs, or the ticket's totalMs. This matches the bound a parsed body (or one read via responseType) already gets for free: it's read, and parse run, inside the attempt; without it, a server that sends headers then stalls the body would let raw.json() or error.response.text() hang past both deadlines. Whichever bound fires aborts the read with Vereda's own error as the rejection — a TimeoutError when attemptMs fired first (or was the only bound), a DeadlineExceededError when totalMs fired first — so the rejection is instanceof the right class and .kind-narrowable, same as any other error the client produces. If you need to stream or read a body later than that — long-lived downloads, deferred processing — set timeout: { attemptMs: Infinity } for that request (and no totalMs), or bring your own AbortSignal via options.signal instead.
partitions()const snapshots = client.partitions();
// [{ partition: "api.example.com", running: 2, queued: 5, concurrency: 5, maxQueueSize: 100, tokens: 3 }, ...]
One entry per partition that has seen a request. Each entry also carries name, the same value as partition; it is deprecated and goes away in 2.0. tokens, present only for a partition with a rateLimit, is how many tokens are free right now; 0 with requests stalling means that partition is held back by its rate limit, not by concurrency. running and queued are point-in-time counts of retry traffic only (first attempts never appear here, by design). Poll this on an interval and feed it into your metrics sink's gauges, or check it directly when debugging why a specific host's requests seem stuck — queued near maxQueueSize means that partition is saturated and new retries for it will start failing with QueueFullError.
circuits()const circuits = client.circuits();
// [{ partition: "api.example.com", state: "open", failures: 5, openedAt: 1767225600000, nextAttemptAt: 1767225630000 }, ...]
One entry per partition with an enabled circuit breaker that has seen a request; [] when no breaker is configured. Pair it with the circuitOpen/circuitClose events: the events tell you when a circuit changes, and the snapshot tells you where every circuit is right now (useful for a health endpoint). An open entry whose nextAttemptAt is in the past hasn't had a request since its reset timeout ran out. The next request becomes the half-open trial.
Implement MetricsSink (counter, histogram, gauge — all synchronous, non-blocking) and pass it as metrics in ClientConfig. Vereda emits the series in METRICS (src/core/metrics.ts), exported so you don't have to hardcode the string names:
| Series | Type | Tags | Meaning |
|---|---|---|---|
METRICS.REQUESTS (vereda.requests) |
counter | partition, method |
One per request initiated |
METRICS.RETRIES (vereda.retries) |
counter | partition, kind |
One per retry attempt, tagged by the error kind that triggered it |
METRICS.DURATION (vereda.duration_ms) |
histogram | partition (absent only when the URL couldn't be resolved), kind (success, cancelled, or the error kind) |
Total ticket duration at settlement |
METRICS.QUEUE_DEPTH (vereda.queue_depth) |
gauge | partition |
Current per-partition queue size — same retries-only caveat as partitions() above: a partition only shows queued first attempts if limitFirstAttempts is enabled for it |
METRICS.GLOBAL_QUEUE_DEPTH (vereda.global_queue_depth) |
gauge | — | Callers currently waiting for a permit under the global concurrency cap (D1) — the signal that you're throttled by concurrency/ClientConfig.concurrency rather than downstream latency |
METRICS.IN_FLIGHT (vereda.in_flight) |
gauge | — | Tickets started but not yet finished, across all partitions. Reported when a ticket starts and when it finishes, so it returns to 0 when the client is idle |
METRICS.CIRCUIT_OPEN (vereda.circuit_open) |
counter | partition |
One per circuit breaker trip to open (opt-in feature; silent unless circuitBreaker.enabled) |
METRICS.RETRY_BUDGET_EXHAUSTED (vereda.retry_budget_exhausted) |
counter | partition |
One per retry the partition's retry budget refused (opt-in feature; silent unless retryBudget is set) |
For OpenTelemetry, otelMetricsSink from @vereda/http/otel is a ready-made MetricsSink: counters and the duration histogram map directly, and gauges become observable gauges reporting the latest value per tag set. See the OpenTelemetry section of the Observability guide, which also covers tracing. Lifecycle events (client.on("request" | "retry" | "success" | "failure" | "cancelled" | "retryBudgetExhausted" | "circuitOpen" | "circuitClose", ...)) are the complementary hook for structured logging or alerting rather than metrics — see Lifecycle events.
QUEUE_DEPTH/GLOBAL_QUEUE_DEPTH are polled, not pushed on change — they're read at request-start and at each ticket's terminal event, not at the moment a caller actually joins or leaves the wait queue. That reliably shows a sustained backlog (a dashboard/alert on "depth > N for 5 minutes" works as expected), but a single request that's briefly queued and released between those two polling points can be invisible to the gauge. For guaranteed per-request visibility regardless of backlog size, use queuedMs on the lifecycle events instead — it's measured at the point of acquisition, not polled.
await client.close({ drain: true, timeoutMs: 30_000 });
close() with no drain (or drain: false) cancels every in-flight ticket immediately and rejects new requests with ConfigurationError("client closed"). Use this for a hard, fast shutdown (e.g. SIGKILL-adjacent paths where you just need the process to exit without lingering timers).close({ drain: true, timeoutMs }) waits for in-flight tickets to settle naturally, up to timeoutMs (a finite number up to 2,147,483,647 ms), then cancels whatever's left. Use this on graceful shutdown (SIGTERM handler) so requests that are about to succeed get the chance to.Either way, all internal timers (backoff sleeps, attemptMs/totalMs deadlines) are unref()ed from the moment they're created, so a Vereda client never by itself keeps the Node event loop alive — close() is about resolving in-flight tickets cleanly, not about unblocking process exit.
Call close() once per client, before the process exits, in whichever shutdown handler your deployment already uses. Extra calls are harmless: a close() that arrives while a drain is in progress returns the same promise, so it resolves only when the shutdown actually finishes. close({ drain: true }) without a finite timeoutMs between 1 and 2,147,483,647 ms (the largest delay a timer honours, about 24.8 days) rejects with a ConfigurationError and leaves the client open.
redactQuery defaults to true. With it on, every URL that appears in a lifecycle event, is passed to your logger, or is embedded in an error (TimeoutError/DeadlineExceededError's message and url, and so MaxRetriesExceededError.message too) has its query values replaced with [redacted] (keys are preserved, so ?token=abc123 becomes ?token=[redacted]), and any userinfo credentials (https://user:pass@host) replaced with [redacted]@. This covers the common case of API keys, session tokens, or PII passed in the URL. Request/response headers are never logged or emitted by Vereda regardless of this setting, so an Authorization header is never at risk of leaking into logs via the library itself.
Redaction only touches what Vereda itself surfaces (event payloads, log calls, error messages). The bundled requestLogger() middleware redacts the same way by default. Middleware can't see the client's config, so pass requestLogger({ redactQuery: false }) to turn it off there too. If your own middleware logs ctx/options directly, or the raw Response (including HttpError.response.url) or request body contains sensitive data, redact that yourself before logging it. redactUrl is exported from @vereda/http for that.
Set redactQuery: false only in trusted environments (e.g. local development against a mock server) where seeing full URLs in logs is worth more than the leak risk.