Retry with backoff is a core resilience pattern: instead of hammering a failing dependency with constant retries, you retry a limited number of times and increase the delay between attempts, often with randomness (jitter), to give the system time to recover. In modern Java microservices, this is as fundamental as timeouts and circuit breakers, and you should treat it as part of your basic “failure budget” design rather than an afterthought.
Why Simple Retries Are Not Enough
If you just “try again” immediately on failure, you run into two systemic issues:
- You amplify load on an already unhealthy dependency, potentially turning a small blip into an outage (classic retry storm or thundering herd).
- You synchronize client behavior: thousands of callers that fail at the same time also retry at the same time, causing periodic waves of load.
Backoff addresses these issues by spreading retries out over time and giving downstream systems breathing room, while still masking short transient failures from end users.
The Core Concept of Backoff
At its heart, retry with backoff is just a loop with three key decisions:
- Should I retry this failure? (Is it transient and safe to repeat?)
- How many times will I retry at most?
- How long will I wait before the next attempt?
Retryable vs non-retryable failures
You normally only retry failures that are likely transient or environmental:
- HTTP: 429, 503, 504, and connection timeouts are typical candidates.
- TCP / OS:
ECONNRESET,ETIMEDOUT,ECONNREFUSED, etc., often indicate temporary network issues.
You usually do not retry:
- Client bugs: 400, 401, 403, validation errors, malformed requests.
- Irreversible business errors, like “insufficient funds”.
The rationale is simple: retried non-transient errors only add load and latency without any chance of success.
Backoff Strategies (Fixed, Exponential, Jitter)
Several backoff strategies are used in practice; the choice affects both user latency and system stability.
Fixed backoff
You wait the same delay before each retry (for example, 1 second between attempts).
- Pros: Simple to reason about.
- Cons: Poor at protecting an overwhelmed dependency; many clients still align on the same intervals.
Exponential backoff (with optional cap)
You grow delays multiplicatively:
- Example: base 200 ms, factor 2 → 200 ms, 400 ms, 800 ms, 1600 ms, … up to some cap (for example 30 s).
This reduces pressure quickly as failures persist, but may produce very long waits unless you cap the maximum delay.
Exponential backoff with jitter
Large-scale systems (AWS and others) recommend adding randomness to each delay, typically “full jitter” where you wait a random time between 0 and the current exponential delay.
- This breaks synchronization between many clients and avoids retry waves.
- Conceptually: delay_n = random(0, min(cap, baseDelay × factor^n)).
From a system-design perspective, exponential backoff with jitter is the default you should reach for in distributed environments.
Design Parameters You Must Choose
When you design a retry-with-backoff policy, decide explicitly:
- Max attempts: How many retries are acceptable before surfacing failure? This is a user-experience vs resilience trade-off.
- Total time budget: How long are you willing to block this call in the worst case? This should be consistent with your higher-level SLAs and timeouts.
- Base delay: The initial wait, often 50–200 ms for low-latency calls or higher for heavily loaded services.
- Multiplier: The growth factor, often between 1.5 and 3; higher factors reduce load faster but increase tail latency.
- Maximum delay (cap): To prevent absurd waits; typical caps are in the 5–60 s range depending on context.
- Jitter mode: Full jitter is usually preferred; “no jitter” is only acceptable when you have few clients.
You should also define per-operation policies: a read-heavy, idempotent query can tolerate more retries than a rare, expensive write.
Java Example: Simple HTTP Client with Exponential Backoff and Jitter
Below is an example using Java 21’s HttpClient and virtual threads. It implements:
- Exponential backoff with full jitter
- A simple notion of retryable HTTP status codes
- A hard cap on attempts and delay
Code
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.Random;
private static final HttpClient CLIENT = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(5))
.build();
private static final Random RANDOM = new Random();
// Policy parameters
private static final int MAX_ATTEMPTS = 5;
private static final long BASE_DELAY_MS = 200; // initial delay
private static final double MULTIPLIER = 2.0; // exponential factor
private static final long MAX_DELAY_MS = 5_000; // cap per attempt
void main() {
String url = "https://httpbin.org/status/503"; // change to /status/200 to see success
try {
String body = getWithRetry(url);
System.out.println("Final response body: " + body);
} catch (Exception e) {
System.err.println("Request failed after retries: " + e.getMessage());
}
}
public static String getWithRetry(String url) throws Exception {
int attempt = 0;
while (true) {
attempt++;
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(url))
.GET()
.timeout(Duration.ofSeconds(3))
.build();
try {
HttpResponse<String> response =
CLIENT.send(request, HttpResponse.BodyHandlers.ofString());
int status = response.statusCode();
if (!isRetryableStatus(status)) {
// Either success or a non-transient error: stop retrying
if (status >= 200 && status < 300) {
return response.body();
}
throw new RuntimeException("Non-retryable status: " + status);
}
if (attempt >= MAX_ATTEMPTS) {
throw new RuntimeException(
"Exhausted retries, last status: " + status
);
}
long delay = computeBackoffDelayMillis(attempt);
System.out.printf("Attempt %d failed with %d, retrying in %d ms%n",
attempt, status, delay);
Thread.sleep(delay);
} catch (Exception ex) {
// Network / IO exceptions
if (attempt >= MAX_ATTEMPTS) {
throw new RuntimeException("Exhausted retries", ex);
}
long delay = computeBackoffDelayMillis(attempt);
System.out.printf("Attempt %d threw %s, retrying in %d ms%n",
attempt, ex.getClass().getSimpleName(), delay);
Thread.sleep(delay);
}
}
}
private static boolean isRetryableStatus(int status) {
// Treat typical transient codes as retryable
return status == 429 || status == 503 || status == 504;
}
private static long computeBackoffDelayMillis(int attempt) {
// attempt is 1-based, but we want exponent starting at 0
int exponent = Math.max(0, attempt - 1);
double rawDelay = BASE_DELAY_MS * Math.pow(MULTIPLIER, exponent);
long capped = Math.min((long) rawDelay, MAX_DELAY_MS);
// Full jitter: random between 0 and capped
return (long) (RANDOM.nextDouble() * capped);
}Why this is structured this way
isRetryableStatuscentralizes policy so you can evolve it without touching the control flow.computeBackoffDelayMillishides the math and encodes base, multiplier, and cap in one place, making it trivial to test in isolation.- The loop is explicit: this makes your retry behavior visible in logs and debuggable, which is important in production troubleshooting.
How to validate the example
- Run it as-is;
https://httpbin.org/status/503will keep returning 503.- You should see multiple attempts logged with growing (but jittered) delays, then a failure after the max attempt.
- Change the URL to
https://httpbin.org/status/200.- The call should succeed on the first attempt with no retries.
- Change to
https://httpbin.org/status/429.- Observe multiple retries; tweak
MAX_ATTEMPTS,BASE_DELAY_MS, andMULTIPLIERand see how behavior changes.
- Observe multiple retries; tweak
Using Libraries: Resilience4j and Friends
In real systems you rarely hand-roll this everywhere; you typically standardize via a library.
A popular option is Resilience4j, where you:
- Configure an
IntervalFunctionfor exponential backoff (and optionally jitter). - Define
RetryConfigwithmaxAttempts,intervalFunction, and error predicates. - Decorate functions or suppliers so retry behavior is applied consistently across the codebase.
Putting It in System Design Context
Retry with backoff must coexist with other resilience mechanisms:
- Timeouts: Every retried call still needs a per-call timeout; otherwise retries just tie up threads.
- Circuit breakers: When a dependency is consistently failing, stop sending it traffic for a while instead of continuously retrying.
- Bulkheads / limits: Cap concurrency so a single broken dependency cannot consume all your resources.
Conceptually, you should design a retry contract per dependency: which operations are idempotent, what latency budget you have, and what backoff profile is acceptable for your users and upstream callers.
A Brief Parameter Guide for Production
As a rule of thumb for synchronous HTTP calls in a microservice:
- Base delay: 50–200 ms for low-latency services, up to 500 ms for heavy operations.
- Multiplier: 2 is a safe starting point; 1.5 if you care more about latency, 3 if you are aggressively protecting a fragile dependency.
- Max delay: 1–5 s for interactive paths, 10–60 s for background jobs.
- Max attempts: 3–5 attempts (including the initial one) is typical for user-facing calls, more for asynchronous jobs.
Always measure: instrument how many retries happen, which status codes cause them, and their impact on latency and error rates.
Recent Comments