🛡️ Resilience & Polly Integration

Automatic retry policies and circuit breakers for building fault-tolerant applications

Error Handling Demo

Demonstrates error handling and retry behavior

This demo intentionally uses an invalid category to show how the library handles failures gracefully.

Expected Result: HTTP 400 Bad Request
The JokeAPI will return a 400 error because "InvalidCategory" is not a valid category.
Valid categories: Programming, Misc, Dark, Pun, Spooky, Christmas, Any

Successful Request Demo

Shows successful request with retry capability

This demo uses a valid category to demonstrate that the same resilience infrastructure works for successful requests.

Expected Result: HTTP 200 OK
The request succeeds on the first attempt, but retry logic is still available if needed.

Why Show an Error Demo?

The error demo is intentional and educational because it demonstrates:

Graceful Error Handling: The application doesn't crash when APIs return errors
Structured Error Information: Error details are captured in ErrorList with correlation IDs
Retry Behavior: Shows how retry logic would work (if Polly is configured)
Real-World Scenarios: In production, APIs fail - you need to handle it properly
Debugging Support: Correlation IDs and detailed error info make troubleshooting easier

🔄 Resilience Patterns

Retry Policy

Automatically retry failed requests with configurable delay and backoff strategies

Exponential backoff
Jitter for avoiding thundering herd
Configurable max attempts
Handles transient failures (5xx errors, timeouts)

Note: Retries are typically for transient failures (500s, timeouts). 400 errors are usually not retried since they indicate client errors that won't resolve with retries.

Circuit Breaker

Prevents cascading failures by stopping requests to failing services

Opens after threshold failures
Automatically recovers
Configurable duration
Protects downstream services

How it works: After multiple failures, the circuit "opens" and requests fail fast without hitting the API. After a timeout, it "half-opens" to test if the service recovered.

⚙️ Configuration in Program.cs

// Configure Polly options
var pollyOptions = new HttpRequestResultPollyOptions
{
    // Retry settings
    MaxRetryAttempts = 3,
    RetryDelay = TimeSpan.FromSeconds(1),
    RetryStrategy = RetryStrategy.Exponential,  // Each retry waits longer
  
    // Circuit breaker settings
    EnableCircuitBreaker = true,
    CircuitBreakerThreshold = 5,    // Open after 5 consecutive failures
    CircuitBreakerDuration = TimeSpan.FromSeconds(30)  // Stay open for 30 seconds
};

// Add Polly decorator to service chain
builder.Services.AddScoped<IHttpRequestResultService>(provider =>
{
    IHttpRequestResultService service = provider.GetRequiredService<HttpRequestResultService>();
    
    // Add Polly decorator (usually after cache, before telemetry)
    service = new HttpRequestResultServicePolly(
      provider.GetRequiredService<ILogger<HttpRequestResultServicePolly>>(),
        service,
        pollyOptions
    );
    
    // Add telemetry (outermost layer)
    service = new HttpRequestResultServiceTelemetry(
        provider.GetRequiredService<ILogger<HttpRequestResultServiceTelemetry>>(),
        service
    );
    
    return service;
});

📝 Usage Example

No Code Changes Required!

// Just use the service normally - resilience is automatic!
var request = new HttpRequestResult<JokeResponse>
{
    RequestPath = "https://v2.jokeapi.dev/joke/Programming?safe-mode",
    RequestMethod = HttpMethod.Get
};

// Polly automatically handles retries and circuit breaking
var result = await _requestResultService.HttpSendRequestResultAsync(request);

// Check if request succeeded
if (result.IsSuccessStatusCode)
{
    var joke = result.ResponseResults;
    
    // Check if any retries occurred
    if (result.RequestContext.TryGetValue("RetryCount", out var retryCount))
    {
        Console.WriteLine($"Request succeeded after {retryCount} retries");
    }
}
else
{
    // Handle failure gracefully
    Console.WriteLine($"Request failed: {result.StatusCode}");
    Console.WriteLine($"Errors: {string.Join(", ", result.ErrorList)}");
    Console.WriteLine($"Correlation ID: {result.CorrelationId}");
    
    // Check circuit breaker state
    if (result.RequestContext.TryGetValue("CircuitState", out var state))
    {
   Console.WriteLine($"Circuit breaker state: {state}");
    }
}

Benefits of Resilience Patterns

Retry Policy Benefits

Handles transient failures: Network blips, timeouts, 5xx errors
Improves reliability: Higher success rate without code changes
Configurable strategy: Linear, exponential, or custom backoff
Automatic: No manual retry loops needed

Circuit Breaker Benefits

Prevents cascade failures: Stops hitting failing services
Fast failure: Fail fast when service is down
Auto recovery: Automatically tries again after timeout
Resource protection: Prevents resource exhaustion

💡 Best Practices

Use Exponential Backoff: Increase delay between retries to avoid overwhelming the service
Set Appropriate Timeouts: Don't retry forever, set reasonable limits
Retry Only Transient Errors: Don't retry 4xx client errors (except 408, 429)
Monitor Circuit State: Log when circuit opens/closes for ops visibility
Combine with Caching: Cache successful responses to reduce retry load
Handle Partial Failures: Consider what to do when some requests succeed and others fail
Test Resilience: Use tools like Chaos Monkey to test fault tolerance
Use Correlation IDs: Track request flows across retries for debugging

📋 HTTP Error Types & Retry Strategy

Status Code	Type	Should Retry?	Reason
`400`	Bad Request	❌ No	Client error - won't resolve with retries
`408`	Request Timeout	✅ Yes	Transient - may succeed on retry
`429`	Too Many Requests	✅ Yes (with delay)	Rate limit - retry after delay
`500`	Internal Server Error	✅ Yes	Server error - may be transient
`502`	Bad Gateway	✅ Yes	Gateway error - may be transient
`503`	Service Unavailable	✅ Yes	Temporary - service may recover
`504`	Gateway Timeout	✅ Yes	Timeout - may succeed on retry