diff --git a/.github/README.md b/.github/README.md index cb7b19b376..5a20c5a535 100644 --- a/.github/README.md +++ b/.github/README.md @@ -688,6 +688,7 @@ Here is a list of middleware that are included within the Fiber framework. | [idempotency](https://github.com/gofiber/fiber/tree/main/middleware/idempotency) | Allows for fault-tolerant APIs where duplicate requests do not erroneously cause the same action performed multiple times on the server-side. | | [keyauth](https://github.com/gofiber/fiber/tree/main/middleware/keyauth) | Adds support for key based authentication. | | [limiter](https://github.com/gofiber/fiber/tree/main/middleware/limiter) | Adds Rate-limiting support to Fiber. Use to limit repeated requests to public APIs and/or endpoints such as password reset. | +| [loadshedding](https://github.com/gofiber/fiber/tree/main/middleware/loadshedding) | Gracefully manages server load by enforcing request timeouts and handling resource-intensive requests during high-traffic periods. | | [logger](https://github.com/gofiber/fiber/tree/main/middleware/logger) | HTTP request/response logger. | | [pprof](https://github.com/gofiber/fiber/tree/main/middleware/pprof) | Serves runtime profiling data in pprof format. | | [proxy](https://github.com/gofiber/fiber/tree/main/middleware/proxy) | Allows you to proxy requests to multiple servers. | diff --git a/docs/middleware/loadshedding.md b/docs/middleware/loadshedding.md new file mode 100644 index 0000000000..646c9b67ca --- /dev/null +++ b/docs/middleware/loadshedding.md @@ -0,0 +1,81 @@ +--- +id: loadshedding +--- + +# Load Shedding + +The **Load Shedding** middleware for [Fiber](https://github.com/gofiber/fiber) helps maintain server stability by applying request-processing timeouts. It prevents resource exhaustion by gracefully rejecting requests that exceed a specified time limit. This is particularly beneficial in high-traffic scenarios, where preventing overload is crucial to sustaining service availability and performance. + +## Features + +- **Request Timeout Enforcement**: Automatically terminates any request that exceeds the configured processing time. +- **Customizable Response**: Enables you to define a specialized response for timed-out requests. +- **Exclusion Logic**: Lets you skip load-shedding for specific requests, such as health checks or other critical endpoints. +- **Enhanced Stability**: Helps avoid server crashes or sluggish performance under heavy load by shedding excess requests. + +## Use Cases + +- **High-Traffic Scenarios**: Safeguard critical resources by rejecting overly long or resource-intensive requests. +- **Health Check Protection**: Exclude monitoring endpoints (e.g., `/health`) to ensure uninterrupted external checks. +- **Dynamic Load Management**: Utilize exclusion logic to adjust load-shedding behavior for specific routes or request types. + +--- + +## Signature + +```go +func New(timeout time.Duration, loadSheddingHandler fiber.Handler, exclude func(fiber.Ctx) bool) fiber.Handler +``` + +## Config + +| Property | Type | Description | Default | +|-----------------------|--------------------|----------------------------------------------------------------------------------|-----------| +| `timeout` | `time.Duration` | The maximum allowed processing time for a request. | Required | +| `loadSheddingHandler`| `fiber.Handler` | The handler invoked for requests that exceed the `timeout`. | Required | +| `exclude` | `func(fiber.Ctx) bool` | Optional function to exclude certain requests from load-shedding logic. | `nil` | + +## Example Usage + +Import the middleware and configure it within your Fiber application: + +```go +import ( + "time" + "github.com/gofiber/fiber/v3" + "github.com/gofiber/fiber/v3/middleware/loadshedding" +) + +func main() { + app := fiber.New() + + // Basic usage with a 5-second timeout + app.Use(loadshedding.New( + 5*time.Second, + func(c fiber.Ctx) error { + return c.Status(fiber.StatusServiceUnavailable).SendString("Service unavailable due to high load") + }, + nil, + )) + + // Advanced usage with an exclusion function for specific endpoints + app.Use(loadshedding.New( + 3*time.Second, + func(c fiber.Ctx) error { + return c.Status(fiber.StatusServiceUnavailable).SendString("Request timed out") + }, + func(c fiber.Ctx) bool { + // Exclude /health from load-shedding + return c.Path() == "/health" + }, + )) + + app.Get("/", func(c fiber.Ctx) error { + // Simulating a long-running request + time.Sleep(4 * time.Second) + return c.SendString("Hello, world!") + }) + + app.Listen(":3000") +} +``` diff --git a/docs/whats_new.md b/docs/whats_new.md index 321df424d6..e786bf0bd7 100644 --- a/docs/whats_new.md +++ b/docs/whats_new.md @@ -31,6 +31,7 @@ Here's a quick overview of the changes in Fiber `v3`: - [Filesystem](#filesystem) - [Monitor](#monitor) - [Healthcheck](#healthcheck) + - [Load shedding](#load-shedding) - [📋 Migration guide](#-migration-guide) ## Drop for old Go versions @@ -810,6 +811,18 @@ The Healthcheck middleware has been enhanced to support more than two routes, wi Refer to the [healthcheck middleware migration guide](./middleware/healthcheck.md) or the [general migration guide](#-migration-guide) to review the changes. +### Load Shedding + +We’ve introduced the **Load Shedding Middleware** to keep your system stable under heavy load. It automatically terminates requests that exceed a specified processing time, enabling the application to gracefully shed excessive load while maintaining responsiveness. + +#### Functionality + +- **Timeout Enforcement**: Automatically terminates requests that exceed the defined maximum processing time. +- **Customizable Response**: Supports a configurable load-shedding handler to define the response for timed-out requests. +- **Exclusion Logic**: Allows certain requests or routes to bypass the load-shedding mechanism based on defined rules. + +By applying timeouts and shedding excess load, this middleware helps your server remain resilient and ensures a smoother user experience during high-traffic periods. + ## 📋 Migration guide - [🚀 App](#-app-1) @@ -1361,6 +1374,35 @@ app.Get(healthcheck.DefaultStartupEndpoint, healthcheck.NewHealthChecker(healthc app.Get("/live", healthcheck.NewHealthChecker()) ``` +#### Load shedding + +This middleware uses `context.WithTimeout` to manage the lifecycle of requests. If a request exceeds the specified timeout, the custom load-shedding handler is triggered, ensuring the system remains stable under stress. + +##### Key Parameters + +`timeout` (`time.Duration`): The maximum time a request is allowed to process. Requests exceeding this time are terminated. + +`loadSheddingHandler` (`fiber.Handler`): A custom handler that executes when a request exceeds the timeout. Typically used to return a `503 Service Unavailable` response or a custom message. + +`exclude` (`func(fiber.Ctx) bool`): A filter function to exclude specific requests from being subjected to the load-shedding logic (optional). + +##### Usage Example + +```go +import "github.com/gofiber/fiber/v3/middleware/loadshedding + +app.Use(loadshedding.New( + 10*time.Second, // Timeout duration + func(c fiber.Ctx) error { // Load shedding response + return c.Status(fiber.StatusServiceUnavailable). + SendString("Service overloaded, try again later.") + }, + func(c fiber.Ctx) bool { // Exclude health checks + return c.Path() == "/health" + }, +)) +``` + #### Monitor Since v3 the Monitor middleware has been moved to the [Contrib package](https://github.com/gofiber/contrib/tree/main/monitor) diff --git a/middleware/loadshedding/loadshedding.go b/middleware/loadshedding/loadshedding.go new file mode 100644 index 0000000000..564c94bfe5 --- /dev/null +++ b/middleware/loadshedding/loadshedding.go @@ -0,0 +1,45 @@ +package loadshedding + +import ( + "context" + "time" + + "github.com/gofiber/fiber/v3" +) + +// New creates a middleware handler enforces a timeout on request processing to manage server load. +// If a request exceeds the specified timeout, a custom load-shedding handler is executed. +func New(timeout time.Duration, loadSheddingHandler fiber.Handler, exclude func(fiber.Ctx) bool) fiber.Handler { + return func(c fiber.Ctx) error { + // Skip load-shedding logic for requests matching the exclusion criteria + if exclude != nil && exclude(c) { + return c.Next() + } + + // Create a context with a timeout for the current request + ctx, cancel := context.WithTimeout(c.Context(), timeout) + defer cancel() + + // Set the new context with a timeout + c.SetContext(ctx) + + // Process the request and capture any error + err := c.Next() + + // Create a channel to signal when request processing completes + done := make(chan error, 1) + + // Send the result of the request processing to the channel + go func() { + done <- err + }() + + // Handle either request completion or timeout + select { + case <-ctx.Done(): // Triggered if the timeout expires + return loadSheddingHandler(c) + case err := <-done: // Triggered if request processing completes + return err + } + } +} diff --git a/middleware/loadshedding/loadshedding_test.go b/middleware/loadshedding/loadshedding_test.go new file mode 100644 index 0000000000..b3e9d3dc2a --- /dev/null +++ b/middleware/loadshedding/loadshedding_test.go @@ -0,0 +1,92 @@ +package loadshedding_test + +import ( + "net/http/httptest" + "testing" + "time" + + "github.com/gofiber/fiber/v3" + "github.com/gofiber/fiber/v3/middleware/loadshedding" + "github.com/stretchr/testify/require" +) + +// Helper handlers +func successHandler(c fiber.Ctx) error { + return c.SendString("Request processed successfully!") +} + +func timeoutHandler(c fiber.Ctx) error { + time.Sleep(2 * time.Second) // Simulate a long-running request + return c.SendString("This should not appear") +} + +func loadSheddingHandler(c fiber.Ctx) error { + return c.Status(fiber.StatusServiceUnavailable).SendString("Service Overloaded") +} + +func excludedHandler(c fiber.Ctx) error { + return c.SendString("Excluded route") +} + +// go test -run Test_LoadSheddingExcluded +func Test_LoadSheddingExcluded(t *testing.T) { + t.Parallel() + app := fiber.New() + + // Middleware with exclusion + app.Use(loadshedding.New( + 1*time.Second, + loadSheddingHandler, + func(c fiber.Ctx) bool { return c.Path() == "/excluded" }, + )) + app.Get("/", successHandler) + app.Get("/excluded", excludedHandler) + + // Test excluded route + resp, err := app.Test(httptest.NewRequest(fiber.MethodGet, "/excluded", nil)) + require.NoError(t, err) + require.Equal(t, fiber.StatusOK, resp.StatusCode) +} + +// go test -run Test_LoadSheddingTimeout +func Test_LoadSheddingTimeout(t *testing.T) { + t.Parallel() + app := fiber.New() + + // Middleware with a 1-second timeout + app.Use(loadshedding.New( + 1*time.Second, // Middleware timeout + loadSheddingHandler, + nil, + )) + app.Get("/", timeoutHandler) + + // Create a custom request + req := httptest.NewRequest(fiber.MethodGet, "/", nil) + + // Test timeout behavior + resp, err := app.Test(req, fiber.TestConfig{ + Timeout: 3 * time.Second, // Ensure the test timeout exceeds middleware timeout + }) + require.NoError(t, err) + require.Equal(t, fiber.StatusServiceUnavailable, resp.StatusCode) +} + +// go test -run Test_LoadSheddingSuccessfulRequest +func Test_LoadSheddingSuccessfulRequest(t *testing.T) { + t.Parallel() + app := fiber.New() + + // Middleware with sufficient time for request to complete + app.Use(loadshedding.New( + 2*time.Second, + loadSheddingHandler, + nil, + )) + app.Get("/", successHandler) + + // Test successful request + resp, err := app.Test(httptest.NewRequest(fiber.MethodGet, "/", nil)) + require.NoError(t, err) + require.Equal(t, fiber.StatusOK, resp.StatusCode) +}