|
| 1 | +--- |
| 2 | +id: extractors |
| 3 | +title: 🔬 Extractors |
| 4 | +description: Learn how to use extractors in Fiber middleware |
| 5 | +sidebar_position: 8.5 |
| 6 | +toc_max_heading_level: 4 |
| 7 | +--- |
| 8 | + |
| 9 | +The extractors package provides shared value extraction utilities for Fiber middleware packages. It helps reduce code duplication across middleware packages while ensuring consistent behavior and security practices. |
| 10 | + |
| 11 | +## Overview |
| 12 | + |
| 13 | +The `github.com/gofiber/fiber/v3/extractors` module provides standardized value extraction utilities integrated into Fiber's middleware ecosystem. This approach: |
| 14 | + |
| 15 | +- **Reduces Code Duplication**: Eliminates redundant extractor implementations across middleware packages |
| 16 | +- **Ensures Consistency**: Maintains identical behavior and security practices across all extractors |
| 17 | +- **Simplifies Maintenance**: Changes to extraction logic only need to be made in one place |
| 18 | +- **Enables Direct Usage**: Middleware can import and use extractors directly |
| 19 | +- **Improves Performance**: Shared, optimized extraction functions reduce overhead |
| 20 | + |
| 21 | +## What Are Extractors? |
| 22 | + |
| 23 | +Extractors are utilities that middleware uses to get values from different parts of HTTP requests: |
| 24 | + |
| 25 | +### Available Extractors |
| 26 | + |
| 27 | +- `FromAuthHeader(authScheme string)`: Extract from Authorization header with optional scheme |
| 28 | +- `FromCookie(key string)`: Extract from HTTP cookies |
| 29 | +- `FromParam(param string)`: Extract from URL path parameters |
| 30 | +- `FromForm(param string)`: Extract from form data |
| 31 | +- `FromHeader(header string)`: Extract from custom HTTP headers |
| 32 | +- `FromQuery(param string)`: Extract from URL query parameters |
| 33 | +- `FromCustom(key string, fn func(fiber.Ctx) (string, error))`: Define custom extraction logic with metadata |
| 34 | +- `Chain(extractors ...Extractor)`: Chain multiple extractors with fallback logic |
| 35 | + |
| 36 | +### Extractor Structure |
| 37 | + |
| 38 | +Each `Extractor` contains: |
| 39 | + |
| 40 | +```go |
| 41 | +type Extractor struct { |
| 42 | + Extract func(fiber.Ctx) (string, error) // Extraction function |
| 43 | + Key string // Parameter/header name |
| 44 | + Source Source // Source type for inspection |
| 45 | + AuthScheme string // Auth scheme (FromAuthHeader) |
| 46 | + Chain []Extractor // Chained extractors |
| 47 | +} |
| 48 | +``` |
| 49 | + |
| 50 | +- **Headers**: `Authorization`, `X-API-Key`, custom headers |
| 51 | +- **Cookies**: Session cookies, authentication tokens |
| 52 | +- **Query Parameters**: URL parameters like `?token=abc123` |
| 53 | +- **Form Data**: POST body form fields |
| 54 | +- **URL Parameters**: Route parameters like `/users/:id` |
| 55 | + |
| 56 | +### Chain Behavior |
| 57 | + |
| 58 | +The `Chain` function creates extractors that try multiple sources in order: |
| 59 | + |
| 60 | +- Returns the first successful extraction (non-empty value with no error) |
| 61 | +- If all extractors fail, returns the last error encountered or `ErrNotFound` |
| 62 | +- **Robust error handling**: Skips extractors with `nil` Extract functions |
| 63 | +- Preserves the source and key from the first extractor for metadata |
| 64 | +- Stores a defensive copy of all chained extractors for introspection via the `Chain` field |
| 65 | + |
| 66 | +## Why Middleware Uses Extractors |
| 67 | + |
| 68 | +Middleware needs to extract values from requests for authentication, authorization, and other purposes. Extractors provide: |
| 69 | + |
| 70 | +- **Security Awareness**: Different sources have different security implications |
| 71 | +- **Fallback Support**: Try multiple sources if the first one doesn't have the value |
| 72 | +- **Consistency**: Same extraction logic across all middleware packages |
| 73 | +- **Source Tracking**: Know where values came from for security decisions |
| 74 | + |
| 75 | +## Usage Examples |
| 76 | + |
| 77 | +### Basic Usage |
| 78 | + |
| 79 | +```go |
| 80 | +// KeyAuth middleware extracts key from header |
| 81 | +app.Use(keyauth.New(keyauth.Config{ |
| 82 | + Extractor: extractors.FromHeader("Middleware-Key"), |
| 83 | +})) |
| 84 | +``` |
| 85 | + |
| 86 | +### Fallback Chains |
| 87 | + |
| 88 | +```go |
| 89 | +// Try multiple sources in order |
| 90 | +tokenExtractor := extractors.Chain( |
| 91 | + extractors.FromHeader("Middleware-Key"), // Try header first |
| 92 | + extractors.FromCookie("middleware_key"), // Then cookie |
| 93 | + extractors.FromQuery("middleware_key"), // Finally query param |
| 94 | +) |
| 95 | + |
| 96 | +app.Use(keyauth.New(keyauth.Config{ |
| 97 | + Extractor: tokenExtractor, |
| 98 | +})) |
| 99 | +``` |
| 100 | + |
| 101 | +## Configuring Middleware That Uses Extractors |
| 102 | + |
| 103 | +### Authentication Middleware |
| 104 | + |
| 105 | +```go |
| 106 | +// KeyAuth middleware (default: FromAuthHeader) |
| 107 | +app.Use(keyauth.New(keyauth.Config{ |
| 108 | + // Default extracts from Authorization header |
| 109 | + // Extractor: extractors.FromAuthHeader("Bearer"), |
| 110 | +})) |
| 111 | + |
| 112 | +// Custom header extraction |
| 113 | +app.Use(keyauth.New(keyauth.Config{ |
| 114 | + Extractor: extractors.FromHeader("X-API-Key"), |
| 115 | +})) |
| 116 | + |
| 117 | +// Multiple sources with secure fallback |
| 118 | +app.Use(keyauth.New(keyauth.Config{ |
| 119 | + Extractor: extractors.Chain( |
| 120 | + extractors.FromAuthHeader("Bearer"), // Secure first |
| 121 | + extractors.FromHeader("X-API-Key"), // Then custom header |
| 122 | + extractors.FromQuery("api_key"), // Least secure last |
| 123 | + ), |
| 124 | +})) |
| 125 | +``` |
| 126 | + |
| 127 | +### Session Middleware |
| 128 | + |
| 129 | +```go |
| 130 | +// Session middleware (default: FromCookie) |
| 131 | +app.Use(session.New(session.Config{ |
| 132 | + // Default extracts from session_id cookie |
| 133 | + // Extractor: extractors.FromCookie("session_id"), |
| 134 | +})) |
| 135 | + |
| 136 | +// Custom cookie name |
| 137 | +app.Use(session.New(session.Config{ |
| 138 | + Extractor: extractors.FromCookie("my_session"), |
| 139 | +})) |
| 140 | +``` |
| 141 | + |
| 142 | +### CSRF Middleware |
| 143 | + |
| 144 | +```go |
| 145 | +// CSRF middleware (default: FromHeader) |
| 146 | +app.Use(csrf.New(csrf.Config{ |
| 147 | + // Default extracts from X-CSRF-Token header |
| 148 | + // Extractor: extractors.FromHeader("X-CSRF-Token"), |
| 149 | +})) |
| 150 | + |
| 151 | +// Form-based CSRF (less secure, use only if needed) |
| 152 | +app.Use(csrf.New(csrf.Config{ |
| 153 | + Extractor: extractors.Chain( |
| 154 | + extractors.FromHeader("X-CSRF-Token"), // Secure first |
| 155 | + extractors.FromForm("_csrf"), // Form fallback |
| 156 | + ), |
| 157 | +})) |
| 158 | +``` |
| 159 | + |
| 160 | +## Security Considerations |
| 161 | + |
| 162 | +### Source Characteristics |
| 163 | + |
| 164 | +Different extraction sources have different security properties and use cases: |
| 165 | + |
| 166 | +#### Headers (Generally Preferred) |
| 167 | + |
| 168 | +- **Authorization Header**: Standard for authentication tokens, widely supported |
| 169 | +- **Custom Headers**: Application-specific, less likely to be logged by default |
| 170 | +- **Considerations**: Can be intercepted without HTTPS, may be stripped by proxies |
| 171 | + |
| 172 | +#### Cookies (Good for Sessions) |
| 173 | + |
| 174 | +- **Session Cookies**: Designed for secure client-side storage |
| 175 | +- **Considerations**: Require proper `Secure`, `HttpOnly`, and `SameSite` flags |
| 176 | +- **Best for**: Session management, remember-me tokens |
| 177 | + |
| 178 | +#### Query Parameters (Use Sparingly) |
| 179 | + |
| 180 | +- **Query parameters**: Convenient for simple APIs and debugging |
| 181 | +- **Considerations**: Always visible in URLs, logged by servers/proxies, stored in browser history |
| 182 | +- **Best for**: Non-sensitive parameters, public identifiers |
| 183 | + |
| 184 | +#### Form Data (Context Dependent) |
| 185 | + |
| 186 | +- **POST Bodies**: Suitable for form submissions and API requests |
| 187 | +- **Considerations**: Avoid putting sensitive data in query strings; ensure request bodies aren’t logged and use the correct content type |
| 188 | +- **Best for**: User-generated content, file uploads |
| 189 | + |
| 190 | +### Security Best Practices |
| 191 | + |
| 192 | +1. **Use HTTPS**: Encrypt all traffic to protect extracted values in transit |
| 193 | +2. **Validate Input**: Always validate and sanitize extracted values |
| 194 | +3. **Log Carefully**: Avoid logging sensitive values from any source |
| 195 | +4. **Choose Appropriate Sources**: Match the source to your security requirements |
| 196 | +5. **Test Thoroughly**: Verify extraction works in your environment |
| 197 | +6. **Monitor Security**: Watch for extraction failures or unusual patterns |
| 198 | + |
| 199 | +### Chain Ordering Strategy |
| 200 | + |
| 201 | +When using multiple sources, order them by your security preferences: |
| 202 | + |
| 203 | +```go |
| 204 | +// Example: Prefer headers, fallback to cookies, then query |
| 205 | +extractors.Chain( |
| 206 | + extractors.FromAuthHeader("Bearer"), // Standard auth |
| 207 | + extractors.FromCookie("auth_token"), // Secure storage |
| 208 | + extractors.FromQuery("token"), // Public fallback |
| 209 | +) |
| 210 | +``` |
| 211 | + |
| 212 | +The "best" source depends on your specific use case, security requirements, and application architecture. |
| 213 | + |
| 214 | +### Common Security Issues |
| 215 | + |
| 216 | +#### Leaky URLs |
| 217 | + |
| 218 | +```go |
| 219 | +// ❌ DON'T: API keys in URLs (visible in logs, history, bookmarks) |
| 220 | +app.Use(keyauth.New(keyauth.Config{ |
| 221 | + Extractor: extractors.FromQuery("api_key"), // PROBLEMATIC |
| 222 | +})) |
| 223 | + |
| 224 | +// ✅ DO: API keys in headers (not visible in URLs) |
| 225 | +app.Use(keyauth.New(keyauth.Config{ |
| 226 | + Extractor: extractors.FromHeader("X-API-Key"), // BETTER |
| 227 | +})) |
| 228 | +``` |
| 229 | + |
| 230 | +#### Session Tokens in Query Parameters |
| 231 | + |
| 232 | +```go |
| 233 | +// ❌ DON'T: Session tokens in URLs (can be bookmarked, leaked) |
| 234 | +app.Use(session.New(session.Config{ |
| 235 | + Extractor: extractors.FromQuery("session"), // PROBLEMATIC |
| 236 | +})) |
| 237 | + |
| 238 | +// ✅ DO: Session tokens in cookies (designed for this purpose) |
| 239 | +app.Use(session.New(session.Config{ |
| 240 | + Extractor: extractors.FromCookie("session_id"), // BETTER |
| 241 | +})) |
| 242 | +``` |
| 243 | + |
| 244 | +#### Form-Only CSRF Tokens |
| 245 | + |
| 246 | +While the default extractor uses headers, some implementations use form fields, which is fine if you don't have AJAX or API clients: |
| 247 | + |
| 248 | +```go |
| 249 | +// ❌ DON'T: CSRF tokens only in forms (breaks AJAX, API calls) |
| 250 | +app.Use(csrf.New(csrf.Config{ |
| 251 | + Extractor: extractors.FromForm("_csrf"), // LIMITED |
| 252 | +})) |
| 253 | + |
| 254 | +// ✅ DO: Header-first with form fallback (works everywhere) |
| 255 | +app.Use(csrf.New(csrf.Config{ |
| 256 | + Extractor: extractors.Chain( |
| 257 | + extractors.FromHeader("X-CSRF-Token"), // PREFERRED |
| 258 | + extractors.FromForm("_csrf"), // FALLBACK |
| 259 | + ), |
| 260 | +})) |
| 261 | +``` |
| 262 | + |
| 263 | +### Understanding Trade-offs |
| 264 | + |
| 265 | +**No extractor is universally "secure" - security depends on:** |
| 266 | + |
| 267 | +- Whether you're using HTTPS |
| 268 | +- How you configure cookies (Secure, HttpOnly, SameSite flags) |
| 269 | +- Your logging and monitoring setup |
| 270 | +- The sensitivity of the data being extracted |
| 271 | +- Your threat model and security requirements |
| 272 | + |
| 273 | +Choose extractors based on your specific use case and security needs, not blanket "secure" vs "insecure" labels. |
| 274 | + |
| 275 | +## Standards Compliance |
| 276 | + |
| 277 | +### Authorization header compliance (RFC 9110; previously RFC 7235) |
| 278 | + |
| 279 | +The `FromAuthHeader` extractor aligns with HTTP authentication semantics: |
| 280 | + |
| 281 | +- **Case-insensitive scheme matching**: `Bearer`, `bearer`, `BEARER` all work |
| 282 | +- **OWS handling**: Supports SP/HTAB around the scheme/value per RFC 9110 |
| 283 | +- **Proper error handling**: Validates header format and content |
| 284 | +- **Security-conscious**: Prevents common parsing vulnerabilities |
| 285 | + |
| 286 | +```go |
| 287 | +// All of these work correctly: |
| 288 | +extractors.FromAuthHeader("Bearer") // Standard case |
| 289 | +extractors.FromAuthHeader("bearer") // Lowercase |
| 290 | +extractors.FromAuthHeader("BEARER") // Uppercase |
| 291 | +extractors.FromAuthHeader("") // No scheme, returns header (or ErrNotFound if empty) |
| 292 | +``` |
| 293 | + |
| 294 | +## Troubleshooting |
| 295 | + |
| 296 | +### Extraction Fails |
| 297 | + |
| 298 | +**Problem**: Middleware returns "value not found" or authentication fails |
| 299 | + |
| 300 | +**Solutions**: |
| 301 | + |
| 302 | +1. Check if the expected header/cookie/query parameter is present |
| 303 | +2. Verify the key name matches exactly (headers are case-insensitive; params/cookies/query keys are case-sensitive) |
| 304 | +3. Ensure the request uses the correct HTTP method (GET vs POST) |
| 305 | +4. Check if middleware is configured with the right extractor |
| 306 | + |
| 307 | +**Debug Example**: |
| 308 | + |
| 309 | +```go |
| 310 | +// Add simple debug logging (avoid logging secrets in production) |
| 311 | +app.Use(func(c fiber.Ctx) error { |
| 312 | + hdr := c.Get("X-API-Key") |
| 313 | + cookie := c.Cookies("session_id") |
| 314 | + if hdr != "" || cookie != "" { |
| 315 | + log.Printf("debug: X-API-Key present=%t, session_id present=%t", hdr != "", cookie != "") |
| 316 | + } |
| 317 | + return c.Next() |
| 318 | +}) |
| 319 | +``` |
| 320 | + |
| 321 | +### Wrong Source Used |
| 322 | + |
| 323 | +**Problem**: Values extracted from unexpected sources |
| 324 | + |
| 325 | +**Solutions**: |
| 326 | + |
| 327 | +1. Check middleware configuration order |
| 328 | +2. Verify chain order (first successful extraction wins) |
| 329 | +3. Use more specific extractors when needed |
| 330 | + |
| 331 | +### Security Warnings |
| 332 | + |
| 333 | +**Problem**: Getting security warnings in logs |
| 334 | + |
| 335 | +**Solutions**: |
| 336 | + |
| 337 | +1. Switch to more secure sources (headers/cookies) |
| 338 | +2. Use HTTPS to encrypt traffic |
| 339 | +3. Review if sensitive data should be in that source |
| 340 | + |
| 341 | +## Advanced Usage |
| 342 | + |
| 343 | +### Custom Extraction Logic |
| 344 | + |
| 345 | +Extractors support custom extractors for complex scenarios: |
| 346 | + |
| 347 | +```go |
| 348 | +// Extract from custom logic (rarely needed) |
| 349 | +customExtractor := extractors.FromCustom("my-source", func(c fiber.Ctx) (string, error) { |
| 350 | + // Complex extraction logic |
| 351 | + if value := c.Locals("computed_token"); value != nil { |
| 352 | + return value.(string), nil |
| 353 | + } |
| 354 | + return "", extractors.ErrNotFound |
| 355 | +}) |
| 356 | +``` |
| 357 | + |
| 358 | +:::warning |
| 359 | +**Custom extractors break source awareness.** When you use `FromCustom`, middleware cannot determine where the value came from, which means: |
| 360 | + |
| 361 | +- **No automatic security warnings** for potentially insecure sources |
| 362 | +- **No source-based logging** or monitoring capabilities |
| 363 | +- **Developer responsibility** for ensuring the extraction is secure and appropriate |
| 364 | + |
| 365 | +**Only use `FromCustom` when:** |
| 366 | + |
| 367 | +- Standard extractors don't meet your needs |
| 368 | +- You've carefully evaluated the security implications |
| 369 | +- You're confident in the security of your custom extraction logic |
| 370 | +- You understand that middleware cannot provide source-aware security guidance |
| 371 | + |
| 372 | +**Note:** If you pass `nil` as the function parameter, `FromCustom` will return an extractor that always fails with `ErrNotFound`. |
| 373 | +::: |
| 374 | + |
| 375 | +### Multiple Middleware Coordination |
| 376 | + |
| 377 | +When using multiple middleware that extract values, ensure they don't conflict: |
| 378 | + |
| 379 | +```go |
| 380 | +// Good: Different sources for different purposes |
| 381 | +app.Use(keyauth.New(keyauth.Config{ |
| 382 | + Extractor: extractors.FromHeader("X-API-Key"), |
| 383 | +})) |
| 384 | +app.Use(session.New(session.Config{ |
| 385 | + Extractor: extractors.FromCookie("session_id"), |
| 386 | +})) |
| 387 | + |
| 388 | +// Avoid: Same source for different middleware |
| 389 | +app.Use(keyauth.New(keyauth.Config{ |
| 390 | + Extractor: extractors.FromCookie("token"), // API auth |
| 391 | +})) |
| 392 | +app.Use(session.New(session.Config{ |
| 393 | + Extractor: extractors.FromCookie("token"), // Session - CONFLICT! |
| 394 | +})) |
| 395 | +``` |
0 commit comments