We are using Tyk's Vault integration to dynamically retrieve secrets for upstream authentication. Our Vault secret keys follow a naming convention that uses underscores to separate parts (e.g., API_KEY, DB_PASSWORD). However, Tyk's current regex pattern for Vault references does not support underscores in the secret path, causing the reference to be truncated at the first underscore character.

For example, when we configure:

$secret_vault.kv-v2/layer1/layer2/{{ .Values.environment }}.API_KEY

Tyk only processes:

$secret_vault.kv-v2/layer1/layer2/{{ .Values.environment }}.API

This forces us to duplicate all secrets with underscore-free names in Vault, which is:

• Operationally inefficient and error-prone
• Creates unnecessary maintenance overhead
• Not feasible to change original keys as they're used by many other systems

The root cause appears to be in the regex pattern definition (from mw_url_rewrite.go):

var vaultMatch = regexp.MustCompile(`\$secret_vault.([A-Za-z0-9\/\-\.]+)`)

The pattern lacks the underscore character (_), while other secret handlers like envValueMatch correctly include it:

var envValueMatch = regexp.MustCompile(`\$secret_env.([A-Za-z0-9_\-\.]+)`)

Describe the solution you'd like

Update the vaultMatch regex pattern to include the underscore character, making it consistent with other secret reference patterns:

var vaultMatch = regexp.MustCompile(`\$secret_vault.([A-Za-z0-9_\/\-\.]+)`)

This would allow Vault references to properly handle secret keys containing underscores, aligning with common naming conventions used in production environments.

Similarly, the consulMatch pattern appears to have the same limitation and should be updated:

var consulMatch = regexp.MustCompile(`\$secret_consul.([A-Za-z0-9_\/\-\.]+)`)

Describe alternatives you've considered

1. Current workaround (unacceptable): Duplicating all secrets in Vault with underscore-free names. This creates maintenance burden and potential security risks due to secret sprawl.
2. Renaming existing secrets: Not feasible as multiple systems depend on the current naming convention.
3. Custom middleware: Would require maintaining custom code and lose official support benefits. we have implemented a plugin for this and it works but it would be much better to support this by default by Tyk.

Additional context

• This affects the Vault integration across all Tyk versions/branches (verified in master branch)
• The inconsistency between envValueMatch (which supports underscores) and vaultMatch (which doesn't) suggests this is an oversight rather than intentional design (maybe other secret engines prohibit this character but surely not Vault)
• Standard secret naming conventions in a lot of organizations use underscores (e.g., DATABASE_URL, API_KEY, AWS_SECRET_KEY)
• This is a breaking limitation for enterprises migrating to Tyk with existing Vault infrastructure

Code references:

• File: mw_url_rewrite.go
• Regex definitions (lines with pattern declarations)
• Variable replacement logic where vaultMatch.FindAllString() is called