DNR: Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

salonichf5 · 2025-11-18T22:45:42Z

Proposed changes

Write a clear and concise description that helps reviewers understand the purpose and impact of your changes. Use the
following format:

Problem: Users want to be able to specify session persistence for their upstreams.

Solution: Add support for session persistence using sticky cookie directives which is only available for NGINX Plus users

Testing: Manual testing

Errors

Spec validation

The HTTPRoute "coffee" is invalid: 
* spec.rules[0].sessionPersistence.sessionName: Too long: may not be more than 128 bytes
* <nil>: Invalid value: null: some validation rules were not checked because the object was invalid; correct the existing errors to complete validation


    sessionPersistence:
      sessionName: "coffee-shop-session-with-very-long-name-for-testing-session-affinity-functionality-in-kubernetes-gateway-api-implementation-v1-hello"

The HTTPRoute "tea" is invalid: spec.rules[0].sessionPersistence.absoluteTimeout: Invalid value: "99999d": spec.rules[0].sessionPersistence.absoluteTimeout in body should match '^([0-9]{1,5}(h|m|s|ms)){1,4}$'

The HTTPRoute "tea" is invalid: spec.rules[0].sessionPersistence.absoluteTimeout: Invalid value: "999d": spec.rules[0].sessionPersistence.absoluteTimeout in body should match '^([0-9]{1,5}(h|m|s|ms)){1,4}$'

httproute.gateway.networking.k8s.io/tea configured
The HTTPRoute "coffee" is invalid: spec.rules[0].sessionPersistence: Invalid value: "object": AbsoluteTimeout must be specified when cookie lifetimeType is Permanent

    sessionPersistence:
      sessionName: "coffee-cookie"
      cookieConfig:
        lifetimeType: Permanent

      Message:               The following unsupported parameters were ignored: [spec.rules[0].sessionPersistence.type: Unsupported value: "Header": supported values: "Cookie", spec.rules[0].sessionPersistence: Invalid value: "spec.rules[0].sessionPersistence": session persistence is ignored because there are errors in the configuration]

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Header
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

httproute.gateway.networking.k8s.io/tea configured
The HTTPRoute "coffee" is invalid: 
* spec.rules[0].sessionPersistence.type: Unsupported value: "cookie": supported values: "Cookie", "Header"
* <nil>: Invalid value: null: some validation rules were not checked because the object was invalid; correct the existing errors to complete validation

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

      Last Transition Time:  2025-11-18T01:28:53Z
      Message:               The following unsupported parameters were ignored: [spec.rules[0].sessionPersistence.idleTimeout: Invalid value: "300s": idleTimeout is not supported, spec.rules[0].sessionPersistence: Invalid value: "spec.rules[0].sessionPersistence": session persistence is ignored because there are errors in the configuration]

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      idleTimeout: 300s
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent

Configuration

HTTPRoutes

Case 1: path and expires specified

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent


upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee_80.conf;
}

Case 2 : no expiry, only path specified

    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Session. —> no expiry



upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie path=/coffee;

    state /var/lib/nginx/state/default_coffee_80.conf;

Regular expression with other path matches (no path set)

rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    - path:
        type: Exact
        value: /coffee/espresso
    - path:
        type: RegularExpression
        value: /coffee/[a-zA-Z0-9]+
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent



upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s;

    state /var/lib/nginx/state/default_coffee_80.conf;

Multiple matches with common prefix

 rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    - path:
        type: Exact
        value: /coffee/espresso
    - path:
        type: PathPrefix
        value: /coffee/tea
    - path:
        type: Exact
        value: /coffee/latte
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent



upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee_80.conf;

multiple matches with no common prefix

  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /1
    - path:
        type: Exact
        value: /2
    - path:
        type: PathPrefix
        value: /3
    - path:
        type: Exact
        value: /4
    backendRefs:
    - name: coffee
      port: 80
    sessionPersistence:
      sessionName: "coffee-cookie"
      type: Cookie
      absoluteTimeout: 600s
      cookieConfig:
        lifetimeType: Permanent


upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s;

    state /var/lib/nginx/state/default_coffee_80.conf;

GRPCRoutes (path is always empty for GRPCRoutes

Hostname matching

Hostname matching

  rules:
  - backendRefs:
    - name: grpc-infra-backend-v2
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent


upstream default_grpc-infra-backend-v2_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v2_8080 1m;
    sticky cookie grpc-cookie expires=1000s;
    
    state /var/lib/nginx/state/default_grpc-infra-backend-v2_8080.conf;

Exact Method matching

rules:
  - matches:
    - method:
        service: helloworld.Greeter
        method: SayHello
    backendRefs:
    - name: grpc-infra-backend-v1
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie-exact"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent



upstream default_grpc-infra-backend-v1_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v1_8080 1m;
    sticky cookie grpc-cookie-exact expires=1000s;
    
    state /var/lib/nginx/state/default_grpc-infra-backend-v1_8080.conf;

Conflicting Cases

HTTPRoute

rules:
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
    backendRefs:
    - name: coffee-v1-svc
      port: 80
    sessionPersistence:
      sessionName: "grpc-cookie-conflicts-diff"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
      headers:
      - name: version
        value: v2
    - path:
        type: PathPrefix
        value: /coffee
      queryParams:
      - name: TEST
        value: v2
    backendRefs:
    - name: coffee-v1-svc
      port: 80
    sessionPersistence:
      sessionName: "grpc-cookie-conflicts"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent
  - matches:
    - path:
        type: PathPrefix
        value: /coffee
      headers:
      - name: headerRegex
        type: RegularExpression
        value: "header-[a-z]{1}"
    - path:
        type: PathPrefix
        value: /coffee
      queryParams:
      - name: queryRegex
        type: RegularExpression
        value: "query-[a-z]{1}"
    backendRefs:
    - name: coffee-v3-svc
      port: 80
    sessionPersistence:
      sessionName: "grpc-cookie-conflicts"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent

Config only generated for v3 no conflict

upstream default_coffee-v3-svc_80 {
    random two least_conn;
    zone default_coffee-v3-svc_80 1m;
    sticky cookie grpc-cookie-conflicts expires=1000s path=/coffee;
    
    state /var/lib/nginx/state/default_coffee-v3-svc_80.conf;
    
    
    
    
}

tatus:
  Parents:
    Conditions:
      Last Transition Time:  2025-11-18T21:57:11Z
      Message:               All references are resolved
      Observed Generation:   3
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-11-18T21:57:11Z
      Message:               Session Persistence configuration ignored: for backendRefs default/coffee-v1-svc:80 due to conflicting configuration across multiple rules
      Observed Generation:   3
      Reason:                InvalidSessionPersistence
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.nginx.org/nginx-gateway-controller
    Parent Ref:
      Group:         gateway.networking.k8s.io
      Kind:          Gateway
      Name:          cafe
      Namespace:     default
      Section Name:  http
Events:              <none>

Traffic splitting

config

upstream default_coffee-v1_80 {
    random two least_conn;
    zone default_coffee-v1_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee-v1_80.conf;

}

upstream default_coffee-v2_80 {
    random two least_conn;
    zone default_coffee-v2_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee-v2_80.conf;

}

Summary (how many times each backend was chosen):
  19 10.244.0.11:8080
  81 10.244.0.10:8080

GRPCRoute

 rules:
  # Matches "version: one"
  - matches:
    - headers:
      - name: version
        value: one
    backendRefs:
    - name: grpc-infra-backend-v1
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent
  # Matches "version: two"
  - matches:
    - headers:
      - name: version
        value: two
    backendRefs:
    - name: grpc-infra-backend-v2
      port: 8080
  # Matches "headerRegex: grpc-header-[a-z]{1}"
  - matches:
    - headers:
      - name: headerRegex
        value: "grpc-header-[a-z]{1}"
        type: RegularExpression
    backendRefs:
    - name: grpc-infra-backend-v2
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie-different"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent
  # Matches "version: two" AND "color: orange"
  - matches:
    - headers:
      - name: version
        value: two
      - name: color
        value: orange
    backendRefs:
    - name: grpc-infra-backend-v1
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent
  # Matches "color: blue" OR "color: green"
  - matches:
    - headers:
      - name: color
        value: blue
    - headers:
      - name: color
        value: green
    backendRefs:
    - name: grpc-infra-backend-v1
      port: 8080
  # Matches "color: red" OR "color: yellow"
  - matches:
    - headers:
      - name: color
        value: red
    - headers:
      - name: color
        value: yellow
    backendRefs:
    - name: grpc-infra-backend-v2
      port: 8080
    sessionPersistence:
      sessionName: "grpc-cookie-conflicts"
      type: Cookie
      absoluteTimeout: 1000s
      cookieConfig:
        lifetimeType: Permanent

  Parents:
    Conditions:
      Last Transition Time:  2025-11-18T21:53:31Z
      Message:               All references are resolved
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
      Last Transition Time:  2025-11-18T21:53:31Z
      Message:               Session Persistence configuration ignored: for backendRefs default/grpc-infra-backend-v2:8080 due to conflicting configuration across multiple rules
      Observed Generation:   1
      Reason:                InvalidSessionPersistence
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.nginx.org/nginx-gateway-controller

Testing Session Persistence

Testing is done using a script by grabbing the cookie session id from the first request and using it in curl request to ensure each request goes to the same backend

HTTPRoutes (coffee -> sticky, tea -> regular)


upstream default_coffee_80 {
    random two least_conn;
    zone default_coffee_80 1m;
    sticky cookie coffee-cookie expires=600s path=/coffee;

    state /var/lib/nginx/state/default_coffee_80.conf; --> 3 endpoints
}

upstream default_tea_80 {
    random two least_conn;
    zone default_tea_80 1m;

    state /var/lib/nginx/state/default_tea_80.conf; --> 3 endpoints
}

curl -v --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
< Set-Cookie: coffee-cookie=2b13cc98f0224fca47885f32115ef236; expires=Tue, 18-Nov-25 22:17:55 GMT; max-age=600; path=/coffee


Summary (how many times each backend was chosen):
 100 10.244.0.111:8080

curl -v --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea
Summary (how many times each backend was chosen):
  32 10.244.0.108:8080
  38 10.244.0.112:8080
  30 10.244.0.113:8080

Still Need to verify traffic splitting works with it or not.

GRPCRoutes (backend v1 --> normal, backend v2 --> sticky)

Each have 3 endpoints


upstream default_grpc-infra-backend-v1_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v1_8080 1m;

    state /var/lib/nginx/state/default_grpc-infra-backend-v1_8080.conf;

}

upstream default_grpc-infra-backend-v2_8080 {
    random two least_conn;
    zone default_grpc-infra-backend-v2_8080 1m;
    sticky cookie grpc-v2-cookie expires=600s;

    state /var/lib/nginx/state/default_grpc-infra-backend-v2_8080.conf;

}

backend 1
grpcurl -v -plaintext -proto grpc.proto -authority bar.com -d '{"name": "bar server"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello
Handling connection for 8080

Resolved method descriptor:
// Sends a greeting
rpc SayHello ( .helloworld.HelloRequest ) returns ( .helloworld.HelloReply );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc
date: Tue, 18 Nov 2025 22:26:41 GMT
server: nginx

Response contents:
{
  "message": "Hello bar server"
}

Response trailers received:
(empty)
Sent 1 request and received 1 response

backend 2

rpcurl -v -plaintext -proto grpc.proto -authority foo.bar.com -d '{"name": "bar server"}' ${GW_IP}:${GW_PORT} helloworld.Greeter/SayHello
Handling connection for 8080

Resolved method descriptor:
// Sends a greeting
rpc SayHello ( .helloworld.HelloRequest ) returns ( .helloworld.HelloReply );

Request metadata to send:
(empty)

Response headers received:
content-type: application/grpc
date: Tue, 18 Nov 2025 22:27:28 GMT
server: nginx
set-cookie: grpc-v2-cookie=cc8db88effa3e2563d0b5b13054dc7cc; expires=Tue, 18-Nov-25 22:37:28 GMT; max-age=600

Response contents:
{
  "message": "Hello bar server"
}

Response trailers received:
(empty)
Sent 1 request and received 1 response
sa.choudhary@N9939CQ4P0 grpc-routing %

Note:

Errors are not added for API spec rules.
For RegularExpression path matches, we leave the cookie Path empty. A regex can match anywhere within the URL path (for example, /coffee/[A-Za-z]+/tea), so deriving a concrete cookie path from it would be misleading and could unintentionally restrict which requests send the cookie.
Will update the design with these new restrictions shortly.
Since each backendRef creates an upstream and an upstream define sticky cookie directive we need to resolve conflicts in session persistence pertaining to each rule that has a SP config and shares the same backend. No conflict is reported when backend ref shared across rules has a same SP config

Please focus on (optional): If you any specific areas where you would like reviewers to focus their attention or provide
specific feedback, add them here.

Closes #4231

Checklist

Before creating a PR, run through this checklist and mark each as complete.

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have checked that all unit tests pass after adding my changes
I have updated necessary documentation
I have rebased my branch onto main
I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NONE

…4251) Problem: Users want to specify load balancing method via Upstream Settings Policy API Solution: Extend Upstream settings policy API to support load balancing method field.

Problem: Users want to be able to specify ip_hash load balancing for upstreams Solution: Add support for session affinity using ip_hash directive in upstreams

salonichf5 · 2025-11-19T14:30:22Z

internal/controller/nginx/config/validation/common.go

+	units := []unit{
+		{"ms", 1},
+		{"s", 1000},
+		{"m", 60 * 1000},
+		{"h", 60 * 60 * 1000},


can switch its order if we want larger units first.

salonichf5 · 2025-11-19T14:43:49Z

internal/controller/state/graph/route_common.go

+func equalSessionPersistenceConfig(a, b *SessionPersistenceConfig) bool {
+	if a == nil || b == nil {
+		return a == b
+	}
+	return a.Name == b.Name &&
+		a.SessionType == b.SessionType &&
+		a.Expiry == b.Expiry &&
+		a.Path == b.Path
+}
+


Thought about merging session persistence configs when overlapping fields are not present like we do for policies. But we do not have a large number of fields that get set here, so it did not make sense to me add more complexity here.

we don't support idleTimeout

absoluteTimeout required whenn cookie lifetime is permanent

session name cannot be empty or else SP is not considered

and we only configure session type cookie

sjberman · 2025-11-19T15:35:41Z

FYI, the release note in the PR description isn't going to be used at all since this is being merged into the feature branch. For the main PR that we merge at the end, we'll want a descriptive release note to discuss the different ways session persistence is supported.

sjberman · 2025-11-19T17:27:51Z

See my comment on the other PR, I think we have to rethink this a bit to support a backend being referenced multiple times, to prevent unintended behavior or blocking desired behavior.

salonichf5 added 2 commits November 17, 2025 18:50

Add a loadBalancingMethod field to the Upstream Settings Policy API (#…

15061f8

…4251) Problem: Users want to specify load balancing method via Upstream Settings Policy API Solution: Extend Upstream settings policy API to support load balancing method field.

Add ip_hash based session affinity support for NGINX OSS (#4257)

2d689a5

Problem: Users want to be able to specify ip_hash load balancing for upstreams Solution: Add support for session affinity using ip_hash directive in upstreams

github-project-automation bot added this to NGINX Gateway Fabric Nov 18, 2025

github-project-automation bot moved this to 🆕 New in NGINX Gateway Fabric Nov 18, 2025

nginx-bot bot added the release-notes label Nov 18, 2025

github-actions bot added the enhancement New feature or request label Nov 18, 2025

salonichf5 changed the title ~~Add session persistence support for NGINX Plus users using the sticky cookie directive~~ Do not review: Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 commented Nov 19, 2025

View reviewed changes

add session persistence support for NGINX plus users

ded1099

salonichf5 force-pushed the feat/plus-session-persistence branch from 622cc92 to ded1099 Compare November 19, 2025 14:47

salonichf5 changed the title ~~Do not review: Add session persistence support for NGINX Plus users using the sticky cookie directive~~ Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 marked this pull request as ready for review November 19, 2025 14:49

salonichf5 requested a review from a team as a code owner November 19, 2025 14:49

nginx-bot bot removed the release-notes label Nov 19, 2025

salonichf5 changed the title ~~Add session persistence support for NGINX Plus users using the sticky cookie directive~~ DNR: Add session persistence support for NGINX Plus users using the sticky cookie directive Nov 19, 2025

salonichf5 force-pushed the feat/session-persistence branch from 2d689a5 to 9863bd1 Compare November 21, 2025 04:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

DNR: Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

DNR: Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

Uh oh!

salonichf5 commented Nov 18, 2025 •

edited

Loading

Uh oh!

salonichf5 Nov 19, 2025

Uh oh!

salonichf5 Nov 19, 2025 •

edited

Loading

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DNR: Add session persistence support for NGINX Plus users using the sticky cookie directive #4305

Are you sure you want to change the base?

DNR: Add session persistence support for NGINX Plus users using the sticky cookie directive #4305

Uh oh!

Conversation

salonichf5 commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Checklist

Release notes

Uh oh!

salonichf5 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

salonichf5 Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

sjberman commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DNR: Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

DNR: Add session persistence support for NGINX Plus users using the `sticky cookie` directive #4305

salonichf5 commented Nov 18, 2025 •

edited

Loading

salonichf5 Nov 19, 2025 •

edited

Loading