Skip to content

[JENKINS-76264] Stuck pod causing Jenkins break #2767

@jenkins-infra-bot

Description

@jenkins-infra-bot

Jenkins agent is removed from jenkins somehow, but agent pod is still running on kubernetes cluster, so k8s pod tries to re-connect every 10s.

Jenkins is refusing connection:

2025-11-04 09:52:00.882+0000 [id=2333]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #1099 from /10.123.2.253:29485 failed: null
2025-11-04 09:52:00.883+0000 [id=2334]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #1100 from /10.123.2.253:6644
2025-11-04 09:52:00.890+0000 [id=2287]    INFO    o.j.r.p.i.ConnectionHeadersFilterLayer#onRecv: [JNLP4-connect connection from core.prod.publishdrive.com/10.123.2.253:6644] Refusing headers from remote: Unknown client name: livesales-43-storytel-1591-wntv6-0q38w 

Connection id is increasing and several threads are open in Java:

Handling GET /tcpSlaveAgentListener/ from 10.123.2.253 : Jetty (winstone)-26 

Something seems to be leaking, because after a while, I can see 100% CPU usage on Garbage Collector threads:

1991964 mainten+  20   0 6099204   2.7g  10604 R  92.4  34.5 215:20.83 GC Thread#1
1991966 mainten+  20   0 6099204   2.7g  10604 R  88.4  34.5 215:26.16 GC Thread#3
1991311 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:21.00 GC Thread#0
1991965 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:13.79 GC Thread#2 

I sent kill -3, here are the stack trace: java.txt

Can you please:

  • fix memory leak
  • help to set connection retry limit in kubernetes agent pod

?

Thanks in advice!

 

BR,

Tamas

 

 


Originally reported by kiskele, imported from: Stuck pod causing Jenkins break
  • status: Open
  • priority: Major
  • component(s): kubernetes-plugin
  • resolution: Unresolved
  • votes: 0
  • watchers: 2
  • imported: 2025-12-02
Raw content of original issue

Jenkins agent is removed from jenkins somehow, but agent pod is still running on kubernetes cluster, so k8s pod tries to re-connect every 10s.

Jenkins is refusing connection:

2025-11-04 09:52:00.882+0000 [id=2333]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Connection #1099 from /10.123.2.253:29485 failed: null
2025-11-04 09:52:00.883+0000 [id=2334]    INFO    h.TcpSlaveAgentListener$ConnectionHandler#run: Accepted JNLP4-connect connection #1100 from /10.123.2.253:6644
2025-11-04 09:52:00.890+0000 [id=2287]    INFO    o.j.r.p.i.ConnectionHeadersFilterLayer#onRecv: [JNLP4-connect connection from core.prod.publishdrive.com/10.123.2.253:6644] Refusing headers from remote: Unknown client name: livesales-43-storytel-1591-wntv6-0q38w 

Connection id is increasing and several threads are open in Java:

Handling GET /tcpSlaveAgentListener/ from 10.123.2.253 : Jetty (winstone)-26 

Something seems to be leaking, because after a while, I can see 100% CPU usage on Garbage Collector threads:

1991964 mainten+  20   0 6099204   2.7g  10604 R  92.4  34.5 215:20.83 GC Thread#1
1991966 mainten+  20   0 6099204   2.7g  10604 R  88.4  34.5 215:26.16 GC Thread#3
1991311 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:21.00 GC Thread#0
1991965 mainten+  20   0 6099204   2.7g  10604 R  82.7  34.5 215:13.79 GC Thread#2 

I sent kill -3, here are the stack trace: java.txt

Can you please:

  • fix memory leak
  • help to set connection retry limit in kubernetes agent pod

?

Thanks in advice!

 

BR,

Tamas

 

 

environment
Official Jenkins Docker image 2.534<br/>
Kubernetes plugin 4384.v1b_6367f393d9
1 attachment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions