Automating OCP-55033 in Origin #30512

asahay19 · 2025-11-21T09:14:36Z

This case is about checking the Kubelet log level is 2.
Here is the test case link : https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-55033

It is duplicate of PR: #30460
PR: 30460 got closed because I had to delete repo, so creating new PR again with addressing all the comments.

PTAL @sairameshv @lyman9966 @cpmeadors

Here is the output for this PR . It got passed successfully while executing it on my local :

./openshift-tests run-test "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]"

Running Suite: - /Users/asahay/newOCP-55033/origin

Random Seed: 1764311478 - will randomize all specs

Will run 1 of 1 specs

[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL
github.com/openshift/origin/test/extended/node/node_e2e/node.go:20
STEP: Creating a kubernetes client @ 11/28/25 12:01:22.453
I1128 12:01:22.455698 58826 discovery.go:214] Invalidating discovery information
STEP: Polling to check kubelet log level on ready nodes @ 11/28/25 12:01:22.455
STEP: Getting all node names in the cluster @ 11/28/25 12:01:32.457
I1128 12:01:35.246189 58826 node.go:30]
Node Names are ip-10-0-21-47.us-east-2.compute.internal ip-10-0-29-148.us-east-2.compute.internal ip-10-0-43-146.us-east-2.compute.internal ip-10-0-53-99.us-east-2.compute.internal ip-10-0-76-236.us-east-2.compute.internal ip-10-0-87-5.us-east-2.compute.internal
STEP: Checking if node ip-10-0-21-47.us-east-2.compute.internal is Ready @ 11/28/25 12:01:35.246
I1128 12:01:36.338169 58826 node.go:37]
Node ip-10-0-21-47.us-east-2.compute.internal Status is True

STEP: Checking KUBELET_LOG_LEVEL in kubelet.service on node ip-10-0-21-47.us-east-2.compute.internal @ 11/28/25 12:01:36.338
STEP: Checking kubelet process for --v=2 flag on node ip-10-0-21-47.us-east-2.compute.internal @ 11/28/25 12:01:44.469
STEP: Verifying KUBELET_LOG_LEVEL is set and kubelet is running with --v=2 @ 11/28/25 12:01:47.469

I1128 12:01:47.470106 58826 node.go:50] KUBELET_LOG_LEVEL is 2.

• [25.041 seconds]

Ran 1 of 1 Specs in 25.042 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
{
"name": "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]",
"lifecycle": "blocking",
"duration": 25042,
"startTime": "2025-11-28 06:31:22.430978 UTC",
"endTime": "2025-11-28 06:31:47.473486 UTC",
"result": "passed",
"output": " STEP: Creating a kubernetes client @ 11/28/25 12:01:22.453\n STEP: Polling to check kubelet log level on ready nodes @ 11/28/25 12:01:22.455\n STEP: Getting all node names in the cluster @ 11/28/25 12:01:32.457\nI1128 12:01:35.246189 58826 node.go:30] \nNode Names are ip-10-0-21-47.us-east-2.compute.internal ip-10-0-29-148.us-east-2.compute.internal ip-10-0-43-146.us-east-2.compute.internal ip-10-0-53-99.us-east-2.compute.internal ip-10-0-76-236.us-east-2.compute.internal ip-10-0-87-5.us-east-2.compute.internal\n STEP: Checking if node ip-10-0-21-47.us-east-2.compute.internal is Ready @ 11/28/25 12:01:35.246\nI1128 12:01:36.338169 58826 node.go:37] \nNode ip-10-0-21-47.us-east-2.compute.internal Status is True\n\n STEP: Checking KUBELET_LOG_LEVEL in kubelet.service on node ip-10-0-21-47.us-east-2.compute.internal @ 11/28/25 12:01:36.338\n STEP: Checking kubelet process for --v=2 flag on node ip-10-0-21-47.us-east-2.compute.internal @ 11/28/25 12:01:44.469\n STEP: Verifying KUBELET_LOG_LEVEL is set and kubelet is running with --v=2 @ 11/28/25 12:01:47.469\nI1128 12:01:47.470106 58826 node.go:50] KUBELET_LOG_LEVEL is 2.\n\n"
}
]%

openshift-ci-robot · 2025-11-21T09:14:38Z

Pipeline controller notification
This repository is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

cpmeadors · 2025-11-21T19:32:52Z

test/extended/node/node_e2e/node.go

+	//author: [email protected]
+	g.It("check KUBELET_LOG_LEVEL is 2", func() {
+		g.By("check Kubelet Log Level\n")
+		assertKubeletLogLevel(oc)


It is a better practice to have all the assertions and test logic in the test. The helper functions should just do stuff and return information. This makes the tests more readable and allows the helper functions to be more reusable.

cpmeadors · 2025-11-21T19:34:46Z

test/extended/node/node_e2e/node_utils.go

+	e2e "k8s.io/kubernetes/test/e2e/framework"
+)
+
+func assertKubeletLogLevel(oc *exutil.CLI) {


Do you expect that this function will be used in another test? Maybe it is not needed. Just put it all in the test. We can refactor later if we discover that is it reusable.

cpmeadors · 2025-11-21T19:37:14Z

test/extended/node/node_e2e/node_utils.go

+	var kublet string
+	var err error
+	waitErr := wait.Poll(10*time.Second, 1*time.Minute, func() (bool, error) {
+		nodeName, nodeErr := oc.AsAdmin().WithoutNamespace().Run("get").Args("nodes", "-o=jsonpath={.items[*].metadata.name}").Output()


Assuming this code gets factored back in to the test, it would be good to see some By()s to clarify what the test is doing.

openshift-ci-robot · 2025-11-25T10:18:41Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

asahay19 · 2025-11-25T13:33:09Z

/test e2e-vsphere-ovn-upi

asahay19 · 2025-11-25T13:33:26Z

/test e2e-vsphere-ovn

cpmeadors · 2025-11-25T15:43:56Z

test/extended/node/node_e2e/node.go

+	e2e "k8s.io/kubernetes/test/e2e/framework"
+)
+
+var _ = g.Describe("[sig-node] Kubelet, CRI-O, CPU manager", func() {


The new labeling method is to use [jira:"Node / Kubelet]"] per https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0#heading=h.v059hn4ylc6b

also "Add the “[OTP]” annotation in the test name to tests that are ported, this is for tracking purposes so we can get a rough idea of how many tests have come over."

cpmeadors · 2025-11-25T21:38:14Z

/lgtm

lyman9966 · 2025-11-26T09:53:27Z

/lgtm

openshift-ci · 2025-11-26T09:53:49Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: asahay19, cpmeadors, lyman9966
Once this PR has been reviewed and has the lgtm label, please assign neisw for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

neisw · 2025-11-27T15:45:15Z

test/extended/node/node_e2e/node.go

+	)
+
+	//author: [email protected]
+	g.It("check KUBELET_LOG_LEVEL is 2", func() {


One thing we advise against is putting specific criteria in test names like '2'. This could be validate KUBELET_LOG_LEVEL and if the desired log level ever changed the test could be updated without changing the test name or losing history.

neisw · 2025-11-27T16:08:59Z

I see where your test ran from the previous pr prior to 2989d03201

But I don't see that it ran in any presubmits on this pr.

openshift-ci · 2025-11-28T06:36:27Z

New changes are detected. LGTM label has been removed.

asahay19 · 2025-11-28T06:36:56Z

I see where your test ran from the previous pr prior to 2989d03201

But I don't see that it ran in any presubmits on this pr.

yes, I have updated the PR description with the latest test output result. Thank you

openshift-ci-robot · 2025-11-28T07:01:40Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

neisw · 2025-11-29T21:42:37Z

Thanks, need to see what the issue is with those failures:

[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 0, Fail: 5, Flake: 0]

asahay19 · 2025-12-01T07:57:07Z

/retest

asahay19 · 2025-12-02T05:55:40Z

/retest

openshift-ci · 2025-12-02T09:28:58Z

@asahay19: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-fips	`94022ca`	link	true	`/test e2e-aws-ovn-fips`
ci/prow/e2e-vsphere-ovn-upi	`94022ca`	link	true	`/test e2e-vsphere-ovn-upi`
ci/prow/e2e-vsphere-ovn	`94022ca`	link	true	`/test e2e-vsphere-ovn`
ci/prow/e2e-gcp-ovn	`94022ca`	link	true	`/test e2e-gcp-ovn`
ci/prow/e2e-metal-ipi-ovn-ipv6	`94022ca`	link	true	`/test e2e-metal-ipi-ovn-ipv6`
ci/prow/e2e-aws-ovn-microshift	`94022ca`	link	true	`/test e2e-aws-ovn-microshift`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-trt · 2025-12-02T10:13:23Z

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 94022ca

Job Name	New Test Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-fips	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that failed 3 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-microshift	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that failed 3 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that was not present in all runs against the current commit, and also failed 2 time(s).
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that failed 3 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that failed 3 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi	High - "[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" is a new test that was not present in all runs against the current commit, and also failed 2 time(s).

New tests seen in this PR at sha: 94022ca

"[sig-node] Kubelet, CRI-O, CPU manager validate KUBELET_LOG_LEVEL [Suite:openshift/conformance/parallel]" [Total: 18, Pass: 2, Fail: 16, Flake: 0]

asahay19 mentioned this pull request Nov 21, 2025

Automation of OCP-55033 #30460

Closed

openshift-ci bot requested review from deads2k and p0lyn0mial November 21, 2025 09:16

cpmeadors reviewed Nov 21, 2025

View reviewed changes

asahay19 force-pushed the new55033 branch from 3ac7058 to 5fc8060 Compare November 25, 2025 06:36

asahay19 requested a review from cpmeadors November 25, 2025 06:55

cpmeadors reviewed Nov 25, 2025

View reviewed changes

openshift-ci bot assigned cpmeadors Nov 25, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 25, 2025

openshift-ci bot assigned lyman9966 Nov 26, 2025

neisw reviewed Nov 27, 2025

View reviewed changes

Automating OCP-55033 in Origin

94022ca

asahay19 force-pushed the new55033 branch from 5fc8060 to 94022ca Compare November 28, 2025 06:36

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 28, 2025

Automating OCP-55033 in Origin #30512

Are you sure you want to change the base?

Automating OCP-55033 in Origin #30512

Conversation

asahay19 commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Running Suite: - /Users/asahay/newOCP-55033/origin

Will run 1 of 1 specs

• [25.041 seconds]

Uh oh!

openshift-ci-robot commented Nov 21, 2025

Uh oh!

cpmeadors Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

cpmeadors Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

cpmeadors Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Nov 25, 2025

Uh oh!

asahay19 commented Nov 25, 2025

Uh oh!

asahay19 commented Nov 25, 2025

Uh oh!

cpmeadors Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

cpmeadors Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

cpmeadors commented Nov 25, 2025

Uh oh!

lyman9966 commented Nov 26, 2025

Uh oh!

openshift-ci bot commented Nov 26, 2025

Uh oh!

neisw Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

asahay19 Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

neisw commented Nov 27, 2025

Uh oh!

openshift-ci bot commented Nov 28, 2025

Uh oh!

asahay19 commented Nov 28, 2025

Uh oh!

openshift-ci-robot commented Nov 28, 2025

Uh oh!

neisw commented Nov 29, 2025

Uh oh!

asahay19 commented Dec 1, 2025

Uh oh!

asahay19 commented Dec 2, 2025

Uh oh!

openshift-ci bot commented Dec 2, 2025

Uh oh!

openshift-trt bot commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

asahay19 commented Nov 21, 2025 •

edited

Loading