Skip to content

Conversation

@Lazin
Copy link
Contributor

@Lazin Lazin commented Nov 7, 2025

The test uses constant timestamp to generate records and query them. But the timestamp is too old so retention policy starts to remove data while the test is running. This commit makes the test use proper base timestamp.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.3.x
  • v25.2.x
  • v25.1.x
  • v24.3.x

Release Notes

  • none

@Lazin Lazin requested review from Copilot and dotnwat November 7, 2025 16:15
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a failing timequery test by updating the base timestamp used for generating test records. The test was using a hardcoded timestamp from 2022 (1664453149000), which is now old enough that Redpanda's retention policy would delete the data while the test is running, causing test failures. The fix changes to using the current system time as the base timestamp.

  • Changed hardcoded old timestamp to dynamically generated current timestamp using int(time.time()) * 1000

@Lazin Lazin changed the title rptest: Fix timequery test [CORE-14434] rptest: Fix timequery test Nov 7, 2025
@ducktape_cluster(num_nodes=7)
def test_timequery(self):
base_ts = 1664453149000
base_ts = int(time.time()) * 1000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot more base_ts = 1664453149000 in this file, do more sites need to be amended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this point I think a base class with one accessible self.base_ts definition might be more ideal but I don't feel strongly about you changing it for this PR.

@Lazin Lazin force-pushed the fix/timequery-test branch from 243c952 to 2bed837 Compare November 7, 2025 17:29
@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Nov 7, 2025

CI test results

test results on build#75855
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
SegmentMsTest test_segment_rolling_with_retention_consumer null integration https://buildkite.com/redpanda/redpanda/builds/75855#019a5f78-71d7-470c-a0dc-5d59527b4e65 FLAKY 20/21 upstream reliability is '94.07191448007775'. current run reliability is '95.23809523809523'. drift is -1.16618 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=SegmentMsTest&test_method=test_segment_rolling_with_retention_consumer
TimeQueryTest test_timequery_with_local_gc null integration https://buildkite.com/redpanda/redpanda/builds/75855#019a5f70-3b0f-4df6-94cc-746726b166cd FLAKY 13/21 upstream reliability is '100.0'. current run reliability is '61.904761904761905'. drift is 38.09524 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=TimeQueryTest&test_method=test_timequery_with_local_gc
test results on build#76111
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ControllerLogLimitMirrorMakerTests test_mirror_maker_with_limits null integration https://buildkite.com/redpanda/redpanda/builds/76111#019a7825-98c7-4a4d-83b9-5ecb7e5f68f4 FLAKY 20/21 upstream reliability is '98.95104895104895'. current run reliability is '95.23809523809523'. drift is 3.71295 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ControllerLogLimitMirrorMakerTests&test_method=test_mirror_maker_with_limits
MountUnmountIcebergTest test_simple_remount {"cloud_storage_type": 1} integration https://buildkite.com/redpanda/redpanda/builds/76111#019a7825-98c5-4515-8fef-f3fcedb42381 FLAKY 18/21 upstream reliability is '87.09394205443371'. current run reliability is '85.71428571428571'. drift is 1.37966 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=MountUnmountIcebergTest&test_method=test_simple_remount
TieredStorageTest test_tiered_storage {"cloud_storage_type_and_url_style": [1, "virtual_host"], "test_case": {"name": "(TS_Read == True, TS_Timequery == True)"}} integration https://buildkite.com/redpanda/redpanda/builds/76111#019a7825-98d9-465c-b8c4-076302f4b1d2 FLAKY 20/21 upstream reliability is '99.77220956719817'. current run reliability is '95.23809523809523'. drift is 4.53411 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=TieredStorageTest&test_method=test_tiered_storage
test results on build#76127
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ShadowLinkingReplicationTests test_replication_basic {"shuffle_leadership": true, "source_cluster_spec": {"cluster_type": "redpanda"}} integration https://buildkite.com/redpanda/redpanda/builds/76127#019a78c2-73d3-467f-9c84-644273d970be FLAKY 18/21 upstream reliability is '88.76551273677336'. current run reliability is '85.71428571428571'. drift is 3.05123 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_replication_basic
WriteCachingFailureInjectionE2ETest test_crash_all {"use_transactions": false} integration https://buildkite.com/redpanda/redpanda/builds/76127#019a78b9-f05c-4f55-8416-d10cd2526864 FLAKY 19/21 upstream reliability is '91.06628242074927'. current run reliability is '90.47619047619048'. drift is 0.59009 and the allowed drift is set to 50. The test should PASS https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionE2ETest&test_method=test_crash_all

@ducktape_cluster(num_nodes=7)
def test_timequery(self):
base_ts = 1664453149000
base_ts = int(time.time()) * 1000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this point I think a base class with one accessible self.base_ts definition might be more ideal but I don't feel strongly about you changing it for this PR.

Copy link
Contributor

@WillemKauf WillemKauf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test uses constant timestamp to generate records and query them. But
the timestamp is too old so retention policy starts to remove data while
the test is running. This commit makes the test use proper base
timestamp.

Signed-off-by: Evgeny Lazin <[email protected]>
@Lazin Lazin force-pushed the fix/timequery-test branch from c9e415a to 033ca19 Compare November 12, 2025 15:19
@Lazin Lazin requested review from WillemKauf and removed request for a team, dotnwat, kbatuigas, michael-redpanda, r-vasquez and rockwotj November 12, 2025 15:19
@WillemKauf
Copy link
Contributor

/ci-repeat 5
dt-repeat=10
skip-units
skip-redpanda-build
tests/rptest/tests/timequery_test.py::TimeQueryTest

Copy link
Contributor

@WillemKauf WillemKauf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice lgtm

@Lazin Lazin merged commit 236fd93 into redpanda-data:dev Nov 12, 2025
17 checks passed
@vbotbuildovich
Copy link
Collaborator

/backport v25.3.x

@vbotbuildovich
Copy link
Collaborator

/backport v25.2.x

@vbotbuildovich
Copy link
Collaborator

/backport v25.1.x

@vbotbuildovich
Copy link
Collaborator

/backport v24.3.x

@vbotbuildovich
Copy link
Collaborator

Branch name "v25.3.x" not found.

Workflow run logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants