Skip to content

Conversation

@oliver-sanders
Copy link
Member

In order to perform memory profiling for Cylc it is generally necessary to disable the lru_cache mechanism that we use to speedup datetime / recurrence operations.

Let's make this configurable via an environment variable rather than performing this hack every time.

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • Changelog entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@oliver-sanders oliver-sanders added this to the 8.7.0 milestone Nov 12, 2025
@oliver-sanders oliver-sanders self-assigned this Nov 12, 2025
Copy link
Contributor

@ChrisPaulBennett ChrisPaulBennett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ChrisPaulBennett
Copy link
Contributor

I closed and reopened the PR to hopefully clear the macOS test that is failing

@oliver-sanders
Copy link
Member Author

Tests happy.

Coverage complaining about pre-existing coverage holes.

@oliver-sanders oliver-sanders requested a review from wxtim November 24, 2025 17:29


# NOTE: We cache some datetime cycling operations to improve compute
# perforance. For profiling, this can be disabled by setting the environment
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# perforance. For profiling, this can be disabled by setting the environment
# performance. For profiling, this can be disabled by setting the environment

@wxtim
Copy link
Member

wxtim commented Nov 25, 2025

It's fine as far as it goes. Do you want to consider these other cases (there aren't many):

cylc/flow/host_select.py:516:@lru_cache()
cylc/flow/host_select.py-517-def _tuple_factory(name, params):
--
cylc/flow/scheduler_cli.py:313:@lru_cache()
cylc/flow/scheduler_cli.py-314-def get_option_parser(add_std_opts: bool = False) -> COP:
--
cylc/flow/util.py:215:@lru_cache(maxsize=100)
cylc/flow/util.py-216-def _serialise_set(flow_nums: tuple) -> str:
--
cylc/flow/util.py:220:@lru_cache(maxsize=100)
cylc/flow/util.py-221-def deserialise_set(flow_num_str: str) -> set:

Copy link
Member

@wxtim wxtim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

@oliver-sanders
Copy link
Member Author

It's fine as far as it goes. Do you want to consider these other cases (there aren't many):

I went with "no" on those:

  • Small cache size.
  • Small object size.
  • Unlikely to ever accumulate (flow usage non-existent, host-select only performed on startup).

@wxtim wxtim merged commit fe21fd9 into cylc:master Nov 25, 2025
47 of 55 checks passed
@oliver-sanders oliver-sanders deleted the lru_cache branch November 25, 2025 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants