Skip to content

Conversation

stefanunity
Copy link
Collaborator

@stefanunity stefanunity commented Sep 5, 2024

Description

This enables our current performance tests to be run continiously (daily on develop, weekly on master).

Changes made

  • created job that runs only the performance tests on all platforms (except tvOS)
    • these can also be run individually
  • performance only test get triggered to run daily on develop and weekly on master branch
  • updated com.unity.test-framework to 1.4.5 (from 1.4.3)
  • updated com.unity.test-framework.performance to 3.0.3 (from 3.0.2)
  • edits to shorten CI job names so they don't get clipped by yamato UI/better readility

Testing

  • run All Tests jobs successfully
  • run all performance jobs tests successfully
    • performance data does get submitted to observer
  • NOTICE: there are some yellow yamato jobs. this happens because I renamed them and the github branch protection still has the old names. see All Tests job for all the green. once this lands I change the required jobs in git branch protection.

Risk

Very little risk, all tests still run, no changes to the tests themselves.

Checklist

Before review:

  • Changelog entry added.
    • Explains the change in Changed, Fixed, Added sections.
    • For API change contains an example snippet and/or migration example.
    • JIRA ticket linked, example (case %%). If it is a private issue, just add the case ID without a link.
    • Jira port for the next release set as "Resolved".
  • Tests added/changed, if applicable.
    • Functional tests Area_CanDoX, Area_CanDoX_EvenIfYIsTheCase, Area_WhenIDoX_AndYHappens_ThisIsTheResult.
    • Performance tests.
    • Integration tests.
  • Docs for new/changed API's.
    • Xmldoc cross references are set correctly.
    • Added explanation how the API works.
    • Usage code examples added.
    • The manual is updated, if needed.

During merge:

  • Commit message for squash-merge is prefixed with one of the list:
    • NEW: ___.
    • FIX: ___.
    • DOCS: ___.
    • CHANGE: ___.
    • RELEASE: 1.1.0-preview.3.

After merge:

  • Create forward/backward port if needed. If you are blocked from creating a forward port now please add a task to ISX-1444.

@stefanunity stefanunity changed the title enable performance test to be run continiously on U6 CHANGE: enable performance test to be run continiously on U6 Sep 5, 2024
Copy link
Collaborator

@jfreire-unity jfreire-unity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change required since we don't have a master branch.

I'm ok with approving once that's fixed.

However, could just explain what's the goal with having this in CI? It's not clear to me.

More precisely, it's unclear to me what is the goal of running this performance jobs daily in case there are no changes to the package? Is the goal to test against different Editor patch versions ?

@stefanunity
Copy link
Collaborator Author

More precisely, it's unclear to me what is the goal of running this performance jobs daily in case there are no changes to the package? Is the goal to test against different Editor patch versions ?

These test are currently almost never run on develop. Since nobody runs test on develop, except for me before a release.
If we want to start making performance improvements and look at these tests for PRs too, then we need a baseline to compare against.
This baseline needs to be more then "one run from 2 months ago". Running these regularily on develop will hopefully give is that baseline.

@jfreire-unity jfreire-unity self-requested a review September 6, 2024 13:10
Copy link
Collaborator

@lyndon-unity lyndon-unity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good update overall.

@stefanunity stefanunity merged commit 1d3505d into develop Sep 6, 2024
77 checks passed
@stefanunity stefanunity deleted the input/performance-testing-changes branch September 6, 2024 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants