Skip to content

Parallelized Simulation Tests#3766

Merged
Andrewyx merged 24 commits into
UBC-Thunderbots:masterfrom
Andrewyx:Andrewyx/parallel_tests
Jun 8, 2026
Merged

Parallelized Simulation Tests#3766
Andrewyx merged 24 commits into
UBC-Thunderbots:masterfrom
Andrewyx:Andrewyx/parallel_tests

Conversation

@Andrewyx

@Andrewyx Andrewyx commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Description

Finally runs our simulated tests in parallel! 50 min -> 15 min CI test times! Almost a 400% reduction! Some of the changes here are to either prevent race conditions or to prevent processes from terminating each other.

  • Adds symlinks to workaround path lengths (i.e. get_runtime_dir())
  • Fixes some flaky tests by extending some test durations

Testing Done

All tests pass.

Resolved Issues

Resolves #2619

Length Justification and Key Files to Review

Review Checklist

It is the reviewers responsibility to also make sure every item here has been covered

  • Function & Class comments: All function definitions (usually in the .h file) should have a javadoc style comment at the start of them. For examples, see the functions defined in thunderbots/software/geom. Similarly, all classes should have an associated Javadoc comment explaining the purpose of the class.
  • Remove all commented out code
  • Remove extra print statements: for example, those just used for testing
  • Resolve all TODO's: All TODO (or similar) statements should either be completed or associated with a github issue

@Andrewyx Andrewyx marked this pull request as ready for review June 7, 2026 05:19
williamckha
williamckha previously approved these changes Jun 8, 2026
Comment thread src/software/thunderscope/binary_context_managers/game_controller.py Outdated
Comment thread src/software/field_tests/field_test_fixture.py
Comment thread src/software/thunderscope/binary_context_managers/game_controller.py Outdated

@someone2060 someone2060 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made an actual review below
Why are some tags arguments completely removed, whilst others only remove the "exclusive" portion?

As an example: src/software/ai/hl/stp/tactic/attacker/BUILD:41 has the tags arg fully removed, but src/software/ai/hl/stp/tactic/chip/BUILD:38 only has "exclusive" removed

@nycrat nycrat left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far; On my macbook the entire simulated test suite ran in less than 3 minutes! I left a few small comments, and also the tags property should be deleted from all of the BUILD files like Daniel pointed out

Comment thread src/software/simulated_tests/simulated_test_fixture.py
Comment thread src/software/simulated_tests/simulated_test_fixture.py
Comment thread src/software/simulated_tests/simulated_test_fixture.py
Comment thread src/software/ai/hl/stp/tactic/pivot_kick/BUILD
Comment thread src/software/simulated_tests/simulated_test_fixture.py
ag_eventually_validation_sequence_set=eventually_validation_sequence_set,
ag_always_validation_sequence_set=always_validation_sequence_set,
test_timeout_s=10,
test_timeout_s=60,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

60 seems a bit too long compared to other tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its very flaky, I think this + ball placement needs dedicated investigation.

@StarrryNight StarrryNight left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! left some comments

@nycrat nycrat left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@someone2060 someone2060 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@williamckha williamckha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes us wonder how we lived with 1 hr+ long CI times...

@Andrewyx Andrewyx merged commit 1761a61 into UBC-Thunderbots:master Jun 8, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simulated Pytests: Freezes when running in parallel

5 participants