Skip to content

[SUREFIRE-1643] surefire junit5 parallel tests#815

Merged
olamy merged 28 commits intomasterfrom
SUREFIRE-1643-junit5-parallel
Mar 23, 2025
Merged

[SUREFIRE-1643] surefire junit5 parallel tests#815
olamy merged 28 commits intomasterfrom
SUREFIRE-1643-junit5-parallel

Conversation

@olamy
Copy link
Member

@olamy olamy commented Mar 17, 2025 •
edited
Loading

  • [SUREFIRE-1643] Make surefire junit5 supporting parallel execution and not mixing result when re running tests

Following this checklist to help us incorporate your
contribution quickly and easily:

  • Make sure there is a JIRA issue filed
    for the change (usually before you start working on it). Trivial changes like typos do not
    require a JIRA issue. Your pull request should address just this issue, without
    pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
    where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
    is to use the JIRA issue title in the pull request title and in the first line of the
    commit message.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean install to make sure basic checks pass. A more thorough check will
    be performed on your pull request automatically.
  • You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

olamy added 15 commits March 18, 2025 09:08
@olamy
[SUREFIRE-1643] Make surefire junit5 supporting parallel execution an…
9385ee5 
…d not mixing result when re running tests

Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
java 21
584fcd6 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
fix typo
81bf277 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
build with mvn 3.8.8
296e96a 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
avoid NPE
72b1c8a 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
use getName if no sourceName
36bdc88 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
restore backward compat
f59cc1d 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
simplify jenkinsfile
d2e7858 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
comment
8e24d2c 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
comment
8b1ee72 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
ignore no sense test
0f2e788 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
do not use LegacyXmlReportGeneratingListener for junit5
74dc32c 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
[SUREFIRE-1643] fix backward compatibility with other providers
ebc4ee9 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
ignore possible flaky test
2123b2e 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
upgrade to last mockito supporting java8 and fix mockito usage
7995ebc 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy olamy force-pushed the SUREFIRE-1643-junit5-parallel branch from 4f3bdd2 to 7995ebc Compare March 18, 2025 00:23
olamy added 10 commits March 18, 2025 10:29
@olamy
use more nodes
2b1f9c2 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
relax this performance test
1c36c16 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
Test IT test for SUREFIRE-1643
6a27e38 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
[SUREFIRE-1643] fix plumbing
1ca4df9 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
dependency not needed
e6fe26c 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
remove not needed change
c26e98d 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
remove not needed change
c41541f 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
test with default forknode impl
92cb07b 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
fix last it
5c7e738 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
reduce number of ITS threads running in parallel
c481a99 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy olamy added the bug Something isn't working label Mar 20, 2025
@olamy olamy changed the title [WIP] [SUREFIRE-1643] surefire junit5 parallel tests [SUREFIRE-1643] surefire junit5 parallel tests Mar 20, 2025
@olamy olamy marked this pull request as ready for review March 20, 2025 01:23
@olamy
remove useless docck-maven-plugin
ae50321 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@slawekjaranowski slawekjaranowski self-requested a review March 20, 2025 06:38
slawekjaranowski
slawekjaranowski reviewed Mar 20, 2025
View reviewed changes
Copy link
Member

@slawekjaranowski slawekjaranowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first question after code read ...
I will try to test on some of my projects

olamy added 2 commits March 20, 2025 17:22
@olamy
fix version in IT file
7402d84 
Signed-off-by: Olivier Lamy <olamy@apache.org>
@olamy
restore a core version we do not even support anymore
cced9cd 
Signed-off-by: Olivier Lamy <olamy@apache.org>
slawekjaranowski
slawekjaranowski approved these changes Mar 23, 2025
View reviewed changes
Copy link
Member

@slawekjaranowski slawekjaranowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on my private project ... looks ok.

@olamy olamy merged commit 5aeca19 into master Mar 23, 2025
39 checks passed
@olamy olamy deleted the SUREFIRE-1643-junit5-parallel branch March 23, 2025 11:50
@github-actions github-actions bot added this to the next-release milestone Mar 23, 2025
@fslevoaca-ionos
Copy link

fslevoaca-ionos commented Mar 31, 2025

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml
Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

@olamy
Copy link
Member Author

olamy commented Mar 31, 2025

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

please create another issue with a reproducer.

@fslevoaca-ionos
Copy link

fslevoaca-ionos commented Mar 31, 2025 •
edited
Loading

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

please create another issue with a reproducer.

Problem is that I do not have the right to open issues inside https://github.com/apache/maven-surefire
Ah I think only in Jira is allowed. On which I have to make a request for Jira account for which I have to wait a few days
Your Jira account request has been successfully verified, and will be reviewed by the project you indicated. Please allow up to a few days for the project to review this request
This is somewhat bureaucratic : ). I might forget to log this issue by the time I get an account : )

@olamy
Copy link
Member Author

olamy commented Mar 31, 2025

approved but definitely need a project to demonstrate the problem you have

@fslevoaca-ionos
Copy link

fslevoaca-ionos commented Mar 31, 2025 •
edited
Loading

Thanks,
The issue is reproducing only when Cucumber is added into the mix.
I've logged an issue on the Cucumber project
cucumber/cucumber-jvm#2984

However, if this proves to be a Surefire issue, I will move it here.

Might be related to https://issues.apache.org/jira/browse/SUREFIRE-2298

Cheers,

@pzygielo
Copy link
Contributor

pzygielo commented Mar 31, 2025 •
edited
Loading

For cucumber report, on surefire side it seems to be: 5aeca19 is the first bad commit indeed. (The one reported in SUREFIRE-2298.)

@fslevoaca-ionos
Copy link

fslevoaca-ionos commented Mar 31, 2025 •
edited
Loading

Opened a separate issue here:
https://issues.apache.org/jira/browse/SUREFIRE-2299
Feel free to close it as duplicated if that is the case.

Thanks,

@hankem hankem mentioned this pull request Apr 8, 2025
Closed
@hankem
Copy link

hankem commented Apr 8, 2025

FYI: This change seems to affect the discovery of ArchUnit tests, see TNG/ArchUnit#1442, where @erwanosouf was kind enough to provide a reproducer.

@olamy
Copy link
Member Author

olamy commented Apr 8, 2025 •
edited
Loading

Yeah sorry for this. I'm looking at some fix for those other cases.

see https://issues.apache.org/jira/browse/SUREFIRE-2298
you might be able to see progress there #828

@jira-importer
Copy link

jira-importer commented Jul 4, 2025

Resolve #2082

@uhafner uhafner mentioned this pull request Aug 18, 2025
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@slawekjaranowski slawekjaranowski slawekjaranowski approved these changes

Assignees

@olamy olamy

Labels

bug Something isn't working

Projects

None yet

Milestone

3.5.3

Development

Successfully merging this pull request may close these issues.

6 participants

@olamy @fslevoaca-ionos @pzygielo @hankem @jira-importer @slawekjaranowski