Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve openjdk exclusion file handling for new versions #5932

Open
sophia-guo opened this issue Jan 30, 2025 · 2 comments
Open

Improve openjdk exclusion file handling for new versions #5932

sophia-guo opened this issue Jan 30, 2025 · 2 comments
Assignees

Comments

@sophia-guo
Copy link
Contributor

Currently, for new jdk version, a new exclusion file ProblemList_openjdkversion.txt is required to be added or the default one ( ProblemList_openjdk11.txt) will be used to avoid breaking the test job. Normally we will copy the most recent previous version (e.g, if the newest is 25, we copy the version 24) and add it as the newest one.

The may cause inaccurate exclusion. The old exclusion file may contain test cases that have been fixed or even enhanced in the latest version. Keeping them excluded unnecessarily reduces test coverage and prevent immediate detection of new failures. (Consider the openjdk developing process, backport always takes time or won't happen at all).

Instead of requiring a manually copied exclusion file for each new version we could modify the process to run without local exclusion file to allow all test cases to execute ( sync with upstream openjdk ) and generate the exclusion file based on actual triage failures. Note generating a new exclusion file may take extra time if many tests fail.

This can be limited to impl=hotspot as openj9 may prefer different option #5890

@sophia-guo
Copy link
Contributor Author

related #1853

@sophia-guo
Copy link
Contributor Author

sophia-guo commented Jan 30, 2025

Running this January release JDK23 linux-x64 with no local problem list PR shows 14 ( sanity https://ci.adoptium.net/view/Test_grinder/job/Grinder/12650/#showFailuresLink ) + 6 (extended https://ci.adoptium.net/view/Test_grinder/job/Grinder/12651/artifact/aqa-tests/TKG/output_17381609376723/Grinder_20250129142723.tap/*view*/) failures ( ignore failures of jdk_net, which rerun passed https://ci.adoptium.net/view/Test_grinder/job/Grinder/12653/ , seems machine related.) = 20 testcases.

The problemlist file includes roughtly 48 testcases linux-all entries.

@sophia-guo sophia-guo self-assigned this Jan 30, 2025
@smlambert smlambert moved this to Todo in 2025 Adoptium Plan Feb 5, 2025
@sophia-guo sophia-guo moved this from Todo to In Progress in 2025 Adoptium Plan Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

1 participant