You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, for new jdk version, a new exclusion file ProblemList_openjdkversion.txt is required to be added or the default one ( ProblemList_openjdk11.txt) will be used to avoid breaking the test job. Normally we will copy the most recent previous version (e.g, if the newest is 25, we copy the version 24) and add it as the newest one.
The may cause inaccurate exclusion. The old exclusion file may contain test cases that have been fixed or even enhanced in the latest version. Keeping them excluded unnecessarily reduces test coverage and prevent immediate detection of new failures. (Consider the openjdk developing process, backport always takes time or won't happen at all).
Instead of requiring a manually copied exclusion file for each new version we could modify the process to run without local exclusion file to allow all test cases to execute ( sync with upstream openjdk ) and generate the exclusion file based on actual triage failures. Note generating a new exclusion file may take extra time if many tests fail.
This can be limited to impl=hotspot as openj9 may prefer different option #5890
The text was updated successfully, but these errors were encountered:
Currently, for new jdk version, a new exclusion file ProblemList_openjdkversion.txt is required to be added or the default one ( ProblemList_openjdk11.txt) will be used to avoid breaking the test job. Normally we will copy the most recent previous version (e.g, if the newest is 25, we copy the version 24) and add it as the newest one.
The may cause inaccurate exclusion. The old exclusion file may contain test cases that have been fixed or even enhanced in the latest version. Keeping them excluded unnecessarily reduces test coverage and prevent immediate detection of new failures. (Consider the openjdk developing process, backport always takes time or won't happen at all).
Instead of requiring a manually copied exclusion file for each new version we could modify the process to run without local exclusion file to allow all test cases to execute ( sync with upstream openjdk ) and generate the exclusion file based on actual triage failures. Note generating a new exclusion file may take extra time if many tests fail.
This can be limited to impl=hotspot as openj9 may prefer different option #5890
The text was updated successfully, but these errors were encountered: