Revive external_custom target

[smlambert]

We had created a prototype under the Outreachy project called Tweetest, to be able to more easily extend external tests that meet a certain criteria (those that pull test material from a single repo, and use maven to run their tests).

Related: #2076

3 parameters were added to be able to trial out this idea (EXTERNAL_CUSTOM_REPO, EXTERNAL_CUSTOM_BRANCH and EXTERNAL_TEST_CMD which defaults to 'mvn clean install').

This issue is to revive the prototype and verify it works for some sample repositories. A Grinder job called Grinder_ext_custom has been created with the additional parameters. A recent run (Grinder_ext_custom/21) with the following parameter settings fails in the following way:

EXTERNAL_CUSTOM_REPO=https://github.com/radsz/jacop.git
EXTERNAL_CUSTOM_BRANCH=4.10.0
EXTERNAL_TEST_CMD=mvn --batch-mode clean install

12:09:05       [exec] #10 [6/9] COPY jacop/test.sh /test.sh
12:09:05       [exec] #10 ERROR: failed to calculate checksum of ref IOHW:E4VQ:KTS2:COOU:5DLG:NW4B:XV2Z:TFDL:SFRV:IF2C:7NZI:V65O::ppf2jo0abn8pypgbxintwpn3o: "/jacop/test.sh": not found
12:09:05       [exec] 
12:09:05       [exec] #4 [1/9] FROM docker.io/library/eclipse-temurin:11-jdk@sha256:bc9f393dff60292b0e7c10e2d8c5263f404672d5ada65eb38d416616d3593e31
12:09:05       [exec] #4 resolve docker.io/library/eclipse-temurin:11-jdk@sha256:bc9f393dff60292b0e7c10e2d8c5263f404672d5ada65eb38d416616d3593e31 0.1s done
12:09:05       [exec] #4 sha256:bc9f393dff60292b0e7c10e2d8c5263f404672d5ada65eb38d416616d3593e31 7.19kB / 7.19kB done
12:09:06       [exec] #4 CANCELED
12:09:06       [exec] ------
12:09:06       [exec]  > [6/9] COPY jacop/test.sh /test.sh:
12:09:06       [exec] ------
12:09:06       [exec] Dockerfile.hotspot.full:68
12:09:06       [exec] --------------------
12:09:06       [exec]   66 |     
12:09:06       [exec]   67 |     # This is the main script to run jacop tests
12:09:06       [exec]   68 | >>> COPY jacop/test.sh /test.sh
12:09:06       [exec]   69 |     COPY test_base_functions.sh test_base_functions.sh
12:09:06       [exec]   70 |     
12:09:06       [exec] --------------------
12:09:06       [exec] ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref IOHW:E4VQ:KTS2:COOU:5DLG:NW4B:XV2Z:TFDL:SFRV:IF2C:7NZI:V65O::ppf2jo0abn8pypgbxintwpn3o: "/jacop/test.sh": not found
12:09:06  

Investigate the failure, and what recent changes to this area of code may have done to affect this target. Suspect we should be copying from external_custom/test.sh to /test.sh to be able to proceed.

[smlambert]

FYI @sophia-guo (so you are aware that there is a Grinder job to test these cases)

[sophia-guo]

The first thing I see is the parameter is not passed

12:09:05       [exec] #7 [3/9] RUN git clone
12:09:05       [exec] #7 CACHED

[smlambert]

https://ci.adoptium.net/view/Test_grinder/job/Grinder_ext_custom/23

As Sophia noted, in the Build stage, EXTERNAL_CUSTOM_REPO is undefined, but also git not present, so the building of the container fails.

15:23:16       [exec] Step 17/24 : RUN git clone
15:23:16       [exec]  ---> Running in 710dd6d443c5
15:23:17       [exec] �[91m/bin/sh: 1: git: not found
15:23:18       [exec] The command '/bin/sh -c git clone' returned a non-zero code: 127

Areas of code related to external_custom:
https://github.com/adoptium/aqa-tests/blob/master/external/external.sh#L119-L125
https://github.com/adoptium/aqa-tests/blob/master/external/external.sh#L361-L363
https://github.com/adoptium/aqa-tests/blob/master/external/build_image.sh#L43-L46
https://github.com/adoptium/aqa-tests/blob/master/external/build_image.sh#L85-L86
https://github.com/adoptium/aqa-tests/blob/master/external/build_image.sh#L96
https://github.com/adoptium/aqa-tests/blob/master/external/dockerfile_functions.sh#L612-L622
https://github.com/adoptium/aqa-tests/blob/master/external/dockerfile_functions.sh#L699-L701

Dockerfile generated from Grinder_ext_custom/23: Dockerfile.hotspot.full.txt

[andrlos]

Hi @smlambert ! Can you please assign the issue to me? I would like to try and resolve it. Hopefully within a reasonable time.

[andrlos]

okay, I am gonna use this as a log of my thoughts, feel free to ignore, unless tagged..
In the code everything looks fine and ready and there is no reason for the argument not to be passed, however I added set -x to the generate_dockerfile function and as suspected the content of check_external_custom_test variable is not what would be expected later in the code.

https://ci.adoptium.net/view/Test_grinder/job/Grinder_ext_custom/24/consoleFull

03:02:38       [exec] ++ file=/home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full
03:02:38       [exec] ++ test=jacop
03:02:38       [exec] ++ version=17
03:02:38       [exec] ++ vm=hotspot
03:02:38       [exec] ++ os=ubuntu
03:02:38       [exec] ++ package=jdk
03:02:38       [exec] ++ build=full
03:02:38       [exec] ++ platform=linux_x86-64
03:02:38       [exec] ++ base_docker_registry_dir=default
03:02:38       [exec] ++ check_external_custom_test=/home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full0
03:02:38       [exec] ++ [[ /home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full0 == \1 ]]
03:02:38       [exec] ++ [[ /home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full0 == \1 ]]
03:02:38       [exec] ++ set_test_info jacop /home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full0
03:02:38       [exec] ++ local test=jacop
03:02:38       [exec] ++ check_external_custom_test=/home/jenkins/workspace/Grinder_ext_custom/jvmtest/external/external_custom/dockerfile/17/jdk/ubuntu/Dockerfile.hotspot.full0

since the check_external_custom_test is tested in a condition to be 1 or 0...

[andrlos]

andrlos@e5ef72a so the check_external_custom vs check_external_custom_test variable missmatch seems to be a part of the problem.. (grinder run)
the other part would be that I do not see the usage of the EXTERNAL_CUSTOM_BRANCH variable, meaning that we are probably on a master branch of the testsuite (usually not the most stable one)..

16:03:12       [exec] #8 DONE 8.7s
16:03:13       [exec] 
16:03:13       [exec] #9 [ 5/11] RUN git clone https://github.com/radsz/jacop.git
16:03:32       [exec] #9 18.02 Cloning into 'jacop'...
16:03:48       [exec] #9 34.99 Updating files:  12% (1231/9795)

[andrlos]

52af986 the variable name missmatch of EXTERNAL_REPO_BRANCH vs EXTERNAL_CUSTOM_BRANCH seems to have been another problem and now we are on a correct version of the testsuite...
grinder run

19:08:03       [exec] +++ echo jacop
19:08:03       [exec] ++ test=JACOP_TAG
19:08:03       [exec] ++ echo -e 'ARG JACOP_TAG=4.10.0\n'