Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure to only build the llvm package and not more #1069

Merged
merged 1 commit into from
Feb 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/actions/generate-matrix/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ runs:
"copr_project_tpl": "llvm-snapshots-big-merge-YYYYMMDD",
"copr_monitor_tpl": "https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots-big-merge-YYYYMMDD/monitor/",
"chroot_pattern": "^(fedora-(rawhide|[0-9]+)|rhel-[8,9]-)",
"packages": "llvm"
}
EOF
)
Expand All @@ -50,7 +49,6 @@ runs:
"copr_project_tpl": "llvm-snapshots-pgo-YYYYMMDD",
"copr_monitor_tpl": "https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots-pgo-YYYYMMDD/monitor/",
"chroot_pattern": "^(fedora-41)",
"packages": "llvm"
}
EOF
)
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/check-snapshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ jobs:
--github-token-env GITHUB_TOKEN \
check "${extra_args}"\
--maintainer-handle ${{matrix.maintainer_handle}} \
--packages ${{matrix.packages}} \
--build-strategy ${{matrix.name}} \
--copr-ownername ${{matrix.copr_ownername}} \
--copr-project-tpl ${{matrix.copr_project_tpl}} \
Expand Down
52 changes: 19 additions & 33 deletions .github/workflows/fedora-copr-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,11 @@ jobs:
today=$(date +%Y%m%d)
yesterday=$(date -d "${today} -1 day" +%Y%m%d)

packages="$(get_packages)"
chroots="$(get_chroots)"

username=@fedora-llvm-team
{
echo "username=$username"
echo "packages=$packages"
echo "chroots=$chroots"
echo "all_chroots=$all_chroots"
echo "project_today=${{ matrix.copr_ownername }}/${{ matrix.copr_project_tpl }}" | sed "s/YYYYMMDD/$today/"
Expand Down Expand Up @@ -163,48 +161,36 @@ jobs:
copr get-chroot "${{ env.project_today }}/$chroot"
done

- name: "Create today's packages: ${{ env.packages }}"
- name: "Create today's package"
shell: bash -e {0}
run: |
source scripts/functions.sh
[[ -n "${{ matrix.extra_script_file }}" ]] && source ${{ matrix.extra_script_file }}

for pkg in ${{ env.packages }}; do
clone_url=$(echo "${{ matrix.clone_url_tpl }}" | sed "s/PKG/$pkg/")
copr add-package-scm \
--clone-url "${clone_url}" \
--commit ${{ matrix.clone_ref }} \
--spec "${pkg}.spec" \
--type git \
--method make_srpm \
--name "${pkg}" \
"${{ env.project_today }}"
done

- name: "Build packages in chroot batches in this order: ${{ env.packages }}"
clone_url=$(echo "${{ matrix.clone_url_tpl }}" | sed "s/PKG/$pkg/")
copr add-package-scm \
--clone-url "${clone_url}" \
--commit ${{ matrix.clone_ref }} \
--spec "llvm.spec" \
--type git \
--method make_srpm \
--name "llvm" \
"${{ env.project_today }}"

- name: "Build llvm package"
shell: bash -e {0}
run: |
source scripts/functions.sh
[[ -n "${{ matrix.extra_script_file }}" ]] && source ${{ matrix.extra_script_file }}

for chroot in ${{ env.chroots }}; do
# Start a new batch
after_build_id=""
for pkg in ${{ env.packages }}; do
if ! is_package_supported_by_chroot "${pkg}" "${chroot}"; then
echo "Package '${pkg}' is not supported by chroot: ${chroot}";
else
copr build-package \
--timeout $((30*3600)) \
--nowait \
--name "$pkg" "${after_build_id}" \
--chroot "${chroot}" \
${{ env.project_today }} \
| tee "${pkg}.log"

after_build_id="--after-build-id $(grep -Po 'Created builds: \K(\d+)' "${pkg}.log")"
fi
done
copr build-package \
--timeout $((30*3600)) \
--nowait \
--name "llvm" \
--chroot "${chroot}" \
${{ env.project_today }} \
| tee "llvm.log"
done

- name: "Delete target Copr project at ${{ env.project_target }} before forking to it"
Expand Down
24 changes: 2 additions & 22 deletions scripts/functions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,34 +57,14 @@ function get_chroots() {
copr list-chroots | grep -P '^(fedora-(rawhide|[0-9]+)|rhel-[8,9]-)' | sort | tr '\n' ' '
}

# Prints the packages we care about
function get_packages() {
echo "llvm"
}

# Returns false if a package needs special handling on certain architectures
function is_package_supported_by_chroot() {
local pkg=$1
local chroot=$2

true
}

# Returns 0 if all packages on all* chroots have successful builds.
#
# *: All supported combinations of package + chroot (see is_package_supported_by_chroot).
# Returns 0 if all llvm packages on all chroots have successful builds.
function has_all_good_builds(){
local project=$1
local extra_packages=$2

copr monitor --output-format text-row --fields state,chroot,name $project | sort -k1 -k2 -k3 > /tmp/actual.txt
truncate -s 0 /tmp/expected.txt
for chroot in $(get_chroots); do
for package in $(get_packages) $extra_packages; do
if is_package_supported_by_chroot "$package" "$chroot"; then
echo "succeeded $chroot $package" >> /tmp/expected.txt
fi
done
echo "succeeded $chroot llvm" >> /tmp/expected.txt
done
sort -k1 -k2 -k3 -o /tmp/expected.txt /tmp/expected.txt
diff -bus /tmp/expected.txt /tmp/actual.txt
Expand Down