Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update and fix rollback workflow #2551

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
89f0470
wip: initial work
k80bowman Oct 17, 2023
825bce3
wip: first effort at retrieving rollback and target tags
k80bowman Oct 19, 2023
fd426bf
chore: fix for getVersionsList step
k80bowman Oct 19, 2023
e55b0a5
chore: fix getVersionsList step and move to end of job
k80bowman Oct 19, 2023
d4f1149
chore: remove json flag from getVersionsList command
k80bowman Oct 19, 2023
df1693b
wip: trying out grep for channel version list
k80bowman Oct 19, 2023
7956e2e
wip: missing parens
k80bowman Oct 19, 2023
2fee63f
wip: add dollar sign to print variable
k80bowman Oct 19, 2023
66c723a
wip: re-add json flag to npm view versions
k80bowman Oct 19, 2023
4da60a3
wip: update rollback variable names and change version list logic to …
k80bowman Oct 19, 2023
165e3fe
wip: checking quotation marks
k80bowman Oct 19, 2023
6dc2cee
wip: missing parens
k80bowman Oct 19, 2023
d51c040
wip: printing the version list
k80bowman Oct 19, 2023
14e2365
wip: trying out jq
k80bowman Oct 19, 2023
21e9225
fix: fix jq command
k80bowman Nov 8, 2023
161ffd1
add logic for finding channel-specific releases
k80bowman Nov 8, 2023
888cc63
fix CHANNEL_VERSION_LIST logic
k80bowman Nov 8, 2023
59b1a9e
syntax fixes for CHANNEL_VERSION_LIST
k80bowman Nov 9, 2023
857802c
add queries for rollback and target versions
k80bowman Nov 9, 2023
6c6e1d6
clean up and complete target and rollback tags
k80bowman Nov 10, 2023
cfde461
add validate-tags job
k80bowman Nov 10, 2023
b5f41f1
move gh script to env variable
k80bowman Nov 10, 2023
787a87e
moving the github command back and fixing
k80bowman Nov 10, 2023
741de5b
cleanup and add logic to validate-tags job
k80bowman Nov 10, 2023
3f2f0f4
only run validate-tags when the rollbackChannel is 'stable'
k80bowman Nov 10, 2023
1f78c50
update promote-previous-tag job to use new vars
k80bowman Nov 13, 2023
81266c4
update invalidate-cdn-cache job to use new vars
k80bowman Nov 13, 2023
3da7980
update deprecate-npm-version job to use new vars
k80bowman Nov 13, 2023
aab6a39
update rollback-brew job to use new vars
k80bowman Nov 13, 2023
ff156ad
move deprecate npm job above promote-target-tag job
k80bowman Nov 13, 2023
8869cb6
update name of promote job
k80bowman Nov 13, 2023
ab2adff
remove unneeded script
k80bowman Nov 13, 2023
dee9ec2
add note to deprecate npm job
k80bowman Nov 13, 2023
f1bd079
change name of validation job
k80bowman Nov 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/promote.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ on:

jobs:
promote:
name: Promote ${{ inputs.version }} to ${{ fromJSON(inputs.isStableRelease) && 'stable' || inputs.channel }}
name: Promote ${{ inputs.version }} to ${{ inputs.channel }}
runs-on: ubuntu-latest
env:
CLOUDFRONT_DISTRIBUTION: ${{ secrets.CLOUDFRONT_DISTRIBUTION }}
Expand Down
117 changes: 72 additions & 45 deletions .github/workflows/rollback.yml
Original file line number Diff line number Diff line change
@@ -1,51 +1,93 @@
name: Rollback release
name: Rollback latest release

on:
workflow_dispatch:
inputs:
version-to-roll-back:
description: version number that needs to be rolled back (format x.x.x)
type: string
rollbackChannel:
description: Which channel needs to be rolled back?
type: choice
required: true
options:
- stable
- beta
- alpha

jobs:
determine-target-tag:
name: Determine target tag for rollback
get-target-and-rollback-tags:
name: Get target tag and rollback tags
runs-on: ubuntu-latest
outputs:
targetTag: ${{ github.ref_name }}
isStableRelease: ${{ steps.isStableRelease.outputs.isStableRelease }}
targetTag: ${{ steps.getRollbackTargetTags.outputs.targetTag }}
rollbackTag: ${{ steps.getRollbackTargetTags.outputs.rollbackTag }}
steps:
- name: is input a tag
if: github.ref_type != 'tag'
run: exit 1
- name: Get target version
id: isStableRelease
shell: bash
- name: install apt-get dependencies
run: |
if [[ ${{ github.ref_name }} =~"beta" ]]
then
echo "isStableRelease=true" >> $GITHUB_OUTPUT
sudo apt-get update
sudo apt-get install -y awscli jq
- name: get rollback and target tags
id: getRollbackTargetTags
run: |
VERSION_LIST=$(npm view heroku versions --json)
CHANNEL_VERSION_LIST=[]
if [[ ${{ inputs.rollbackChannel }} == 'stable' ]]; then
CHANNEL_VERSION_LIST=$(echo ${VERSION_LIST} | jq '[.[] | select(test("-") | not )]')
else
echo "isStableRelease=false" >> $GITHUB_OUTPUT
CHANNEL_VERSION_LIST=$(echo ${VERSION_LIST} | jq '[.[] | select(test("'${{ inputs.rollbackChannel }}'"))]')
fi
ROLLBACK_VERSION=$(echo ${CHANNEL_VERSION_LIST} | jq '.[-1]')
echo "rollbackTag=${ROLLBACK_VERSION}" >> $GITHUB_OUTPUT
TARGET_VERSION=$(echo ${CHANNEL_VERSION_LIST} | jq '.[-2]')
echo "targetTag=${TARGET_VERSION}" >> $GITHUB_OUTPUT

Comment on lines +37 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final sanity check: how worried are we that we can not specify the version to rollback to?

promote-previous-tag:
needs: determine-target-tag
name: Promote ${{ needs.determine-target-tag.outputs.targetTag }} to ${{ needs.determine-target-tag.outputs.isStableRelease && 'stable' || 'beta' }}
validate-stable-tags:
name: Validate rollback tag
# pub-hk-ubuntu-22.04- due to IP allow list issues with public repos: https://salesforce.quip.com/bu6UA0KImOxJ
runs-on: pub-hk-ubuntu-22.04-small
needs: [get-target-and-rollback-tags]
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- if: ${{ inputs.rollbackChannel == 'stable' }}
run: |
GITHUB_LATEST_RELEASE=$(gh release view --repo heroku/cli --json tagName -q '.tagName.[1:]')
if [[ "${GITHUB_LATEST_RELEASE}" != "'${{ needs.get-target-and-rollback-tags.outputs.rollbackTag }}'"* ]]; then
echo "The Github latest release (${GITHUB_LATEST_RELEASE}) and the NPM latest release ("${{ needs.get-target-and-rollback-tags.outputs.rollbackTag }}") do not match."
exit 1
fi

# This job needs to go before the promote job, as the promote job looks for the current channel release on NPM.
deprecate-npm-rollback-tag:
needs: [get-target-and-rollback-tags, validate-stable-tags]
name: Deprecate rollback version on NPM
# if statement only here to protect our prod release while we test this functionality
if: ${{ !fromJSON(needs.determine-target-tag.outputs.isStableRelease) }}
if: ${{ inputs.rollbackChannel != 'stable' }}
# pub-hk-ubuntu-22.04- due to IP allow list issues with public repos: https://salesforce.quip.com/bu6UA0KImOxJ
runs-on: pub-hk-ubuntu-22.04-small
steps:
- name: set NPM auth
run: echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_PUBLISH_KEY }}" > ~/.npmrc
- name: deprecate NPM release and set previous release to latest/prerelease channel
run: |
npm dist-tag add heroku@${{ needs.get-target-and-rollback-tags.outputs.targetTag }} ${{ inputs.rollbackChannel }}
npm deprecate heroku@${{ needs.get-target-and-rollback-tags.outputs.rollbackTag }} "Version is deprecated due to problems with the release"

promote-target-tag:
needs: [get-target-and-rollback-tags, validate-stable-tags, deprecate-npm-rollback-tag]
name: Promote ${{ needs.get-target-and-rollback-tags.outputs.targetTag }} to ${{ inputs.rollbackChannel }}
# if statement only here to protect our prod release while we test this functionality
if: ${{ inputs.rollbackChannel != 'stable' }}
uses: ./.github/workflows/promote.yml
with:
version: ${{ needs.determine-target-tag.outputs.targetTag }}
isStableRelease: ${{ needs.determine-target-tag.outputs.isStableRelease }}
version: ${{ needs.get-target-and-rollback-tags.outputs.targetTag }}
isStableRelease: ${{ inputs.rollbackChannel == 'stable' }}
channel: ${{ inputs.rollbackChannel }}
secrets: inherit

invalidate-cdn-cache:
needs: [determine-target-tag, promote-previous-tag]
needs: [get-target-and-rollback-tags, validate-stable-tags, promote-target-tag]
name: Invalidate Cloudfront cache
# if statement only here to protect our prod release while we test this functionality
if: ${{ !fromJSON(needs.determine-target-tag.outputs.isStableRelease) }}
if: ${{ inputs.rollbackChannel != 'stable' }}
runs-on: ubuntu-latest
env:
CLOUDFRONT_DISTRIBUTION: ${{ secrets.CLOUDFRONT_DISTRIBUTION }}
Expand All @@ -60,32 +102,17 @@ jobs:
aws configure set preview.cloudfront true
- run: ./scripts/postrelease/invalidate_cdn_cache

deprecate-npm-version:
needs: determine-target-tag
name: Deprecate rollback version on NPM
# if statement only here to protect our prod release while we test this functionality
if: ${{ !fromJSON(needs.determine-target-tag.outputs.isStableRelease) }}
# pub-hk-ubuntu-22.04- due to IP allow list issues with public repos: https://salesforce.quip.com/bu6UA0KImOxJ
runs-on: pub-hk-ubuntu-22.04-small
steps:
- name: set NPM auth
run: echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_PUBLISH_KEY }}" > ~/.npmrc
- name: deprecate release
run: |
npm dist-tag add heroku@${{ needs.determine-target-tag.outputs.targetTag }} ${{ needs.determine-target-tag.outputs.isStableRelease && 'latest' || 'beta' }}
npm deprecate heroku@${{ inputs.version-to-roll-back }} "Version is deprecated due to problems with the release"

rollback-brew:
name: Rollback brew release
needs: determine-target-tag
if: ${{ !fromJSON(needs.determine-target-tag.outputs.isStableRelease) }}
needs: [get-target-and-rollback-tags, validate-stable-tags]
if: ${{ inputs.rollbackChannel != 'stable' }}
# pub-hk-ubuntu-22.04- due to IP allow list issues with public repos: https://salesforce.quip.com/bu6UA0KImOxJ
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this.

Suggested change
if: ${{ inputs.rollbackChannel != 'stable' }}
if: ${{ inputs.rollbackChannel == 'stable' }}

There will be nothing to rollback in our brew repo unless it was a stable release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only there for testing, to make sure we don't actually roll back a stable release. I'll remove this check after the test.

runs-on: pub-hk-ubuntu-22.04-small
environment: ReleaseHomebrew
steps:
- name: yarn install
run: yarn --immutable --network-timeout 1000000
- name: rollback homebrew
run: node ./scripts/rollback/homebrew.js
run: ./scripts/rollback/homebrew.js
env:
ROLLBACK_VERSION: ${{ inputs.version-to-roll-back }}
ROLLBACK_VERSION: ${{ needs.get-target-and-rollback-tags.outputs.rollbackTag }}
Loading