Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VPA: Create a build and deploy locally script #7654

Merged
merged 4 commits into from
Jan 8, 2025
+112 −14
Merged

VPA: Create a build and deploy locally script #7654

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0d46bfa
VPA: Create a build and deploy locally script
adrianmoisey Jan 4, 2025
b8851df
Linting
adrianmoisey Jan 4, 2025
40f0c9b
Update copyright date
adrianmoisey Jan 4, 2025
8c1671f
Fix typo
adrianmoisey Jan 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion vertical-pod-autoscaler/common/version.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,26 @@ limitations under the License.

package common

// gitCommit is the commit used to build the VPA binaries, if available.
// It is injected at build time.
var gitCommit = ""

// VerticalPodAutoscalerVersion is the version of VPA.
const VerticalPodAutoscalerVersion = "1.2.0"
const versionCore = "1.2.0"

func VerticalPodAutoscalerVersion() string {
v := versionCore
if gitCommit != "" {
// NOTE: use 14 character short hash, like Kubernetes
v += "+" + truncate(gitCommit, 14)
}

return v
}

func truncate(s string, maxLen int) string {
if len(s) < maxLen {
return s
}
return s[:maxLen]
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this is copied from kind: https://github.com/kubernetes-sigs/kind/blob/1c5a56b30145d98a61c0538975ca0894ad65e0d0/pkg/cmd/kind/version/version.go#L42

Ideally I'd like to copy what Kubernetes does, but it's a larger project with a more complicated build process that I'm unfamiliar with

70 changes: 70 additions & 0 deletions vertical-pod-autoscaler/hack/dev-deploy-locally.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/bin/bash

# Copyright 2023 The Kubernetes Authors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are in 2025 :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 40f0c9b

#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o nounset
set -o pipefail

SCRIPT_ROOT=$(dirname ${BASH_SOURCE})/..

SUITE=full-vpa
REQUIRED_COMMANDS="
docker
git
go
kubectl
make
"

for i in $REQUIRED_COMMANDS; do
if ! command -v $i > /dev/null 2>&1
then
echo "$i could not be found, please ensure it is installed"
echo
echo "The following commands are required to run these tests:"
echo $REQUIRED_COMMANDS
exit 1;
fi
done

if ! docker ps >/dev/null 2>&1
then
echo "docker isn't running"
echo
echo "Please ensure that docker is running"
exit 1
fi

if ! kubectl version >/dev/null 2>&1
then
echo "Kubernetes isn't running"
echo
echo "Please ensure that Kubernetes is running"
exit 1
fi

COMMIT=$(git rev-parse HEAD 2>/dev/null)
COMMIT=${COMMIT:0:14}
export BUILD_LD_FLAGS="-s -X=k8s.io/autoscaler/vertical-pod-autoscaler/common.gitCommit=$COMMIT"
export TAG=$COMMIT


echo " ** Deploying building and deploying all VPA components"
${SCRIPT_ROOT}/hack/deploy-for-e2e-locally.sh full-vpa

echo " ** Restarting all VPA components"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have to restart all VPA components? Just curious

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the script add the latest git commit as a tag, which will cause the pods to restart if a new commit is added.
But, if local changes were made without being committed, then the tag won't change, and pods won't restart

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. It makes sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There may be other ways to handle that, such as hashing all uncommitted files and adding that to the tag. After a quick google it seemed to be more effort than it's worth

Copy link
Member

@omerap12 omerap12 Jan 5, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree. good job :)

for i in admission-controller updater recommender; do
kubectl -n kube-system rollout status deployment/vpa-$i
done
3 changes: 2 additions & 1 deletion vertical-pod-autoscaler/pkg/admission-controller/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ COPY common common
COPY pkg pkg

ARG TARGETOS TARGETARCH
ARG BUILD_LD_FLAGS="-s"

RUN CGO_ENABLED=0 LD_FLAGS=-s GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/admission-controller -o admission-controller-$TARGETARCH
RUN CGO_ENABLED=0 GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/admission-controller -ldflags="${BUILD_LD_FLAGS}" -o admission-controller-$TARGETARCH
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that LD_FLAGS=-s ever worked:

$ docker create --platform linux/amd64 --name vpa-recommender registry.k8s.io/autoscaling/vpa-recommender:1.2.1
999058e6c14330101bf0ce4e916f5dedbc4d1005d6ff9643033db0289624a7ee
$ docker cp vpa-recommender:/recommender .
Successfully copied 58.2MB to /Users/Adrian/.
$ file recommender
recommender: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=1NTySJ_TLdHBpDxDiF3X/gpLdixJiYC-YAHAH54Dt/1soeNlTu_Tj_uuiTrLyP/hqX1KbQPpE4_hW1xIaiQ, with debug_info, not stripped

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before:

$ GOOS=linux go build -C pkg/admission-controller -o admission-controller-not-stripped
$ file pkg/admission-controller/admission-controller-not-stripped
pkg/admission-controller/admission-controller-not-stripped: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=FdIwuV800tRp_DwpAJve/rSttx_oRFM1ycMHfuEdQ/-anDCNZimL-FExp30NhX/EBATf_tSCu-ZUkhthpUD, with debug_info, not stripped
$ ls -lh pkg/admission-controller/admission-controller-not-stripped
-rwxr-xr-x 1 adrian staff 67M Jan  4 17:22 pkg/admission-controller/admission-controller-not-stripped

After:

$ GOOS=linux go build -C pkg/admission-controller -ldflags="-s" -o admission-controller-stripped
$ file pkg/admission-controller/admission-controller-stripped
pkg/admission-controller/admission-controller-stripped: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=MBFlhlA4r-VBNuUueZHw/jA8qtsf8vlLZF32hl2vw/-anDCNZimL-FExp30NhX/HIDLyY7TlMNMbxcIptrl, stripped
$ ls -lh pkg/admission-controller/admission-controller-stripped
-rwxr-xr-x 1 adrian staff 46M Jan  4 17:22 pkg/admission-controller/admission-controller-stripped


FROM gcr.io/distroless/static:nonroot

Expand Down
5 changes: 3 additions & 2 deletions vertical-pod-autoscaler/pkg/admission-controller/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ GOOS?=linux
COMPONENT=admission-controller
FULL_COMPONENT=vpa-${COMPONENT}

BUILD_LD_FLAGS?=-s
ALL_ARCHITECTURES?=amd64 arm arm64 ppc64le s390x
export DOCKER_CLI_EXPERIMENTAL=enabled

Expand All @@ -35,7 +36,7 @@ ifndef TAG
ERR = $(error TAG is undefined)
$(ERR)
endif
docker buildx build --pull --load --platform linux/$* -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../
docker buildx build --pull --load --platform linux/$* --build-arg BUILD_LD_FLAGS="${BUILD_LD_FLAGS}" -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../

.PHONY: docker-push
docker-push: $(addprefix do-push-,$(ALL_ARCHITECTURES)) push-multi-arch;
Expand Down Expand Up @@ -91,4 +92,4 @@ format:

.PHONY: document-flags
document-flags:
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-admission-flags.md
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-admission-flags.md
2 changes: 1 addition & 1 deletion vertical-pod-autoscaler/pkg/admission-controller/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ func main() {
klog.InitFlags(nil)
common.InitLoggingFlags()
kube_flag.InitFlags()
klog.V(1).InfoS("Starting Vertical Pod Autoscaler Admission Controller", "version", common.VerticalPodAutoscalerVersion)
klog.V(1).InfoS("Starting Vertical Pod Autoscaler Admission Controller", "version", common.VerticalPodAutoscalerVersion())

if len(commonFlags.VpaObjectNamespace) > 0 && len(commonFlags.IgnoredVpaObjectNamespaces) > 0 {
klog.Fatalf("--vpa-object-namespace and --ignored-vpa-object-namespaces are mutually exclusive and can't be set together.")
Expand Down
3 changes: 2 additions & 1 deletion vertical-pod-autoscaler/pkg/recommender/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ COPY common common
COPY pkg pkg

ARG TARGETOS TARGETARCH
ARG BUILD_LD_FLAGS="-s"

RUN CGO_ENABLED=0 LD_FLAGS=-s GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/recommender -o recommender-$TARGETARCH
RUN CGO_ENABLED=0 GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/recommender -ldflags="${BUILD_LD_FLAGS}" -o recommender-$TARGETARCH

FROM gcr.io/distroless/static:nonroot

Expand Down
5 changes: 3 additions & 2 deletions vertical-pod-autoscaler/pkg/recommender/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ FULL_COMPONENT=vpa-${COMPONENT}
# localhost registries need --insecure for some docker commands.
INSECURE=$(subst localhost,--insecure,$(findstring localhost,$(REGISTRY)))

BUILD_LD_FLAGS?=-s
ALL_ARCHITECTURES?=amd64 arm arm64 ppc64le s390x
export DOCKER_CLI_EXPERIMENTAL=enabled

Expand All @@ -37,7 +38,7 @@ ifndef TAG
ERR = $(error TAG is undefined)
$(ERR)
endif
docker buildx build --pull --load --platform linux/$* -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../
docker buildx build --pull --load --platform linux/$* --build-arg BUILD_LD_FLAGS="${BUILD_LD_FLAGS}" -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../

.PHONY: docker-push
docker-push: $(addprefix do-push-,$(ALL_ARCHITECTURES)) push-multi-arch;
Expand Down Expand Up @@ -93,4 +94,4 @@ format:

.PHONY: document-flags
document-flags:
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-recommender-flags.md
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-recommender-flags.md
2 changes: 1 addition & 1 deletion vertical-pod-autoscaler/pkg/recommender/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ func main() {
componentbaseoptions.BindLeaderElectionFlags(&leaderElection, pflag.CommandLine)

kube_flag.InitFlags()
klog.V(1).InfoS("Vertical Pod Autoscaler Recommender", "version", common.VerticalPodAutoscalerVersion, "recommenderName", *recommenderName)
klog.V(1).InfoS("Vertical Pod Autoscaler Recommender", "version", common.VerticalPodAutoscalerVersion(), "recommenderName", *recommenderName)

if len(commonFlags.VpaObjectNamespace) > 0 && len(commonFlags.IgnoredVpaObjectNamespaces) > 0 {
klog.Fatalf("--vpa-object-namespace and --ignored-vpa-object-namespaces are mutually exclusive and can't be set together.")
Expand Down
3 changes: 2 additions & 1 deletion vertical-pod-autoscaler/pkg/updater/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ COPY common common
COPY pkg pkg

ARG TARGETOS TARGETARCH
ARG BUILD_LD_FLAGS="-s"

RUN CGO_ENABLED=0 LD_FLAGS=-s GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/updater -o updater-$TARGETARCH
RUN CGO_ENABLED=0 LD_FLAGS=-s GOARCH=$TARGETARCH GOOS=$TARGETOS go build -C pkg/updater -ldflags="${BUILD_LD_FLAGS}" -o updater-$TARGETARCH

FROM gcr.io/distroless/static:nonroot

Expand Down
5 changes: 3 additions & 2 deletions vertical-pod-autoscaler/pkg/updater/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ GOOS?=linux
COMPONENT=updater
FULL_COMPONENT=vpa-${COMPONENT}

BUILD_LD_FLAGS?=-s
ALL_ARCHITECTURES?=amd64 arm arm64 ppc64le s390x
export DOCKER_CLI_EXPERIMENTAL=enabled

Expand All @@ -35,7 +36,7 @@ ifndef TAG
ERR = $(error TAG is undefined)
$(ERR)
endif
docker buildx build --pull --load --platform linux/$* -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../
docker buildx build --pull --load --platform linux/$* --build-arg BUILD_LD_FLAGS="${BUILD_LD_FLAGS}" -t ${REGISTRY}/${FULL_COMPONENT}-$*:${TAG} -f ./Dockerfile ../../

.PHONY: docker-push
docker-push: $(addprefix do-push-,$(ALL_ARCHITECTURES)) push-multi-arch;
Expand Down Expand Up @@ -91,4 +92,4 @@ format:

.PHONY: document-flags
document-flags:
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-updater-flags.md
go run ../../hack/vpa-generate-flags.go . ../../docs/vpa-updater-flags.md
2 changes: 1 addition & 1 deletion vertical-pod-autoscaler/pkg/updater/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ func main() {
componentbaseoptions.BindLeaderElectionFlags(&leaderElection, pflag.CommandLine)

kube_flag.InitFlags()
klog.V(1).InfoS("Vertical Pod Autoscaler Updater", "version", common.VerticalPodAutoscalerVersion)
klog.V(1).InfoS("Vertical Pod Autoscaler Updater", "version", common.VerticalPodAutoscalerVersion())

if len(commonFlags.VpaObjectNamespace) > 0 && len(commonFlags.IgnoredVpaObjectNamespaces) > 0 {
klog.Fatalf("--vpa-object-namespace and --ignored-vpa-object-namespaces are mutually exclusive and can't be set together.")
Expand Down
Loading