Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Propagate stop channel to prevent resource leaks in VPA components #7661

Merged

Conversation

omerap12
Copy link
Member

@omerap12 omerap12 commented Jan 6, 2025

What type of PR is this?

/kind bug

What this PR does / why we need it:

Fixes potential resource leaks and improves shutdown handling in VPA components by properly propagating stop channels. Previously, some components were creating their own stop channels which could lead to improper cleanup during shutdown. Adds proper cache synchronization checks.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NONE

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 6, 2025
@voelzmo
Copy link
Contributor

voelzmo commented Jan 7, 2025

I'm curious, did you observe any actual issues with the current implementation? Can you help me understand which error scenario this change covers?

My understanding is that if the run method only ever terminates on a fatal/os.Exit, which would in theory also tear down all the other routines as well?

@omerap12
Copy link
Member Author

omerap12 commented Jan 7, 2025

I'm curious, did you observe any actual issues with the current implementation? Can you help me understand which error scenario this change covers?

My understanding is that if the run method only ever terminates on a fatal/os.Exit, which would in theory also tear down all the other routines as well?

I didn't observe any specific issues, and you're mostly correct. but, for graceful shutdowns (e.g., during rolling updates or controlled scaling down), it's better to ensure proper cleanup. By adding WaitForCacheSync in the newPodClients function, we ensure that the cache is fully synchronized before the lister starts being used. Without this check, there's a potential race condition where the lister might be used before its cache is fully populated.

@voelzmo
Copy link
Contributor

voelzmo commented Jan 8, 2025

Thanks for helping me understand! I guess we can make a mental note to switch this to context instead of using stop channels once client-go 0.33.0 has been tagged: kubernetes/client-go@5d289bc#diff-dbac68d65bbc9814c2336df6ee91f22e807d84a9127a4d34a8b39c5657d0e447R369

@voelzmo
Copy link
Contributor

voelzmo commented Jan 8, 2025

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: omerap12, voelzmo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 8, 2025
@omerap12
Copy link
Member Author

omerap12 commented Jan 8, 2025

Thanks for helping me understand! I guess we can make a mental note to switch this to context instead of using stop channels once client-go 0.33.0 has been tagged: kubernetes/client-go@5d289bc#diff-dbac68d65bbc9814c2336df6ee91f22e807d84a9127a4d34a8b39c5657d0e447R369

I agree. Ill open an issue so we won't forget about this :)

@adrianmoisey
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 8, 2025
@k8s-ci-robot k8s-ci-robot merged commit 72206c1 into kubernetes:master Jan 8, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/vertical-pod-autoscaler cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants