Change conda env validity checking to activate and run Python #48552
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Peter Wang <[email protected]>
|
cc @jjyao |
jjyao
reviewed
Nov 20, 2024
| Check if the given conda environment exists and is valid. | ||
| We try to `conda activate` it and check that python is runnable | ||
| """ | ||
| activate_cmd = get_conda_activate_commands(conda_env_name) + ["python", "--version"] |
There was a problem hiding this comment.
Do we need to deactivate it afterwards?
There was a problem hiding this comment.
I don't believe so, because conda activate sets environment variables and other things that aren't global state, which will be reset when the subprocess exits.
However, I did recently do some thinking about it, and I think conda activate does a lot of shell-specific things, so I've changed it to execute in a shell.
Signed-off-by: Peter Wang <[email protected]>
|
Can you also update unit tests? I think this test will now fail (changed error msg) |
added 2 commits
December 13, 2024 10:59
…se_conda_env_without_list Signed-off-by: Peter Wang <[email protected]>
Signed-off-by: Peter Wang <[email protected]>
|
Oops, I guess I didn't notice that! I've made the change. Sorry for the delay. |
…se_conda_env_without_list Signed-off-by: Peter Wang <[email protected]>
dayshah
approved these changes
Jan 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Currently, we validate conda environments by checking the output of
conda info --jsonto see if the environment is the list. Sometimes, one might install conda environments on some filesystem mounted across multiple machines. These environments can be activated with the absolute path:conda activate /path/to/environmentbut don't show up under the local conda installation'sconda info --json. I figure that what we really care about is that one can activate the environment and run Python in it, as that is what the RuntimeEnv plugin ultimately does, so this is what the check has become.Related issue number
None
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.