Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Don't rely on configs in prepare_for_gfn #62

Open
wants to merge 13 commits into
base: uncertainty
Choose a base branch
from

Conversation

vict0rsch
Copy link
Collaborator

Not tested yet, feel free to do it yourself in the mean time.

This would be my preferred direction if we don't hit more configs-related obstacles.

@vict0rsch
Copy link
Collaborator Author

Make loading the data optional ; MAKE SURE this does not break GflowNet validation procedure which could expect the dataloaders

@vict0rsch
Copy link
Collaborator Author

vict0rsch commented Sep 24, 2024

Implemented a first version refactoring BaseTrainer.load(...) to use a new kwarg called prevent_load which is used in prepare_for_gfn as

config["prevent_load"] = {
    "logger": True,
    "loss": True,
    "datasets": True,
    "optimizer": True,
    "extras": True,
}

@lpodina
Copy link
Collaborator

lpodina commented Oct 1, 2024

On origin/catalyst_parallel for gflownet-dev and origin/faster_surface for ocp, and this command can trigger a permission denied error:
HYDRA_FULL_ERROR=1 python -W ignore main.py user=elena.podina +experiments=catalysts/asir_magpie logger.debug=True env.atomgraphconverter.n_cpu_threads=64 gflownet.optimizer.batch_size.forward=64 gflownet.optimizer.n_train_steps=3 +env.atomgraphconverter.sample_timeout=2 +env.atomgraphconverter.pyx_timeout=1

@vict0rsch vict0rsch marked this pull request as ready for review October 8, 2024 13:43
@vict0rsch vict0rsch requested a review from lpodina October 8, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants