Running on DIDE cluster (ICL only)

To be able to run model simulations on the DIDE cluster, we first have to set up the model on a filesystem that is visible to the cluster. In most cases, this will be your "Q:"

Connecting to Q:

You should be on Imperial's network (either Imperial Wi-fi, office, or connected via Zscaler). (For Mac Users: pause Private-Relay to enable this connection.)

On Mac, press command+K and type "smb://wpia-san04.dide.ic.ac.uk/homes/", replacing username with your DIDE username. For example, my username is sbhatia.

On Windows, you will need to map a network drive to \wpia-san04.dide.ic.ac.uk\homes\username Remember to tick the box to “connect using different credentials” and put your username as “dide\username and use your DIDE password to connect.

Cloning TLO repo

Navigate to the Q: on your machine and clone the TLO repo.

git clone https://github.com/UCL/TLOmodel.git

Checkout the branch on which your analysis script is available.

Setting up the virtual environment on Q:

The first time you are running an analysis, create a bat file that sets up the virtual environment. Save it in the TLOmodel directory with any name (e.g. install.bat).

call setGit
call set_python_311_64
python -m venv venv-tlo
call venv-tlo2\Scripts\activate
pip install -r requirements/dev.txt
pip install -e .
tlo

Then in your browser, navigate to https://mrcdata.dide.ic.ac.uk/hpc/

Login with your DIDE username (without using the prefix dide) and password.

Click on "submit jobs".

Give the job a name.

Working dir is the directory where you have cloned the repo and saved the above bat file e.g., for me it is \\qdrive.dide.ic.ac.uk\homes\sbhatia\TLOmodel

In the field Job(s) to run, give the name of the bat file above e.g. for me it was install.bat.

Press submit.

You can now click on manage jobs and check the status of the submitted job. This page refreshes automatically and only lists jobs that are currently running. If you don't see your job listed here, click on the dropdown above to filter the list to the jobs that have finished/failed/queued etc.

If the job has finished successfully, you will see the output of the command tlo in the output box.

Running model analysis

The virtual environment set-up needs to happen only once. If the job has run successfully, you can now run your actual analysis. Create a bat file with the following contents and save it in the TLOModel directory.

call setGit
call set_python_311_64
call venv-tlo2\Scripts\activate

net use Q: \\qdrive.dide.ic.ac.uk\homes\sbhatia
Q:
cd TLOmodel

tlo scenario-run src/scripts/dev/scenarios/cluster-run.py

Replace the last line with the file you wish to use. Now submit your jobs as in the previous step where you can also choose the number of cores. The log files will be written in the usual place in the TLOModel directory; this will also give you an indication of whether the job is running successfully.