Difference between revisions of "Environment"
(Python venv primer for cluster users.) |
|||
| (One intermediate revision by one other user not shown) | |||
| Line 1: | Line 1: | ||
| + | === Bash === | ||
| + | |||
| + | There are several useful environment variables provided by SLURM when submitting jobs: | ||
| + | SLURM_JOB_ID - Job ID. | ||
| + | SLURM_JOB_NAME - Job name. | ||
| + | CUDA_VISIBLE_DEVICES - Specifies the GPU devices for the job allocation. | ||
| + | TMPDIR - Local temporary directory reserved for the job. This should be used instead of /tmp. | ||
| + | |||
| + | Refer to [https://slurm.schedmd.com/prolog_epilog.html SLURM docs] for complete list. | ||
| + | |||
=== Python === | === Python === | ||
| Line 9: | Line 19: | ||
=== Virtual environment === | === Virtual environment === | ||
| − | The recommended way is to use a virtual environment. | + | The recommended way is to use python through a virtual environment. |
You need to decide which PYTHON_VERSION you want to use and where do you want to store your virtual environment (VENV_PATH). Then you can create it: | You need to decide which PYTHON_VERSION you want to use and where do you want to store your virtual environment (VENV_PATH). Then you can create it: | ||
Latest revision as of 14:34, 9 April 2024
Bash
There are several useful environment variables provided by SLURM when submitting jobs:
SLURM_JOB_ID - Job ID. SLURM_JOB_NAME - Job name. CUDA_VISIBLE_DEVICES - Specifies the GPU devices for the job allocation. TMPDIR - Local temporary directory reserved for the job. This should be used instead of /tmp.
Refer to SLURM docs for complete list.
Python
If you need to use a different version of python than the default one installed with node OS you can use pre-installed versions found at:
/opt/python
If you need some specific version not found in this directory please contact the AIC administrators.
Virtual environment
The recommended way is to use python through a virtual environment. You need to decide which PYTHON_VERSION you want to use and where do you want to store your virtual environment (VENV_PATH). Then you can create it:
/opt/python/<PYTHON_VERSION>/bin/python3 -m venv <VENV_PATH>
You need to activate the environment before you actually use it:
source <VENV_PATH>/bin/activate
Then you should be able to use the python version of your choice. More about virtual environments can be found here
