Python¶
Pip/Virtualenv¶
Pip is the preferred package manager for Python and each cluster provides several Python versions through the associated module which comes with pip. In order to install new packages, you will first have to create a personal space for them to be stored. The preferred solution (as it is the preferred solution on Compute Canada clusters) is to use virtual environments.
First, load the python module you want to use:
module load python/3.6
Then, create a virtual environment in your home
directory:
virtualenv $HOME/<env>
where <env>
is the name of your environment. Finally, activate the environment:
source $HOME/<env>/bin/activate
You can now install any python package you wish using the pip
command, e.g. pytorch:
pip install torch torchvision
or Tensorflow:
pip install tensorflow-gpu
Conda¶
Another solution for Python is to use miniconda or
anaconda which are also available through the module
command:
(the use of conda is not recommended for Compute Canada Clusters due to the availability of custom-built packages for pip)
module load miniconda/3
[=== Module miniconda/3 loaded ===]
To enable conda environment functions, first use:
conda-activate
Then like advised, if you want to enable conda activate/deactivate
functions, start the following alias once
conda-activate
To create an environment (see here for details) do:
conda create -n <env> python=3.6
where <env>
is the name of your environment. You can now activate it by doing:
conda activate <env>
You are now ready to install any python package you want in this environment. For instance, to install pytorch, you can find the conda command of any version you want on pytorch’s website, e.g:
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
Don’t forget to clean the environment after each install:
conda clean --all