In order to properly use the Jupyter notebook/lab on the HPC you will have to make some modifications in your workflow.

To have a singular resource for each user we had to make a global install of conda so every user would have conda installed regardless of their own personal install. 

This global install of conda has been configured to allow users to change environments on the fly.


To access this installation you need to run the following command:

/opt/conda/bin/conda init bash


This modifies how you will be able to call your own existing envs from command line.

Namely all names are removed (i.e. if you had an env made with '-n my_env' typing `conda activate my_env` would no longer activate the environment) as the conda binary is looking in /opt/conda and /home/${USER}/.conda/envs for environments.

To activate the environments you have created you’ll need to use the full path (i.e. `conda activate /home/${USER}/*conda/envs/my_env`). 


This will also change where you create environments. When you use `conda create -n my_env` it will place it in a new folder located at /home/<username>/.conda/envs/ and you can activate it using the typical `conda activate my_env`. 

If you want to install in the old location instead of using `conda create -n my_env` you will need to use the prefix option to point to your env folder: `conda create -p /home/${USER}/*conda/envs/my_env`. All environments created with the prefix option will have to be activate with the full path instead of the name. 


In order to leverage the environment switching inside Jupyter Notebook you will need to install the appropriate kernels for the languages you have inside the environment in order to be able to switch kernels.

You can do this with the following commands within the environment you want to use:

To install the python kernel run: conda install -c anaconda ipykernel

To install the R kernel run: conda install -c r r-irkernel


You can change the active environment once inside the notebook (or when creating one) via kernels: Kernel > Change kernel > whatever_you_name_your_env


Note: If for some reason you want to switch back to your personal conda setup and loose the use of environment switching using Jupyter Notebooks/Lab on the HPC simply run:

/home/${USER}/*conda/bin/conda init bash

You will need to re-run /opt/conda/bin/conda init bash to enable environment switching again.


If you do not want to change permissions to your /home/ directory and subdirectories, please refer to these instructions on how to create a shareable environment: Creating shareable environments with conda outside of /home/

If you want to have all shared environments visible and able to be activated by name alone, simply run the following command

conda config --append envs_dirs /home/SE
  • No labels