Skip to main content

Home Directories

A user’s home data space is accessed by /home/{username} from the command line. Permissions are set to only allow that {username} to access their own directory. Each home directory:

  • has a quota of 100GB
  • is backed up daily
  • has hardware failure protection
  • is housed in a traditional ZFS file system on a remote server

The user’s home data space is NOT to be used to process data or as a job’s working directory but is used to store permanent files.

What kinds of files should be stored in my home directory?

Examples of files to store in the home directory:

  • Code repositories, scripts, or notes written by the user
  • Compiled programs/software
  • Final results from simulations/analyses

Examples of data that should NOT be stored in the home directory:

  • Intermediate data generated during simulations/analyses
  • Read-only data that will be processed to produce some sort of results
Watch Your Quota!

You cannot log in to the system if your home directory is full (over quota). If this happens to you, please submit a support ticket and a system administrator will help you fix the overage.

Check Quota​

You can check the quota and usage for your home directory using the checkquota command on the login node.

user@login001 $ checkquota

HOMEDIR QUOTA for USER example

Max  quota = 100G
Used quota = 1.44G

(used quota is refreshed every 2 minutes)

To investigate which files and folders are using large amounts of storage, you can use the du command. For example, to check the sizes of all files and folders in your home directory (non-recursively), run:

du -h -d 1 $HOME

If you find a folder you want to investigate (say /home/<user>/.local), you can run du on that directory too:

du -h -d 1 $HOME/.local

You can perform this recursively, moving down the directory tree looking for where all your space is being used. For example, if within your .local folder you found most of the space was in share, you could see where in share space is being used with:

du -h -d 1 $HOME/.local/share

Once you find where all your space is being used, it may become clear what is causing the issue. We have a few common cases listed below. If you still can't figure out what needs to be cleaned up or how, please reach out to us by submitting a support ticket.

Jupyter Trash (~/.local/share/Trash)​

When you delete things in a Jupyter notebook, it is not actually removed from disk. Instead it is moved to a Trash folder. You can check the size of the folder with:

du -sh ~/.local/share/Trash

To empty the trash and permanently delete all files within in it, run:

rm -rf ~/.local/share/Trash/*

Pip Cache (~/.cache/pip)​

If you install pip packages in the default Anaconda environment, pip will use the ~/.cache/pip directory to cache downloads and wheels.

To check how much cache pip is using, run:

qsub -I -l select=1:ncpus=4:mem=8gb,walltime=2:00:00
module load anaconda3/2022.05-gcc/9.5.0
pip cache info

To clear the cache, run:

pip cache purge

Anaconda (~/.conda)​

You may notice that your .conda folder is large. This folder stores your conda environments. Anaconda comes with a handy command to clean up unneeded packages and caches. To run it, get on a compute node, load anaconda, and then run the clean command:

qsub -I -l select=1:ncpus=4:mem=8gb,walltime=2:00:00
module load anaconda3/2022.05-gcc/9.5.0
conda clean --all

If this doesn't clean up enough, you may want to delete any unneeded environments. First, you may want to see which environments are taking up the most space. Unfortunately, this is not trivial since data for environments are split between two locations. Packages installed with pip will be placed in ~/.conda/envs/<env-name>, where as conda packages will be placed in ~/.conda/pkgs and shared between any conda environment that needs it.

So the following command will display the size of the pip packages installed, but not necessarily the full size of the environment -- some data will be stored in ~/.conda/pkgs:

du -sh ~/.conda/envs/*

Once you decide you want to delete an environment, you can use conda remove to delete the environment and conda clean to clean up any packages that became unused:

conda remove --name <env-name> --all
conda clean --all