Mike Slinn
Mike Slinn

A Python Virtual Environment For Every Project

Published 2021-04-09. Last modified 2021-04-22.
Time to read: about 3 minutes.

This article is categorized under Bash, Python.

Python virtual environments are cheap to make and use. I have adopted the habit of making a Python virtual environment (venv) for each significant Python project, plus a default venv for trivial Python work.

Dedicating a venv for each Python project means that dependencies for any given Python project do not impact the dependencies for any other Python projects. Things just work better.

VEnvs are Nearly Free

The cost of a venv is virtually free. This is because by default, the executable images are linked, so they do not require much storage space. The ls command below shows that the python program in the aw venv is linked to /usr/bin/python3.8.

Shell
$ ls -go ~/venv/aw/bin/python
lrwxrwxrwx 1 18 Apr  9 06:01 /home/mslinn/venv/aw/bin/python -> /usr/bin/python3.8

Standard Procedure For Creating a VEnv

I name each venv the same as my python project. My projects are stored under the directory pointed to by $work.

My standard procedure when making a Python project called $work/blah is to also create a venv for it at ~/venv/blah. A bash alias could be defined called blah that activates the venv and cds into the project directory:

~/.bash_aliases
alias blah="source ~/venv/blah/bin/activate; cd $work/blah"
😁

Now you could type blah at a shell prompt and you would be working on that project. Boom!

Script For Creating a VEnv

Here is a bash script that creates the venv and changes ~/.bashrc and ~/.bash_aliases for you. It assumes that you keep your projects under $work.

newVenv
#!/bin/bash

function help {
  echo -e "$1$(basename $0) - Create a Python virtual environment with a given name.

Usage:
$(basename $0) venv_name

The new virtual environment will be created under ~/venv/.
If a project directory called \$work/venv_name exists before this script runs,
then a bash alias is created named after the venv."
  exit 1
}


if [ -z `which virtualenv` ]; then sudo apt install virtualenv; fi

if [ -z "$1" ]; then help "Please specify a name for the virtual environment.\n\n"; fi
if [ "$1" == -h ]; then help; fi
VENV="$1"
shift

mkdir -p "$HOME/venv"
cd "$HOME/venv"
virtualenv "$VENV"
DIR="$HOME/venv/$VENV"

echo "source $DIR/bin/activate" >> $HOME/.bashrc

echo
echo "Activation for "$VENV" in future shells was appended to $HOME/.bashrc"
echo "To activate the "$VENV" venv in this shell right now, type: source ~/venv/$VENV/bin/activate"

if [ "$work" ] && [ -d "$work/$VENV" ]; then
  echo "alias $VENV='source $DIR/bin/activate; cd $work/$VENV'" >> $HOME/.bash_aliases
  echo "An alias called $VENV for future shells was appended to $HOME/.bash_aliases"
  echo "To define the alias in this shell right now, type: alias $VENV='source $DIR/bin/activate; cd $work/$VENV'"
else
  echo "To define an alias, type something like this: alias $VENV=\"source $DIR/bin/activate; cd $work/$VENV\""
fi

This is the help message for the script:

newVenv help message
$ newVenv -h
newVenv - Create a Python virtual environment with a given name.

Usage:
newVenv venv_name

The new virtual environment will be created under ~/venv/.
If a project directory called $work/venv_name exists before this script runs,
then a bash alias is created named after the venv.

Let's use the script to create a venv called aw:

newVenv help message
$ newVenv aw
created virtual environment CPython3.8.6.final.0-64 in 528ms
  creator CPython3Posix(dest=/home/mslinn/venv/aw, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mslinn/.local/share/virtualenv)
    added seed packages: pip==20.1.1, pkg_resources==0.0.0, setuptools==44.0.0, wheel==0.34.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

Script for Using a VEnv

Here is a script that can display the available Python virtual environments, and optionally activates one them. It does not use bash aliases.

use
#!/bin/bash

function help {
  echo "Usage:"
  for f in $HOME/venv/*; do
    if [ -d "$f" ]; then echo "  . $(basename $0) $(basename $f)"; fi
  done
  return 2
}

unset PV
if [ "$1" == -h ]; then 
  help
elif [ "$1" ]; then 
  PV="$1"
else 
  PV="default"
fi

if [ "$PV" ]; then
  DIR="$HOME/venv/$PV"
  if [ ! -d "$DIR" ]; then
    echo "Error: $DIR does not exist."
    return 1
  fi

  if [ ! -f "$DIR/bin/python" ]; then
    echo "Error: No Python virtual environment is installed in $DIR"
    return 1
  fi
  echo "Setting Python virtual environment to $DIR"
  source "$DIR/bin/activate"
fi

Here are examples of using the script to change virtual environments:

Shell
$ . use -h
Usage:
  . bash aw
  . bash default 

$ . use
Setting Python virtual environment to /home/mslinn/venv/default 

(default) $ . use aw
Setting Python virtual environment to /home/mslinn/venv/aw 

(aw) $

Notice that the last command above changed the shell prompt, in that (aw) was prepended to the normal prompt. To cause all future shells to use this virtual environment by default, the script adds a line to ~/.bashrc that looks like this:

Shell
$ echo "source ~/venv/aw/bin/activate" >> ~/.bashrc

At this point the virtual environment just contained executable images for Python.

Shell
$ ls ~/venv/aw/**
~/venv/aw/pyvenv.cfg

~/venv/aw/bin:
activate       activate.ps1      chardetect      distro      easy_install      pip      pip3.8   python3.8  wheel3
activate.csh   activate.xsh      chardetect-3.8  distro-3.8  easy_install-3.8  pip-3.8  python   wheel
activate.fish  activate_this.py  chardetect3     distro3     easy_install3     pip3     python3  wheel-3.8

~/venv/aw/lib:
python3.8

Deactivate a VEnv

Stop using venvs with `deactivate`:

Shell
(aw) $ deactivate
$

Activate With an Alias

Once again we can use a bash alias, this time to invoke the use script. We can call the alias use, because bash aliases have precedence over bash scripts. This alias removes the need to type . or source before the script name (which you know is use, if you have been following along).

Shell
$ alias use="source use"
$ use
(default) $ use aw
(aw) $

You can add the alias to bash_aliases:

Shell
$ echo 'alias use="source use"' >> ~/.bash_aliases
😁

Directory-Locked Python Virtualization

After setting up a Python virtual environment, a quick examination of the pip script shows that it is hard-coded to the directory that it was made for:

Shell
$ head -n 1 ~/venv/aw/bin/pip
  #!/home/mslinn/venv/aw/bin/python

For virtualized environments, such as Docker, this means that a Python virtual environment created without Docker can only be used within a Docker image if the path to it is the same from within the Docker image as when it was created.

For Further Reading

Python Best Practices for a New Project in 2021

Summary

  • Demonstrated how to make an alias for working with Python virtual environments (venvs) that are coupled with Python projects.
  • The newVenv bash script was demonstrated for making new Python virtual environments.
  • The use bash source script was demonstrated for activating a venv.
  • Deactivating the current venv was demonstrated using the deactivate command, provided with every venv.
  • The use alias for source use was demonstrated for more conveniently selecting a venv.
  • Locked directories mean that Python virtual environments should normally only be created in the same environment they are intended to be used.