The pip package manager makes it easy to install third-party Python modules along with their dependencies.
The installation is then usually done for the Python interpreter installed in the system, and the installed package is available to any Python program from then on.
Let’s suppose we were to develop two Python programs A and B on the same system, each with very different requirements. In such (quite common) scenarios, there are multiple conceivable reasons that a simple, system-wide installation of all dependencies of A and B could be an issue:
- The dependencies of A and B could be mutually exclusive—for example, via different version requirements—making simultaneous installation impossible.
- Installing a dependency required for A could change the behavior of B in a way we don’t want.
- A dependency of A might be experimental or unsafe, so we don’t want to make it available system-wide.
- We may be concerned that future updates of dependency packages we make for B will disrupt the functionality of A, which is particularly important to us.
For such use cases, Python provides the concept of virtual environments. A virtual environment is a directory that contains its own Python interpreter and pip package manager. Python programs can be run and Python packages can be installed in this directory without affecting the system-wide installation of Python.
In this way, a virtual environment can be set up for each of the imaginary Python programs A and B mentioned previously, in which the respective dependencies of A and B are installed completely independently of each other in the appropriate versions.
Note: We’ll first look at the venv module, which allows you to set up and use a virtual environment within a shell.
Especially for Windows users, the graphical user interface of Anaconda Navigator for managing virtual environments is much more convenient. Development environments such as PyCharm also provide such a graphical interface.
Then, we’ll briefly review the management of virtual environments in Anaconda.
Using Virtual Environments: venv
A virtual environment named test_environment can be created using the venv module of the standard library:
$ python -m venv test_environment
This command creates a test_environment directory that contains the new virtual environment. Within this directory, the bin subfolder contains the executable components relevant to the virtual environment, including the Python interpreter and the pip package manager.
Activating a Virtual Environment
Aside from the fact that a virtual environment was created, nothing else happened. A Python program would still run in the context of the system-wide Python installation at this point. To use a virtual environment, it must first be activated.
To activate a virtual environment on Windows, you must run the activate script in the Scripts subdirectory of the virtual environment:
$ test_environment\Scripts\activate
On Linux and macOS, the script can be found in the bin subdirectory. With the additional source command, we enable the script to modify the environment variables of the currently running shell:
$ source test_environment/bin/activate
Working in a Virtual Environment
The activate script of the virtual environment sets all Python-specific environment variables of the current shell to reference the Python installation of the virtual environment. In this way, the virtual environment can be used in the current shell as if it were the system-wide Python installation. In particular, Python packages can be installed via pip:
(test_environment) $ pip install setuptools
The (test_environment) annotation is used in most shells to indicate which virtual environment is currently being used after the activation script is executed. Python programs can also be executed in virtual environments as usual:
(test_environment) $ python test.py
In this case, the execution takes place in the context of the virtual environment—that is, in particular, in the context of the packages installed there.
Deactivating a Virtual Environment
You can leave a virtual environment using the deactivate command. This has been made universally available on both Windows and Linux via the activate script within the running shell, so a simple execution is all that’s required:
(test_environment) $ deactivate
The command changes all Python-specific environment variables of the current shell back to referencing the system-wide Python installation. To permanently remove a virtual environment, it suffices to delete the directory where it’s located.
Note: In addition to the venv module of the standard library for using virtual environments, the third-party virtualenv module is also very popular. For more information on virtualenv, you can visit https://virtualenv.pypa.io.
Virtual Environments in Anaconda
Anaconda also supports the creation and management of virtual environments. This can either be done via the graphical Anaconda Navigator or within a shell, similar to the usage of venv. An Anaconda environment can be created via the conda create command:
$ conda create --name test_environment2
This command doesn’t create the environment as a folder in the local working directory, but in a central collection of virtual environments organized by Anaconda. Activating and deactivating an environment involves the same semantics as working with venv and is done via conda activate and conda deactivate respectively:
$ conda activate test_environment2
$ conda deactivate
A major advantage of Anaconda is that it can also create virtual environments that use a version of the Python interpreter that differs from the system installation, whereupon all required packages are automatically downloaded from the Anaconda package index:
$ conda create --name test_environment3 python=3.10
These defaults can also be made when configuring virtual environments via the graphical user interface in Anaconda Navigator. Especially because of the easier handling, we recommend the use of Anaconda on Windows.
Note: Many developers make their Python programs directly executable on Unix-based operatingsystems using a shebang line. If your program needs to run in the context of an Anaconda environment test_environment, the shebang line can be adjusted accordingly:
#!/usr/bin/env conda run -n test_environment python
This shebang line makes the program run in the context of the enabled test_environment environment using Anaconda. Note that Anaconda must be contained in the system path for this to work.
Editor’s note: This post has been adapted from a section of the book Python: The Comprehensive Guide by Johannes Ernesti and Peter Kaiser.
Comments