Python Virtual Environments

Python programming language supports multiple technology use cases which span across the fields of blockchain, artificial intelligence, machine learning, scientific computing, web development (my favorite), and several others. Have you ever wondered, how is the language able to meet the requirements of all the applications in the listed tech sectors and beyond? We agree that the requirements are very many and not any particular programming language can be able to meet them. Putting it into context, when given two applications, X and Z, have conflicting requirements for the same module, such as X needs version 1.0 of the module while Z needs version 2.0 of the module, let's say when the software complexity increases, another application Y is integrated and requires version 3.0 of the module. Installing any version will prevent one application from functioning due to the mentioned conflicting requirements based on these versions. You may have come across the term backward compatibility in software development or in another related field, where a module version 2.0 could not be compatible with its predecessor version 1.0 which is a concept that comes up in the Software Development Life Cycle (SDLC).

The Python Virtual Environment

To address the issue above, Python has Virtual Environments which is a self-contained directory tree that includes several extra packages in addition to a Python installation for a specific version of Python. An existing Python installation in your computer is used as the "base" for the creation of a virtual environment, which can optionally be separated from the base environment's packages to make only those expressly installed in the virtual environment available. With this in place, various apps can utilize various virtual environments. Application X can have its own virtual environment with version 1.0 installed while Application Z has a different virtual environment with version 2.0 installed to resolve the preceding example of conflicting requirements. Application X's environment will not be impacted if Application Z needs a library to be from Y at version 3.0.

Why Do You Need Virtual Environments?

Some of the reasons that you may need to create isolated and self-contained Python environments for your projects include:

• Dependency Isolation: Your project or app could require a different version of an external library for one of your projects or apps than for another. There is no way to utilize two distinct versions of the same library if there is only one place to install packages. One of the most obvious justifications for suggesting using a Python virtual environment is this.
• Create Shareable Apps/Projects: As we will demonstrate, packages in a given virtual environment can be shared with collaborators using the requirements.txt file making it simple for others to replicate the environment.
• Not Interfere with System Packages: Virtual environments help improve security by limiting access to the system-wide Python installation. This is especially crucial when working on web applications or running untrusted code.
• Version Management: With virtual environments, you can use different versions of Python for different projects. This is particularly useful when you need to work on projects that are compatible with different Python versions.
• Testing and Debugging: Isolating your project in a virtual environment allows you to test and debug your code in an environment that precisely matches your project's dependencies. This ensures that the code behaves as expected for all users.

Types of Virtual Environments

Global Environments

Python interpreter is the default operating system environment well referred to as the global environment. If you have installed Python on your PC then you can run it by just typing python, python3, or py in the command prompt or terminal

Local Environments

These types of environments allow you to install packages without affecting other environments that are on your PC by isolating your package installations specific to a given virtual environment.

Examples of local environments include:

Virtual Environment: A built-in method for creating an environment is the use of a Python virtual environment. A Python virtual environment makes a copy of a particular Python interpreter in a folder that it generates. It will end up in this new folder when you install packages into a virtual environment, keeping them separate from other workspace-specific packages.

Conda Environment: Another method for creating an environment is the use of a Conda package manager. Conda installs runs, and updates packages as well as their dependencies. On your own PC, Conda makes creating, saving, loading, and switching between environments simple. You can read more about it from the conda documentation.

How To Use Virtual Environments

As of now, we are aware of what virtual environments are and their purposes. We will learn how to create, activate, and generally interact with virtual environments in this section of the lesson. For this article, we will be making use of Python's Venv module because it comes packaged with Python installation on your PC. If you have not installed Python, make a reference to how to in the following articles:

• Installing Python on Windows
• Installing Python on Linux

For MacOS users, make a reference here. Let's get going!

Creating a Python Virtual Environment

We shall be using Linux in this article, current Windows also comes with Windows Subsystem for Linux which you may use. We shall first create a project folder as shown below. Open the terminal and write the following command, hitting enter

mkdir project_folder

Run the venv command to create a virtual environment inside the just created folder as shown below

python3 -m venv project_folder/proj-env

Below is the directory tree structure of the created virtual environment folder.

project_folder/proj-env/
├── bin
├── include
├── lib
│   └── python3.10
│       └── site-packages
├── lib64 -> lib
└── pyvenv.cfg

bin - This directory contains executable scripts related to the virtual environment, such as the Python interpreter (python) and pip package manager specific to the virtual environment. It also contains activator scripts for PowerShell and Fish.

include - This directory contains header files that might be needed when building C extensions for Python packages (optional and not always present).

lib - This directory contains the Python standard library and installed packages specific to the virtual environment.

lib64 - This directory might exist on some 64-bit systems, and it also contains Python libraries (similar to lib).

pyvenv.cfg - This contains various settings and information about the virtual environment, including the Python interpreter used, the path to the base Python installation, and other options related to the environment.

NOTE We may use virtualenv and venv, two programs for creating virtual environments, almost interchangeably. Older Python versions are supported through virtualenv, which must be installed using the pip command. The Python standard library already has venv, which can only be used with Python versions 3.3 or higher and doesn't need to be installed.

Activating Python Virtual Environment

Run the following command to activate the virtual environment

source project_folder/proj-env/bin/activate

The terminal will appear with the name of the virtual environment inside parentheses at the start of the terminal prompt as shown below.

(proj-env) $

We could also check the location of the Python interpreter

(proj-env) $which python
/home/testuser/project_folder/proj-env/bin/python

Or the location of the pip package manager

(proj-env) $which pip
/home/testuser/project_folder/proj-env/bin/pip

Installing Packages in a Python Virtual Environment

Right now, the only tools installed by default in our confined virtual environment are pip and setup. Let's use the pip list command to look at the pre-installed packages on the virtual environment.

(proj-env) $pip list
Package    Version
---------- -------
pip        22.0.2
setuptools 59.6.0

We can also use the pip show command to get detailed information about a specific Python package in our virtual environment, in his case we are checking for setuptools.

(proj-env) $pip show setuptools
Name: setuptools
Version: 59.6.0
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License: UNKNOWN
Location: /home/testuser/project_folder/proj-env/lib/python3.10/site-packages
Requires: 
Required-by:

We can also install a package, in this case, let us install the Django package, you will notice that additional packages (dependencies) are also installed

(proj-env) $python3 -m pip install django
Collecting django
  Using cached Django-4.2.3-py3-none-any.whl (8.0 MB)
Collecting asgiref<4,>=3.6.0
  Using cached asgiref-3.7.2-py3-none-any.whl (24 kB)
Collecting sqlparse>=0.3.1
  Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Collecting typing-extensions>=4
  Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Installing collected packages: typing-extensions, sqlparse, asgiref, django
Successfully installed asgiref-3.7.2 django-4.2.3 sqlparse-0.4.4 typing-extensions-4.7.1

Now if we list the packages using pip list command, this is what we get.

(proj-env) $pip list
Package           Version
----------------- -------
asgiref           3.7.2
Django            4.2.3
pip               22.0.2
setuptools        59.6.0
sqlparse          0.4.4
typing_extensions 4.7.1

There are other pip commands which you can refer to from the pip documentation.

Reproducing a Python Virtual Environment

Replicating a virtual environment is common. Assume that the project you have been working on for weeks will be completed by your coworker. On her workstation, she must install the exact same software in the proper versions in a virtual environment. You must first use the pip freeze command to identify all the dependencies that have been installed in the project's virtual environment in order to build identical environments.

(proj-env) $pip freeze
asgiref==3.7.2
Django==4.2.3
sqlparse==0.4.4
typing_extensions==4.7.1

Although the output  pip freeze is relatively similar to that of pip list, it returns the list of packages installed on an environment in the proper format so that the environment can be replicated with the precise package versions the project requires. The export of the package list into the requirements.txt file comes next. Run this command to accomplish that:

(proj-env) $pip freeze > requirements.txt

This creates the requirements.txt file containing all the packages and their exact versions. The file appears as shown below printed out using the cat command:

(proj-env) $cat requirements.txt 
asgiref==3.7.2
Django==4.2.3
sqlparse==0.4.4
typing_extensions==4.7.1

The next step entails deactivating a virtual environment.

Deactivating a Python Virtual Environment

When you are done working with the virtual environment, or if switching to another virtual environment using the same terminal window, it is best to deactivate it. This can be achieved by just running the deactivate command.

(proj-env) $deactivate


Deleting a Python Virtual Environment

One can simply delete the directory for the virtual environment by running the rm -rf command.

$rm -Rf proj-env

Using Third-Party Tools

Numerous other tools have been developed by the Python community that make use of virtual environments as one of its features and let you easily manage multiple virtual environments. You might be curious about each tool's purpose and potential to assist you in managing your virtual environments because they are frequently mentioned in online discussions and tutorials. Although going into detail about each of them is outside the purview of this lesson, you'll receive an overview of the most well-known projects, what they do, and where you can find out more:

1. Pipenv
2. Virtualenvwrapper
3. Poetry
4. Pipx
5. Pyenv

We shall be looking at some of the tools listed above in future posts. I trust that you find this article informative and helpful and now that you have a basic understanding of the Python Virtual Environment you can create Python projects, install packages and reproduce them and share them with fellow developers or colleagues. Check out the next article on managing Python virtual environments using virtualenv.

Related articles

• Introduction To Django
• How To Learn Python in 2023

Comments

Name*

Email*

Body*

Captcha*

Submit