Python Virtual Environments
Python programming language supports multiple technology use cases which span across the fields of blockchain, artificial intelligence, machine learning, scientific computing, web development (my favorite), and several others. Have you ever wondered, how is the language able to meet the requirements of all the applications in the listed tech sectors and beyond? We agree that the requirements are very many and not any particular programming language can be able to meet them. Putting it into context, when given two applications, X and Z, have conflicting requirements for the same module, such as X needs version 1.0 of the module while Z needs version 2.0 of the module, let's say when the software complexity increases, another application Y is integrated and requires version 3.0 of the module. Installing any version will prevent one application from functioning due to the mentioned conflicting requirements based on these versions. You may have come across the term backward compatibility in software development or in another related field, where a module version 2.0 could not be compatible with its predecessor version 1.0 which is a concept that comes up in the Software Development Life Cycle (SDLC).
The Python Virtual Environment
To address the issue above, Python has Virtual Environments which is a self-contained directory tree that includes several extra packages in addition to a Python installation for a specific version of Python. An existing Python installation in your computer is used as the "base" for the creation of a virtual environment, which can optionally be separated from the base environment's packages to make only those expressly installed in the virtual environment available. With this in place, various apps can utilize various virtual environments. Application X can have its own virtual environment with version 1.0 installed while Application Z has a different virtual environment with version 2.0 installed to resolve the preceding example of conflicting requirements. Application X's environment will not be impacted if Application Z needs a library to be from Y at version 3.0.
Why Do You Need Virtual Environments?
Some of the reasons that you may need to create isolated and self-contained Python environments for your projects include:
- Dependency Isolation: Your project or app could require a different version of an external library for one of your projects or apps than for another. There is no way to utilize two distinct versions of the same library if there is only one place to install packages. One of the most obvious justifications for suggesting using a Python virtual environment is this.
- Create Shareable Apps/Projects: As we will demonstrate, packages in a given virtual environment can be shared with collaborators using the requirements.txt file making it simple for others to replicate the environment.
- Not Interfere with System Packages: Virtual environments help improve security by limiting access to the system-wide Python installation. This is especially crucial when working on web applications or running untrusted code.
- Version Management: With virtual environments, you can use different versions of Python for different projects. This is particularly useful when you need to work on projects that are compatible with different Python versions.
- Testing and Debugging: Isolating your project in a virtual environment allows you to test and debug your code in an environment that precisely matches your project's dependencies. This ensures that the code behaves as expected for all users.
Types of Virtual Environments
Python interpreter is the default operating system environment well referred to as the global environment. If you have installed Python on your PC then you can run it by just typing python, python3, or py in the command prompt or terminal
These types of environments allow you to install packages without affecting other environments that are on your PC by isolating your package installations specific to a given virtual environment.
Examples of local environments include:
Virtual Environment: A built-in method for creating an environment is the use of a Python virtual environment. A Python virtual environment makes a copy of a particular Python interpreter in a folder that it generates. It will end up in this new folder when you install packages into a virtual environment, keeping them separate from other workspace-specific packages.
Conda Environment: Another method for creating an environment is the use of a Conda package manager. Conda installs runs, and updates packages as well as their dependencies. On your own PC, Conda makes creating, saving, loading, and switching between environments simple. You can read more about it from the conda documentation.
How To Use Virtual Environments
As of now, we are aware of what virtual environments are and their purposes. We will learn how to create, activate, and generally interact with virtual environments in this section of the lesson. For this article, we will be making use of Python's Venv module because it comes packaged with Python installation on your PC. If you have not installed Python, make a reference to how to in the following articles:
For MacOS users, make a reference here. Let's get going!
Creating a Python Virtual Environment
We shall be using Linux in this article, current Windows also comes with Windows Subsystem for Linux which you may use. We shall first create a project folder as shown below. Open the terminal and write the following command, hitting enter
Run the venv command to create a virtual environment inside the just created folder as shown below
python3 -m venv project_folder/proj-env
Below is the directory tree structure of the created virtual environment folder.
│ └── python3.10
│ └── site-packages
├── lib64 -> lib
bin - This directory contains executable scripts related to the virtual environment, such as the Python interpreter (
pip package manager specific to the virtual environment. It also contains activator scripts for PowerShell and Fish.
include - This directory contains header files that might be needed when building C extensions for Python packages (optional and not always present).
lib - This directory contains the Python standard library and installed packages specific to the virtual environment.
lib64 - This directory might exist on some 64-bit systems, and it also contains Python libraries (similar to
pyvenv.cfg - This contains various settings and information about the virtual environment, including the Python interpreter used, the path to the base Python installation, and other options related to the environment.
NOTE We may use virtualenv and venv, two programs for creating virtual environments, almost interchangeably. Older Python versions are supported through virtualenv, which must be installed using the pip command. The Python standard library already has venv, which can only be used with Python versions 3.3 or higher and doesn't need to be installed.
Activating Python Virtual Environment
Run the following command to activate the virtual environment
The terminal will appear with the name of the virtual environment inside parentheses at the start of the terminal prompt as shown below.
We could also check the location of the Python interpreter
(proj-env) $which python
Or the location of the pip package manager
(proj-env) $which pip
Installing Packages in a Python Virtual Environment
Right now, the only tools installed by default in our confined virtual environment are pip and setup. Let's use the
pip list command to look at the pre-installed packages on the virtual environment.
(proj-env) $pip list
We can also use the pip show command to get detailed information about a specific Python package in our virtual environment, in his case we are checking for setuptools.
(proj-env) $pip show setuptools
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Author: Python Packaging Authority
We can also install a package, in this case, let us install the Django package, you will notice that additional packages (dependencies) are also installed
(proj-env) $python3 -m pip install django
Using cached Django-4.2.3-py3-none-any.whl (8.0 MB)
Using cached asgiref-3.7.2-py3-none-any.whl (24 kB)
Using cached sqlparse-0.4.4-py3-none-any.whl (41 kB)
Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Installing collected packages: typing-extensions, sqlparse, asgiref, django
Successfully installed asgiref-3.7.2 django-4.2.3 sqlparse-0.4.4 typing-extensions-4.7.1
Now if we list the packages using
pip list command, this is what we get.
(proj-env) $pip list
There are other pip commands which you can refer to from the pip documentation.
Reproducing a Python Virtual Environment
Replicating a virtual environment is common. Assume that the project you have been working on for weeks will be completed by your coworker. On her workstation, she must install the exact same software in the proper versions in a virtual environment. You must first use the
pip freeze command to identify all the dependencies that have been installed in the project's virtual environment in order to build identical environments.
(proj-env) $pip freeze
Although the output
pip freeze is relatively similar to that of
pip list, it returns the list of packages installed on an environment in the proper format so that the environment can be replicated with the precise package versions the project requires. The export of the package list into the requirements.txt file comes next. Run this command to accomplish that:
(proj-env) $pip freeze > requirements.txt
This creates the
requirements.txt file containing all the packages and their exact versions. The file appears as shown below printed out using the cat command:
(proj-env) $cat requirements.txt
The next step entails deactivating a virtual environment.
Deactivating a Python Virtual Environment
When you are done working with the virtual environment, or if switching to another virtual environment using the same terminal window, it is best to deactivate it. This can be achieved by just running the deactivate command.
Deleting a Python Virtual Environment
One can simply delete the directory for the virtual environment by running the rm -rf command.
$rm -Rf proj-env
Using Third-Party Tools
Numerous other tools have been developed by the Python community that make use of virtual environments as one of its features and let you easily manage multiple virtual environments. You might be curious about each tool's purpose and potential to assist you in managing your virtual environments because they are frequently mentioned in online discussions and tutorials. Although going into detail about each of them is outside the purview of this lesson, you'll receive an overview of the most well-known projects, what they do, and where you can find out more:
We shall be looking at some of the tools listed above in future posts. I trust that you find this article informative and helpful and now that you have a basic understanding of the Python Virtual Environment you can create Python projects, install packages and reproduce them and share them with fellow developers or colleagues. Check out the next article on managing Python virtual environments using virtualenv.
Joseph Kariuki on Aug. 6, 2023
Thank you for your comment. I am happy to hear that you have been learning something from my posts. This encourages me to keep on sharing content. Congratulations on your new chapter in the software engineering journey. Let us keep in touch.
Jeevanananda Dalai on July 22, 2023
Wahh! Great Sir. I learn from somewhere practice makes perfect. Your continuously hard work and dedication reach out there. I always admire you and also try to learn everything from you. Finally your student will be a software engineer in future. A good teacher is a candle 🕯️ always shows bright future to her students. Really appreciating on your motivational speech.
Hi, thanks for visiting! My name is Joseph Kariuki, a software developer and this is my website. I am consistently seeking solutions to various problems using coding. This site does not contain ads, trackers, affiliates, and sponsored posts.
Published: July 20, 2023