Virtual Environment for Python Workspace – Part 3/3 – venv

In the earlier two posts of this series, I discussed about virtualenv  and conda  and their usage along with some examples. I this post I am going to introduce you guys with another package and environment manager called venv  which comes with standard Python3.X by default. So, nothing to worry about its installation.

Virtual Environment for Python Workspace – Part 1/3 – virtualenv
Virtual Environment for Python Workspace – Part 2/3 – conda

Creating an Environment
For example, If you have Python3.X installed in your system and you can run it’s REPL by issuing python3 in the terminal, then you can create a virtual environment based on this Python version by issuing the following command:

This will create the myvenv  directory if it doesn’t exist, and also create directories inside it containing a copy of the Python interpreter, the standard library, and various supporting files. By the way, you got the idea of using different versions of Python to use its venv  module for creating different environments, right? Just find out an exact Python 3.X (I showed how, in earlier posts) and use its binary/executable in the command <active python> -m venv <environment name> .

Activating the Environment
Its similar to the previous two workarounds and again simple as below. I assumed you are inside the newly created myvenv  directory.

Managing Packages with pip
As venv  is native with Python, you can use pip  for installing packages in your virtual environment which will pull down packages from Python’s official package repository, PyPI. For example, to install our very favorite package numpy , use the following command,

Also, for sure you can use commands like pip search <packagename> , pip list in each newly created virtual environment.

Did you know, you can use pip show <packagename>  to see detail of an installed package in an active environment?

Sharing Environment
In the earlier post, I have discussed why and how you can share your environment made using  conda  with others. If you are using venv  or virtualenv  then you can export an environment’s state/configuration to a file, issuing the following command,

Here, the file name requirements.txt , that contains environment’s package information could be anything. But people have been using this name conventionally for a long time.

So, after exporting the environment information, you can share this file to any one who can install exact same packages in his newly created virtual environment by using the following command:

By the way, sharing environment in this way is similar if you use virtualenv


Do you love Watching than Reading? Subscribe & Get Connected with my YouTube channel for Video version of all my Blog posts.

Virtual Environment for Python Workspace – Part 2/3 – conda

Hope you already got the idea behind the necessity of virtual environment in Python ecosystem in my previous post. So, in this post I will directly go into the detail of conda  and its usage.

What is conda?
Just like virtualenv , this is also a package, dependency & environment management tool. Unlike virtualenv , conda  can work not only with Python but also with R, Javascript, Ruby, Lua, Scala etc. But, we will only focus on Python ecosystem for the sake of this series.

There are some significant benefits of conda  over virtualenv . For example – in its default configuration, conda  can install and manage thousand of packages at that are built, reviewed and maintained by Anaconda. Also, conda  can be combined with Continous Integration tools for better test & deployment.

How to install conda?
The easiest way to get conda  is to install miniconda . Yes, its sounds like Anaconda. But we don’t need to install that big fat ready-made environment. Rather we can install miniconda  that will come with conda  as package manager and its dependencies and the Python. By the way,

You do not need to uninstall other Python installations or packages in order to use conda. Even if you already have a system Python, another Python installation from a source such as the macOS Homebrew package manager and globally installed packages from pip such as pandas and NumPy, you do not need to uninstall, remove, or change any of them before using conda.

So, just download miniconda  from here and install in your system. You can choose either Python 2.X or Python 3.X version (I prefer to install miniconda  of Python 3.X version). We will talk about that very soon. The installer will add the  conda  installation of Python to your PATH environment variable. To verify that conda  is installed successfully, issue the following command.

Creating Virtual Environment
Before creating an environment, please keep in mind that, there are two variants of the installer: Miniconda is Python 2.X based and Miniconda3 is Python 3.X based. Note that the choice of which Miniconda is installed only affects the root environment. Regardless of which version of Miniconda you install, you can still install both Python 2.x and Python 3.x based environments.

Suppose you installed Python 3.X version of Miniconda and then if you issue the following command to create an environment,

the environment will be created based on Python 3.X version. On the other hand if you issue the following command, it will create a virtual environment based on Python 2.X.

Anyway, for now, forget about the mypy2env . Let’s activate the myenv  environment and start using that.

Activating an Environment 
To activate the new environment, run the appropriate command for your operating system:

  • Linux and macOS: source activate myenv
  • Windows: activate myenv

As my miniconda  version was of Python 3.X version and I created the environment myenv  with default Python, so this environment is based on Python 3.X. We can also check that by activating and running the Python REPL in it, maybe. Just check out the following Terminal activity and hope you will get it 🙂

Type exit()  in the above REPL to quit it and simply get back to the environment.

Deactivating an Environment 

Deactivating an environment is as simple as issuing the following command,

So, the above command will deactivate the myenv  environment. Oh, by the way, before deactivating, let’s check which packages are installed in this new virtual environment by the following command:

Want to install packages in this virtual environment for your next project?

Managing Packages in an Environment
If you want to install packages using the conda  package manager then it can be done simply by conda install <packagename> . For example, let’s install the package beautifulsoup4  in the environment myenv :

A good thing about conda  package manager is that, if you don’t find a package using conda install  then you can look into which is another product of the same vendor of miniconda and Anaconda. For example, to install the package bottleneck , you can issue the following command:

Keep in mind that, you have to search for the desired package in the site and there you will get the appropriate installation command for conda .

pip in conda
If you don’t find a package in conda or in, then still you can install the package inside this conda  virtual environment by pip . Using pip  will download and install the package from Python’s official package repository PyPI . For example, let’s install numpy  in this virtual environment by the following command:

Now, let’s check the packages that are installed so far in the environment by the following command again:

which will show something like following:

In the above output, we can see that both beautifulsoup4  & numpy  exist together, though we installed beautifulsoup4  using conda  from their repository and numpy  from PyPI using pip .

Removing an Environment
Before removing any environment, you need to see all existing conda environments, right? For that, simply issue the following command which will show you all the existing virtual environments made by conda:

Lets, remove the mypy2env  which we don’t want to use.

Sharing Environment to Other
Say, you have prepared a very good Python environment with all the necessary packages installed in it and made sure that the environment is working fine for your project/app. Now, you want to share this ready-made environment with your friend who is struggling to prepare such an environment where he wants to work on the same Python project/app. Yes, you can do that by simply exporting your environment configuration in a file and sharing that file with your friend.

First, you need to activate the environment you want to share. Then, to export our myenv  environment’s configuration, issue the following command:

So, the shareable file is called environment.yml  and want to see whats in this file? It should contain something like following:

Pretty self-explanatory, right? Now, you can share this file to your friend and if your friend wants to prepare such a ready-made virtual environment, then he needs to install conda  first and the issue the following command:

conda is the package manager. Anaconda is a set of about a hundred packages including conda, numpy, scipy, ipython notebook, and so on.

Did you know, Once you have Miniconda, you can easily install Anaconda into it with conda install anaconda ?


Do you love Watching than Reading? Subscribe & Get Connected with my YouTube channel for Video version of all my Blog posts.

Virtual Environment for Python Workspace – Part 1/3 – virtualenv

If you are planning to work or have been working in Python stack for web development, machine learning, data analysis etc. then you must have already come through the term “virtual environment”. In this article I will try to explain what it is, why its needed and how to acheive this workaround in three different ways. I will give you a basic overview of creating and managing Python virtual environment with virtualenv , venv  and conda  (with miniconda). So, lets start.

What is Python virtual environment and why it’s needed?
Suppose, you have only one computer for development or even a single VPS for deployment. Obviously you will not develop a single application in your whole life, right? As a software developer you will have to develop bunch of tools, modules, softwares, scripts in your entire lifecycle. Let’s not go that much deeper for now 😛

Assume that, you are developing an application which needs version 1.0 of a certain library/package and so far its working nice based on its dependency(s). Here by dependency, I mean that specific library/package. What if, you start developing another app or got responsibility to contribute in an ongoing application and fortunately you found that, this app also uses that exact same library/package you already have installed in your system; and unfortunately you also found that it works only for a different version of that so called library? So, its quite normal that, an application can depend on different packages of different versions and there is nothing wrong with this situation. Even, different app can depend of different versions of Python (interpreter), seems fair, right?

Here comes the necessity of app specific dependency management. This dependency is not only on library or packages but also could on Python versions. Another important issue arises while developing in Python stack is, the chance of messing up your system’s native Python installation. Every *Nix like operating system comes with Python pre-installed. You already know that 🙂 Even, recent distributions are coming with two or more versions of Python pre-installed (You can access those different REPL by issuing commands like python  or python3  in the terminal).

But why so? Because, it could be happened that, some OS specific tools or programs use Python 2.X for their own needs. On the other hand users can expect to have an updated version of Python in that system for their custom development. Now, if you install all your required libraries and packages in global package (in site-packages  directory if specifically said) scope, then very soon it will be a mess up. What you could do is to install needed packages to the location where Python 3.X resides. Still that above mentioned situation could be raised where you may need different versions of libraries for different developing apps.

So, need for having isolated Python environments is so casual. If we can make different Python environments; based on different versions of Python and install very specific libraries and packages in it, then everything will be more organized and loosely coupled with the native system. virtualenv  is such a package that helps you to make unlimited number of virtual environments and configure them according to your various needs. It creates an environment that has its own installation directories, that doesn’t even share libraries with other virtualenv  environments (and optionally doesn’t access the globally installed libraries either).

Installing virtualenv
To install globally with pip (if you have pip 1.3 or greater installed globally), issue the following command in your terminal:

Creating New Virtual Environment(s)
To create a virtual enviornment named myenvironment  for example, issue the following command in the terminal:

As soon as the above environment is created, several directories and files are also created inside the myenvironment  folder. You will see directory like myenvironment/bin where the python interpreter or executeable for this specific environment is placed. Directories like myenvironment/lib & myenvironment/include  contains supporting libraries for this python environment. Thus, for obvious reason – packages that will be installed in this new environment will live under myenvironment/lib/pythonX.X/site-packages/ directory. Its worth mentioning that, with each new virtual environment creation, it automatically installs the pip , wheel  & setuptools  package.

Activating a Virtual Environment
If you are inside the environment directory which means inside the myenvironment  folder as per this example, then issuing the following command will activate the virtual environment:

The activate script will also modify your shell prompt to indicate which environment is currently active. If you see the following terminal activity then you will understand what I am talking about.

In the above logs, first I created the virtual environment. Then while creating the environment, pip , wheel , & setuptools  are also being installed in the new environment. After that, I issued command to activate the environment and it prepended the environment name infront of the command prompt which indicates that the shell has been updated to use the new python environment from now on.

As this environment is now active, why not check what packages are installed and ready to use in this brand new fresh python environment? For that, issue the following command:

Which will output something like below,

Which clearly shows that we have only three packages installed in this new environment. By the way, would you like to check which pip  it is? I mean is this pip  belong to this environment or its something else? Check that by:

and you will see something like following,

that clearly shows that, its belong to the newly created virtual environment. Even you can check whats the responsible Python here in this environment by the following command:

and you will see output similar to below,

which indicates that the python in this context is also belongs to the virtual environment. You are already feeling isolation, right? 🙂 So, lets install few packages here inside the virtual environment using it’s pip.

Again, lets check package list of this new environment:

Impressed? We are installing only our needed packages in this new virtual environment without messing around the native python environment or global package scope.

Using the Environment
Just after activating the environment, its ready to use. For example, lets create a Python script like following:

Keep in mind that, you are not bound to create Python scripts/apps inside the environment. You are free to create those files anywhere in your system. You just make use of a virtual environment by activating it and run any Python scripts/apps through it’s python executable. Check the following terminal activity:

Here I have moved to Desktop, created a file called  and put the above code in it. Finally run it with python interpreter and got expected output. Keep in mind, my virtual environment is still in active mode.

Deactivating the Environment
It’s as easy as issuing the following command:

Did you notice, how the prepended environment just gone? It means you are now back not the normal shell.

Creating Environment based on Different Python Versions
So far have you ever think when we are creating a virtual environment, we are not downloading Python from anywhere? Still we are getting python executable inside the virtual environment. How does it happen? Well, it’s actually making a symbolic link of the Python  to which this  virtualenv  package belongs to. Lets check this out – enter into the virtual environment folder and look into all the files including hidden ones by issuing the command:

I hope, you can see a link to my system’s default Python which is located in /System/Library/Frameworks/Python.framework/Versions/2.7/Python . But, just like you, I don’t also want to use that old Python for my fresh new virtual environment, rather I want to use Python 3 with some fresh packages that I need for a project.

Fortunately its possible. Say, you have downloaded Python 3.X from the official website and installed it in your computer. Its also possible that, you downloaed anaconda or miniconda and that also installed another version of Python in your computer. But so far, you have come to know that, you just need the Python executable and you are free to make isolated environments. But, may be you don’t know where all the different Python versions actually reside. Let’s find those.

Yeah, I found some 2.X versions of Python. Lets try to find more:

Lets say, we want to make a new virtual environment based on Python 3.6 & from the above search, we know that our system has that installed already (or may be I once installed and forgot). So, issuing the following command will make a brand new Python virtual environment that will be based on Python 3.6:

Now, activate the new environment and check it’s Python’s version and identity,

Here, we can clearly see that, this environment’s Python is actually a Python3.6 that we wanted it to be. Sure, we can also check whether this is an isolated Python or not. Also, we can check which is the base of this Python interpreter.

and lastly, as its a new virtual environment and totally separated from the earlier one named myenvironment , is should show us nothing but some default package installed in a fresh mode ( I mean it should not contain the numpy  package that we installed on myenvironment  environment).

Managing an Environment
As we made a new virtual environment with our desired Python 3.6 and started again with a fresh ecosystem, let’s delete the old virtual environment. Oh yes, its as simple as deleting a directory:

As, I mentioned before, you are not supposed to put your actual app files inside the environment’s folder. Environment is just an environment definition, it should not contain your actual files and scripts. So, by deleting that unnecessary environment folder, you can get rid off that messed up Python environment. Is not it easy to throw away a messed up virtual environment that throwing away a full OS?

In the next two posts, I will discuss about conda  & venv  for similar purpose 🙂


Do you love Watching than Reading? Subscribe & Get Connected with my YouTube channel for Video version of all my Blog posts.