Before we get into the heavy duty tools, I wanted to mention two tools that are used independently of virtual environments:
If you have been programming in python for a while, you probably have many projects on your computer. Most likely, each of these projects use different versions of Python. At this point you encounter the first pain point as a developer - how to easily switch versions between projects.
pyenv to the rescue.
pyenv allows us to manage our python installations, easily installing different versions and switching between them. So when you need Python 3.7 for a work project, but want to play around with Python 3.12 on your side project then
pyenv can help you do that.
pyenv from the link below.
There is one problem if you are working on Windows -
pyenv does not support Windows (though you can still use it on Windows Subsystem for Linux if you work in that environment). But not to worry; there is a fork called
pyenv-win that provides the same functionality for windows devs.
pyenv is quite straightforward.
pyenv install xyzto install Python version
xyz. You can install multiple versions of Python this way.
- When you want to select a particular python version, you can do
pyenv global xyzto set the default version of python globally,
- Another useful feature is to use
pyenv local xyz4to set the python version to be used for the current directory. This makes it really easy to use different versions of python as required by each project
- And if you need to just temporarily use a version of python, you can use the command
pyenv shell xyzto get a shell configured with that version of python. The old configuration will still be in use outside the shell
Another situation that comes up from time to time is when you want to run a command line tool written in python.
To do this you would normally do a
pip install <tool> into the global python
site-packages folder. But that tool might in turn require dependencies that also need to be downloaded and installed, and this can end up with many packages getting installed in the global
Having many packages installed globally is problematic because you might end up with version conflicts with other dependencies for other tools. You can also mistakenly end up importing the wrong package than what you indended to.
The best solution is if you could create a separate virtual environment for each tool so that they would all be isolated from each other. This is exactly what
pipx is just like using
pip. You do
pipx install <tool> and that will create a new virtualenv just for this tool. The tool and all its dependencies will be installed into this virtualenv. Finally, the tool's binary will be added to the
Path variable so that it can be run just as if it was installed globally.
Difference between pip and pipx
It might look like
pipx do the same thing. You say
pip/pipx install <dependency> and ig will download and install the dependency. While they look similar, the intent for both tools is quite different.
pip will download the package and all its dependencies into the current environment (whether global or within an activated virtual environment)
pipx will create a new virtual environment and download the tool and its dependencies into the newly created environment. So every tool will have its own environment and will be isolated from other tools as well as the project environment.
pip is mainly used to install packages that are required for your project.
pipx is used to install command line tools that live outside of any project environment.
There is nothing more frustrating than facing hard to debug bugs that turn out to be due to dependencies from different projects interfering with each other.
pipx are two tools that are super useful for setting up your python environments in a clean way.
Did you like this article?
If you liked this article, consider subscribing to this site. Subscribing is free.
Why subscribe? Here are three reasons:
- You will get every new article as an email in your inbox, so you never miss an article
- You will be able to comment on all the posts, ask questions, etc
- Once in a while, I will be posting conference talk slides, longer form articles (such as this one), and other content as subscriber-only