Virtual Environment and package installation
Notes based on the live Ambientes virtuais e instalação de bibliotecas - Live de Python #191 from the youtube channel Eduardo Mendes - @Dunossauro.
- Python has a large and diverse package ecosystem.
- We just install a package using
pip
, but sometimes we want to use different versions from the same package in different projects, for example. - And, how to install and manage different package versions? It's messy:
PIP
- PIP stands for Pip Install Package and just installs external packages from PyPI.
- It gets the package, unzip it and store it in a global folder.
- Show where the python installed the package:
- To import the package, python looks inside some specific paths.
- Show the paths python looks for the packages:
- When stored globally, two versions from the same package can't live together
- Show package details:
- Show dependency tree:
- Show package list:
- Different python packages may have the same dependency, but they depend on different versions of the shared package.
VENV
- Virtual environment tries to isolate the environment.
- It hacks the
site-packages
creating a local environment isolated from the global one. - It is plug-and-play from python.
- In the same project, you can have multiple virtual environments.
- Important commands:
VENV OR VIRTUALENV?
- virtualenv is external to python, and distributed by PyPA (Python Packaging Authority).
- venv is part of the virtualenv inside python.
- virtualenv advantages:
- It's faster;
- New releases are independent from python releases;
- It's extensible;
- Code API;
- It supports older python versions.
requirements
requirements.txt
allows you to list all packages required in your application.- Install requirements:
freeze
buildsrequirements.txt
file automatically. However,freeze
gets all installed packages, not only the ones you need for your project. In that way, some unnecessary packages may be written into the requirements.
- Remove the package and its dependencies:
$ pip uninstall <package-name> # don't uninstall the package and its dependencies
$ pip install pip-autoremove
$ pip-autoremove <package-name> # uninstall the package and its dependencies
- Create a
requirements_dev.txt
requirements_dev.txt
# install production libs
-r requirements.txt
# list bellow all dev packages
package_1
package_2
package_3
- Sometimes we need some packages to be in a specific version, mainly if that package is a sub-dependency. That's what
constaints.txt
is for. It works in any dependency tree level and constrains the package version. - It does not overwrite the
requiments.txt
file, it just indicates whether a specific package is going to be installed, and then install it in a specific version.
Other tools
- pip's friends:
- pip-autoremove: removes the package and its dependencies;
- pipdeptree: dependency tree.
- venv's friends:
- virtualenvwrapper: wraps venv commands.
- python's friends:
- pyenv: installs different python versions;
- tox: runs tests in different python versions;
- pipx: installs CLI tools apart from the global environment.
- Scientific world:
- Conda: package manager just like pip; it has its repository different from PyPI; it downloads packages from other languages, like R, fortran, C, etc.
- miniconda: python + conda;
- anaconda: gets all the scientific world;
- mamba: like miniconda but in C++ aiming performance.
Future
pyproject.toml
: replacesrequirements.txt
+requirements_dev.txt
+ `setup.py``- poetry:
- Handles the virtual environment (venv);
- Handles setup.py + setup.cfg;
- Package metadata (setup.py);
- Package installation (pip);
- Handles package versions (
requirements.txt
);
- flit:
- Just like poetry.
- pdm:
- PEP 582 (Python Enhancement Proposal);
- Handles packages without the need for the virtual environment.