From requirements.txt Lock Files to pyproject.toml
Understand the difference between abstract dependencies in pyproject.toml and concrete lock files from pip freeze. Learn the modern Python lock file ecosystem.
Detailed Explanation
A key concept in Python packaging is the distinction between abstract dependencies (what your project needs) and concrete dependencies (the exact resolved versions). pyproject.toml is meant for abstract dependencies; lock files are for concrete ones.
pip freeze output (lock file):
certifi==2023.11.17
charset-normalizer==3.3.2
idna==3.6
requests==2.31.0
urllib3==2.1.0
Appropriate pyproject.toml (abstract):
[project]
dependencies = [
"requests>=2.31",
]
The lock file ecosystem:
| Tool | Lock File | From pyproject.toml? |
|---|---|---|
pip-tools |
requirements.txt |
Yes (pip-compile pyproject.toml) |
pdm |
pdm.lock |
Yes (native) |
poetry |
poetry.lock |
Yes (native) |
hatch |
Uses pip-tools | Yes (via plugin) |
pip freeze |
stdout | No (from installed) |
Best practice workflow:
- Define abstract dependencies in
pyproject.tomlwith flexible ranges - Use a lock tool to resolve and pin all transitive dependencies
- Commit the lock file for reproducible installs
- Update the lock file periodically to get security patches
# pip-tools workflow
pip-compile pyproject.toml -o requirements.lock
pip install -r requirements.lock
# pdm workflow
pdm lock
pdm install
# poetry workflow
poetry lock
poetry install
Important: Do NOT put pip freeze output directly into pyproject.toml. The frozen output includes transitive dependencies (dependencies of your dependencies) that should not be declared explicitly. Only list your direct dependencies in pyproject.toml and let the lock tool handle the rest.
Use Case
Understanding the relationship between requirements.txt, pyproject.toml, and lock files when migrating a project to modern Python tooling with pip-tools or pdm.