Optional Dependency Groups in pyproject.toml
Learn how to convert multiple requirements files (requirements-dev.txt, requirements-test.txt) into organized optional-dependency groups in pyproject.toml.
Advanced Features
Detailed Explanation
The [project.optional-dependencies] section in pyproject.toml replaces the common pattern of having multiple requirements-*.txt files. Each group can be installed independently using pip install -e ".[group]".
Traditional multi-file approach:
# requirements.txt (production)
flask>=3.0
sqlalchemy>=2.0
redis>=5.0
# requirements-test.txt
-r requirements.txt
pytest>=7.4
pytest-cov>=4.1
factory-boy>=3.3
# requirements-lint.txt
ruff>=0.1.6
mypy>=1.7
types-redis>=4.6
# requirements-docs.txt
sphinx>=7.2
sphinx-rtd-theme>=2.0
Consolidated pyproject.toml:
[project]
dependencies = [
"flask>=3.0",
"sqlalchemy>=2.0",
"redis>=5.0",
]
[project.optional-dependencies]
test = [
"pytest>=7.4",
"pytest-cov>=4.1",
"factory-boy>=3.3",
]
lint = [
"ruff>=0.1.6",
"mypy>=1.7",
"types-redis>=4.6",
]
docs = [
"sphinx>=7.2",
"sphinx-rtd-theme>=2.0",
]
dev = [
"my-project[test,lint,docs]",
]
Installation commands:
# Install production only
pip install .
# Install with test dependencies
pip install ".[test]"
# Install with all dev dependencies
pip install -e ".[dev]"
# Install specific groups
pip install -e ".[test,lint]"
Advantages of optional groups over multiple files:
- Single source of truth -- all dependencies in one file
- Self-referencing groups -- the
devgroup can reference other groups - No
-rincludes -- TOML doesn't need file includes - Build tool integration --
hatch,pdm, andpoetryunderstand groups natively - Discoverable -- users can see all available extras in one place
Use Case
Consolidating a project with four separate requirements files (production, test, lint, docs) into a clean pyproject.toml with named optional-dependency groups.