Skip to content

Task Execution#

Recurring tasks, such as running the test suite, generating a coverage report or previewing the documentation locally should be easy to do for any developer.

Therefore, a unified way of running those tasks is preferable over remembering the command for each of those tasks. Especially when these commands differ across projects, a common way of calling them without remembering their exact syntax reduces the mental overhead of everyday development tasks significantly.

This project template relies on GNU make as a task runner. While it was designed as a build tool, it is available across many systems already, which helps with bootstrapping a project environment without any additional dependencies.

Makefile#

An overview of the included Makefile targets and what they do can be obtained using make help:

make[1]: Entering directory '/home/runner/work/python-project-template/python-project-template/docs/examples/default'
install-dev          install project including all development dependencies
maintainability      run maintainability checks
coverage             collect coverage data and open report in browser
lint                 run static code checks
docs                 build documentation
docs-live            serve documentation
make[1]: Leaving directory '/home/runner/work/python-project-template/python-project-template/docs/examples/default'

Installation#

install-dev:    ## install project including all development dependencies
    pip install -e .[test,dev]
    pip install -r docs/requirements.txt

The project is being installed in place (using pip's -e option) including all optional requirements (given in square brackets).

The project template keeps development requirements as optional requirements of the Python package in the pyproject.toml, so these can be installed alongside the project.

[project.optional-dependencies]
dev = ["black", "radon", "ruff"]
test = ["pytest", "pytest-cov", "coverage[toml]"]

The advantage of this is that development dependencies are handled exactly the same as other dependencies. A possible downside of this approach is that these optional dependencies are also included in the package built for users. If this is undesirable, an alternative approach would be to keep development requirements in a separate file (e.g. dev-requirements.txt) or use tooling that manages development requirements (e.g. pipenv).

Static Analysis#

Maintainability#

A key aspect of maintainability is reducing accidental complexity1. This means not allowing complexity to accumulate that is not inherent to the problem to be solved. During development, accidental complexity arises in many forms, some of which may be caught by the right tooling.

Radon#

Documentation

maintainability:  ## run maintainability checks
    @radon cc --total-average -nB -s src

One such tool to estimate complexity is radon, which can be used to calculate the average cyclomatic complexity (cc) for your project:

make[1]: Entering directory '/home/runner/work/python-project-template/python-project-template/docs/examples/default'
make[1]: radon: No such file or directory
make[1]: *** [Makefile:8: maintainability] Error 127
make[1]: Leaving directory '/home/runner/work/python-project-template/python-project-template/docs/examples/default'

Code Linters#

Another type of static analysis is code linting i.e. analyzing source code for potential errors, code style violations, and programming best practice adherence.

lint:   ## run static code checks
    @ruff src tests
ruff#

Documentation

Testing#

coverage:   ## collect coverage data and open report in browser
    @pytest --doctest-modules --cov --cov-config=pyproject.toml --cov-branch --cov-report term --cov-report html:build/coverage
    @test -z "$(CI)" \
        && ( echo "Opening 'build/coverage/index.html'..."; open build/coverage/index.html )\
        || echo ""
coverage-ci:
    @CI=true $(MAKE) coverage

Documentation#

Makefile - Documentation Targets 
DOCS_TARGET?=build/docs
MKDOCS_BIN?=mkdocs
docs:   ## build documentation
    ${MKDOCS_BIN} build --site-dir ${DOCS_TARGET}/html
docs-live:  ## serve documentation
    mkdocs serve
Example Makefile 
.PHONY: install-dev
install-dev:    ## install project including all development dependencies
    pip install -e .[test,dev]
    pip install -r docs/requirements.txt

.PHONY: maintainability
maintainability:  ## run maintainability checks
    @radon cc --total-average -nB -s src

.PHONY: coverage coverage-ci
coverage:   ## collect coverage data and open report in browser
    @pytest --doctest-modules --cov --cov-config=pyproject.toml --cov-branch --cov-report term --cov-report html:build/coverage
    @test -z "$(CI)" \
        && ( echo "Opening 'build/coverage/index.html'..."; open build/coverage/index.html )\
        || echo ""
coverage-ci:
    @CI=true $(MAKE) coverage

.PHONY: lint
lint:   ## run static code checks
    @ruff src tests

.PHONY: docs docs-live
DOCS_TARGET?=build/docs
MKDOCS_BIN?=mkdocs
docs:   ## build documentation
    ${MKDOCS_BIN} build --site-dir ${DOCS_TARGET}/html
docs-live:  ## serve documentation
    mkdocs serve


.PHONY: help
# a nice way to document Makefiles, found here: https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
help:
    @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

pre-commit#

Example .pre-commit-config.yaml 
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
        exclude: .*\.dat
      - id: check-yaml
      - id: check-added-large-files

While pre-commit is primarily designed to run checks against your repository including changes you are about to commit, it can also be used to run those checks manually

  1. sometimes also called incidental complexity ↩