Getting Started With Pre-commit in Python and Django projects (Part 1)

Mastering Git hooks: Pre-commit and beyond for your Python/Django project

Posted by Pierre on May 31, 2024, 8:58 a.m.
en
fr

This article is the first part of a series of articles about “how to add pre-commit hooks to a Python / Django project”.

Table of content

About Git hooks

Git comes with a way to fire off custom scripts when certain important actions occur. This feature is called Git Hooks. As an example, the following hooks are called when invoking git commit:

  • pre-commit: the first hook that run during git commit (before even type a commit message). It can be used to perform checks on edited files, to reformat those files, etc …
  • prepare-commit-msg: let you edit the default commit message before the commit author sees it.
  • commit-msg: allows you to validate the commit message (useful, as an example, if you defined a particular policy regarding commit messages on your project).

The complete list of available Git Hooks is actually quite long! Some hooks are executed on the client side (eg: when the developer execute an action) and others are server side (eg: when an event occurred on the git remote server).

One of the most popular (and useful) hook, in the context of a team of developers, is surely the pre-commit hook. In this article, we will see how to setup pre-commit hooks for a Python (and Django) project that:

  • Are easy to share with all the contributors of your project.
  • Will help developers to get rid of common mistake before pushing new code to your repository.
  • Will help to normalize your code by automatically format it.

Sources for this article can be found on Github. Don’t hesitate to pull the project and to modify the code in order to test the hooks.

The pre-commit hook

As said above, the pre-commit hook is the first hook than run during git commit. It allows you to perform processing and verifications on each file that is modified by a commit. Common scenarios are:

  • Automatically format the edited files (as an example by using black).
  • Automatically runs code linters on edited files (as an example by using flake8) in order to detect common errors. If an error is found during the execution of a git hook, then the commit is aborted and the developer has to fix it before being able to commit its changes.

As we will see later, in this series of article, it is also possible to run global checks (that are not related to edited code).

Of course the usage of linters and formatters can also be done:

  • manually (by invoking commands in a console).
  • through an IDE (such as PyCharm or VisualStudio).

But pre-commit hooks are easier to share, it ensure that the same checks are run for each of your collaborator regardless of their environment or of the code editor they use.

Note that pre-commit hooks do not replace checks done in a continuous integration pipeline. They can be considered optional since collaborators cannot be forced to use them (and the --no-verify flag of the git commit command allows bypassing hooks). However, using pre-commit hooks will save you time during code reviews as collaborators can focus on what the code does rather than correcting typo, formatting, and common errors.

Pre-commit hooks should really be considered as an helpful tool for the developers.

Git hooks are ordinary scripts and resides in the .git/hooks directory of your local repository. When initializing a git repository, git creates the .git/hooks directory and put sample files in it:

foo@bar:~$ 
total 64
-rwxr-xr-x 1 pierre pierre  478 Mar  4 11:33 applypatch-msg.sample
-rwxr-xr-x 1 pierre pierre  896 Mar  4 11:33 commit-msg.sample
-rwxr-xr-x 1 pierre pierre 4726 Mar  4 11:33 fsmonitor-watchman.sample
-rwxr-xr-x 1 pierre pierre  189 Mar  4 11:33 post-update.sample
-rwxr-xr-x 1 pierre pierre  424 Mar  4 11:33 pre-applypatch.sample
-rwxr-xr-x 1 pierre pierre  641 Mar  4 11:40 pre-commit
-rwxr-xr-x 1 pierre pierre 1643 Mar  4 11:33 pre-commit.sample
-rwxr-xr-x 1 pierre pierre  416 Mar  4 11:33 pre-merge-commit.sample
-rwxr-xr-x 1 pierre pierre 1492 Mar  4 11:33 prepare-commit-msg.sample
-rwxr-xr-x 1 pierre pierre 1374 Mar  4 11:33 pre-push.sample
-rwxr-xr-x 1 pierre pierre 4898 Mar  4 11:33 pre-rebase.sample
-rwxr-xr-x 1 pierre pierre  544 Mar  4 11:33 pre-receive.sample
-rwxr-xr-x 1 pierre pierre 2783 Mar  4 11:33 push-to-checkout.sample
-rwxr-xr-x 1 pierre pierre 3650 Mar  4 11:33 update.sample

Any file suffixed with .sample is automatically generated by git, as an example that you can re-use, but not executed until your create a file named without this suffix (or until you renamed it).

Here is an example of a valid .git/hooks/pre-commit file:

#!/usr/bin/env bash

# If any command fails, exit immediately with that command's exit status
set -eo pipefail

flake8

This script will automatically run flake8 any time you run git commit. For now this script is pretty simple but, if you want to add more checks, its complexity will quickly increase. Some checks can also be quite hard to write.

The pre-commit library

Thankfully, we can use the pre-commit library in order to write pre-commit hooks more easily. This library bills itself as “A framework for managing and maintaining multi-languages pre-commit hooks”. It runs on Python but you can use it on any project regardless of the project’s main language.

The library can be installed by using pip:

$ pip install pre-commit

Hooks should be described in a .pre-commit-config.yaml file. You can invoke pre-commit to generate a sample configuration file:

$ pre-commit sample-config
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.2.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files

As this sample is a good basis to start, you can use it to create the .pre-commit-config.yaml file:

$ pre-commit sample-config >> .pre-commit-config.yaml

Then run pre-commit install to set up the git hook scripts:

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

Finally, it is possible to run the hooks against all of the files by using pre-commit run --all-files. This is useful when adding new hooks to test their behavior on the whole project. It is important to note that most of the hooks are only run on the changed files when creating a new commit.

$ pre-commit run --all-files

The pre-commit config file syntax

As said above, the pre-commit library is designed as a framework. It brings:

  • one way to describes pre-commit hooks in a configuration file (written in yaml).
  • a generic way to import and to run scripts to be executed as pre-commit hooks (named plugins and described with the repo keyword).

We won´t go in depth into the syntax of the .pre-commit-config.yaml file in this section as we will have the opportunity to see some keywords while integrating pre-commit hooks later in this article. Don’t hesitate to read the “Adding pre-commit plugins to your project” section of the documentation but, for now, you just have to know that:

  • A .pre-commit-config.yaml file describe a list of repos.
  • Each repo is a mapping to a Git repository.
    • It must declare a rev which is the revision or the tag to clone at.
    • Its value can be local, allowing you to run local scripts (in this situation, rev is not necessary).
    • Each repo must declare a list of hooks describing the list of pre-commit hooks to execute (a same repository may contains more than one available hook).
      • Each hook to execute is described with an id.

A sample repository:

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v1.2.3
    hooks:
    -   id: trailing-whitespace

Hooks from the pre-commit library

Out of the box, the library comes with its own repository and a bunch of available hooks. I will describe some that I like to use but feel free to add more! You can have a look to the list of supported hooks.

  • check-yaml: Ensure that .yaml files are correctly formatted. As an example, it should prevent you to introduce a syntax error in your .pre-commit-config.yaml file. Similar hooks also exists for other format so feel free to add it if intend to add this king of files to your project:
  • json
  • toml
  • xml
  • check-added-large-files: prevent giant files to being committed.
  • check-merge-conflict: checks for files that contain merge conflict strings.
  • debug-statements: checks for debugger imports and py37+ breakpoint() calls in python source.
  • detect-private-key: detects the presence of private keys.
  • If you are using AWS, you might be interested by the detect-aws-credentials hook.
  • end-of-file-fixer: ensures that a file is either empty, or ends with one newline.
  • trailing-whitespace: trims trailing whitespace.

As an example, you can have a look to the commit on the example repository that add the .pre-commit-config.yaml file.

What next?

In this article we:

  • learned about Git hooks and about the pre-commit hook.
  • added the pre-commit library to our project in order to easily share pre-commit hooks with our teammates.
  • added our first hooks by using the pre-commit repository.

Don’t hesitate to have a look to the repository related to the article. The upcoming articles will describe how to add the following kind of hooks:

  • formatters: hooks that automatically format files modified by a commit.
  • linters: hooks that performs checks on modified files.
  • local checks: hooks that fire off custom scripts each time we do a commit.