Getting Started With Pre-commit in Python and Django projects (Part 2)

This article is the second part of a series of articles about “how to add pre-commit hooks to a Python / Django project”.

• Part I: Git hooks and the pre-commit library
• Part II: Use code formatters libraries as Git pre-commit hooks
• Part III: Use code linters libraries as Git pre-commit hooks
• Part IV: Fire off local scripts as pre-commit hooks

In a previous article we learned how to add pre-commit hooks to a Python project and how to share them easily with every contributor of the project. This article will now focus on code formatters that you can add, as pre-commit hooks, to your project.

Table of content

Introduction to code formatter libraries

A code formatter library can apply changes to a file and format your code in order for it to be normalized according to the specifications defined on your repository. By using it as a pre-commit hook, it becomes easier for contributors to normalize their code, before to push it on your repository, without taking the risk to forget about code format and being frustrated by being blocked:

• By a comment about format during a code review.
• An automatic check made by a continuous integration pipeline.

In this article, I will present to you Black and isort which are the formatters that I use in almost all of my Python projects. In addition to be very effective, those two libraries are also easy to manipulate which is important when introducing new processes within an existing team.

As seen in the part 1 of this article we will use the pre-commit Python library to manage our collection of pre-commit hooks and to add new one.

Black

Black describes itself as “The Uncompromising Python Code Formatter”. The idea behind it is to unify the style of your code base. Blake is compliant with PEP 8 and style configuration options are deliberated limited. By using Black, you will focus more on “what my code do” rather than on “how my code looks like”.

If you never used Black, it can be frustrating at first. As an example, I really like to write a list by putting a newline at the end of each element:

colors = [
    'red',
    'green',
    'blue',
]

I find it easier to read that way. But, if the instruction can feet on a single line, Black will reformat it this way:

colors = ['red', 'green', 'blue']

But, with the time, I find that using Black increase the velocity of the team (we almost never have to think about code formatting) and make the code reviews quicker. The Black code style is described in this article.

isort

isort is a Python utility that sort imports alphabetically and automatically separate them into sections and by type and that is part of the Django coding style guide for contributors. Here is a quote from the guide:

Put imports in these groups: future, standard library, third-party libraries, other Django components, local Django component, try/excepts.

And here is an example that illustrate this quote:

# future
from __future__ import unicode_literals

# standard library
import json
from itertools import chain

# third-party
import bcrypt

# Django
from django.http import Http404
from django.http.response import (
    Http404,
    HttpResponse,
    HttpResponseNotAllowed,
    StreamingHttpResponse,
    cookie,
)

# local Django
from .models import LogEntry

# try/except
try:
    import yaml
except ImportError:
    yaml = None

Asking to contributors of a project to follow this rule will make the import section of each Python file easier to read and more predictable, and, by using isort as a pre-commit hook, it is really easy for developers to write code that is compliant with this guideline.

Other formatters

There is a lot of code formatters available in Python. In this section I will list some that, I think, may be interesting depending on your organization and on your project. Please note that using some of them may increase the complexity of onboarding new people into your team.

• ruff: both a linter and code formatter. It is used in many open-source projects, intend to run 10-100x faster than other libraries and can be used as a replacement of both isort and Black. The reason why I don’t use it is that, IMHO, the speed execution is not that relevant when using a formatter as a pre-commit hook because, very often, we don’t modify a massive amount of files in a single commit. However, this library is very interesting and I can see more and more people starting to using it!
• docformatter: a library that automatically formats docstrings. It is compatible with Black, supports sphinx and epytext (but plan to recognize numpy and google in the future). It can be a great addition to your pre-commits as it can be quite hard to perfectly normalize docstrings in a project that have a lot of contributors.
• pyupgrade: a tool that automatically upgrade syntax for newer versions of Python. Using it can make your life easier when needed to bump the version of Python used to run a project!

Using code formatters as pre-commit hooks

Installation and configuration

Adding a new code formatter library as a pre-commit hook can (usually) be done by following those steps:

1. Add the library to your Python requirements and install it.

2. (optional) Add a configuration file (you can, very often, stick with the default configuration).

3. Add the library to the .pre-commit-config.yaml file.

4. Run the library on the whole project (black ., isort .) in order:

   • to make your code up to date with the style of the new formatter.
   • to prevent the risk of having a very large diff the next time one of your collaborator update a file in the context of its work (which can make a pull request very hard to be reviewed).

5. Commit your changes and push it your repository.

6. (optional) If some pull requests are opened on your repository, ask the contributor to integrate the changes into their branches.

You can have a look to the commit that introduced Black and to the commit that introduced isort on the example repository.

---

Both Black and isort can be configured by using a pyproject.toml file so I will do it that way. But both libraries offer many way to handle their configuration via a file:

• Black: https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#configuration-via-a-file
• Isort: https://pycqa.github.io/isort/docs/configuration/config_files.html

Configuring Black

The way to configure black is to create a [tool.black] section in a pyproject.toml file. As an example you can define a specific line length for your project (the default value is 88). But you can, usually, stick with default values and use black without modifying its configuration. This will ensure your code to be compliant with many other Black formatted projects.

As an example, here is the content of my [tool.black] section:

[tool.black]
line-length = 88
target-version = ['py310']

I would recommend to specify, at least, the target-version. I choose to specify the line-length explicitly but its value is equal to the default value from Black.

Configuring isort

Regarding isort, there is a little more to do than for Black (mostly because both library will have an impact on the import section of Python files). In the same way as for Black, isort can be configured by creating a [tool.isort] section in your pyproject.toml file:

[tool.isort]
line_length = 88
py_version=310
profile = "black"
src_paths = ["test_project"]

• profile: by setting it to black, we ensure that isort will work according to the style of the Black library. If you want more information regarding each individual built-in profiles for isort, please have a look to this link.
• line_length: The max length of an import line (used for wrapping long imports). It is here optional as using the black profile set this value to 88 (but can be useful if you want to use a different value for both Black and isort).
• src_paths: Add an explicitly defined source path (Cf: isort Src Paths).

Ignoring commits in git blame

It is essential to recognize that integrating Black (or any other code formatter) into an existing project may cause numerous files to be modified and impact your git blame. This can lead to frustration, as changes made in the past will appear different in the commit history. Fortunately, a colleague of mine has shared an excellent article on how to avoid this issue.

Working with code formatters as pre-commit hooks

Working with code formatters as pre-commit hooks, that automatically reformat the code during a commit, may be (at first) a bit tricky. It is a new process that follow particular rules: when running git commit hooks can reformat any modified Python file and produce a content that is slightly different from the one that you staged to be committed. If a file is modified by a pre-commit hook, the developer will have to stage the changes produced by the formatters (by using git add) by himself.

To demonstrate it, I modified two files in the example repository:

• test_project/apps/core/factories.py in a way that I know to be incorrect for Black.
• test_project/apps/core/constants.py: in a way that I know to be correct for Black.

$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   test_project/apps/core/constants.py
    modified:   test_project/apps/core/factories.py

no changes added to commit (use "git add" and/or "git commit -a")

$ git diff
diff --git a/test_project/apps/core/factories.py b/test_project/apps/core/factories.py
index a607766..261679a 100644
--- a/test_project/apps/core/factories.py
+++ b/test_project/apps/core/factories.py
@@ -24,3 +24,6 @@ class EntryFactory(factory.django.DjangoModelFactory):
     content = fuzzy.FuzzyText(length=2000)

     author = factory.SubFactory(UserFactory)
+    author_2 = factory.SubFactory(
+        UserFactory
+    )
diff --git a/test_project/apps/core/views.py b/test_project/apps/core/views.py
index fd0e044..b9906df 100644
--- a/test_project/apps/core/views.py
+++ b/test_project/apps/core/views.py
@@ -1,3 +1,6 @@
 # from django.shortcuts import render

+
 # Create your views here.
+def my_view():
+    return

I first stage the two files by using the git add command in order to tell to git that I want to include those changes into my commit:

$ git add test_project/apps/core/factories.py test_project/apps/core/views.py 
$ git status
On branch chore/pierre/pre-commit-black-test
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    modified:   test_project/apps/core/factories.py
    modified:   test_project/apps/core/views.py

Then, I try to commit the changes:

$ git commit -m "Update factories"
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for added large files..............................................Passed
black (python)...........................................................Failed
- hook id: black
- files were modified by this hook

reformatted test_project/apps/core/factories.py

All done! ✨ 🍰 ✨
1 file reformatted, 1 file left unchanged.

isort (python)...........................................................Passed
flake8 (python)..........................................................Passed
bandit (python)..........................................................Passed
Check forgotten migrations (django)......................................Passed
Check outdated po files (django).........................................Passed

• git commit -m (message): use the given <msg> as the commit message.

As we can see, the “black (python)” step failed because the factories.py file needed to be reformatted by Black. As expected, the views.py already meet the expectations of Black and was left intact. We can verify it by running git status again:

$ git status
On branch chore/pierre/pre-commit-black-test
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    modified:   test_project/apps/core/factories.py
    modified:   test_project/apps/core/views.py

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   test_project/apps/core/factories.py

We can see that the changes applied by Black on factories.py are not staged for commit. This is normal because those changes has been done automatically by the script and it is of your responsibility to control what you are about to commit and to send on the repository.

In order to finalize your commit, you have run git add test_project/apps/core/factories.py again or use the -a flag of the git commit command in order to automatically stage the file:

$ git commit -am "Update factories"
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...........................................(no files to check)Skipped
Check for added large files..............................................Passed
black (python)...........................................................Passed
isort (python)...........................................................Passed
flake8 (python)..........................................................Passed
bandit (python)..........................................................Passed
Check forgotten migrations (django)......................................Passed
Check outdated po files (django).........................................Passed
[chore/pierre/pre-commit-black-test 8c2ca60] Update factories
 2 files changed, 4 insertions(+)

• git commit -a (add): tell the command to automatically stage files that has been modified or deleted.
• git commit -m (message): use the given <msg> as the commit message.

Our changes are now ready to be pushed on the repository as its content passed both Black and isort hooks!

What’s next?

In the previous article we setup the pre-commit library with our first hooks.

Today we added two new hooks, Black and isort, and seen how to work with code formatters in order to automatically reformat our code each time we commit a change.

Don’t hesitate to have a look to:

• the repository related to this series of articles
• the commit that introduced Black
• the commit that introduced isort

The upcoming articles will describe how to add the following king of pre-commit hooks:

• linters: hooks that performs checks on modified files.
• local checks: hooks that fire off custom scripts each time we do a commit.