How to Use Git Hooks for Shift Left on Continuous Integration

Githooks serve a cross-cutting role, where any operation can be performed in Git, which is one of the most used version control systems. With these hooks, developers can perform the stages of Continuous Integration Pipelines on their machines. In this way, they can catch errors that may be encountered during any step as early as possible. Hence, they can apply the Shift Left Principle which is to take a task that's traditionally done at a later stage of the process and perform that task at earlier stages. For example, I manage the CI process with GitLab and ensure that the tests I have written are running. Running tests can be done before committing or pushing the changes to the remote repository automatically. As a result, Git Hooks are an ideal solution to implement the Shift Left principle in the CI process.

Git offers client-side hooks such as pre-commit, prepare-commit-msg, pre-push, post-checkout, pre-rebase, pre-applypatch, post-applypatch, post-rewrite, post-commit, post-merge . With these hooks, you can build your code, run your tests, and analyze code with SonarQube before pushing any changes to your git repo or committing to your local environment. In fact, decisions can be made about the Commit message format like a policy, and the commit message can be automatically edited with the prepare-commit-msg hook. In this way, developers will be able to take faster action regarding the content of the commits and the issue in which the change was made.

Hooks can work on the developer's machine, or they can work on the machine where your remote repo is located. These are called Server-side Hooks. There are hooks such as pre-receive, update, post-receive. Server hooks are used to enforce that commits and changes conform to the project’s policies. Hence, any unwanted code will not make it to your remote repo. To give an example, if the branch that the developer has just opened in her local does not comply with the project’s branch name policy, you can prevent this branch from being pushed to the remote repo with the update hook.

Installing Hooks

Hooks are located in the hooks sub-folder under the .git folder in the main directory of the repository. These files are created automatically when the git repo is initiated. If you make these files executable and delete the .sample extension, they will start working. While the hook samples are shell scripts, they can be written with any language.

Let’s see Git Hooks in action!

​​In this example, I have developed a client-side hook on the kloia_exporter pip package repository, which enables us to easily target the Prometheus Monitoring toolkit and create a client that yields our metrics, with the Prometheus Monitoring toolkit I developed under kloia. I will ensure that unit tests run through hooks.

   This is the final structure of my directory.

Requirements

pre-commit(optional). You can use the pip package if you want to write your pre-commit hooks or to create your own executable files without using the package. In the following steps, both methods are described.

With flake8, I will deal with formatting issues of Python codes.

     pip3 install flake8 pre-commit

I have two hook samples. The first one is prepare-commit-msg. I will ensure that the name of the branch is automatically included in the commit messages and that the commit message is automatically formatted, such as what the change is (feature/fix/refactor). In my pre-commit hook, I will expect whether the Python code I have developed complies with the flake8 rules and the unit tests written will run successfully.

prepare-commit-msg

Suppose my branch names contain the type of enhancement to be made and the Issue number. (like fix/SRE-150). My goal is to make the commit message "[SRE-150] : fix - change collector name" together with the message entered by the developer when I commit the changes. Git will automatically give the commit-msg as a parameter to the hook I wrote. In the script below, I split the branch name according to the “/” delimiter and place them in the commit message and update our file. Thus, my commit message will be formatted according to the structure I have determined.

   #!/bin/bash


COMMIT_MSG_FILE=$1
BRANCH_NAME=$(git symbolic-ref --short HEAD)
hint=$(cat "$COMMIT_MSG_FILE")

delimiter="/"
s=$BRANCH_NAME$delimiter 
array=(); 

while [[ $s ]]; 
do 
array+=( "${s%%"$delimiter"*}" ); 
s=${s#*"$delimiter"};
done;

if [ -z ${array[0]} ] || [ -z ${array[1]} ]
then
   echo "${hint}" > "$COMMIT_MSG_FILE"
else
   echo "[${array[1]}]: ${array[0]} - ${hint}" > "$COMMIT_MSG_FILE"
fi

pre-commit

I will deal with the unit tests and formatting of the Python code I have written in the pre-commit part. There are two different methods here. The first one is to create your own executable files and give them as a hook. The second one is to create a YAML config file with pre-commit, which is a Pip package, and to run my hook. 

Let's go step by step.

Way 1 - Writing your executable files 

I run our tests written with the unit test library, which comes as a built-in package on Python3, by searching the Python files that start with "test_" recursively in the directory. Then I run flake8. If there is an error in the output of these two functions, I write "exit 1" and prevent the commit process from continuing. If there is no problem, if the tests are running properly and there is no problem with formatting, my hook will return "exit 0" and will continue the commit process.

   #!/bin/bash

function test_code {
    python3 -m unittest discover -s . -v -f -p "test_*.py"
}

function check_format_of_code {  
    flake8
}

test_code && check_format_of_code
result=$?
if [[ $result -ne 0 ]];
then
    exit 1
fi

First of all, I make the scripts (hooks) I have written executable. Then I copy them under the “.git/hooks” folder, which is a hidden file in my repository. I checkout a branch from the Master according to the branch name I have determined. I name it “fix/SRE-150”. Then I check my Git stage/cache.

I create a config file named .flake8 in the main directory and let Flake8 ignore my comments and exclude the init file of the kloia_exporter module.

   [flake8]

ignore = E501, E123
exclude = __init__.py

I am making some changes to our files. Next, I examine the Stage and my Logs.

It's time for the commit part. I enter my commit message as “test”. Then I see that there are 24 unit tests written and they all executed properly with the pre-commit hook. There were no formatting issues. (flake8 did not give any output). Then, with the prepare-commit-msg hook, I can see that my commit message is formatted as “[SRE-150]: fix - test”.

When I view my logs, I see that I could format my commit properly.

Way 2 - Using Pre-commit Pip Package

Another method of using a pre-commit hook is to take the configurations from the YAML file with pre-commit, which is already a pip package, and to run my hook according to entry points.

First, I create a YAML file named .pre-commit-config in the main directory of my repo and give my repo list to it. In the YAML file, I can use the already existing repos on Github, as well as run scripts or processes that I wrote in my local environment. In my example, I will use flake8’s own Github repository. Also, I will define an entry for unit tests to be able to run my shell command.

repos:
- repo: https://github.com/pycqa/flake8
  rev: ‘4.0.1’
  hooks:
  - id: flake8
- repo:
  hooks:
  - id: unit-test
    name: unit-test-hook
    entry: python3 -m unittest discover -s . -v -f -p “test_*.py”
    always_run: true
    pass_filenames: false

Then, I give my configuration with the install command of the pip package I have downloaded. This command will run my repos in the yaml file one by one.

pre-commit install
 

First of all, I examine my stage for changes. I have some changed files. When I commit them, I see that my flake8 and unit tests are running and outputting, just like pipeline stages, step by step.

Finally, when I examine my git logs, I see that my changes have been successfully committed.

Conclusion

You can avoid sending unformatted and untested code by using Git Hooks. Just create your pre-commit, pre-push, prepare-commit-msg files and put them under the .git/hooks folder of your repository. Thus, On the more left pipeline of CI, you will be able to face the problems. You will apply the Shift Left principle.

You can give Git Hooks a try.