GitHub Automation with Terraform Pt 3

This is the third post in a series about creating a GitHub repository and subcomponents in the style of Azure Verified Modules (AVM). In part one, I set the scene, in part two, I wrote about submodules.

In this part, we’re going to explore linting in the context of AVM.

Backgrounder - what is linting?

Linting is the process of running lint checks (how gloriously unhelpful!).

Wikipedia defines lints as:

…the computer science term for a static code analysis tool used to flag programming errors, bugs, stylistic errors and suspicious constructs.

Terraform has some built in linting capabilities, such as terraform validate (self-explanatory!) and terraform fmt (to format code in an opinionated way).

Linting is often done as a pre-commit (i.e. locally, before pushing code), and during the Pull Request continuous integration stage, to check the health of the code before it is posted to the main branch.

Why lint?

Lint helps drive uniformity and consistency, making it easier for a team to read and maintain projects written by many contributors.

Linting for Azure Verified Modules

The AVM team has put a lot of work into the automated linting of modules to make governance at scale more practical.

Each AVM module has a script that is used to perform linting checks, amongst other things. There is a shell version, and a batch version that do the same thing. This script is part of the AVM template which every AVM module should use as their starting point.

The easy way to run the linting is to use GitHub Codespaces (there’s a couple videos below!).

If you’d prefer to run locally, you need to have the Docker Engine installed. The AVM team recommends the use of Windows Subsystem for Linux on Windows, which means from a licensing perspective you can use the open-source Docker Engine, there’s no need to install the commercially licensed Docker Desktop.

Running lint checks

Lint checks for AVM run whenever you raise a pull request (PR), module owners will typically look for this to be passing before starting the human review process.

It is useful to run these locally when writing modules so you can be confident your code is ready prior to raising the PR. Two key checks that are useful when writing modules are:

• avm pre-commit
• avm pr-check

The “pre-commit” target is the faster of the two, I usually run it just before I commit blocks of work locally. I talk about pre-commit hooks later.

The “pr-check” target is best run just before submitting a pull request.

AVM pre-commit

This quicker-to-run target does the following things at the time of writing:

• checks the formatting of Terraform code, and Terraform code in any markdown (e.g. example docs)
• runs terraform-docs to create the automated documentation.
• runs avmfix (autofix), this performs some stylistic changes such as standardising the order of parameters within resource and data blocks.

The above checks run not only in the root module, but also any submodules, and any examples.

AVM pr-check

This longer-run target does the following things at the time of writing:

• Formatting and doc checks (to make sure the pre-commit checks have been run)
• A terraform validate across each example, module & submodule
• TFLint, including the specific TFLint ruleset for Azure Verified Modules.
• Any defined go-based unit or integration tests.

Interesting in knowing more?

Hopefully the above provides some useful feedback on how to run lint for AVM, and what action it performs when it runs.

For the curious amongst you, we’ll explore some more details

• How to create an override (with thanks to Matt White for the pointer)
• How the checks work, and how to quickly run individual checks
• How to leverage this using a local pre-commit hook.

Overriding tflint rules

On rare occasions, it is necessary to override the AVM tflint ruleset. The example for the GitHub Repository module is the check to verify that created resources output their resource ID. Components such as secrets, branches, etc, do not have a resource ID.

The AVM script downloads the reference tflint rules, if you want to change these, you create one or both of these “override” files:

• avm.tflint.override.hcl - applies to the root of the module.
• avm.tflint_module.override.hcl - applies to submodules (the one we use).

Here is the example linked above:

1
2
3

rule "required_output_rmfr7" {
  enabled = false
}

The contents of the override file are merged with the original, hence the original can be a useful reference when working out what the syntax is!

This isn’t the only way to apply ruleset overrides, you can also use the built-in “tflint:ignore” decorations, however in this case the output variable doesn’t exist, thus there is no target resource to apply this to, hence it must be done by overriding the rule (I believe!).

What drives the AVM lint checks?

The AVM script is a shell or batch script that downloads a makefile (avmmakefile).

This makefile is sourced from the “Azure/tfmod-scaffold” repository on GitHub, with targets that call contents from the avm_scripts folder.

By reviewing the avmmakefile, you can see the individual targets and run checks individually, for instance, ‘avm docs’ or ‘avm tflint’:

1
2
3

.PHONY: tflint
tflint:
  curl -H 'Cache-Control: no-cache, no-store' -sSL "$(REMOTE_SCRIPT)/run-tflint.sh" | bash

This can be very useful when focusing on specific areas, such as tflint checks, without having to wait for the rest of the pr-check process.

The azterraform container

The docker container used by the lint checks (azterraform) is also defined in tfmod-scaffold/Dockerfile.

The versions of tools in this container are sourced from the version.env file, and the readme provides guidance if you’d like to customise you own version.

The container used by AVM tests is hosted on the Microsoft Artifact Registry, and it is this same container that is used to create the Codespaces dev container definition for this project.

Optional - using a local pre-commit

The GitHub Repository module has an example using an automated pre-commit hook:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

repos :
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.86.0
    hooks :
    - id: terraform_fmt
  - repo: local
    hooks :
      - id: avm_docs
        name: AVM Pre-Commit Hook
        entry: bash -c '[ "$OSTYPE" = "msys" ] && ./avm. bat docs | ./avm docs'
        language: system
        types: [terraform]

Grept - the AVM “hive mind”

One of the more interesting lint checks ran by the AVM core team is Azure/grept. This is used to perform linting checks across all the various AVM modules stored on the Azure organisation.

A GitHub bot periodically runs against each AVM repo, performing various checks (e.g. the presence of key files and file contents, including the files responsible for the linting process - how satisfyingly recursive 😺). If it finds gaps, it raises a PR for the module owner to review.

You could think of it being like a form of dependabot, but aimed at governing lint checks at scale.

You can also run grept as follows:

1
2
3

export GITHUB_REPOSITORY=Azure/<the local repo>
export GITHUB_REPOSITORY_OWNER=Azure
./avm grept-apply

Grept will automatically apply the required changes, so make sure you’re got your code committed locally before you run this.

In closing

The AVM project has a lot of linting checks to enable them to perform governance at scale.

These are also a very useful foundation when writing your own modules, and I encourage reviewing their capability and consider adopting them.

Developing and maintaining processes to govern Terraform modules at scale is not an easy undertaking, the challenges often do not appear until you find yourself with too many modules using too many approaches, which is one of the reasons I believe AVM is a very important initiative within the Azure community ecosystem.

At the time of writing, you can see the results of the lints running on the GitHub Repository module here:

https://github.com/kewalaka/terraform-github-avm-res-githubrepository/actions/runs/13597718396

As usual, feedback and comments are most welcome 😺.