GitHub Automation with Terraform Pt 1

I just started a personal project yesterday to build a GitHub Repository module in the style of Azure Verified Modules (AVM). This post is the first in a series where I will share the process of creating this in the open.

This a very early share of something that is far from complete.

Why

A code repository is a foundational component if you wish to deploy workloads using infrastructure as code.

Microsoft’s Azure Landing Zone (ALZ) solution accelerator provides a glimpse into a future state - this not only creates the resources but also the GitHub or Azure DevOps resources require to house and deploy this solution.

Public comments indicate a desire to take this approach beyond the ALZ, i.e. to bootstrap similar pre-requisites for application workloads.

We then need a more generic mechanism to create the GitHub repository and associated services, the design approach for Resource Modules recommended by AVM fits this well and provides some seed guidance on how to structure this work.

There are various ways this could be added into the solution workflow, for my case, I’m targeting this to be ran as part of solution vending.

This is the path we will follow.

Prior influences

Before starting on any task, I like to look around for inspiration & see if I should contribute or carry on.

• The aforementioned ALZ accelerator is a good starting point with very sensible defaults around CI policy and governance. It is specific for the ALZ solution.
• Reviewing the Hashicorp gallery I found recent solution by the UK’s Ministry of Justice, which I borrow from, along with a couple solutions from well-respected teams (e.g. Mineoris), which have had limited development recently.
• Azure Terraformer’s AT-AT solution seeks to combine the power of Github + GH actions and Terraform to make a deployable unit that includes both code repo and Azure resources. This is a viable alternative approach, but for my case, it does not provide the required responsibility split, and it doesn’t fit with the way I compose solutions.
• The design specification for Azure Verified Modules (obviously!).

There’s great stuff out there, and I’ve spotted several reference issues relating to Terraform & GitHub bugs to follow up on, however I didn’t find something close enough for my needs, so we carry on!

First steps

AVM has a Process Overview for contributors.

• Check the Issues · Azure-Verified-Modules & https://aka.ms/AVM/ModuleIndex/Terraform for existing suggestions.
• Given GitHub sits outside the official scope of AVM, I asked the team if they see work on this as worthwhile and received positive feedback.

Determining scope & name

AVM has guidance around the scope for a resource module, which in turn defines the required name.

Since GitHub is not an Azure resource & sits outside the existing guidance, I proposed the following:

In the above, the surrounding boxes illustrate the top-level resource in the module, and the internal boxes represent proposed child modules. I’ll go into more detail about why the child submodule pattern is used in AVM in a subsequent post.

My current requirements focus on the GitHub repository rather than the organisation, that is where we will start.

Cutting code

We create a repository from the template repository. This provides a standardised layout, and has valuable recommended practices and functionality baked into it.

As an example, there is a Dev Container definition for GitHub Codespaces with pre-baked tooling to support writing high quality Terraform, allowing us to develop this GitHub solution on the GitHub platform (how pleasingly meta 🙂).

Newres?

Side note! If you’re writing a new Terraform module, I highly recommend checking out newres. This uses the provider schema to write the main and variable files for you, along with high quality types and descriptions, you can read about it on this blog.

This supports many providers, not just AzureRM (e.g. AzAPI, AWS, GCP, Kubernetes, TLS).

Unfortunately, GitHub is not one of them, so we can’t use it on this occasion.

Implementation

The GitHub repository is a fairly straightforward Terraform resource, we start by creating the following:

• the resource configuration in main.repository.tf
• relevant variables in variables.repository.tf
• a “default example” to deploy the simplest form of the module (examples\default)

The “examples” in Azure Verified Modules are not simply documentation. The end-to-end GitHub workflow that runs during CI actually deploys these resources, ensuring each example is functional.

Pre-commit check

The repository has a makefile that can run various lint checks. Some of these take a while to run, so for now the pre-commit just runs terraform fmt and avm docs.

Linting isn’t happy, because I haven’t tidied up the vars I’m not using. I said this was an early share already 😺. We’ll come back to this.

To keep the checks snappy I run avm pre-commit periodically. This additionally runs an “autofix” check which further standardises the layout of the code

Initial local deployment

In a future post, we will switch to using GitHub Actions to run the examples.

For the time being, I have the GitHub command line (GH CLI) running on my machine. Apparently, the provider should be able to use the GH CLI to authenticate, but I haven’t been able to make this work, so I do the following:

1
2
3
4
5
6
7

# authenticate with github (follow the prompts )
gh auth
# get the token
gh auth status -t
# set the following env variables:
export GITHUB_OWNER =< username or GitHub org>
export GITHUB_TOKEN=gho _< token provided by the CLI>

Happy days

My first aim is to make something simple deploy, one that starting post is in place, then it is time to commit and start on the next feature.

Well, here’s the start:

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

The work in progress

Just reminding, this is an early share, and it’s not ready for detailed feedback yet. I know many things are messy, like unused variables, which are the product of me thinking about scope & layout, and there is still plenty functionality to add.

However, if you want to have a look around and provide some high-level guidance, that is very welcome.

https://github.com/kewalaka/terraform-github-avm-res-githubrepository

Next time

In no specific order, depending on when I get to writing things up & coding things 🙂

• Adding the first child modules (e.g. for things like branches & protection policies) - actually just landed a start! edit - available at part 2
• Switch from local deployment to Github CI.
• Removing dead code from various borrowed bits and making the linting checks happy. edit - available at part 3

If there is more or different things you would like to see, please leave me a comment!