GitHub Automation with Terraform Pt 4

This continues my personal learning journey to create a Terraform module for GitHub repositories in the style of Azure Verified Modules, that started back in part 1.

In this article we’ll look how we can re-use the continuous integration (CI) testing approach used by AVM modules.

Background

AVM has a GitHub Action for modules that tests them end-to-end, which runs as part of the pull request process.

For each example, it will:

• deploy the workload in the example.
• re-run a plan & check for idempotency (i.e. the second plan should not report any pending changes).
• destroy the workload (tidy up the resources)

Since creating resources costs time and money, Microsoft use an approval gate to run these tests after linting and code review has been completed.

This means any code on main should have examples that have actually been deployed and are known to be working, helping to maintain the quality of the module.

Since we’re deploying resources to GitHub rather than Azure, I thought a good place to start is to understand how to authenticate with the GitHub API!

GitHub API authentication

The documentation provides the options for authenticating to the GitHub API.

We assume that multi-factor authentication is enabled.

• Authenticating to the API with a GItHub App is the recommended approach when acting on behalf of an organization or another user, it offers more control over access and permissions and is suited to non-interactive flows. Don’t worry, no application coding is required!

For completeness, other less suitable options for our use case are:

• Authenticating to the API with a personal access token (PAT) is recommended for personal use. The newer, “fine-grained” PATs avoid the issue of excessive scope, but these are tied to a user’s identity.
• Authenticating with GitHub App via OAuth is a user-flow which would require the use of device-codes to authorise situations where the token needs to be refreshed, or MFA is required (“sudo mode”)
• Authenticating in a GitHub Actions workflow uses the built-in GITHUB_TOKEN which is scoped to the repository; for changes beyond that, a PAT or GitHub App is needed.

Creating the application

We’ll create a GitHub app within an organisation. If you don’t already have an org and want to follow along, you can sign up for a free one.

1. Go to https://github.com/settings/organizations
2. Locate your organisation and select Settings
3. Scroll down the settings to Developer settings -> GitHub Apps
4. Select “New GitHub App” (you may be asked to re-authenticate at this point)

The following must be filled in:

• GitHub App name (e.g. ‘my-org-github-automation’)
• Homepage URL (e.g. your company GitHub profile page)
• Turn off the Webhook unless you want an event to fired when this is called
• Choose permissions. I’ll cover more detail later, for now, go with “Repository -> Administration -> Write” & “Organisation” -> “Administration” -> “Read”

I suggest scoping apps to “Only on this account” to limit their blast radius and scope for misuse.

After pressing “Create GitHub App”, you’re prompted to create a private key:

Carefully store the private key, this key is used to sign the request for each short-lived installation token, which will have the permissions selected above.

The key is generated and downloaded in your browser when clicking the link.

Scroll back up the page and note down the App ID:

Click “Install App” on the left-hand menu and select the organisation you want to use this app with, you’ll be prompted to configuration permissions then be redirected to a page that looks like this:

Take a note of the number at the end of this URL, this is the installation ID, e.g. 12345678 in the example below:

https://github.com/organizations/kewalaka-org/settings/installations/12345678

Using the GitHub App with the Terraform provider

We’re now going to update the default example in the module to make use of this application.

We’re going to use a TFVAR approach to do this rather than the alternative (environment variables) for reasons that will become apparent.

Here’s the updates to the provider block, adding the required “app_auth” section:

1
2
3
4
5
6
7
8

provider "github" {
  owner = var.github_organization_name
  app_auth {
    id              = var.github_app_id
    installation_id = var.github_app_installation_id
    pem_file        = var.github_app_pem_file
  }
}

In the above:

• id is the AppID from the “General” page
• installation_id is the id from the installation URL. If you have misplaced this, you can go to: Third-party Access - GitHub Apps -> select Configure against the app just created.
• pem_file is the contents of the private key generated after making the GitHub App.

These are passed as variables and we can simulate this happening locally:

1
2
3

export TF_VAR_github_app_id=123456
export TF_VAR_github_app_installation_id=12345678
export TF_VAR_github_app_pem_file="$(cat ~/src/my-app.2025-03-01.private-key.pem)"

If you’re not aware of this trick - a variable can be passed to Terraform by making an environment variable with the same name, prefixed by “TF_VAR_”.

So, to labour the point, TF_VAR_github_app_id is passed to the variable “github_app_id”.

You still need to have these defined in Terraform in the usual way, i.e.:

1
2
3
4
5
6

variable "github_app_id" {
  type        = string
  description = "The AppId of the GitHub App, used for authentication. "
  nullable    = false
# etc for other variables
}

Wiring in the AVM E2E process

Initially I expected to have to create a different version of the end-to-end flow, but, as I was exploring how best to do this, I found another way.

The existing AVM E2E action calls a re-useable workflow that is imported from the template, just prior to running the examples it calls a script (“prepare-credential.sh”) that does two things:

• Adds the necessary environment variables to authenticate to Azure
• Adds any secrets or variables inside the repository that are prefixed with TF_VAR_

To mimic the approach in the last step, we simply need to add the necessary secrets to the “test” environment pre-created by the template:

Perfect!

Here it is running on the default example:

So there we have it - CI using the AVM workflow.

I’m going to finish by documenting a few learnings that I bumped into along the way - I’ve done these after the main article because otherwise it would interrupt the flow.

This hopefully illustrates that it didn’t go smooth sailing - sometimes attempts to do what seems simple things can be tripped up by entertaining quirks and misunderstandings.

Learning #1 - creating personal repos with IAT flow

The installation access token only supports the creation of repositories in an organisation. This can be checked from the Permissions required for GitHub Apps (as an aside - this is a very useful for covering what specific permissions your GitHub App requires).

If you compare this to the organisation equivalent, you see it supports the IAT token type as well:

This is true of many permissions under the ‘personal context’.

This GitHub module can still be used in these scenarios, but you will need to authenticate via a fine-grained PAT token or use the OAuth device flow, which is less ideal for automation.

If you attempt to use the IAT, you’ll receive an error that may lead you to think there is a permissions issue with the token:

1
2
3

github_repository.github_repository: Creating...
╷
│ Error: POST <https://api.github.com/user/repos>: 403 Resource not accessible by integration []

I faced a similar error when attempting to use a GitHub App registered against my personal profile with a GitHub organisation on the same account.

Learning #2 - re-usable workflows & inherited secrets

Re-useable workflows allow you to define centralised workflow(s) in yaml files that you can then call. This helps standardise CI between projects.

The GitHub Repository uses the end-to-end testing workflow from the AVM template:

1
2
3
4
5
6
7
8

run-e2e-tests :
  if: github. event.pull_request.head.repo.fork == false
  uses: Azure/terraform-azurerm-avm-template/.github/workflows/test-examples-template.yml@main
  name: end to end
  secrets: inherit
  permissions:
    id-token: write
    contents: read

You’ll see on line 25 secrets: inherit - this is designed as a convenience to allow secrets defined in the calling repository to be passed to the centralised workflow.

However, for security reasons, this does not work across organisation boundaries. i.e. in this case, I can’t have GitHub Repository module in my own account (‘kewalaka-org’) pass secrets to the AVM template hosted in the Azure org.

Since this repository is a clone of the AVM template a simple workaround is to change the uses: statement so that “run-e2e-test” uses the local workflow:

1
2
3
4
5
6
7
8

run-e2e-tests :
  if: github.event.pull_request.head.repo. fork == false
  uses: . /.github/workflows/test-examples-template.yml
  name: end to end
  secrets: inherit
  permissions:
    id-token: write
    contents: read

Learning #3 - storing the private key contents in a GitHub secret

When storing the private key in a GitHub secret as a multi-line string, I received the following error:

1
2
3
4
5
6
7

Error: Error
│
│   with provider["registry.terraform.io/integrations/github"],
│   on main.tf line 15, in provider "github":
│   15: provider "github" {
│
│ no decodeable PEM data found

The solution is to store it as a single line, using “\n” to terminate line endings. This approach is confirmed as acceptable in the provider documentation.

Learning #4 - don’t forget to approve permission changes

When setting application permissions, besides adding them to the “Permissions & Events” settings within the app, don’t forget you have to “approve” changes by going to the app installation page in your organisation.

Otherwise, you’ll do as me, and raise a bug saying the permissions don’t work as expected 😹

Learning #5 - idempotency checks

Some features are only available on the free plan if the visibility of the repository is public. This will not throw an error, but will affect subsequent plans - for example the “has_wiki” flag will continually try to set to “false” if you’re using a free organisation with private visibility.

Remember, the AVM E2E test checks the terraform plan after the apply and expects to find “no changes are required”.

An exploration

When I first started to hit issues with getting the auth to work, I created a separate simple approach to check the behaviour. It’s available here:

<kewalaka/terraform-github-githubapp>

Inside there, as a little freebie, you’ll find a script called “fetch-github-iat.sh” that I used to check the GitHub app was returning a token.

Permissions required

To run all the examples, the following is required:

• Organization Administration: Read - is needed to determine the organisation plan type (e.g. free vs enterprise, which is used to determine feature availability).
• Repository Administration: Write - is needed to create new repos and certain sub-resources such as branches
• Content: write (code) - is needed to add file contents (in addition to “Administration”)
• Workflows: write - is needed to write contents to the actions workflow folder (in addition to “Administration”)
• Variables & Secrets: write - for environmental or repository scoped secrets (in addition to “Administration”)

As mentioned earlier, GitHub’s permissions requirements are documented here:

https://docs.github.com/en/rest/authentication/permissions-required-for-github-apps

I’ve noted what I think these need to be in the appropriate example readme.

Are we at the end?

I’ve raised some issues on the module repository, if there is anything of interest, please upvote them. No guarantees of timeframes, remembering this is a personal endeavour. These are mostly minor-works, and I can’t see any good articles in them, but who knows.

My next thoughts tend towards how to fit this into the solution vending workflow, but I think that is going to be a new adventure.

Of course, I will continue the conversation with the AVM team, to see if we can get this added to the official library (and if so, that would encourage me to fix some of the functional gaps too 😺).

I’ll take onboard comments for further topics, but perhaps this is the end, and if so, I thank those of you who followed along, and hope it was helpful!

As Hannibal used to say, “I love it when a CI comes together”