In this article I’m going to show you a mechanism to test for destructive changes when writing Terraform modules. You could re-use this capability in your own module development process, for transparency I’m going to do this in the context of a public Azure Verified Module.

Some of you may be familiar with my previous post on converting the AVM virtual network module resource to be 100% AzAPI, where I mentioned an experimental PR.

I also mentioned the use of moved() blocks to reduce breaking changes, but given this is a broadly used resource, one piece of PR feedback was how do we evidence this?

What you see here is a proposal, it is not part of AVM (but I’m hopeful 😸).

Example-based testing

A bit of background context. If you know how AVM end-to-end testing works, feel free to skip this section!

To test upgrades, we need examples that we can deploy and then check for breaking changes.

Luckily, the AVM framework already includes a process for testing all examples during a pull request.

Let’s walk through it:

• The AVM Template contains re-useable GitHub workflow templates; the end-to-end one uses test-examples-template.yml.
• This template relies on a container (“azterraform”), which has all the various tools required (e.g. tflint, avmfix, terraformdocs, grept). The Dockerfile for this container is in the ’tfmod-scaffold’ repository.
• The template also fetches the avmmakefile, which has a list of targets that call various test processes - e.g. lint, test-examples.
• The container runs the make target ’test-examples'.

Local testing experience

You can also interact with this makefile from a resource module too, using the avm shell script or avm.bat, i.e. the above process powers the checks that you may already be familiar with:

• avm pre-commit
• avm lint
• avm docs
• avm pr-check
• avm test-examples

Onward to upgrade testing

What does an upgrade test look like? Here’s our target state:

• We deploy the last tagged version of the module into a sandbox (’terraform apply')
• We take the proposed changes and run a terraform plan
• We pass only if there are no destructive changes (i.e. no deletes or replaces).
• We tidy up with a terraform destroy.

Ideally, there would be ’no changes detected’ - however this isn’t always practical, especially when migrating between providers (AzureRM to AzAPI in this case). So, in this scenario we do accept additions and ‘updates in place’.

How does this get called? A new make target test-non-destructive.

Usage

If you want to try it out, you can add this GitHub workflow:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

---
name: test-non-destructive
on:
  workflow_dispatch:

jobs:
  test-non-destructive:
    uses: kewalaka/terraform-azurerm-avm-template/.github/workflows/test-non-destructive-template.yml@feat/test-non-destructive
    name: test changes are non-destructive
    secrets: inherit
    permissions:
      id-token: write
      contents: read

You’ll also need to add a GitHub Environment named “test” (as per usual, for AVM), with these secrets set:

• ARM_TENANT_ID
• ARM_SUBSCRIPTION_ID
• ARM_CLIENT_ID

The above should be the details of a managed identity with Contributor and RBAC Admin, which will be used to run the tests.

Obviously, the intention is to get this added to the Azure org, in the meantime, please don’t use it from this location for production purposes!

Later on, I go into more details on how it works, if you want to make your own version.

Results

This is the result of it running against the vnet module:

As you can see - that’s a lot of examples to work through, I’m pleased I didn’t have to run them all manually!

In the output, you see content like this:

1
2
3
4
5

--- PASS: TestDestructiveUpgradeExample (0.00s)
--- PASS: TestDestructiveUpgradeExample/complete_azurerm_v4 (255.13s)
PASS
ok
github.com/Azure/avmtester 255.177s

The above is a successful test, which indicates there are no destructive changes

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

Error:  Received unexpected error:
        terraform configuration contains destructive operations (breaking changes)

<followed by the plan>

--- FAIL: TestDestructiveUpgradeExample (0.00s)
      --- FAIL: TestDestructiveUpgradeExample/complete_azurerm_v4 (255.13s)

FAIL
FAIL  github.com/Azure/avmtester  255.177s
FAIL

The above illustrates the failed test, where some resources will be replaced or deleted.

Following the breadcrumbs

If you would like to see how this is wired in, here is the entry point for the GitHub Action in my fork of the AVM vnet module:

https://github.com/kewalaka/terraform-azurerm-avm-res-network-virtualnetwork/blob/main/.github/workflows/destructivechanges.yml

This calls the following workflow in my fork of the AVM template:

https://github.com/kewalaka/terraform-azurerm-avm-template/blob/feat/test-non-destructive/.github/workflows/test-non-destructive-template.yml

This has been adjusted to use a local runner and execute directly rather than in a container (to save me having to build a forked container image).

It uses a forked repo with an updated makefile and make target:

1
2
3
4
5
6
7

export REMOTE_MAKEFILE_URL="https://raw.githubusercontent.com/kewalaka/tfmod-scaffold/refs/heads/feat/
test-non-destructive/avmmakefile"
curl -H 'Cache-Control: no-cache, no-store' -SSL "$REMOTE_MAKEFILE_URL" -o avmmakefile
export AVM_MOD_PATH=$( pwd )
export AVM_EXAMPLE=${ { matrix. example }}
export CURRENT_MAJOR_VERSION=0
make -f avmmakefile test-non-destructive

If you’d like to continue down the rabbit hole, read on!

Deeper into the engine room we go

For those that are curious how the cake is made, let us explore what the E2E make target does, as that is what we’re building on.

We add the following:

• A couple new functions to avmtester ("TestDestructiveUpgradeExample" and “TestBreakingChangeDetection”)
• & a corresponding helpers in terraform-module-test-helpers (upgradedestructivetest.go)
• & a test for the helper (upgradedestructivetest_test.go)
• & some terraform that exercises a destructive upgrade (/example/upgrade_destructive and example/after_destructive_upgrade)

Credits

The jedi master known on GitHub as lonegunmanb is responsible for the majority of this capability, kudos, mighty one!

My contribution has been to add the destructive test capability and associated testing.

In summary

Using the existing framework within AVM, I have illustrated how to use Gruntwork’s terratest framework to check for breaking changes when updating modules.

I’ve shown how you can use my temporary version to try this out, just by adding a workflow to your AVM module.

I aspire to get a version of this landed in AVM, in the meantime hopefully this provides inspiration and help if you’re considering how to automate your own module upgrade processes!