AVM Contributor Agent Pt 1 — What and Why

I’ve started building an agent, in the open, to help with contributions to Azure Verified Modules (AVM). The aim is for it to do the grunt work: write code, run tests, document any breaking changes, and raise a draft pull request with CI results attached.

This is part one of a series. I’ll start with the what and the why, and dig into the individual bits in later posts.

This is very much a work in progress. The CLI version works end-to-end, but there’s still plenty to figure out.

Why

Azure Verified Modules is Microsoft’s programme for publishing opinionated, high quality Terraform (and Bicep) modules for Azure. There are a lot of modules now, each with its own repo, its own pile of open issues, and each at a different point on the AVM journey.

As an external contributor, I can’t run tests in Microsoft’s internal CI. The guidance says to run the tests locally, but this makes it hard to evidence the changes to maintainers.

Additionally, the roadmap to v1.0 includes many alignment fixes, some of which are breaking changes (e.g. aligning variable names with the REST API spec). These need evidencing with an UPGRADE.md.

The idea

Use an agent to do most of the coding, cross refernce against the AVM guidance and then use real CI as a guardrail and integration test case.

I use a mixture of agents model for best results, with one writing, another reviewing and the deterministic CI as the measure of quality.

I tried a few options along the way:

• Microsoft Conductor,
• Brady Squads and
• GitHub agentic workflows

In the end, I decided to go with my own approach, partly because I wanted to play around with Foundry too!

How it works

The agents are built in Python using the Microsoft Agent Framework, they can run locally or from Azure AI Foundry.

There are two agents and a CI loop:

• Developer agent: reads the GitHub issue, creates fixes using the AVM skill included in the repository
• Reviewer agent: focuses on AVM-specific review criteria and specifications, this avoids it having to deal with implementation context, and runs using a different model. The reviewer can send work back to the developer three times until it will escalate to a humna.
• CI dispatch: once the Reviewer is happy, a repository_dispatch event kicks off a test run in a companion repo (kewalaka/avm-contributions). This runs the standard deterministic AVM checks, in addition to module upgrade testing, posting results to a summary.json artifacts.

If everything looks good, it opens a draft PR on the upstream repostiory and links the evidence.

Three ways to start

At this point, the process can be invoked from the CLI in one of three ways

• from an existing issue (--issue N)
• from an existing PR (--pr N)
• directly to the repository (--existing-repo PATH)

This allows the process to be able to pick up from whereever work has started.

What works today

Currently, I have the pipeline running end to end, it’s been useful for testing a number of my contributions (in particular, the deterministic CI).

I might move this to Foundry in the future, for now I’m focused on whether it works functionally.

Guardrails

These are the controls I have in place at the moment:

• The branch name must match a specific regex.
• The owner must match to the fork
• A separate PAT test runs, scoped only to the avm-contributions respository, with repository_dispatch permission.

I have also decorated agent commits with Agent-Run-Id: for traceability.

Where is this going?

Not sure, I’m quite happy about the CI deterministic elements, so look out for a post on that. If anything else is of interest, reach out to me.

Where’s the code?

Find it here: kewalaka/avm-contributor-agent.

It’s rough around the edges, but have a poke around and feel free to get in touch if you have thoughts or ideas 🙂.