Dynamic Matrices in GitHub Actions

I often use GitHub Actions to deploy Azure resources, and recently, I needed a job to run multiple times based on how many WebApps needed deploying. To handle this, I set up a dynamic matrix in GitHub Actions, which allowed the job to run the required number of times. In this post, I will show you how to do this, with an example using folders in a repository instead of WebApps.

Why Dynamic Matrices Are Beneficial

As I mentioned above, creating a Dynamic matrix lets you run how many times depending on the requirement – in this example, folder count.. even better than manually having a static matrix eh? I thought it was a great addition!

Dynamic matrices in GitHub Actions lets you:

• Automatically adjust number of jobs (in this example) based on how many directories rather than constantly updating static values
• Adapt to changes in your repo or configuration without modifying your workflow manually
• Adding dynamic possibilities lets you scale your CI/CD processes more efficiently
• Run multiple operations in parallel within the same job (in this example, against multiple directories)

The Workflow

My workflow is broken down into two jobs:

• prepare: In this example, gets a list of folder directories
• per-folder: Dynamic matrix job that runs X times depending on how my folders are in the source repository

name: Dynamic matrix

on:
  pull_request:
    branches:
      - main

jobs:
  prepare:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Get test directories
        run: |
          DIRECTORIES=$(ls -d */ | jq -R -s -c 'split("\n")[:-1]')
          echo "DIRECTORIES=$DIRECTORIES" >> $GITHUB_ENV

    outputs:
      directories: ${{ env.DIRECTORIES }}

  per-folder:
    needs: prepare
    runs-on: ubuntu-latest
    strategy:
      matrix:
        dir: ${{fromJson(needs.prepare.outputs.directories)}}
    steps:
      - name: Run echo on directories
        run: |
          echo ${{ matrix.dir }}

Workflow Breakdown

Lets breakdown the workflow into the two jobs

Prepare Job

• Checkout the GitHub repository
• Lists all directories within the repository and uses jq to covert list of folders into a JSON array, in the example I have three folders:

• Store this array in the EXAMPLE_DIRS environment variable & sets this as an output that can be picked up by another job

Per-folder Job

• Has a dependency on prepare job running
• Uses a matrix strategy & populates the matrix with the directories from the prepare job (The fromJson function parses the JSON string into an array that GitHub Actions can iterate over. For each directory, a separate job will be executed.)
• Echo each directory name

Awesome, notice below each folder has a job ran:

Conclusion

Dynamic matrices in GitHub Actions offer a powerful way to create flexible, efficient workflows that can adapt to your requirements, a great feature to use when required. By using techniques like this, you can significantly reduce the maintenance overhead of your CI/CD pipelines and ensure that your automation scales with your project.

Have you used dynamic matrices in your workflows? Feel free to share your experiences or challenges in the comments!

GitHub repository containing example from above