Deploying Azure AKS with Application Gateway and Flux extension – An introduction to GitOps

In this blog post/tutorial, we’ll dive into a range of topics that are all deployed using GitHub Actions and Terraform. The focus is on deploying key components:

1. Azure AKS Cluster: The foundation for the next components, will deploy to AKS using GitOps
2. Azure Application Gateway: Seamlessly connected to the AKS cluster as an Ingress Controller as part of the Terraform deployment
3. Flux GitOps Extension on AKS: Unleashing the potential of GitOps for streamlined cluster management.
4. Example Application Deployment with Flux: Taking a hands-on approach to deploying applications onto the AKS cluster.
5. Sample Flux Configuration Deployment: Understanding the practical application of Flux in a real-world scenario.

This is also part of the Festive Tech Calendar – do check it out for even more awesome content!

Prerequisites

Before diving into Terraform deployments and application setups, let’s go over the prerequisites:

1. Azure Provider Registration:

• In your chosen Azure subscription, make sure to register the following providers:
  • Microsoft.ContainerService
  • Microsoft.Kubernetes
  • Microsoft.KubernetesConfiguration
• Execute the following commands using the az cli:

az provider register --namespace 'Microsoft.Kubernetes' 
az provider register --namespace 'Microsoft.ContainerService' 
az provider register --namespace 'Microsoft.KubernetesConfiguration' 

2. Ensure your GitHub environment repository secrets are set up as follows:

• CLIENT_ID: Client ID of the Azure AD Service Principal responsible for Terraform deployment.
• CLIENT_SECRET: Client Secret linked to the Azure AD Service Principal for Terraform deployment.
• DEPLOYMENT_SUBSCRIPTION_ID: Azure subscription ID where the Terraform storage account is located for storing the .tfstate file.
• SUBSCRIPTION_ID: Azure subscription ID where the Terraform code is intended to be deployed.
• TENANT_ID: Azure AD tenant ID associated with the deployment.

Leveraging GitHub repository secrets guarantees the confidentiality and security of these critical credentials throughout the deployment process.

3. Recommended Reading:

As we navigate through this blog/tutorial, certain areas won’t be explored in-depth. For a comprehensive understanding, consider reviewing the following resources:

• Configurations I recommend you setup to deploy your Terraform into Azure at scale using GitHub Actions
• GitHub Actions quickstart

Time to deploy

Terraform Deployment

The terraform deployment has been broken into various stages due to various dependencies of Azure resources – while an all-encompassing Terraform block is an option, the multi-stage approach offers flexibility. Below is the breakdown of the deployment, with the reminder that Terraform modules play a pivotal role in this process. (Please note: Terraform modules will be used during the deployment)

Core

Will deploy:

• The resource group, serving as the container for all resources (excluding AKS nodes, which have a dedicated resource group).
• Log Analytics resource, establishing a connection with AKS.
• Container Registry, designated for storing the Docker image essential for the upcoming flux deployment.

Network

Will deploy:

• Virtual network and subnets utilised by AKS and the application gateway.

AKS

Will deploy:

• AKS Cluster
• Application Gateway and associate with the AKS cluster
• Flux extension and configuration to allow two example applications be deployed

Lets deploy the Terraform

As mentioned, I will deploy in multiple stages.

• Deploy core stage

As mentioned earlier, this will be carried out in multiple stages. The first one being the “Core Stage.” For each subsequent stage, I’ll update this entry, triggering the GitHub Action workflow automatically.

Once core stage has been deployed successfully, we will need to add additional GitHub repository secrets for the Azure Container Registry admin user login – this will be used later during the docker stage to successfully upload the image to the container registry.

Add 3 secrets as below from the access keys tab above:

• REGISTRY_LOGIN_SERVER – Login server
• REGISTRY_USERNAME – Username
• REGISTRY_PASSWORD – Password

• Deploy network stage

Core & network stages deployed successfully, now ready for the AKS deployment

• Deploy AKS stage

As part of this, it will also install & configure a flux deployment.

Prior to this though, two things to do:

• Run this stage to allow a successful docker build & deployment of an example application. AKS/Flux will make use of this image during the AKS stage.

You will see a new repository / image tag within the created Azure Container Registry

• A flux configuration is required, I have used a seperate repository for this

The flux configuration is stored in this repository

• Update the flux configuration with the new image tag, here is the image location that needs updated

Now deploy the AKS stage

Reviewing Flux setup and configuration

Once the microsoft.flux cluster extension is successfully installed, you gain the ability to establish one or multiple fluxConfigurations resources. These resources facilitate the synchronization of your Git repository sources with the cluster and ensure the cluster is continually reconciled to its desired state. By adopting GitOps principles, your Git repository serves as the authoritative source for both cluster configuration and application deployment, providing a streamlined and efficient approach to managing your cluster’s state.

For this all to happen, a Kubernetes Kustomization is requried – Think of a Kustomization as a tool within Flux that acts as a guide for the cluster. It represents a collection of manifests in a Git repository, which can be either regular Kubernetes YAML files or Kustomize overlays. Essentially, it tells Flux what to look for in the repository and how to bring those configurations into sync with the cluster.

It’s important to note that you need at least one Kustomization defined to specify the repository path to reconcile and the details of the reconciliation process. However, the real power comes into play when you have multiple Kustomizations. In this scenario, you can configure dependencies between them, allowing for a more sophisticated and interconnected setup.

With a successful setup, we can review the GitOps configuration via the Azure Portal (can be somewhat useful if you are not too familiar with the Kubernetes commands). You can also create a Managed Grafana instance to take monitoring to the next level of your GitOps & flux configurations

The configuration objects are useful, we can see 3 are deployed

• Flux-system is the name of the GitRepository Kind
• flux-system-kustomization-2 is the kustomization that is created from the extension in Terraform as reference here
• voting-app is another kustomization, that is referenced from above, what is useful here is that you can have one kustomization above deploy multiple other kustomizations!

Accessing the public IP of the Application Gateway, we can see the application is accessible and working as expected – awesome!

To summarise how this current implementation of GitOps works – lets look at the below diagram from https://learn.microsoft.com/

1. The developer commits configuration changes to the GitHub repository.
2. Flux identifies configuration drift in the Git repository and fetches the configuration updates.
3. Flux aligns the state within the Kubernetes cluster.

Lets Conclude

In conclusion, once the microsoft.flux cluster extension is successfully installed, you unlock the capability to establish fluxConfigurations resources, enabling seamless synchronization between your Git repository sources and the Kubernetes cluster. This integration ensures continuous reconciliation, maintaining the cluster in its desired state. Embracing GitOps principles streamlines cluster and application management, designating the Git repository as the authoritative source.

A pivotal component in this process is the Kubernetes Kustomization, acting as a guiding tool for Flux. It defines the repository path for reconciliation and orchestrates the synchronization of configurations with the cluster. While a single Kustomization is essential, the true power lies in leveraging multiple ones, allowing for intricate setups with configurable dependencies.

After successful setup, monitoring GitOps configurations can be conveniently done through the Azure Portal, offering a user-friendly alternative to Kubernetes commands. Additionally, creating a Managed Grafana instance enhances monitoring capabilities.

Inspecting the deployed configuration objects, such as Flux-system and voting-app Kustomizations, reveals the versatility of the system. The diagram from Microsoft’s documentation illustrates the seamless workflow, from developer commits to Flux detecting drift and aligning the cluster state. This implementation proves effective, ensuring accessibility and optimal performance of applications through GitOps practices – Happy GitOps’n 🙂

GitHub repository for Terraform

GitHub repository for sample flux configuration