Chaminda's DevOps Journey with MSFT: Deploying AKS with Kubenet and Application Gateway Ingress Controller (AGIC) with Terraform

Saturday, 10 September 2022

Deploying AKS with Kubenet and Application Gateway Ingress Controller (AGIC) with Terraform - Simplest Way

We have discussed how to setup AGIC for AKS in the simplest way as an addon to AKS using Azure portal ina previous post. In this post lets look at how we can get a new AKS cluster setup with kubent networking and add Application Gateway Ingress Controller as addon to it.

Lets get started with setting the required providers in terraform. We need to use the azurerm provider. When you use azurerm always use the latest version.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.23.0"
    }
  }
}

provider "azurerm" {
  features {}
}

Then we can create a resource group for the aks cluster as shown below.

# resource group for aks
resource "azurerm_resource_group" "aks_rg" {
  name     = "rg-kubenetdemo-dev01"
  location = "westeurope"
}

The next step is creating the AKS cluster with the kubenet as networking profile. Note the comments in ingress_application_gateway block.

# aks cluster
resource "azurerm_kubernetes_cluster" "aks" {

  # any autoscaling should not be reset by TF after intial setup
  lifecycle {
    ignore_changes = [default_node_pool[0].node_count]
  }

  name                = "aks-kubenetdemo-dev01"
  kubernetes_version  = "1.23.8"
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  dns_prefix          = "aks-kubenetdemo-dev01-dns"
  

  network_profile {
    load_balancer_sku = "standard"
    network_plugin    = "kubenet"
  }

  default_node_pool {
    name                = "demo01linux"
    enable_auto_scaling = true
    node_count          = 1
    min_count           = 1
    max_count           = 5
    max_pods            = 110
    vm_size             = "Standard_B4ms"
  }

  identity {
    type = "SystemAssigned"
  }

  # this will deploy an app gatway in the same vnet
  # of the AKS cluster and apply user assigned identity
  # of the AKS cluster for AGIC to the app gateway
  # so that the app gateway configurations can be managed by the AGIC
  # based on AKS cluster ingress requirements
  ingress_application_gateway {
    gateway_name = "agw-aksingress-demo01"
    subnet_cidr = "10.225.0.0/24"
  }
}

This is similar to what we have done in the portal as explained in the post here. The app gatway is added as addon to AKS clsuter as shown below.

Let's verify if our AKS and AGIC deployment with terraform works as expected by deploying a sample app. We can deploy a test app to the AKS cluster with command below

kubectl apply -f https://raw.githubusercontent.com/Azure/application-gateway-kubernetes-ingress/master/docs/examples/aspnetapp.yaml

We can see the app gatway is updating based on ingress defined in AKS.

Ingress in AKS.