Saturday, 16 March 2024

Deploying Kubernetes Event Drivern Autoscaling (KEDA) with Azure Pipelines Using Helm

 We have discussed how to deploy KEDA using helm in the post "Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity" .  Instead of deploying KEDA manually it is better to automate the deployment. Let's look at the steps to get KEDA deployed using Azure pipelines.

As the first step we need to have kubectl and helm installed in the pipeline agent.

      - task: KubectlInstaller@0
        displayName: 'Install Kubectl latest'
     
      - task: HelmInstaller@0
        displayName: 'Install Helm latest'
        inputs:
          helmVersion: latest

With terraform apply step we have to ensure, we are creating federated identity credentials with terraform for the user assigend identity for the keda-oprator as described in "Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity".

# Federated identity credential for AKS user assigned id - used with workload identity service account for KEDA
resource "azurerm_federated_identity_credential" "keda" {
  name                = "${var.prefix}-${var.project}-${var.environment_name}-aks-keda-fic-${var.deployment_name}"
  resource_group_name = var.rg_name
  audience            = ["api://AzureADTokenExchange"]
  issuer              = azurerm_kubernetes_cluster.aks_cluster.oidc_issuer_url # Open id connect issue url from AKS
  parent_id           = var.user_assigned_identity                             # user assigned identity id (Azure resource id)
  subject             = "system:serviceaccount:keda:keda-operator"             # system:serviceaccount:aksapplicationnamespace:workloadidentityserviceaccountname (to be created after AKS cluster is setup)

  depends_on = [
    azurerm_kubernetes_cluster.aks_cluster
  ]
  lifecycle {
    ignore_changes = []
  }
}


We need to setup service account via pipeline step by aplying below yaml via a kuber

---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: "${workload_id_client_id}$"
    azure.workload.identity/tenant-id: "${tenantid}$"
  name: keda-operator # Referred by AKS user assigned identity federated credential
  namespace: keda

The next step is to use an Azure CLI task and execute below set of steps.

  • Obtain aks admin credentials
  • update helm repo for KEDA
  • Install KEDA
    • we need to provide existing service account of the workload identity
    • since we are using existing service account we have to --set serviceAccount.create=false

      - task: AzureCLI@2
        displayName: 'Deploy KEDA'
        inputs:
          azureSubscription: '${{ parameters.serviceconnection }}'
          scriptType: pscore
          scriptLocation: inlineScript
          inlineScript: |
            $rgName = 'my-demo-rg';
            $aksName = 'my-demo-aks';
            
            Write-Host $aksName
            
            az aks get-credentials -n $aksName -g $rgName --admin --overwrite-existing

            helm repo add kedacore https://kedacore.github.io/charts
            helm repo update

            helm upgrade keda kedacore/keda --install `
              --namespace keda `
              --version 2.14.2 `
              --set serviceAccount.operator.create=true `
              --set serviceAccount.operator.name=keda-operator `
              --set podIdentity.azureWorkload.enabled=true `
              --set podIdentity.azureWorkload.clientId=$(sys_aks_uai_client_id) `
              --set podIdentity.azureWorkload.tenantId=$(tenantid)

            kubectl config delete-context (-join($aksName,'-admin'))

Once we execute all above in pipeline steps, we will have KEDA running in AKS.





No comments:

Popular Posts