High Availability Deployment of Nginx Gateway Fabric Replacing Retired Ingress Nginx in AKS - Part 5 - Cleanup Ingress-Nginx from the AKS Cluster

 We have switched traffic from ingress-nginx to nginx-gateway as discussed in the previous blog. The next step is to gent ingress-nginx setup cleaned up from AKS env. This can be done manually. However, when there are mutiple production regions deployed with same deployment, it is easier if we programatically remove all ingress-nginx related routing and setupp, so it can be deployed to all production environments, same way, without issues.

The expectation is to get ingrss nginx removed, and the namespace ingress-nginx also cleaned up from the AKS cluster. Therefore, we can remove the namespace creation yaml as shown below.

Then elastic search and kibana routes created with kind: Ingress should be removed from code.

Next we should remove the ingress-nginx helm setup from code. See how we have deployed ingress-nginx previously here.

Even though we have removed the code as above it will not remove existing deployments from AKS. Below is section setup in PowerShell script, which removes ingress-nginx routes , helm installation and namepsace. It is written in a way thet it can be rerun without casuing any issues, to support autmated pipelines. We can remove below section of code once, all env are deployed with it and ingress-nginx is fully cleaned up.

#region Nginx Ingress Controller
Write-Host ('Cleaning up nginx ingress controller if exists...');
Write-Host ('----------------------------------------------------------');
Write-Host ('Checking if nginx ingress controller related routes exist in elastic-stack namespace ...');
$nginxIngressRoutes = kubectl get Ingress -n elastic-stack -o json | ConvertFrom-Json;
if (($null -ne $nginxIngressRoutes) -and ($nginxIngressRoutes.items.Count -gt 0))
{
    Write-Host ('Nginx ingress controller related routes exist in elastic-stack namespace. Deleting ...');
    kubectl delete Ingress --all -n elastic-stack;

    do
    {
        Write-Host ('Waiting for nginx ingress controller related routes to be deleted in elastic-stack namespace ...');
        Start-Sleep -Seconds 10;
        $nginxIngressRoutes = kubectl get Ingress -n elastic-stack -o json | ConvertFrom-Json;
    } until(($null -eq $nginxIngressRoutes) -or ($nginxIngressRoutes.items.Count -eq 0))

    Write-Host ('Nginx ingress controller related routes are deleted successfully in elastic-stack namespace.');
}
else
{
    write-Host ('Nginx ingress controller related routes do not exist in elastic-stack namespace. Skipping deletion.');
}

Write-Host ('----------------------------------------------------------');
Write-Host ('Checking if nginx ingress controller helm release exists ...');
$nginxHelmInstall = helm list -n ingress-nginx -o json | ConvertFrom-Json;

if (($null -ne $nginxHelmInstall) -and ($nginxHelmInstall.Count -gt 0))
{
    Write-Host ('Nginx ingress controller helm release exists. Uninstalling ...');
    helm uninstall ingress-nginx -n ingress-nginx;

    do
    {
        Write-Host ('Waiting for nginx ingress controller helm release to be uninstalled ...');
        Start-Sleep -Seconds 10;
        $nginxHelmInstall = helm list -n ingress-nginx -o json | ConvertFrom-Json;
    } until(($null -eq $nginxHelmInstall) -or ($nginxHelmInstall.Count -eq 0))

    Write-Host ('Nginx ingress controller helm release is uninstalled successfully.');
}
else
{
    Write-Host ('Nginx ingress controller helm release does not exist. Skipping uninstallation.');
}

Write-Host ('----------------------------------------------------------');
Write-Host ('Checking if namespace ingress-nginx exists ...');
$namespaces = kubectl get ns -o json | ConvertFrom-Json;

if ($namespaces.items.metadata.name.Contains('ingress-nginx'))
{
    Write-Host ('Namespace ingress-nginx exists. Deleting ...');
    kubectl delete ns ingress-nginx;
    
    do
    {
        Write-Host ('Waiting for namespace ingress-nginx to be deleted ...');
        Start-Sleep -Seconds 10;
        $namespaces = kubectl get ns -o json | ConvertFrom-Json;
    } until(-not $namespaces.items.metadata.name.Contains('ingress-nginx'))

    Write-Host ('Namespace ingress-nginx is deleted successfully.');
}
else
{
    Write-Host ('Namespace ingress-nginx does not exist. Skipping deletion.');
}
Write-Host ('=========================================================');
#endregion Nginx Ingress Controller

The Azure piline task can be used like below to call the script above. With all this we can cleanup ingress-nginx properly from all environments.

- task: AzureCLI@2

  displayName: 'Deploy services'

  inputs:

    azureSubscription: '${{ parameters.serviceconnection }}'

    scriptType: pscore

    scriptLocation: inlineScript

    inlineScript: |

      $rgName = 'ch-demo-$(envname)-rg';

      $aksName = 'ch-demo-$(envname)-aks';

      $serviceDeployNodepool = '$(sys_sh_svc_deploy_nodepool_suffix)';

      $bluegreenMode = '${{ parameters.deployinfra }}';

      $nginxGatewayLoadBalancerIp = '$(sh_private_ip_nginx_gateway)';

      $elasticUrl = 'http://es-search.sh.aks.ch-demo-$(envname).net';

      $keyvaultName = 'ch-demo-$(envname)-kv';

      $appCofigName = 'ch-demo-$(env)-appconfig-ac';

      az aks get-credentials -n $aksName -g $rgName --admin --overwrite-existing

      $(System.DefaultWorkingDirectory)/pipelines/scripts/deploy_services.ps1 `

        -manifestPath '$(System.DefaultWorkingDirectory)/pipelines/aks_manifests/' `

        -bluegreenMode $bluegreenMode `

        -serviceDeployNodepool $serviceDeployNodepool `

        -nginxGatewayLoadBalancerIp $nginxGatewayLoadBalancerIp `

        -elasticUrl $elasticUrl `

        -keyvaultName $keyvaultName `

        -appConfigName $appCofigName `

        -rgName $rgName

     

      kubectl config delete-context (-join($aksName,'-admin'))