Path Based Routing to AKS Ingress with Application Gateway Ingress Controller

 We have discussed getting Application Gateway Ingress Controller (AGIC) deployed using AGIC addon method in the post "Enable Application Gateway Ingress Controller (AGIC) for AKS - SImplest Way with a New App Gatway in Same vNet of AKS Cluster". We have tried a sample app deployment and added a routing ingress rule, to route to root, and reach the sample application toverify the working state of the AGIC in AKS cluster. However, generally in the APIs we deploy to AKS cluster, would require path based routing to reach, diffrent APIs. Let's discuss how to create ingress path based routing to AKS deployed APIs using AGIC.

Firts lets undersdtand how APIs are implemneted with path based routing. As described in the post "Enable Swagger UI with an .NET 6 API having Custom Routing"  even the swagger UI can be set to use custom routing with APIs. Now lets take four APIs customer, order, invoice and payment. Each API route is supposed to have a prefix as below.

• /customer/api/
• /order/api/
• /invoice/api/
• /payment/api/

To implement in each API for swagger the changes should be done in program.cs  (.NET 6) as described in post "Enable Swagger UI with an .NET 6 API having Custom Routing". Routing of each API is done as shown in below sample. Note that we are using weatherforecast controller created by defualt with dotnet new webapi command to create the project using template.

To create such routing with AGIC the Kubernetes ingress can be defined as shown below. Note that for health probe path we are using swagger UI path as it would return healthy to APP gateway health probe.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: customerapi
  namespace: demo
  annotations:
    # --------------
    # AGIC 
    kubernetes.io/ingress.class: azure/application-gateway 
    appgw.ingress.kubernetes.io/health-probe-path: /customer/api/swagger
    # --------------
spec:
  rules:
  - http:
      paths:
      - path: /customer/api/*
        pathType: Prefix
        backend:
          service:
            name: customer-clusterip # Name of the cluster IP service of API
            port:
              number: 8091 # cluster ip port

We can use replaceable tokens instead of hardcoded values as shown below and use a replace tokens task in Azure DevOps pipline to appy changes to the deployment yaml file before deploying to AKS. Full API deployment yaml to Kubernetes namespace named demo, using resource limits and horizontal pod autoscaling setup is below. In the below deployment yaml, a Kubernetes config map usage to apply application configs to APIs is used. We can further discuss about onfig maps,  horizontal pod autoscaling and applying resource limits to continers in Kubernetes in future posts.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${aks.appName}$
  namespace: demo
  labels:
    app: ${aks.appName}$
spec:
  # replicas: 1
  selector:
    matchLabels:
      service: ${aks.appName}$
  template:
    metadata:
      labels:
        app: ${aks.appName}$
        service: ${aks.appName}$
    spec:
      nodeSelector:
        "kubernetes.io/os": ${aks.nodeSelector}$
      volumes:
        # `name` here must match the name
        # specified in the volume mount
        - name: demo-configmap-${aks.appName}$-volume
          #
          configMap:
            # `name` here must match the name
            # specified in the ConfigMap's YAML
            name: demo-configmap
      containers:
        - name: ${aks.appName}$
          # when aks
          image: ${aks.containerRegistry}$.azurecr.io/demo/${aks.appName}$:${Build.BuildId}$
          # when k3d
          # image: demo-k3d-registry:44263/${aks.appName}$:dev
          imagePullPolicy: Always
          volumeMounts:
          - mountPath: /etc/config
            name: demo-configmap-${aks.appName}$-volume
          ports:
            - containerPort: 80
              protocol: TCP
          env:
            - name: ASPNETCORE_URLS
              value: http://+:80
            - name: ASPNETCORE_ENVIRONMENT
              value: ${aks.aspNetCoreEnv}$
            - name: DEMO_CONFIG_PATH
              value: /etc/config/config_${envname}$.json
          resources:
                limits:
                  memory: ${aks.container.limits.memory}$
                  cpu: "${aks.container.limits.cpu}$"
                requests:
                  memory: ${aks.container.requests.memory}$
                  cpu: "${aks.container.requests.cpu}$"
---
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: ${aks.appName}$-hpa
  namespace: demo
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ${aks.appName}$
  maxReplicas: ${aks.hpa.maxReplicas}$
  minReplicas: ${aks.hpa.minReplicas}$
  behavior:
    scaleDown:
      stabilizationWindowSeconds: ${aks.hpa.scaleDown.stabilizationWindowSecs}$
      policies:
      - type: Percent
        value: ${aks.hpa.scaleDown.percentage}$
        periodSeconds: ${aks.hpa.scaleDown.periodSecs}$
    scaleUp:
      stabilizationWindowSeconds: ${aks.hpa.scaleUp.stabilizationWindowSecs}$
      policies:
      - type: Percent
        value: ${aks.hpa.scaleUp.percentage}$
        periodSeconds: ${aks.hpa.scaleUp.periodSecs}$
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: ${aks.hpa.metrics.cpu.averageUtilization}$
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: ${aks.hpa.metrics.memory.averageUtilization}$
---
apiVersion: v1
kind: Service
metadata:
  name: ${aks.appName}$-clusterip
  namespace: demo
  labels:
    app: ${aks.appName}$
    service: ${aks.appName}$
spec:
  type: ClusterIP
  ports:
    - port: ${aks.clusterIp.port}$
      targetPort: 80
      protocol: TCP
  selector:
    service: ${aks.appName}$

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ${aks.appName}$
  namespace: demo
  annotations:
    # kubernetes.io/ingress.class: nginx # for nginx
    # --------------
    # when AGIC is ready
    kubernetes.io/ingress.class: azure/application-gateway 
    appgw.ingress.kubernetes.io/health-probe-path: ${aks.appHeathProbePath}$
    # --------------
spec:
  rules:
  - http:
      paths:
      - path: ${aks.appRoutePrefix}$
        pathType: Prefix
        backend:
          service:
            name: ${aks.appName}$-clusterip
            port:
              number: ${aks.clusterIp.port}$

In the build piline Library variables are used to apply the required values in above YAML.

Customer API specific variables.

Once the above mentioned ingress is deployed to the AKS cluser will have ingress gor customer API with the public Ip of the app gatway as shown below.

App gatway will be setup autmatically with backend settings for Customer API with custom probe. 

Custom probe will have the Swagger UI path of the Customer API set to it as we have defined in the Ingress. However, proper liveness, readiness and startup health probes should be defined for an API and Liveness and readiness probes should be ideally validated to check the pod health via app gateway. We can dicuss such topics in later posts. 

Routing rules will be defined autmatically facilitating routing to the Customer API via app gatway.

Once all 4 APIs mentioned above deployed with ingress in AKS the rule will be updated by AGIC as shown below, and the required probes and backend settings will be added to app gateway automatically.

Backend health will be shown as success in app gateway.

We can define an alias to the app gatwway public IP to access it via the alias rather than using IP address in url to access APIs. You can define requierd A records and setup a custom domain route to app gatway as well.  

With that we can access all APIs deployed to AKS via app gatwway path based routing as shown below.