RabbitMQ KEDA Trigger for AKS Deployed Apps to Scale Out the Apps Accurately

 We have discussed "Setting Up RabbitMQ Cluster in AKS Using RabbitMQ Cluster Operator" in a previous post. Baed on rabbitmq messages we may want to setup a trigger to auto scale our apps to handle the workloads similar to what we have done with eventhubs, azure service bus etc, as described in below posts.

• Setting Up (KEDA) Authentication Trigger for Azure Storage Queue/Service Bus in AKS
• Scale Pods in AKS with Kubernetes Event Driven Autoscaling (KEDA) ScaledJob Based on Azure Service Bus Queue as a Trigger
• Multiple KEDA Triggers for a Scaled Job with Event Hubs in AKS

In this post, let's look at usage of keda scale objects and how we can setup to trigger scaling with rabbit mq messages accurately for AKS deployed apps. The expectation is to scale out apps based on number of messgaes in rabbitmq queues and effective process the the work, while scaling in when there is not much workload on the system.

Depending on above message loads the consumer apps scale out and in accrdingly.

Multiple KEDA Triggers for a Scaled Job with Event Hubs in AKS

 Kubernetes scaled job helps us running one job per event/message we recive from the queue/even hub. We can have an event handler job which can handle more than one type of event messages or  queue messages. Let's look at what we need to consider when we are defining more than one trigger, with kubernetes event drivern autoscaler (KEDA) for a scaled job.

Setting Up (KEDA) Authentication Trigger for Azure Storage Queue/Service Bus in AKS

We have discussed setting up Kubernetes Event Drivern Autoscaling (KEDA) with AKS workload identity in the post, "Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity". Purpose of KEDA is to once we receive messages in a queue, such as Azure storage queue or Azure service bus queue we have to scale a scaledjob/deployment in kubernetes.

To setup authentication for the KEDA to communicate and monitor such a queue to scale a job or deployment, it should authentication to access the queue. We can set up the required authentication using using connnection strings for Azure service bus or storage queue . Instead of using such connection strings or shared access keys we can authenticate to the queue using the workload identity, since we have already enabled wrkload identity in KEDA as described in "Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity".

Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity

 We have discuss setting up workload identity in AKS to be used with application containers we deploy to AKS in the post "Setting Up Azure Workload Identity for Containers in Azure Kubernetes Services (AKS) Using Terra- Improved Security for Contianers in AKS". Kubernetes Event Drivern Autoscaling (KEDA) is the mechanism we need to use when we want to scale our deployments, or specially kubernetes jobs (A pod that runs for completion). In this post let's look at how to setup KEDA with workload identity, so that we can use KEDA in a later posts to run a Kubernets job, autoscaled based on the messages received to a storage queue or Azure service bus.