Create AKS Cluster to Host Valkey Cluster (Open Source Redis)

 Since bitnami redis images have gone commercial now the best open source redis clsuter setup option is to use Valkey, which is guranteed to be open source forever. The source code repos for Valkey is here. We have discussed deploying redis cluster on AKS with bitnami in "Setup Redis Cluster with JSON and Search Modules on AKS with Binami Redis Using Custom Image" However, if we want to deploy Valkey/Redis in cluster in AKS and want to access the Valkey/Redis from outside the AKS cluster, we have to setup AKS networking in a way that each pod is allocated with an IP from the Azure virtual network subnet. In this post let's explore how to setup such an AKS cluster correctly.

Why we need pods in AKS to have IPs from virtual network subnet to deploy Valkey?

When we connect to a Valkey/Redis cluster it always provide diffrent IPs of master nodes to client. Therefore, clients need to acess to IPs of Valkey node pods.

We cannot use Azure CNI overlay for this AKS cluster as it would setup internal network CIDR for pod IPs. Such IPs are not visible outside of AKS. Therefore it is not possible use Azure CNI overlay for AKS clsuter, if we are to deploy Valkey clsuter that should be allowed to access outside of AKS.

We can Azure node subnet mode. But this is not ideal as we have to have a large subnet planned to support boith the node needs as well as pods IP requirement.

The best option is to use Azure CNI Pod Subnet, which would be really useful for this need.

Using Managed Prometheus Data to Setup Monitoring for AKS Deployed Bitnami Redis Cluster or Standalone

 Based on "Enable Prometheus Data Scraping for Bitnami Redis Cluster Deployed on AKS with Managed Prometheus" and "Enable Prometheus Data Scraping for Bitnami Redis Standalone Deployed on AKS with Managed Prometheus" discussed in the previous posts, let's look at using the managed prometheus data source in Azure Manged Grafana and how to setup monitoring for AKS deployed Bitnami Redis, with few examples.

The expectation is to have charts as shown below.

Enable Prometheus Data Scraping for Bitnami Redis Standalone Deployed on AKS with Managed Prometheus

 We have discussed "Enable Prometheus Data Scraping for Bitnami Redis Cluster Deployed on AKS with Managed Prometheus" in the previous post. Similar way we can setup Prometheus data scraping for standalone redis deployments on AKS as well. Here are the steps.

The expectaion is to have the metrics sidecar run with the standalone redis master and replica pods as shown below and setup a service monitor so that redis metrics data is made available in Azure managed grafana via managed prometheus on AKS. 

Enable Prometheus Data Scraping for Bitnami Redis Cluster Deployed on AKS with Managed Prometheus

 "Setup Redis Cluster with JSON and Search Modules on AKS with Binami Redis Using Custom Image" and "Setup Managed Prometheus for AKS via Terraform" are explained in the previous posts. To enable setting up monitoring and alerting for the Redis Custer deployed in AKS the first step is to enable Prometheus data scraping in the Redis cluster we deployed on AKS. Let's look at the steps in this post.

The expectation is to have redis metrices available to Azure managed grafana via the managed prometheus in AKS as shown below.

Deploy Redis Insight on AKS to View/Update Data in Redis Cluster Deployed on AKS

 "Setup Redis Cluster with JSON and Search Modules on AKS with Binami Redis Using Custom Image" is explained in the prevous post. We can use redis insight to connect to redis cluster on AKS to explore data and add or update data. Let's look at how we can setup redis insight on AKS in this post.

The expectation is to have redis insight connected to redis clsuter on aks as shown below.

PART2 - Using Bash Instead of PowerShell for - Setting Up Azure Managed Redis with Terraform Using AzApi

 We have discussed "Setting Up Azure Managed Redis with Terraform Using AzApi" , since direct terraform resources are not yet available until the pull request here is released. We used a PowerShell script to extract access key and output it from terraform output. In this post let's look at how to get the same steps with bash script instead of PowerShell and get the Azure Managed Redis deployed as shown below.

Setting Up Azure Managed Redis with Terraform Using AzApi

 The new Azure Managed Redis can be deployed with balanced compute and memory with high availability, and useful modules such as RedisJson and RedisSearch. This is really a useful and good pricing options to use Redis as a managed service in Azure. Note that it is not the enterprise redis offering in Azure and managed redis for Azure has more flexible pricing options. However, terraform support for this is yet to be added and will be available in another month or so as per the pull request here. Let's see how to get a Azure Managed Redis deployed with terraform for no using AzAPI.

The expectaion is to have a deployed Azure Managed Redis as shown below.

Setup Redis Cluster with JSON and Search Modules on AKS with Binami Redis Using Custom Image

 We have discussed "Build Custom Docker Images with Redis Json and Search Module Support for deploying Bitnami Redis Cluster and Standalone in AKS" previously. We have cfreated a cutom redis cluster image including the redis modules json, search etc. In this post let's explore how to setup Redis cluster on AKS using binami helm chart, while using the custom built image. The usage of custom built image is required to have the json and search modules available as described in the post "Build Custom Docker Images with Redis Json and Search Module Support for deploying Bitnami Redis Cluster and Standalone in AKS".

Note that the redis cluster deployed to AKS or to kubernetes can be only accessed withing the kubernetes cluster. Therefore, only the apps deployed to AKS/kubernetes can only access the redis endpoint in redis cluster on AKS. For allowing local development with redis on AKS, we need to setup bitnami redis standalone mode, which we will discuss in a future post.

Build Custom Docker Images with Redis Json and Search Module Support for deploying Bitnami Redis Cluster and Standalone in AKS

 Redis cluster deployed in AKS (kubernetes) is a really useful way to use Redis cache in dotnet projects. To depoy Redis on AKS we can use Redis bitnami cluster or standalone. Cluster mode deployment is only accessible within the kubernetes clsuter, therefore for development environments, to allow local machine access to Redis in an AKS clsuter you need the standalone mode of deployment, which we will discuss in the next post. The bitnnami Redis images are not included with the json module and search module which are useful to store json documents and search them in a Redis setup. In this post let's explore how to include addtional modules in Bitnami Redis to build a custom image so that the Json and Search of Redis available in AKS once deployed.

The expection is to build custom Bitnami Redis images with module support similar to Redis 8 and get them added to Azure Container registry as shown below.

 

Azure Redis Connectivity Checker App

Here is a simple .NET application to test connectivity to Azure Redis Cache. The application code is available in GitHub chamindac/RedisConnectionChecker. Refer the video for more details.

Resolve C# Connection Issue "StackExchange.Redis.RedisConnectionException: No connection is active/available to service this operation: It was not possible to connect to the redis server(s). There was an authentication failure; check that passwords (or client certificates) are configured correctly." to Azure Redis cache Connection via Private Endpoint

 Using Azure Redis cache with C# throws an error message such as below while accessing the Redis cache via a private endpoint.

StackExchange.Redis.RedisConnectionException: 

No connection is active/available to service this operation: 

GET Zdddddddddddddddddddddddddd; It was not possible to connect to the redis server(s). 

There was an authentication failure; check that passwords (or client certificates) are 

configured correctly. ConnectTimeout, mc: 1/1/0, mgr: 10 of 10 available,

 clientName: DESKTOP-XXXXX, IOCP: (Busy=1,Free=999,Min=16,Max=1000), WORKER: 

(Busy=3,Free=32764,Min=16,Max=32767), v: 2.2.88.56325
 ---> StackExchange.Redis.RedisConnectionException: It was not possible to connect to the 

redis server(s). There was an authentication failure; check that passwords 

(or client certificates) are configured correctly. ConnectTimeout
   --- End of inner exception stack trace ---