Azure File Share with DefaultAzureCredential in .NET with Azure.Storage.Files.Shares - Is it possible?

 Using DefaultAzureCredential with most of the Azure resources is straight forward and simple with most of the Azure resources with relevant Azure .NET SDKs (We can use nuget packages Azure.Storage.Blobs and Azure.Identity). For example, with storage blob we can easily use DefaultAzureCredential as shown in below code.

    private static BlobServiceClient GetBlobServiceClient(string accountName)

    {

        return new(new Uri($"https://{accountName}.blob.core.windows.net"),

            new DefaultAzureCredential());

    }

However, we cannot simply create ShareClient with .NET SDK Azure.Storage.Files.Shares to use DefaultAzureCredential . as shown below.

ShareClient share = new(new Uri(fileShareUri),

    new DefaultAzureCredential());

With the above setup, we will get runtime errors when we try to perform operations with the Azure file share. As per the GitHub issue here it is not possible to use DefaultAzureCredential with Azure File Share with .NET SDK Azure.Storage.Files.Shares  due to "SMB Files cannot authenticate with a TokenCredential". So is it impossible to use DefaultAzureCredential  to perform operations with an Azure File Share using Azure.Storage.Files.Shares ? Let's look at a wokaround, which can help if desperately need to use DefaultAzureCredential with Azure.Storage.Files.Shares .

Setting Up Azure Workload Identity for Containers in Azure Kubernetes Services (AKS) Using Terraform - Improved Security for Containers in AKS

 Azure Workload Identity allows your containers in AKS touse amanaged identity to access Azure resources securely without having to depend on connection strings, passwords, access keys or secrets. In other works you can just use DefaultAzureCredential in your containers running in AKS, which will be using workload identity assigned to the container, to get access to the required Azure resource. The roale based access permissions will be in effect and the user assigned managed identity (we can use AD app registration as well bu user assigned managed identity is recommended) used to setup the workload identity in AKS should be given the necessary roles in the target Azure resource. This is far better than having to store secrets or connection stigs to utilized by the dotnet applications. In this post let's understand how to setup workload identity in AKS deployed containers and explore how it simplifies the dotnet application code allowing the application to access Azure resources securely with a managed identity.

Full example source code with terraform and a .NET application using default credentials to access app config service and keyvault is available here in my GitHub repo,

Setting Up Helm in WSL

 Kubernetes applications can be deployed easily with helm. Meny useful tools such as KEDA (Kubernetes Event Driven Autoscaler) deployments can be done with helm, using few simple steps. In this post let's look at how to setup helm in WSL so that we can use it to setup applications using helm charts.

Installing .NET 8 Runtime on Ubuntu 22.04 Docker Image

 .NET 8 was release on November 14. There are docker container images for .NET 8 available for dotnet runtime and can be found with tag list here https://mcr.microsoft.com/v2/dotnet/runtime/tags/list . However, if you want to setup .NET 8 runtime on another specific Linux docker image for example on ubuntu:jammy , amd64/ubuntu:22.04 or with a special image such as ffmpeg Linux server image linuxserver/ffmpeg:amd64-version-6.0-cli, where you might want to run your .NET app to use ffmpeg, In such cases, where you you might have to setup .NET 8 runtime on a specific docker image, with your other tools readily available, details mentioned may come in handy.   Lets, see how we can install .NET 8 runtime on base Ubuntu 22.04 images, using a docker file.

Installing .NET 8.0 SDK on WSL

 WSL (Windows subsystem for Linux) is a great way to work with Linux on Windows. .NET 8 is released on November 14th, and let's see how we can get .NET 8 SDK setup on WSL to build and test our .NET 8 apps on Linux on a windows machine.

What we want is when we do a dotnet --list-sdks to see the .NET 8 available in WSL. 

Terraform vs Azure Portal Defaults for Azure Storage Soft Delete

 Azure storage support soft deletion of blobs, blob containers and file shares. When we create a storage account usng Azure portal, by default soft deletion will be enabled with 7 day retention.

Install Kubectl on WSL (Windows Subsystem for Linux)

 Kubectl is the command line which will help to do everything with AKS or any other kubernetes set up. To setup or update kubectl on WSL follow the steps below. 

Transform json Files in Azure Pipelines

 In .NET aplication we use the appsettings.json to keep values such as connection string etc for local development mostly. In the context of Azure app servies or containeried envronments etc, we configure mostly app config service (with key vault to manage secrets) to keep our connection information, and any other config values. Howver, sometimes legecy deployments which are still done targeting on premise servers etc might need us to update actual appsetting.josn files or web config files etc. Let's look at how to transform josn files in an Azure pipeline.

Code Unit Test Coverage with Azure Pipelines

 Unit tests are essntial to ensure the code we develop is working as intended. Running the unit tests Azure pipelines is really helpful to not to miss the unit test failures. However, to give afurther assuarance we need to check the coverage of code with unit tests in our projects. Let's look at steps required to obtain a code unit tests coverage report in Azure piplines in  cobertura format.

The expected outcome

The coverage here shows the first library, all code lines (100%) are covered with tests and the the other library has only 22.2% code coverage. 

Conditionally Passing Different Values to Template Parameters in Azure DevOps Pipelines

Parameters Azure pipeline template helps to achieve dynamic behaviours in pipeline templates. Passing different values for template parameters based on conditions, via a single usage of teplate will help to reduce duplication of usage of template in the pipeline. Let's look at how to conditionally pass different values to template paramters with a practical example.

The expected behavoir

Let's assume there are three stages in the pipeline.

• Unit test stage - to build and unit test
• Build and push Docker image stage
• Deploy app stage

If we are in develop branch (refs/heads/develop) we want to run unit tests, build dcoker image and depending on both stages want to run a deploy.

Reducing Log Analytics Cost by Preventing Container Logs Ingession from Azure Kubernetes Services (AKS)

 Monitoring is an essential part of a deployment of software on aplatform such as AKS. However, once monitoring enabled there could be significant cost involved for monitoring data. When we enable log analytics workspace to ingest monitoring data from AKS, by default AKS will ingest all container logs, except for kube-system and gatekeeper-system namespaces. If our application are having large amout of container logs generated, then the cost will be lot higher for log analytics workspace. In case we are using app insights/ or an alternative system to monitor the application logs, for the applications deployed to AKS, we have enuough information to diagnose issues etc. Therefore, to reduce unnecessary cost log analytics can be done, by preventing AKS ingestion of the container logs to log analytics workspace. Let's explore the steps.

Expected outcome

As shown in the below figure, the log analytics workspace ingestion over time chart, is indicating the container logs data is no longer getting ingested after the change is applied.

 

Passing Environment Variables to Reusable Workflows in GitHub Actions

 We have discueed, that we have to use an environment variable to handle input parameter default values, if we are using trigger for workflow on push in the post "Setting Workflow Environment Variable Based on Input Parameter in GitHub Actions - on workflow_dispatch and Use a Default Value on push". If we have to pass on the input paramter value from a workflow to a reusable workflow, it does not work as expected and it is a limitation of reusable workflows  as explained in here. Let's try to understand with an example how to pass an env variable to a reusable workflow.

Setting Workflow Environment Variable Based on Input Parameter in GitHub Actions - on workflow_dispatch and Use a Default Value on push

 GitHub actions workflows support input only on manual trigger workflow_dispatch. What if we need to use default value in other triggers such as on push and use input parameter in case of manual trigger workflow_dispatch? Let's explore our options with an example.

Avoid GitHub Action Workflow Code Duplication - Using Composite Actions to Create Reusable Templates

 Azure DevOps yaml files for piplines can be created as templates to avoid duplicating same set of tasks in multiple areas of the pipelines. For GitHub actions workflow, we have seen here in "Build and Unit Tests for .NET Apps with GitHub Actions" the same steps are repeated in both Windows and Linux, build and unit test run. The solution to avoid duplicating same steps in multiple jobs in a GitHub actions workflow is the usage of composite actions. Let's modify our workflow implemented in "Build and Unit Tests for .NET Apps with GitHub Actions" to use composite action to build and unit test, and reuse that in both Windows and Linux job.

Running Pod Counts for All Services Summary - Grafana Chart with Azure Monitor for AKS

Monitoring the actual running pod count, against the desired pod count by each horizontal pod autoscaler, for your application is useful to get a summarize view, that will help to understand ow the pods are really scaling based on  scale out demand of each individual service. We can enable monitoring AKS cluster with Managed Grafana in Azure with Azure Monitor and Log Analytics Workspace. Let's write a query to and create Grafana chart for this purpose.

Expected Outcome

Each service actual scaling over time is shown as summary. This indicate based on load how each service in your application scale out based on the demanded by horizontal pod autoscaler.

Build and Unit Tests for .NET Apps with GitHub Actions

 Executing unit tests and viewing results in unit tests in a build pipeline is essential to keep high quality in an application deployment via any pipeline system. GitHub actions are the way forward to implement pipelines with repositories in GitHub. Let's explore how to run unit tests and view results in GitHub actions workflow.

Importing a Git Repo with LFS (Large Files) to Azure GIt Repos

 Importing a git repo to Azure DevOps Git is straight forward using the Azure DevOps portal. However, if the importing repo has large files with git lfs then the cloning of the repo after import fails due to the large files are not imported. Let's see how to get the large files fixed for the imported repo.

Using Object Type Azure DevOps YAML Pipeline Parameter in PowerShell Task

 Using a YAML pipeline paramter in a PowerShell task is straight forward for types such as string. For example, if there is a YAML pipeline parameter named env of type string, we can read it in PowerShell task with $envName = '${{ parameters.env }}; without any issue. However, if the parameter is type of object we cannot read, the paramter the same way we do with string parameters. If we try to read object parameter named apps, $apps = '${{ parameters.apps}}; , there will be an YAML validation error staring the pipeline such as below. 

/azure-pipelines.yml (Line: 57, Col: 23): Unable to convert from Array to String. Value: Array

Even if we try to use below it will be same issue.

[String[]]$apps = '${{ parameters.apps}};

[PSObject[]]$apps = '${{ parameters.apps}};

Let's explore the issueand solution in detail.

Mapping SQL Database SKU Bicep Specs with Available SKUs in a Region

 How to fnd SKU information for SQL databses is documented here. The command suggested to use is az sql db list-editions -l region -o table . This would provide available list of SQL database SKU optons to chose form for a given region. However, the headings and parameters required in bicep is bit confusing to figure out intially. Let's look at how to map values for available SKUs provided by az sql db list-editions -l region -o table  command, and parmeters in Bicep SKU for SQL database.

Ensure Azure App Config Refresh for Keyvault Secret Updates in Terraform

Keyvault secrets can be used in Azure app conciguratoins and can be setup with terraform. However, if the secret is modified then modified secret reference is not get updated to the app configuration. There are two ways to fix this issue in terraform. Let's explore them.

Fix Terraform Azure AD App Registration (SPN) Read Permssions Running with Azure DevOps Pipelines

 Azure DevOps use service principals (SPN or Azure AD app registration) to make a service connection to Azure to be able to run Terraform or other IaC based resource deployments targeting Azure. You may run into issue while trying to read another Azure AD app registration information, within terraform. For example consider below code segment.

# aks kv app

data "azuread_application" "akskv" {

  display_name = "${var.PREFIX}-${var.PROJECT}-aks-kv-app"

}

data "azuread_service_principal" "akskv" {

  application_id = data.azuread_application.akskv.application_id

}

Generate KVSet json Format Using appsettings json for Updating Azure App Configurations

 To apply Azure App Configurations including key vault secrets reference key values and normal key values, using a single file requires to use KVSet file as described in the Article here.  With Azure pipelines updating the app config values require, two seperate files to be used in default mode to update, app configs for non secrets and secrets. However, developers of .NET applications would prefer to keep the appsettings file for keeping configurations for development purpose, rather than keeping seperate file to keep references to secret key values. Therefore, in Azure pipline implmentation, it would be required to generate a KVSet file using an app setting file.

Zero Downtime Blue-Green Deployment for AKS with Terraform - Simulation Using a Resource Group with Pipeline Steps

In this post we use a resource group to demo a blue green deployment scenario for AKS with Terraform IaC, to understand the steps required for successful pipline implementation. Same pattern can be applied to deploy other Azure resources well. Instead of two resource groups used here to run demo faster, actual implementation can be two AKS clusters (blue and green) withing a single resource group or in two resource groups. Full pipeline with Terrafrom IaC for AKS blue greeen with application deployments will be dicussed in a future post(s). For now let's loo at the example blue green, using resource group to represent AKS cluster, which is used to validate blue green deployment algorithm for AKS.

Full algorithm test information and terraform files available in below links.

• ReadMe file describing algorithim and test steps in detail.
• Excel file containing algorithm steps and variable behaviour which will help to understand the algorith decribed in ReadMe.
• Terraform files used to simulate the AKS deployment using a resource group.

Fix Terraform Azure AD Group Read with "403 Insufficient privileges" in Azure Pipelines

 For deploying Azure resources  with Terraform via Azure pipelines we use service principals (SPN) to connect to Azure from Azure DevOps. You might encounter "403 Insufficient privileges" errors while trying to read Azure AD groups data, which you might want to use to create role assignment in the new resources you are provsioning with Terraform. For example it can be a Azure AD group referred as data as shown below.

# refer to sub_owners AD group to assign as aks admins

data "azuread_group" "myteam" {

  display_name     = "sub_owners"

  security_enabled = true

}

 Let's look at the how to resolve the exception "403 Insufficient privileges".

Fix Azure Advisor "Update VNet permission of Application Gateway users" in AKS Application Gateway Ingress Controller (AGIC) App Gateway

 In last two days there was an alert in the AKS ingress app gateway, recomending to  "Update VNet permission of Application Gateway users". Then starting on 20th June 2023, any newly created  AKS cluster with AGIC, started to fail to manage ingess settings on app gatway for eastus region. Same behavoiur is shown in westeurope region today 21 June 2023. 

ch-demo-dev-euw-005-aks-agw-snet to perform action Microsoft.Network/virtualNetworks/subnets/join/action. For details on the required permissions, 

HPA Desired Pod Counts for All Services Summary - Grafana Chart with Azure Monitor for AKS

 Monitoring the desired pod count by each horizontal pod autoscaler for your application is useful to get a summarize view, that will help to understand scale out demand of each individual service. We can enable monitoring AKS cluster with Managed Grafana in Azure with Azure Monitor and Log Analytics Workspace. Let's write a query to and create Grafana chart for this purpose.

Dynamically Adding Virtual Network Rules to Azure Cognitive Account Using Terraform

 To enable access to various Azure resources via given virtual networks and subnets, we have to setup allowd subnets using virtual network rules in Azure resources. For example Azure Cognitive Account can be restricted to access only from given virtual networks and subnets. This requirement of allowed virtual networks and subnets may changed based on the deploying environment such as develop, qa or production. Let's look at how to write a terraform dynamic block to handle such scenarios, using Azure Cognitive Account virtual network rule as example, which can be used in other Azure resources as well.

Full sample code with  Azure Cognitive Account is available here in GitHub.

Store Azure SignalR Connection String as a Secret in Azure Key Vault with Bicep IaC

  We can create Azure SignalR and Azure key vault as resources using Bicpe IaC (Infrastructure as Code). Setting up of SignalR connection string as a Secret in Azure Key Vault via IaC is important, so that it can be used by applications by refering to key vault secret. Let's look at the steps required to store the Azure SignalR connection string as a secret in Azure key vault with Bicep.

Create Key Vault Reference in Azure App Config Service with Bicep IaC

 We have discussed "Store Azure Service Bus Connection String  as a Secret in Azure Key Vault with Bicep IaC" previously. Setting such a secret in Azure App configuration service as a key vault reference will help the applications to access the connection string to Azure service bus via the app config service. This is applicable to setting up reference to any secret in Azure key vault, as Azure key vault reference in Azure app configuration service, via Bicep as Infrastructure as Code (IaC). Let's look at the steps required.

Store Azure Service Bus Connection String as a Secret in Azure Key Vault with Bicep IaC

 We can create Azure Service Bus and Azure key vault as resources using Bicpe IaC (Infrastructure as Code). Setting up of Service Bus connection string as a Secret in Azure Key Vault via IaC is important, so that it can be used by applications by refering to key vault secret. Let's look at the steps required to store the Azure service bus connection string as a secret in Azure key vault with Bicep.

Restart Count Details of Each Pod and Container for App in Grafana Chart with Azure Monitor for AKS

 Pod and container restart counts monitoring are discussed in posts "Pod Restart Counts Grafana Chart with Azure Monitor for AKS" and "Container Restart Counts Grafana Chart with Azure Monitor for AKS". Those two charts show summarized view per service. To view details of pod and container restarts for an individual service we can create another chart as described in this post.

Workaround Solution for Intermittent CrashLoopBackOff in Windows Containers Running on AKS (.NET 6 Apps with System.Net.Sockets.SocketException 11001 and 10060)

Let's look at a temporary solution to the issue Intermittent CrashLoopBackOff in Windows Containers Running on AKS (.NET 6 Apps with System.Net.Sockets.SocketException 11001 and 10060). Same issue is asked in stackoverflow here. Instead of manually deleting pods that run into the issue maually, the cleaner app implemented in this repo is doing autmatic deletion of pods CrashLoopBackOff state with known exception reported in the container log. If the exeception is unknown the pod in CrashLoopBackOff state will not be deleted, and the container log output is printed, in cleaner app logs to show the exception of the pod having CrashLoopBackOff state.

Docker Alpine (mcr.microsoft.com/dotnet/runtime) - Resolve "Microsoft.WindowsAzure.Storage.StorageException: Only the invariant culture is supported in globalization-invariant mode."

.NET app running with mcr.microsoft.com/dotnet/runtime docker alpine images, might run into issue "Microsoft.WindowsAzure.Storage.StorageException: Only the invariant culture is supported in globalization-invariant mode" if your app is requiring globalization. Let's look at what needs to be done to get the issue fixed.

Resolve Windows Docker Build "C:\Program Files\dotnet\sdk\6.0.407\Microsoft.Common.CurrentVersion.targets(3262,5): error MSB4018: System.Runtime.InteropServices.COMException (0x80040154): Retrieving the COM class factory for component with CLSID {7Bxxxxxxxxxxxx} failed due to the following error: 80040154 Class not registered (0x80040154 (REGDB_E_CLASSNOTREG))"

 Building .NET 6 project with Resource files is running into build issue as shown below, when building with .NET 6 nanoserver build images.

C:\Program Files\dotnet\sdk\6.0.407\Microsoft.Common.CurrentVersion.targets(3262,5): error MSB4018: The "GenerateResource" task failed unexpectedly. [C:\src\Demo\Demo.csproj]

C:\Program Files\dotnet\sdk\6.0.407\Microsoft.Common.CurrentVersion.targets(3262,5): error MSB4018: System.Runtime.InteropServices.COMException (0x80040154): Retrieving the COM class factory for component with CLSID {7Bxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx} failed due to the following error: 80040154 Class not registered (0x80040154 (REGDB_E_CLASSNOTREG)). [C:\src\Demo\Demo.csproj]

Container Restart Counts Grafana Chart with Azure Monitor for AKS

 We have disucussed pod restart Grafana panel in AKS in the post "Pod Restart Counts Grafana Chart with Azure Monitor for AKS". We can create similar panel for container restarts using managed Grafana in Azure for AKS using Azure monitor.

Expected Outcome

Panel similar to below showing contianer restarts for a applications over time with a table showing last container(s) of application restarts, maximum restarts for each application.

Pod Restart Counts Grafana Chart with Azure Monitor for AKS

 If your pods are often restarting it might indicate a problem you might have in your application deployed to AKS. For example, there was some significant number of restarts seen in .NET 3.1 applications deployed to AKS (reason found as .NET 3.1 issue which is supposedly fixed in .NET 5, so the approach to fix was to update the applications to .NET 6), which were only appearing in development and in staging envronments, while QA environment haven't shown a single restart. Threfore, it is important to monitor the restart counts in pods to identify issues you might not see in development or QA envronements, but may occur in production environments. Let's see how we can create a pod restart count panel in managed Grafana in Azure for AKS using Azure monitor.

Pod Counts Grafana Chart with Azure Monitor for AKS

 We have discussed how to create a Grafana panel for view Horizontal Pod Autoscaler (HPA) desired pod counts in AKS (Azure Kubernetes Services) in the post "Horizontal Pod Autoscaler Desired Replica Count Grafana Chart with Azure Monitor for AKS". As desired count is changing in HPA for an app monitoring how the actual pod count is changing would be really useful insights. Let's look at steps required to implement Grafana chart for actual pod counts over time using Azure monitor fror AKS.

Horizontal Pod Autoscaler Desired Replica Count Grafana Chart with Azure Monitor for AKS

 Horizontal Pod Autoscaler (HPA) in Kubernetes manage the number of instances for each pod deployment based on the scale settings specified. Itw would be good insights to monitor the demand for scaling of each pod. as a prerequicite we need to setup monitoring AKS cluster with Managed Grafana in Azure with Azure Monitor and Log Analytics Workspace.  Let's see how we can create a Grafana pannel to monitor the HPA behaviour over time.

Creating AKS Node Count Grafana Panel with Azure Monitor

 AKS can be setup with automatic node scaling using VM scalesets in node pools. It will be useful to monitor how the nodes are scaling over time to have  a good insights, in to the AKS cluster nodes behaviour, while applications deployed in the cluster getting heavy traffic. We can enable monitoring AKS cluster with Managed Grafana in Azure with Azure Monitor and Log Analytics Workspace.  Let's look at query to setup node count over time panel in Managed Grafana in Azure using Azure Monitor.

Set Value for a Secret Variable in Azure Release Pipelines while Triggering from Another Release

 You might want to trigger a release from another release pipeline, specially if you are deploying to a multi tenant application environment. In such cases when you trigger a pipeline from another pipeline the variables you are setting dynamically cannot be a secret variable. If you set a variable that should be settable at release time as a secret, the release cannot be triggerd via the Azure DevOps REST API call, as it will throw an exception such as below.

The 'secret' property of

variables cannot be altered while creating a release. Verify the value

provided for variables UserPwd at scope Release and try again.  

Let's try to understand bit more of the requirement.