tag:blogger.com,1999:blog-46848744659879516012024-03-15T18:13:09.007-07:00Chaminda's DevOps Journey with MSFTDevOps with Azure DevOpsChaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.comBlogger421125tag:blogger.com,1999:blog-4684874465987951601.post-950870566473082692024-01-20T12:22:00.000-08:002024-02-24T08:39:49.050-08:00Scale Pods in AKS with Kubernetes Event Drivern Autoscaling (KEDA) ScaledJob Based on Azure Service Bus Queue as a Trigger<p> In previous posts we discussed "<a href="https://chamindac.blogspot.com/2023/11/setting-up-keda-in-aks.html" target="_blank">Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity</a>" and how to "<a href="https://chamindac.blogspot.com/2024/01/setting-up-keda-authentication-trigger.html" target="_blank">Set Up (KEDA) Authentication Trigger for Azure Storage Queue/Service Bus in AKS</a>". With that now we can proceed to setup kubernetes scaled job in AKS to run a pod when the Azure service bus queue received a message. Using scaled job we are going to start a job (pod) once a messsage is received in the queue and then receive the massage in the pod container app, process and complete the message and complete the job execution with a pod complete. So, there will be a different pod and a container (kubernetes job) processing each message recived in the Azure service bus queue.<span></span></p><a name='more'></a><p></p><p>So the purpose is to create a kubernetes job which can be scaled using KEDA based on the messages recived in the queue as shown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgGNMVSmN6swh2Sdtjk5xNGbFJSgYD1LA-snUUhmwoHSw8BCjzeyhpH6fr1bu9kDqiJbvWahgfnBJPWIk6IoX1J3qS_aEpULn1Q5CFE4YeixDvc2MkJeWL9ICmG0eVpFPKIvnrM_DPEOfw61YBMuEHtaYvqUuggxdOJOR2AujCizAyGqN0IdFLSrvzTvihD" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="881" data-original-width="1262" height="428" src="https://blogger.googleusercontent.com/img/a/AVvXsEgGNMVSmN6swh2Sdtjk5xNGbFJSgYD1LA-snUUhmwoHSw8BCjzeyhpH6fr1bu9kDqiJbvWahgfnBJPWIk6IoX1J3qS_aEpULn1Q5CFE4YeixDvc2MkJeWL9ICmG0eVpFPKIvnrM_DPEOfw61YBMuEHtaYvqUuggxdOJOR2AujCizAyGqN0IdFLSrvzTvihD=w615-h428" width="615" /></a></div><br /><a href="https://github.com/chamindac/ffmpeg_aks/tree/main/src/videoprocessor.xabe" target="_blank">The example .NET 8 application code is available here in GitHub</a>, which is performing a video transcoding as its process using ffmpeg. The dcker image can be built using the docker file in the project here and the image can be <a href="https://chamindac.blogspot.com/2022/09/manually-push-net-app-docker-image-to.html" target="_blank">pushed to Azure container registry</a>.<p></p><p><br /></p><p>We can use below shown yaml (deploy with kubectl apply) to deploy the .NET app as a scaled job with KEDA scaling the job when we send messages to the Azure service bus queue. You can find comments describing purpose of each setting in below yaml, and some important settings are described latter in this post below for further clarification.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
name: dotnet-video-processor-scaled-job
namespace: media
<span style="color: #888888;"># annotations:</span>
<span style="color: #888888;"># autoscaling.keda.sh/paused: true # Optional. Use to pause autoscaling of Jobs</span>
spec:
jobTargetRef:
parallelism: 1 <span style="color: #888888;"># [max number of desired pods](https://kubernetes.io/docs/concepts/workloads/controllers/job/#controlling-parallelism)</span>
completions: 1 <span style="color: #888888;"># [desired number of successfully finished pods](https://kubernetes.io/docs/concepts/workloads/controllers/job/#controlling-parallelism)</span>
activeDeadlineSeconds: 15000 <span style="color: #888888;"># Should be higher than maxProcessWaitTime (14400) in video processor # Specifies the duration in seconds relative to the startTime that the job may be active before the system tries to terminate it; value must be positive integer</span>
backoffLimit: 6 <span style="color: #888888;"># Specifies the number of retries before marking this job failed. Defaults to 6</span>
template:
<span style="color: #888888;"># ==========================================================</span>
<span style="color: #888888;"># describes the [job template](https://kubernetes.io/docs/concepts/workloads/controllers/job)</span>
metadata:
labels:
app: dotnet-video-processor
service: dotnet-video-processor
azure.workload.identity/use: <span style="background-color: #fff0f0;">"true"</span>
spec:
serviceAccountName: ch-video-wi-sa
nodeSelector:
<span style="background-color: #fff0f0;">"kubernetes.io/os"</span>: linux
priorityClassName: media-highest-priority-linux
<span style="color: #888888;">#------------------------------------------------------</span>
<span style="color: #888888;"># setting pod DNS policies to enable faster DNS resolution</span>
<span style="color: #888888;"># https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy</span>
dnsConfig:
options:
<span style="color: #888888;"># use FQDN everywhere </span>
<span style="color: #888888;"># any cluster local access from pods need full CNAME to resolve </span>
<span style="color: #888888;"># short names will not resolve to internal cluster domains</span>
- name: ndots
value: <span style="background-color: #fff0f0;">"2"</span>
<span style="color: #888888;"># dns resolver timeout and attempts</span>
- name: timeout
value: <span style="background-color: #fff0f0;">"15"</span>
- name: attempts
value: <span style="background-color: #fff0f0;">"3"</span>
<span style="color: #888888;"># use TCP to resolve DNS instad of using UDP (UDP is lossy and pods need to wait for timeout for lost packets)</span>
- name: use-vc
<span style="color: #888888;"># open new socket for retrying</span>
- name: single-request-reopen
<span style="color: #888888;">#------------------------------------------------------</span>
volumes:
<span style="color: #888888;"># # `name` here must match the name</span>
<span style="color: #888888;"># # specified in the volume mount</span>
<span style="color: #888888;"># - name: demo-configmap-video-processor-volume</span>
<span style="color: #888888;"># configMap:</span>
<span style="color: #888888;"># # `name` here must match the name</span>
<span style="color: #888888;"># # specified in the ConfigMap's YAML</span>
<span style="color: #888888;"># name: demo-configmap</span>
- name: media-data-volume
persistentVolumeClaim:
claimName: media-azurefile-video-processor <span style="color: #888888;"># PersistentVolumeClaim name in aks_manifests\prerequisites\k8s.yaml</span>
terminationGracePeriodSeconds: 90 <span style="color: #888888;"># This must be set to a value that is greater than the preStop hook wait time.</span>
containers:
- name: dotnet-video-processor
lifecycle:
preStop:
exec:
command: [<span style="background-color: #fff0f0;">"sleep"</span>,<span style="background-color: #fff0f0;">"60"</span>]
image: chdemosharedacr.azurecr.io/media/chdotnetmediaservice:1.5
imagePullPolicy: Always
volumeMounts:
<span style="color: #888888;"># - mountPath: /etc/config</span>
<span style="color: #888888;"># name: demo-configmap-video-processor-volume</span>
- mountPath: /media/data
name: media-data-volume
env:
- name: MEDIA_PATH
value: /media/data
resources:
limits:
memory: 4Gi <span style="color: #888888;"># the memory limit equals to the request!</span>
<span style="color: #888888;"># no cpu limit! this is excluded on purpose</span>
requests:
memory: 4Gi
cpu: <span style="background-color: #fff0f0;">"2"</span>
<span style="color: #888888;"># ==========================================================</span>
pollingInterval: 5 <span style="color: #888888;"># Optional. Default: 30 seconds</span>
successfulJobsHistoryLimit: 20 <span style="color: #888888;"># Optional. Default: 100. How many completed jobs should be kept.</span>
failedJobsHistoryLimit: 20 <span style="color: #888888;"># Optional. Default: 100. How many failed jobs should be kept.</span>
<span style="color: #888888;"># envSourceContainerName: {container-name} # Optional. Default: .spec.JobTargetRef.template.spec.containers[0]</span>
minReplicaCount: 0 <span style="color: #888888;"># Optional. Default: 0</span>
maxReplicaCount: 10 <span style="color: #888888;"># Optional. Default: 100</span>
<span style="color: #888888;"># rolloutStrategy: gradual # Deprecated: Use rollout.strategy instead (see below).</span>
rollout:
strategy: gradual <span style="color: #888888;"># We should not delete existing jobs. So gradual. # Optional. Default: default. Which Rollout Strategy KEDA will use.</span>
propagationPolicy: foreground <span style="color: #888888;"># Optional. Default: background. Kubernetes propagation policy for cleaning up existing jobs during rollout.</span>
scalingStrategy:
strategy: <span style="background-color: #fff0f0;">"default"</span> <span style="color: #888888;"># We use default as locked messages should not appear in queue # Optional. Default: default. Which Scaling Strategy to use. </span>
<span style="color: #888888;"># customScalingQueueLengthDeduction: 1 # Optional. A parameter to optimize custom ScalingStrategy.</span>
<span style="color: #888888;"># customScalingRunningJobPercentage: "0.5" # Optional. A parameter to optimize custom ScalingStrategy.</span>
<span style="color: #888888;"># pendingPodConditions: # Optional. A parameter to calculate pending job count per the specified pod conditions</span>
<span style="color: #888888;"># - "Ready"</span>
<span style="color: #888888;"># - "PodScheduled"</span>
<span style="color: #888888;"># - "Pending"</span>
<span style="color: #888888;"># - "ContainerCreating"</span>
<span style="color: #888888;"># multipleScalersCalculation : "max" # Optional. Default: max. Specifies how to calculate the target metrics when multiple scalers are defined.</span>
triggers:
- type: azure-servicebus
metadata:
queueName: dotnetvideoqueue
namespace: ch-video-dev-euw-001-sbus-blue
messageCount: <span style="background-color: #fff0f0;">"1"</span>
authenticationRef:
name: video-processor-queue-auth
<span style="color: #888888;">#--------------</span>
<span style="color: #888888;"># Below is how the storage queue is used as trigger</span>
<span style="color: #888888;">#--------------</span>
<span style="color: #888888;"># - type: azure-queue</span>
<span style="color: #888888;"># metadata:</span>
<span style="color: #888888;"># queueName: dotnetvideoqueue</span>
<span style="color: #888888;"># queueLength: '1'</span>
<span style="color: #888888;"># accountName: chvideodeveuw001queuest # storage-account-name</span>
<span style="color: #888888;"># # activationQueueLength: '50'</span>
<span style="color: #888888;"># authenticationRef:</span>
<span style="color: #888888;"># name: video-processor-queue-auth</span>
<span style="color: #888888;">#--------------</span>
</pre></div>
<p>Once we deploy the scale job we can find it is ready to create jobs if any messages received in the queue.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhzXXzbgwMKmTnXAPZUoz9OBdnmmzqDpsUAKwg1yqbh3l6KEoU-UApGloqlfeH7DHs6WCbX3ScVKOAWBgisGtmJ2EJKxKAVz3FCFYrde6mliCj17AC6-Cg14TWVzzzAPvNHcIcRvYg1hGB9WkoTwCMqF7uqM75OJmXfmnd4DNuXWJnpNc0pGgjfrQmiwjDs" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="5864" data-original-width="1936" height="1971" src="https://blogger.googleusercontent.com/img/a/AVvXsEhzXXzbgwMKmTnXAPZUoz9OBdnmmzqDpsUAKwg1yqbh3l6KEoU-UApGloqlfeH7DHs6WCbX3ScVKOAWBgisGtmJ2EJKxKAVz3FCFYrde6mliCj17AC6-Cg14TWVzzzAPvNHcIcRvYg1hGB9WkoTwCMqF7uqM75OJmXfmnd4DNuXWJnpNc0pGgjfrQmiwjDs=w648-h1971" width="648" /></a></div><br /><p></p><p>Once we send messages to the queue for example, <a href="https://github.com/chamindac/ffmpeg_aks/blob/main/src/servicebus.q.messagesender/Program.cs" target="_blank">the code here in GitHub can be used to send 5 messages to the Azure service bus queue </a>(the message content hardcoded for the demo purpose). KEDA will trigger the jobs as shown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjuJaEg8M4Pd62RlE_ONb9HyPd4KdMwp7TIP37GcEUVo0lGjdVlqRFWKYmveFPEcJ1XegP3ZZLxvCXrEPmBaS7BJGvIvtNKxgjjcAu0fusLZ0L06jz4i7eaVl8blaQqruND1gVxcrUY_VvNtylKGIIoZVPOJodx5OiS67gnqFRRzX6E-gUGSJV9scrS7fkt" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="881" data-original-width="1262" height="437" src="https://blogger.googleusercontent.com/img/a/AVvXsEjuJaEg8M4Pd62RlE_ONb9HyPd4KdMwp7TIP37GcEUVo0lGjdVlqRFWKYmveFPEcJ1XegP3ZZLxvCXrEPmBaS7BJGvIvtNKxgjjcAu0fusLZ0L06jz4i7eaVl8blaQqruND1gVxcrUY_VvNtylKGIIoZVPOJodx5OiS67gnqFRRzX6E-gUGSJV9scrS7fkt=w627-h437" width="627" /></a></div><br />Jobs should be executed to the completion as messages recived in the queue.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjw3UyAxIRuHHu1uaoRan2BdKXQPC_97lvZo0EkTsSWPlLL43guGnfHrwhRkiEjnS4QQ9CefSpuOY39uMrVoEWMRkFHFu5N1WcysdmEpXgoXcliL9KKOGhWpq2vUixYmgCnIQwdd2zFqYWS5kVJ41C5LTTStIXiFI55AJE7wLtWMlnC7anXfo5Oxs1p6Gue" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="881" data-original-width="1262" height="389" src="https://blogger.googleusercontent.com/img/a/AVvXsEjw3UyAxIRuHHu1uaoRan2BdKXQPC_97lvZo0EkTsSWPlLL43guGnfHrwhRkiEjnS4QQ9CefSpuOY39uMrVoEWMRkFHFu5N1WcysdmEpXgoXcliL9KKOGhWpq2vUixYmgCnIQwdd2zFqYWS5kVJ41C5LTTStIXiFI55AJE7wLtWMlnC7anXfo5Oxs1p6Gue=w558-h389" width="558" /></a></div><br /><a href="https://github.com/chamindac/ffmpeg_aks" target="_blank">Complete infrastructure deployment terraform, with AKS, workload identity and KEDA, and required deployment manifests to deploy apps and scaled job are available in the GitHub repo here</a>.<p></p><p>Now let's have a detailed look at the scaled job deployment YAML above.</p><p>The kid of deployment is a ScaledJob with API version set as keda.sh/v1alpha1. Notice that we have commented the annotation which can be used to disable the autoscaling of the job with KEDA.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">apiVersion</span>: <span style="color: #ce9178;">keda.sh/v1alpha1</span></div><div><span style="color: #569cd6;">kind</span>: <span style="color: #ce9178;">ScaledJob</span></div><div><span style="color: #569cd6;">metadata</span>:</div><div> <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">dotnet-video-processor-scaled-job</span></div><div> <span style="color: #569cd6;">namespace</span>: <span style="color: #ce9178;">media</span></div><div> <span style="color: #6a9955;"># annotations:</span></div><div> <span style="color: #6a9955;"># autoscaling.keda.sh/paused: true </span></div></div><p>Next important settings are job target reference. Here we define the requested parallelism as 1. However, this will not prevent the job from scaling based on the max scale setting we define later. We are defning that we need job completions to be successful if the pod (the pod one queued for message) is completed. Back off limit allows restart of the contianer in case of startup failures to be max count to 6 here, before deciding the pod has failed (job failed). The deadline below is allowing the pod to keep runing from start time the given number of seconds, befre kubernetes decides job is timed out and not cmpleted succefully. For long running jobs should be set to a larger value. Here it is set to more than 4 hous to accomdate of proessing of larger video files. </p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">jobTargetRef</span>:</div><div> <span style="color: #569cd6;">parallelism</span>: <span style="color: #b5cea8;">1</span> </div><div> <span style="color: #569cd6;">completions</span>: <span style="color: #b5cea8;">1</span> </div><div> <span style="color: #569cd6;">activeDeadlineSeconds</span>: <span style="color: #b5cea8;">15000</span> </div><div> <span style="color: #569cd6;">backoffLimit</span>: <span style="color: #b5cea8;">6</span> </div></div><p><br />Polling interval below says for each how many seconds the queue is checked for message availablility. Job hostory limits allow to keep the history of success and failed job pods, for inspection, if required. Since, we are using a single container defalt is fine for the source container. Min replica count set to 0 here so that no pods will run there is no messages. If there are 100 messages in the queue, max replicas count controls how many jobs running as maximum job count at agiven time. So, the 10 jobs at a time will be allowed to run and the next mssages will sart new pods as and when other jobs completed if there are large number of messages are still in the queue.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">pollingInterval</span>: <span style="color: #b5cea8;">5</span> </div><div><span style="color: #569cd6;">successfulJobsHistoryLimit</span>: <span style="color: #b5cea8;">20</span> </div><div><span style="color: #569cd6;">failedJobsHistoryLimit</span>: <span style="color: #b5cea8;">20</span> </div><div><span style="color: #6a9955;"># envSourceContainerName: {container-name} </span></div><div><span style="color: #569cd6;">minReplicaCount</span>: <span style="color: #b5cea8;">0</span> </div><div><span style="color: #569cd6;">maxReplicaCount</span>: <span style="color: #b5cea8;">10</span> </div></div><p>Rollout stategy is useful to define how the current running jobs would behave when a new job deployment spec is applied. The default k8s behaviour of terminating existing pods when new ones with new spec appear, should not happen here. So we are setting it to gradual strategy and the KEDA will let the existing running jobs to complete. We are choosing propagation policy as foreground, instead of default background to ensure cleanup of all objects when pods are deleted.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">rollout</span>:</div><div> <span style="color: #569cd6;">strategy</span>: <span style="color: #ce9178;">gradual</span> </div><div> <span style="color: #569cd6;">propagationPolicy</span>: <span style="color: #ce9178;">foreground</span> </div></div><p>As scaling strategy we are using the default scaling strategy of KEDA. If you need to cange the behavoir you can change strategy to custom and play around with the other commented settings.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> </div><div><span style="color: #569cd6;">scalingStrategy</span>:</div><div> <span style="color: #569cd6;">strategy</span>: <span style="color: #ce9178;">"default"</span> </div><div> <span style="color: #6a9955;"># customScalingQueueLengthDeduction: 1 </span></div><div> <span style="color: #6a9955;"># customScalingRunningJobPercentage: "0.5" </span></div><div> <span style="color: #6a9955;"># pendingPodConditions: </span></div><div> <span style="color: #6a9955;"># - "Ready"</span></div><div> <span style="color: #6a9955;"># - "PodScheduled"</span></div><div> <span style="color: #6a9955;"># - "Pending"</span></div><div> <span style="color: #6a9955;"># - "ContainerCreating"</span></div><div> <span style="color: #6a9955;"># multipleScalersCalculation : "max" # Optional. Default: max. </span></div></div><p>For the trigger we use the service bus queue as trigger. The <a href="https://chamindac.blogspot.com/2024/01/setting-up-keda-authentication-trigger.html" target="_blank">trigger authentication we have created with Azure worload identity</a> is used in this trigger.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">triggers</span>:</div><div> - <span style="color: #569cd6;">type</span>: <span style="color: #ce9178;">azure-servicebus</span></div><div> <span style="color: #569cd6;">metadata</span>:</div><div> <span style="color: #569cd6;">queueName</span>: <span style="color: #ce9178;">dotnetvideoqueue</span></div><div> <span style="color: #569cd6;">namespace</span>: <span style="color: #ce9178;">ch-video-dev-euw-001-sbus-blue</span></div><div> <span style="color: #569cd6;">messageCount</span>: <span style="color: #ce9178;">"1"</span></div><div> <span style="color: #569cd6;">authenticationRef</span>:</div><div> <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">video-processor-queue-auth</span></div></div><p>For further information look at <a href="https://keda.sh/docs/2.13/concepts/scaling-jobs/" target="_blank">KEDA documentation for scaled jobs</a> here.</p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-76888029682720154912024-01-13T09:38:00.000-08:002024-03-10T10:10:36.314-07:00Setting Up (KEDA) Authentication Trigger for Azure Storage Queue/Service Bus in AKS<p>We have discussed setting up Kubernetes Event Drivern Autoscaling (KEDA) with AKS workload identity in the post, "<a href="https://chamindac.blogspot.com/2023/11/setting-up-keda-in-aks.html" target="_blank">Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity</a>". Purpose of KEDA is to once we receive messages in a queue, such as Azure storage queue or Azure service bus queue we have to scale a scaledjob/deployment in kubernetes.</p><p>To setup authentication for the KEDA to communicate and monitor such a queue to scale a job or deployment, it should authentication to access the queue. We can set up the required authentication using using connnection strings for Azure <a href="https://keda.sh/docs/2.13/scalers/azure-service-bus/#authentication-parameters" target="_blank">service bus</a> or <a href="https://keda.sh/docs/2.13/scalers/azure-storage-queue/#authentication-parameters" target="_blank">storage queue</a> . Instead of using such connection strings or shared access keys we can authenticate to the queue using the workload identity, since we have already enabled wrkload identity in KEDA as described in "<a href="https://chamindac.blogspot.com/2023/11/setting-up-keda-in-aks.html" target="_blank">Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity</a>".<span></span></p><a name='more'></a><p></p><p>A triger authentication for KEDA with workload identity can be specified as shown below in a yaml file let's say k8s.yaml. We need to ensure the trigger authntication is defined in the same namespace which we are going to later deploy a scaled job or a scaled deployment. KEDA is deployed at keda namespace and the trigger authentication does not need to be in that namespace.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #0e84b5; font-weight: bold;">---</span>
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: video-processor-queue-auth
namespace: media
spec:
podIdentity:
provider: azure-workload
</pre></div>
<p>Then we can apply the above trigger with <span style="font-family: courier;">kubectl apply -f k8s.yaml</span>. Once we applied the we can see it is created as shown below. In order for the successful authentication to the queue, the storage queue or service bus queue should have required permision granted to the user assigned identity federated for workload identity as described in the "<a href="https://chamindac.blogspot.com/2023/11/setting-up-keda-in-aks.html" target="_blank">Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity</a>".</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgwPC9KGDmm6oh054apodD2tMMNeqfaIQfu0YQHyssNHWK_5ewumN-ETYhalLS5i3sNtGXpk3ARCqZhO8TwfDUNp_O63Ho6uCI1_W6hECRuu-7Ag5ek5MBO-4iddyRD3D4dG-yBcAAFOERiXBy8s61jBxF4ekrZZwq3NLz3IPQcjSquWTYoBLZz3yEFdXDC" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="2095" data-original-width="1721" height="749" src="https://blogger.googleusercontent.com/img/a/AVvXsEgwPC9KGDmm6oh054apodD2tMMNeqfaIQfu0YQHyssNHWK_5ewumN-ETYhalLS5i3sNtGXpk3ARCqZhO8TwfDUNp_O63Ho6uCI1_W6hECRuu-7Ag5ek5MBO-4iddyRD3D4dG-yBcAAFOERiXBy8s61jBxF4ekrZZwq3NLz3IPQcjSquWTYoBLZz3yEFdXDC=w614-h749" width="614" /></a></div><br />In the <a href="https://chamindac.blogspot.com/2024/01/scale-pod-in-aks-with-kubernetes-event.html" target="_blank">next post let's discuss how to scale a kubernetes scaled job</a> with Azure service bus trigger utilizing the trigger authentication with workload identity, that we have created as shown above.<p></p><p><br /></p><p><br /><br /></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-33883187773318556272024-01-06T05:31:00.000-08:002024-02-12T08:36:11.712-08:00Setting Up Kubernetes Event Drivern Autoscaling (KEDA) in AKS with Workload Identity<p> We have discuss setting up workload identity in AKS to be used with application containers we deploy to AKS in the post "<a href="https://chamindac.blogspot.com/2023/12/setting-up-azure-workload-identity-for.html" target="_blank">Setting Up Azure Workload Identity for Containers in Azure Kubernetes Services (AKS) Using Terra- Improved Security for Contianers in AKS</a>". Kubernetes Event Drivern Autoscaling (KEDA) is the mechanism we need to use when we want to scale our deployments, or specially kubernetes jobs (<a href="https://kubebyexample.com/concept/jobs" target="_blank">A pod that runs for completion</a>). In this post let's look at how to setup KEDA with workload identity, so that we can use KEDA in a later posts to run a Kubernets job, autoscaled based on the messages received to a storage queue or Azure service bus.<span></span></p><a name='more'></a><p></p><p>To get KEDA setup on AKS we can use helm. We have discussed <a href="https://chamindac.blogspot.com/2023/12/setting-up-helm-in-wsl.html" target="_blank">setting up helm in WSL in the post here</a>. As the first step we need to setup federated identity credential with the user assigned identity used for AKS, which we have discussed in detail in "<a href="https://chamindac.blogspot.com/2023/12/setting-up-azure-workload-identity-for.html" target="_blank">Setting Up Azure Workload Identity for Containers in Azure Kubernetes Services (AKS) Using Terra- Improved Security for Contianers in AKS</a>". Note that the namespace we are using for Keda is <span style="font-family: courier;">keda </span>and the service account we are going to setup later is named as <span style="font-family: courier;">keda-operator</span>.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;"># Federated identity credential for AKS user assigned id - used with workload identity service account for KEDA</span>
resource "azurerm_federated_identity_credential" "keda" {
name = "${var.prefix}-${var.project}-${var.environment_name}-aks-keda-fic-${var.deployment_name}"
resource_group_name = var.rg_name
audience = ["api://AzureADTokenExchange"]
issuer = azurerm_kubernetes_cluster.aks_cluster.oidc_issuer_url <span style="color: #888888;"># Open id connect issue url from AKS</span>
parent_id = var.user_assigned_identity <span style="color: #888888;"># user assigned identity id (Azure resource id)</span>
subject = "system:serviceaccount:keda:keda-operator" <span style="color: #888888;"># system:serviceaccount:aksapplicationnamespace:workloadidentityserviceaccountname (to be created after AKS cluster is setup)</span>
depends_on = [
azurerm_kubernetes_cluster.aks_cluster
]
lifecycle {
ignore_changes = []
}
}
</pre></div>
<p>Terraform will setup federated identity credential as shown below.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEip_TeOUutwSN0et4iBqVVUHHIkW3d5yLb6W9-6-_4l7g-k3XJX3x_f5oXBfqMCsOeQ73T8B8XJ_rpcQn_QtarAH0htyCG7e7rNFmBXIgQA5_GIwb_OtkotUtoFzEhHb6YEQcpn2aq1jILj_9AKV7UoegUsJDDLQoM0bZvqQfUKhezSs00kboFBdREzUz9S" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1503" data-original-width="1985" height="430" src="https://blogger.googleusercontent.com/img/a/AVvXsEip_TeOUutwSN0et4iBqVVUHHIkW3d5yLb6W9-6-_4l7g-k3XJX3x_f5oXBfqMCsOeQ73T8B8XJ_rpcQn_QtarAH0htyCG7e7rNFmBXIgQA5_GIwb_OtkotUtoFzEhHb6YEQcpn2aq1jILj_9AKV7UoegUsJDDLQoM0bZvqQfUKhezSs00kboFBdREzUz9S=w568-h430" width="568" /></a></div><br /><p></p><p>Next we need to setup namespace <span style="font-family: courier;">keda </span>once the cluster is up and running. We can use kubectl to apply below yaml to get the namespace <span style="font-family: courier;">keda </span>created.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">apiVersion</span>: <span style="color: #ce9178;">v1</span></div><div><span style="color: #569cd6;">kind</span>: <span style="color: #ce9178;">Namespace</span></div><div><span style="color: #569cd6;">metadata</span>:</div><div> <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">keda</span></div></div><p>We need to setup the service account as well using below yaml. Replace ${sys_aks_uai_client_id}$ in the yaml to the client id of the user assigned identity used for AKS. Replace ${tenantid}$ with the Azure tenant id.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: ${sys_aks_uai_client_id}$ <span style="color: #888888;"># ${sys_aks_uai_client_id}$</span>
azure.workload.identity/tenant-id: ${tenantid}$ <span style="color: #888888;"># "${tenantid}$"</span>
name: keda-operator <span style="color: #888888;"># Referred by AKS user assigned identity federated credential</span>
namespace: keda
</pre></div>
<p>We can get the user assigned identity from terraform using output variables as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_user_assigned_identity"</span> <span style="color: #4fc1ff;">"aks"</span> {</div><div> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">ENVNAME</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-aks-uai</span><span style="color: #ce9178;">"</span></div><div> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div>}</div></div><p><br /></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">output</span> <span style="color: #4fc1ff;">"aks_uai_client_id"</span> {</div><div> <span style="color: #9cdcfe;">value</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_user_assigned_identity</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">aks</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">client_id</span></div><div>}</div></div><p>Then we can use below .sh file in WSL to run helm install for KEDA. Again replace ${sys_aks_uai_client_id}$ and ${tenantid}$ in the .sh file. </p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #dcdcaa;">helm</span> <span style="color: #ce9178;">repo</span> <span style="color: #ce9178;">add</span> <span style="color: #ce9178;">kedacore</span> <span style="color: #ce9178;">https://kedacore.github.io/charts</span></div><div><span style="color: #dcdcaa;">helm</span> <span style="color: #ce9178;">repo</span> <span style="color: #ce9178;">update</span></div><br /><div><span style="color: #dcdcaa;">helm</span> <span style="color: #ce9178;">install</span> <span style="color: #ce9178;">keda</span> <span style="color: #ce9178;">kedacore/keda</span> <span style="color: #569cd6;">--namespace</span> <span style="color: #ce9178;">keda</span> <span style="color: #d7ba7d;">\</span></div><div>--set <span style="color: #ce9178;">serviceAccount.create=</span><span style="color: #569cd6;">false</span> <span style="color: #d7ba7d;">\</span></div><div>--set <span style="color: #ce9178;">serviceAccount.name=keda-operator</span> <span style="color: #d7ba7d;">\</span></div><div>--set <span style="color: #ce9178;">podIdentity.azureWorkload.enabled=</span><span style="color: #569cd6;">true</span> <span style="color: #d7ba7d;">\</span></div><div>--set <span style="color: #ce9178;">podIdentity.azureWorkload.clientId=</span>${<span style="color: #9cdcfe;">sys_aks_uai_client_id</span>}$ <span style="color: #d7ba7d;">\</span></div><div>--set <span style="color: #ce9178;">podIdentity.azureWorkload.tenantId=</span>${<span style="color: #9cdcfe;">tenantid</span>}$</div></div><p>As you can see in above we are setting up Keda in <span style="font-family: courier;">keda </span>namespace. We are requesting to use the service account we have created previously, instead of creating a new one as the service account we have created correctly setup with the user assigned identity of AKS as workload identity.</p><p>With this our intial setup requirement for KEDA is completed and we can see below deployments running in keda namespace.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi0Wm-IWyp1OluHMj2esLrKotiGQ-rtE0YwVn7FVJhdgwRrapdtlp3lmYOv_rJ0_vblv0BB7W12uDByZAf-iUv3ODUoUc1xKWxzaCSHacSptEbo0aCrZR5iIhaIqPrupTUq3ZC4PheGSSOD-HF3JtBbNZUHDXVRtyQYsFfnV-3uSrTumH7TpxBnv7gizQy7" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="843" data-original-width="1831" height="282" src="https://blogger.googleusercontent.com/img/a/AVvXsEi0Wm-IWyp1OluHMj2esLrKotiGQ-rtE0YwVn7FVJhdgwRrapdtlp3lmYOv_rJ0_vblv0BB7W12uDByZAf-iUv3ODUoUc1xKWxzaCSHacSptEbo0aCrZR5iIhaIqPrupTUq3ZC4PheGSSOD-HF3JtBbNZUHDXVRtyQYsFfnV-3uSrTumH7TpxBnv7gizQy7=w613-h282" width="613" /></a></div><br /><br /><p></p><p>The keda-oprator and keda-oprator-metrics-apiserver containers will have folowing shown environment variables injected, so that the scaling with Aure service bus queue or Azure storage queue can be done with KEDA autorizing with <a href="https://azure.github.io/azure-workload-identity/docs/concepts.html" target="_blank">Azure workload identity for AKS</a>.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiYbIPZnuBpA0Tz_nQj9Tjl4ZsoNnzVXb44dO35YTXmbMuTVJl48XNbg7dbb0RdrazfUB2jvcSDgmpUpgb6-Zkt1gNl4LloQa3D7zDz0TqLl7C1a4oLl5C3NPA4j6oAdom5fSK5NDSp1_k1nzuwCgIwBliWgdY_H8EIlRfPdBHmjkvnRt-4JtD6Z9NsM1Ku" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1455" data-original-width="2109" height="430" src="https://blogger.googleusercontent.com/img/a/AVvXsEiYbIPZnuBpA0Tz_nQj9Tjl4ZsoNnzVXb44dO35YTXmbMuTVJl48XNbg7dbb0RdrazfUB2jvcSDgmpUpgb6-Zkt1gNl4LloQa3D7zDz0TqLl7C1a4oLl5C3NPA4j6oAdom5fSK5NDSp1_k1nzuwCgIwBliWgdY_H8EIlRfPdBHmjkvnRt-4JtD6Z9NsM1Ku=w623-h430" width="623" /></a></div><p><br /></p>For storage queue based KEDA scaling we need user assigned identity assigned to storage queue as shown below.<p></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_account"</span> <span style="color: #4fc1ff;">"queue"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">queuest</span><span style="color: #ce9178;">"</span></div><div> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div> <span style="color: #9cdcfe;">account_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Standard"</span></div><div> <span style="color: #9cdcfe;">account_replication_type</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"LRS"</span></div><div> <span style="color: #9cdcfe;">account_kind</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"StorageV2"</span></div><div> <span style="color: #9cdcfe;">access_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Hot"</span></div><div> <span style="color: #9cdcfe;">allow_nested_items_to_be_public</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><div> <span style="color: #9cdcfe;">min_tls_version</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"TLS1_2"</span></div><div> <span style="color: #9cdcfe;">cross_tenant_replication_enabled</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><div>}</div><br /><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_queue"</span> <span style="color: #4fc1ff;">"video"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"demovideoqueue"</span></div><div> <span style="color: #9cdcfe;">storage_account_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_storage_account</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">queue</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div>}</div><br /><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_queue"</span> <span style="color: #4fc1ff;">"dotnet_video"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"dotnetvideoqueue"</span></div><div> <span style="color: #9cdcfe;">storage_account_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_storage_account</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">queue</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div>}</div><br /><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_role_assignment"</span> <span style="color: #4fc1ff;">"storage_q_contributor"</span> {</div><div> <span style="color: #9cdcfe;">principal_id</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_user_assigned_identity</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">aks</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">principal_id</span></div><div> <span style="color: #9cdcfe;">role_definition_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Storage Queue Data Contributor"</span></div><div> <span style="color: #9cdcfe;">scope</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_storage_account</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">queue</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">skip_service_principal_aad_check</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">true</span></div><div>}</div></div><p><br /></p><p>For service bus it should be as below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_servicebus_namespace"</span> <span style="color: #4fc1ff;">"demo"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">ENVNAME</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-sbus</span><span style="color: #ce9178;">"</span></div><div> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">sku</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Standard"</span></div><br /><div> <span style="color: #9cdcfe;">tags</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #dcdcaa;">merge</span>(<span style="color: #dcdcaa;">tomap</span>({</div><div> <span style="color: #9cdcfe;">Service</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"service_bus"</span></div><div> }), <span style="color: #9cdcfe;">local</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">tags</span>)</div><div>}</div><br /><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_servicebus_queue"</span> <span style="color: #4fc1ff;">"dotnetvideo"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"dotnetvideoqueue"</span></div><div> <span style="color: #9cdcfe;">namespace_id</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_servicebus_namespace</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">demo</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">lock_duration</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"PT5M"</span> </div><div> <span style="color: #9cdcfe;">requires_duplicate_detection</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">true</span></div><div> <span style="color: #9cdcfe;">enable_partitioning</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">true</span></div><div>}</div><br /><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_role_assignment"</span> <span style="color: #4fc1ff;">"sbq_contributor"</span> {</div><div> <span style="color: #9cdcfe;">principal_id</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_user_assigned_identity</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">aks</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">principal_id</span></div><div> <span style="color: #9cdcfe;">role_definition_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Azure Service Bus Data Owner"</span></div><div> <span style="color: #9cdcfe;">scope</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_servicebus_queue</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">dotnetvideo</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">skip_service_principal_aad_check</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">true</span></div><div>}</div></div><p><br /></p><p><br /></p><p><br /></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-40863606077973271862023-12-30T04:04:00.000-08:002024-03-10T09:56:18.261-07:00Azure File Share with DefaultAzureCredential in .NET with Azure.Storage.Files.Shares - Is it possible?<p> Using <span style="font-family: courier;">DefaultAzureCredential</span> with most of the Azure resources is straight forward and simple with most of the Azure resources with relevant Azure .NET SDKs (We can use nuget packages <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Storage.Blobs</span> and <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Identity</span>). For example, with storage blob we can easily use <span style="font-family: courier;">DefaultAzureCredential </span>as shown in below code.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> <span style="color: #569cd6;">private</span> <span style="color: #569cd6;">static</span> <span style="color: #4ec9b0;">BlobServiceClient</span> <span style="color: #dcdcaa;">GetBlobServiceClient</span>(<span style="color: #569cd6;">string</span> <span style="color: #9cdcfe;">accountName</span>)</div><div> {</div><div> <span style="color: #c586c0;">return</span> <span style="color: #569cd6;">new</span>(<span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">Uri</span>(<span style="color: #ce9178;">$"https://{</span><span style="color: #9cdcfe;">accountName</span><span style="color: #ce9178;">}.blob.core.windows.net"</span>),</div><div> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">DefaultAzureCredential</span>());</div><div> }</div></div><p>However, we cannot simply create <span style="background-color: #1f1f1f; color: #4ec9b0; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">ShareClient</span><span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> </span><a href="https://learn.microsoft.com/en-us/azure/storage/files/storage-dotnet-how-to-use-files?wt.mc_id=DT-MVP-5000590" target="_blank">with .NET SDK <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Storage.Files.Shares </span>to</a> use <span style="font-family: courier;">DefaultAzureCredential</span> . as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">ShareClient</span> <span style="color: #9cdcfe;">share</span> <span style="color: #d4d4d4;">=</span> <span style="color: #569cd6;">new</span>(<span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">Uri</span>(<span style="color: #9cdcfe;">fileShareUri</span>),</div><div> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">DefaultAzureCredential</span>());</div></div><p>With the above setup, we will get runtime errors when we try to perform operations with the Azure file share. As <a href="https://github.com/Azure/azure-sdk-for-net/issues/17000" target="_blank">per the GitHub issue here it is not possible to use DefaultAzureCredential with Azure File Share with .NET SDK </a><span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Storage.Files.Shares </span> due to "SMB Files cannot authenticate with a TokenCredential". So is it impossible to use <span style="font-family: courier;">DefaultAzureCredential </span>to perform operations with an Azure File Share using <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Storage.Files.Shares </span>? Let's look at a wokaround, which can help if desperately need to use <span style="font-family: courier;">DefaultAzureCredential </span>with <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.Storage.Files.Shares </span>.<span></span></p><a name='more'></a><p></p><p>We will have to use <span style="background-color: #1f1f1f; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Azure.ResourceManager.Storage</span> and use it with <span style="font-family: courier;">DefaultAzureCredential </span>to obtain the storage key and then create a <span style="background-color: #1f1f1f; color: #4ec9b0; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">StorageSharedKeyCredential </span>to access the file share. This is not an ideal solution, but it makes it possible, to not to store, the access key or connection string of storage account in the application config or in a key vault and use <span style="font-family: courier;">DefaultAzureCredential</span> to access an Azure file share with .NET SDK for Azure files .</p><p>So what we need to achive this?</p><p>Use below nuget packages </p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: grey;"><</span><span style="color: #569cd6;">PackageReference</span> <span style="color: #9cdcfe;">Include</span>=<span style="color: #ce9178;">"Azure.Identity"</span> <span style="color: #9cdcfe;">Version</span>=<span style="color: #ce9178;">"1.10.4"</span> <span style="color: grey;">/></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">PackageReference</span> <span style="color: #9cdcfe;">Include</span>=<span style="color: #ce9178;">"Azure.ResourceManager.Storage"</span> <span style="color: #9cdcfe;">Version</span>=<span style="color: #ce9178;">"1.2.0"</span> <span style="color: grey;">/></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">PackageReference</span> <span style="color: #9cdcfe;">Include</span>=<span style="color: #ce9178;">"Azure.Storage.Files.Shares"</span> <span style="color: #9cdcfe;">Version</span>=<span style="color: #ce9178;">"12.17.1"</span> <span style="color: grey;">/></span></div></div><p>Then, obtain the Azure file share storage account resource ID and use <span style="background-color: #1f1f1f; color: #4ec9b0; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">ArmClient </span> with <span style="font-family: courier;">DefaultAzureCredential</span> to recive the storage access key. Then, use a <span style="background-color: #1f1f1f; color: #4ec9b0; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">StorageSharedKeyCredential </span> and create the <span style="background-color: #1f1f1f; color: #4ec9b0; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">ShareClient</span><span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> </span> .</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg9oh2bdswKiw6ggKg7P8uHtheZnaA1kREDXwd7aEh0rqvos7AgyH9L4OhWrMxGnWDQ5tqLEj0xDEhhgzcDmYEs7w8gCy8BOq4FoZ6fGIZCiORV6SeBe5A_rMa-DgbfHvGbOqCsKoZx8xBLnn8aKB6CqUMqgihWYHUcNU7SXo1njkxoFjcxdPGMdFDHKcvD" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="956" data-original-width="2202" height="272" src="https://blogger.googleusercontent.com/img/a/AVvXsEg9oh2bdswKiw6ggKg7P8uHtheZnaA1kREDXwd7aEh0rqvos7AgyH9L4OhWrMxGnWDQ5tqLEj0xDEhhgzcDmYEs7w8gCy8BOq4FoZ6fGIZCiORV6SeBe5A_rMa-DgbfHvGbOqCsKoZx8xBLnn8aKB6CqUMqgihWYHUcNU7SXo1njkxoFjcxdPGMdFDHKcvD=w626-h272" width="626" /></a></div><br /><p></p><p>Full example code is shown below.</p><p><br /></p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #557799;">#region AzureFileShare SDK with Default Azure Creds Try</span>
<span style="color: #888888;">//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span>
<span style="color: #888888;">//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span>
<span style="color: #888888;">// Example of Azure file share client SDK try with default Azure creds</span>
<span style="color: #888888;">//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span>
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Identity</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Storage.Files.Shares.Models</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Storage.Files.Shares</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Storage</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.ResourceManager</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.ResourceManager.Storage</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Core</span>;
<span style="color: #888888;">// -------------This is using connection string we know it works--------------</span>
<span style="color: #888888;">//string connectionString = "DefaultEndpointsProtocol=https;AccountName=chazfsdeveuw001fsst;AccountKey=storagekey;EndpointSuffix=core.windows.net";</span>
<span style="color: #888888;">//string shareName = "aksfileshare";</span>
<span style="color: #888888;">//ShareClient share = new(connectionString, shareName);</span>
<span style="color: #888888;">// ---------------------------------------------------------------------------</span>
<span style="color: #888888;">//====================================================================</span>
<span style="color: #888888;">// Below is the only way to use default Azure creds to work with file share</span>
<span style="color: #888888;">// SMB Files cannot authenticate with a TokenCredential refer</span>
<span style="color: #888888;">// https://github.com/Azure/azure-sdk-for-net/issues/17000</span>
<span style="color: #888888;">//--------------------------------------------------------------------</span>
ArmClient client = <span style="color: #008800; font-weight: bold;">new</span> ArmClient(<span style="color: #008800; font-weight: bold;">new</span> DefaultAzureCredential());
<span style="color: #333399; font-weight: bold;">string</span> resourceId = <span style="background-color: #fff0f0;">"/subscriptions/subscriptionid/resourceGroups/ch-azfs-dev-euw-001-rg/providers/Microsoft.Storage/storageAccounts/chazfsdeveuw001fsst"</span>;
StorageAccountResource storageAccount = client.GetStorageAccountResource(<span style="color: #008800; font-weight: bold;">new</span> ResourceIdentifier(resourceId));
<span style="color: #333399; font-weight: bold;">string</span> storageKey = storageAccount.GetKeys().FirstOrDefault().Value;
<span style="color: #333399; font-weight: bold;">string</span> fileShareUri = <span style="background-color: #fff0f0;">"https://chazfsdeveuw001fsst.file.core.windows.net/aksfileshare"</span>;
ShareClient share = <span style="color: #008800; font-weight: bold;">new</span>(<span style="color: #008800; font-weight: bold;">new</span> Uri(fileShareUri),
<span style="color: #008800; font-weight: bold;">new</span> <span style="color: #0066bb; font-weight: bold;">StorageSharedKeyCredential</span>(<span style="background-color: #fff0f0;">"chazfsdeveuw001fsst"</span>, storageKey));
<span style="color: #888888;">//====================================================================</span>
<span style="color: #888888;">// Track the remaining directories to walk, starting from the root</span>
<span style="color: #333399; font-weight: bold;">var</span> remaining = <span style="color: #008800; font-weight: bold;">new</span> Queue<ShareDirectoryClient>();
remaining.Enqueue(share.GetRootDirectoryClient());
<span style="color: #008800; font-weight: bold;">while</span> (remaining.Count > <span style="color: #6600ee; font-weight: bold;">0</span>)
{
<span style="color: #888888;">// Get all of the next directory's files and subdirectories</span>
ShareDirectoryClient dir = remaining.Dequeue();
<span style="color: #008800; font-weight: bold;">foreach</span> (ShareFileItem item <span style="color: #008800; font-weight: bold;">in</span> dir.GetFilesAndDirectories())
{
<span style="color: #888888;">// Print the name of the item</span>
Console.WriteLine(item.Name);
<span style="color: #888888;">// Keep walking down directories</span>
<span style="color: #008800; font-weight: bold;">if</span> (item.IsDirectory)
{
remaining.Enqueue(dir.GetSubdirectoryClient(item.Name));
}
}
}
<span style="color: #888888;">//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span>
<span style="color: #557799;">#endregion</span>
</pre></div>
<p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-75827683118140996762023-12-16T06:34:00.000-08:002024-02-04T07:20:04.831-08:00Setting Up Azure Workload Identity for Containers in Azure Kubernetes Services (AKS) Using Terraform - Improved Security for Containers in AKS<p> <a href="https://azure.github.io/azure-workload-identity/docs/" target="_blank">Azure Workload Identity</a> allows your containers in AKS touse amanaged identity to access Azure resources securely without having to depend on connection strings, passwords, access keys or secrets. In other works you can just use <a href="https://learn.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?wt.mc_id=DT-MVP-5000590&view=azure-dotnet#defaultazurecredential" target="_blank">DefaultAzureCredential</a> in your containers running in AKS, which will be using workload identity assigned to the container, to get access to the required Azure resource. The roale based access permissions will be in effect and the user assigned managed identity (we can use AD app registration as well bu user assigned managed identity is recommended) used to setup the workload identity in AKS should be given the necessary roles in the target Azure resource. This is far better than having to store secrets or connection stigs to utilized by the dotnet applications. In this post let's understand how to setup workload identity in AKS deployed containers and explore how it simplifies the dotnet application code allowing the application to access Azure resources securely with a managed identity.</p><p><a href="https://github.com/chamindac/aks_workloadidentity" target="_blank">Full example source code with terraform and a .NET application using default credentials to access app config service and keyvault is available here in my GitHub repo</a>,<span></span></p><a name='more'></a><p></p><p>The first step of setting up the workload identity in AKS is to enable the OIDC (open id conectivity) issuer and workload identity. In terraform resource <span face="ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace" style="background-color: rgba(175, 184, 193, 0.2); color: #1f2328; font-size: 13.6px; white-space-collapse: break-spaces;"><a href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster" target="_blank">azurerm_kubernetes_cluster</a>.</span></p><pre class="notranslate" style="background-color: #f6f8fa; border-radius: 6px; box-sizing: border-box; color: #1f2328; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 13.6px; line-height: 1.45; margin-bottom: 0px; margin-top: 0px; overflow-wrap: normal; overflow: auto; padding: 16px;"><code style="background: transparent; border-radius: 6px; border: 0px; box-sizing: border-box; display: inline; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 13.6px; line-height: inherit; margin: 0px; overflow-wrap: normal; overflow: visible; padding: 0px; word-break: normal;"> oidc_issuer_enabled = true
workload_identity_enabled = true</code></pre><p>Example AKS cluster tf code is below</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">resource "azurerm_kubernetes_cluster" "aks_cluster" {
lifecycle {
ignore_changes = [default_node_pool[0].node_count]
}
name = "${var.prefix}-${var.project}-${var.environment_name}-aks-${var.deployment_name}"
kubernetes_version = local.kubernetes_version
sku_tier = "Standard"
location = var.location
resource_group_name = var.rg_name
dns_prefix = "${var.prefix}-${var.project}-${var.environment_name}-aks-${var.deployment_name}-dns"
node_resource_group = "${var.prefix}-${var.project}-${var.environment_name}-aks-${var.deployment_name}-rg"
image_cleaner_enabled = false <span style="color: #888888;"># As this is a preview feature keep it disabled for now. Once feture is GA, it should be enabled.</span>
image_cleaner_interval_hours = 48
network_profile {
network_plugin = "azure"
load_balancer_sku = "standard"
}
storage_profile {
file_driver_enabled = true
}
default_node_pool {
name = "chlinux"
orchestrator_version = local.kubernetes_version
node_count = 1
enable_auto_scaling = true
min_count = 1
max_count = 7
vm_size = "Standard_DS4_v2"
os_sku = "Ubuntu"
vnet_subnet_id = var.subnet_id
max_pods = 30
type = "VirtualMachineScaleSets"
scale_down_mode = "Delete"
zones = ["1", "2", "3"]
}
oidc_issuer_enabled = true <span style="color: #888888;"># Allow creating open id connect issue url to be used in federated identity credential</span>
workload_identity_enabled = true <span style="color: #888888;"># Enable workload identity in AKS</span>
identity {
type = "SystemAssigned"
}
ingress_application_gateway {
gateway_id = azurerm_application_gateway.aks.id
}
key_vault_secrets_provider {
secret_rotation_enabled = false
}
azure_active_directory_role_based_access_control {
azure_rbac_enabled = false
managed = true
tenant_id = var.tenant_id
# add sub owners as cluster admin
admin_group_object_ids = [
var.sub_owners_objectid] <span style="color: #888888;"># azure AD group object ID</span>
<span style="background-color: #ffaaaa; color: red;">}</span>
oms_agent {
log_analytics_workspace_id = var.log_analytics_workspace_id
}
depends_on = [
azurerm_application_gateway.aks
]
tags = merge(tomap({
Service = "aks_cluster"
}), var.tags)
}
</pre></div>
<p>As the next step we hve to setup a user assigned managed identity, which will be used as workload identity in AKS.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># User assigned identity to use as workload identity in AKS</span></div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_user_assigned_identity"</span> <span style="color: #4fc1ff;">"aks"</span> {</div><div> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">ENVNAME</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-aks-uai</span><span style="color: #ce9178;">"</span></div><div> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div>}</div></div><p>Then we need to setup a federated identity credential for user assigned identity for the AKS service account which will be used to assigned the identity for each pod. We need to use OIDC issuer url of the AKS cluster in the federated identity credential setup. To understand the <a href="https://azure.github.io/azure-workload-identity/docs/concepts.html" target="_blank">concepts in detail read the docs here</a>.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #888888;"># Federated identity credential for AKS user assigned id - to be used with workload identity service account</span>
resource "azurerm_federated_identity_credential" "aks" {
name = "${var.prefix}-${var.project}-${var.environment_name}-aks-fic-${var.deployment_name}"
resource_group_name = var.rg_name
audience = ["api://AzureADTokenExchange"]
issuer = azurerm_kubernetes_cluster.aks_cluster.oidc_issuer_url
parent_id = var.user_assigned_identity
subject = "system:serviceaccount:widemo:wi-demo-sa" <span style="color: #888888;"># system:serviceaccount:aksapplicationnamespace:workloadidentityserviceaccountname</span>
depends_on = [
azurerm_kubernetes_cluster.aks_cluster
]
lifecycle {
ignore_changes = []
}
}
</pre></div>
<p><br /></p><p>We can use terraform output to obtain the client id of the user assigned managed identity we created. This is required for later use with the service account creation.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">output</span> <span style="color: #4fc1ff;">"aks_uai_client_id"</span> {</div><div> <span style="color: #9cdcfe;">value</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_user_assigned_identity</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">aks</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">client_id</span></div><div>}</div></div><p>Once the terrafomr code is deployed and the AKS cluster is created we can use kubectl to create the service account as shown below.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: <span style="background-color: #fff0f0;">userassignedidentitycientid</span> <span style="color: #888888;">#${USER_ASSIGNED_CLIENT_ID}$ # user Assigned identity client ID (aks_uai_client_id output from Terraform)</span>
azure.workload.identity/tenant-id: <span style="background-color: #fff0f0;">tenantid</span> <span style="color: #888888;">#${AZURE_TENANT_ID}$ # Azure tenant id</span>
<span style="color: #888888;"># azure.workload.identity/service-account-token-expiration: "3600" # Default is 3600. Supported range is 3600-86400. Configure to avoid down time in token refresh. Setting in Pod spec takes precedence.</span>
name: wi-demo-sa
namespace: widemo
</pre></div>
<p>We can deploy our application pods enabling use of workload identity as shown below in pod template.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">template:
metadata:
labels:
app: wi-api
service: wi-api
azure.workload.identity/use: <span style="background-color: #fff0f0;">"true"</span> <span style="color: #888888;"># Required to make the contianers in the pod to use the workload identity</span>
<span style="color: #888888;"># annotations:</span>
<span style="color: #888888;"># azure.workload.identity/service-account-token-expiration: "3600" # Configure to avoid down time in token refresh. Takes precedence over servie acount setting. Default 3600, acceptable range: seconds 3600 - 86400.</span>
<span style="color: #888888;"># azure.workload.identity/skip-containers: "container1:container2" # Containers o skip using workload identity. By default all containers in pod will use workload identity when pod is labeled with azure.workload.identity/use: true </span>
<span style="color: #888888;"># azure.workload.identity/inject-proxy-sidecar: "true" # Default true. The proxy sidecar is used to intercept token requests to IMDS (Azure Instance Metadata Service) and acquire an AAD token on behalf of the user with federated identity credential.</span>
<span style="color: #888888;"># azure.workload.identity/proxy-sidecar-port: "8000" # Port of the proxy sidecar. Default 8000</span>
spec:
serviceAccountName: wi-demo-sa <span style="color: #888888;"># Service account (see aks_manifests\prerequisites\k8s.yaml) will provide identity to the pod https://azure.github.io/azure-workload-identity/docs/concepts.html</span>
</pre></div>
<p>A full example deployment is below.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">apiVersion: apps/v1
kind: Deployment
metadata:
name: wi-api
namespace: widemo
labels:
app: wi-api
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 50%
maxUnavailable: 25%
minReadySeconds: 0
selector:
matchLabels:
service: wi-api
template:
metadata:
labels:
app: wi-api
service: wi-api
azure.workload.identity/use: <span style="background-color: #fff0f0;">"true"</span> <span style="color: #888888;"># Required to make the contianers in the pod to use the workload identity</span>
<span style="color: #888888;"># annotations:</span>
<span style="color: #888888;"># azure.workload.identity/service-account-token-expiration: "3600" # Configure to avoid down time in token refresh. Takes precedence over servie acount setting. Default 3600, acceptable range: seconds 3600 - 86400.</span>
<span style="color: #888888;"># azure.workload.identity/skip-containers: "container1:container2" # Containers o skip using workload identity. By default all containers in pod will use workload identity when pod is labeled with azure.workload.identity/use: true </span>
<span style="color: #888888;"># azure.workload.identity/inject-proxy-sidecar: "true" # Default true. The proxy sidecar is used to intercept token requests to IMDS (Azure Instance Metadata Service) and acquire an AAD token on behalf of the user with federated identity credential.</span>
<span style="color: #888888;"># azure.workload.identity/proxy-sidecar-port: "8000" # Port of the proxy sidecar. Default 8000</span>
spec:
serviceAccountName: wi-demo-sa <span style="color: #888888;"># Service account (see aks_manifests\prerequisites\k8s.yaml) will provide identity to the pod https://azure.github.io/azure-workload-identity/docs/concepts.html</span>
nodeSelector:
<span style="background-color: #fff0f0;">"kubernetes.io/os"</span>: linux
priorityClassName: widemo-highest-priority-linux
<span style="color: #888888;">#------------------------------------------------------</span>
<span style="color: #888888;"># setting pod DNS policies to enable faster DNS resolution</span>
<span style="color: #888888;"># https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy</span>
dnsConfig:
options:
<span style="color: #888888;"># use FQDN everywhere </span>
<span style="color: #888888;"># any cluster local access from pods need full CNAME to resolve </span>
<span style="color: #888888;"># short names will not resolve to internal cluster domains</span>
- name: ndots
value: <span style="background-color: #fff0f0;">"2"</span>
<span style="color: #888888;"># dns resolver timeout and attempts</span>
- name: timeout
value: <span style="background-color: #fff0f0;">"15"</span>
- name: attempts
value: <span style="background-color: #fff0f0;">"3"</span>
<span style="color: #888888;"># use TCP to resolve DNS instad of using UDP (UDP is lossy and pods need to wait for timeout for lost packets)</span>
- name: use-vc
<span style="color: #888888;"># open new socket for retrying</span>
- name: single-request-reopen
<span style="color: #888888;">#------------------------------------------------------</span>
volumes:
<span style="color: #888888;"># `name` here must match the name</span>
<span style="color: #888888;"># specified in the volume mount</span>
- name: widemo-configmap-wi-api-volume
configMap:
<span style="color: #888888;"># `name` here must match the name</span>
<span style="color: #888888;"># specified in the ConfigMap's YAML. See aks_manifests\prerequisites\k8s.yaml</span>
name: widemo-configmap
terminationGracePeriodSeconds: 90 <span style="color: #888888;"># This must be set to a value that is greater than the preStop hook wait time.</span>
containers:
- name: wi-api
lifecycle:
preStop:
exec:
command: [<span style="background-color: #fff0f0;">"sleep"</span>,<span style="background-color: #fff0f0;">"60"</span>]
image: chdemosharedacr.azurecr.io/widemo/wi-api:1.1
imagePullPolicy: Always
<span style="color: #888888;"># probe to determine the stratup success</span>
startupProbe:
httpGet:
path: /api/health
port: container-port
initialDelaySeconds: 30 <span style="color: #888888;"># give 30 seconds to get container started before checking health</span>
failureThreshold: 30 <span style="color: #888888;"># max 300 (30*10) seconds wait for start up to succeed</span>
periodSeconds: 10 <span style="color: #888888;"># interval of probe (300 (30*10) start up to succeed)</span>
successThreshold: 1 <span style="color: #888888;"># how many consecutive success probes to consider as success</span>
timeoutSeconds: 10 <span style="color: #888888;"># probe timeout </span>
terminationGracePeriodSeconds: 30 <span style="color: #888888;"># restarts container (default restart policy is always)</span>
<span style="color: #888888;"># readiness probe fail will not restart container but cut off traffic to container with one failure </span>
<span style="color: #888888;"># as specified below and keep readiness probes running to see if container works again</span>
readinessProbe: <span style="color: #888888;"># probe to determine if the container is ready for traffic (used by AGIC)</span>
httpGet:
path: /api/health
port: container-port
failureThreshold: 1 <span style="color: #888888;"># one readiness fail should stop traffic to container</span>
periodSeconds: 20 <span style="color: #888888;"># interval of probe</span>
<span style="color: #888888;"># successThreshold not supported by AGIC</span>
timeoutSeconds: 10 <span style="color: #888888;"># probe timeout</span>
<span style="color: #888888;"># probe to determine the container is healthy and if not healthy container will restart</span>
livenessProbe:
httpGet:
path: /api/health
port: container-port
failureThreshold: 3 <span style="color: #888888;"># tolerates three consecutive faiures before restart trigger</span>
periodSeconds: 40 <span style="color: #888888;"># interval of probe</span>
successThreshold: 1 <span style="color: #888888;"># how many consecutive success probes to consider as success after a failure probe</span>
timeoutSeconds: 10 <span style="color: #888888;"># probe timeout </span>
terminationGracePeriodSeconds: 60 <span style="color: #888888;"># restarts container (default restart policy is always)</span>
volumeMounts:
- mountPath: /etc/config
name: widemo-configmap-wi-api-volume
ports:
- name: container-port
containerPort: 80
protocol: TCP
env:
- name: ASPNETCORE_URLS
value: http://+:80
- name: ASPNETCORE_ENVIRONMENT
value: Production
- name: CH_WIDEMO_CONFIG
value: /etc/config/config_dev-euw-001.json
resources:
limits:
memory: 1Gi <span style="color: #888888;"># the memory limit equals to the request!</span>
<span style="color: #888888;"># no cpu limit! this is excluded on purpose</span>
requests:
memory: 1Gi
cpu: <span style="background-color: #fff0f0;">"500m"</span>
<span style="color: #0e84b5; font-weight: bold;">---</span>
apiVersion: v1
kind: Service
metadata:
name: wi-api-clusterip
namespace: widemo
labels:
app: wi-api
service: wi-api
spec:
type: ClusterIP
ports:
- port: 8091
targetPort: 80
protocol: TCP
selector:
service: wi-api
<span style="color: #0e84b5; font-weight: bold;">---</span>
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wi-api
namespace: widemo
annotations:
<span style="color: #888888;"># --------------</span>
<span style="color: #888888;"># AGIC</span>
appgw.ingress.kubernetes.io/connection-draining: <span style="background-color: #fff0f0;">"true"</span>
appgw.ingress.kubernetes.io/connection-draining-timeout: <span style="background-color: #fff0f0;">"120"</span>
appgw.ingress.kubernetes.io/use-private-ip: <span style="background-color: #fff0f0;">"true"</span>
appgw.ingress.kubernetes.io/request-timeout: <span style="background-color: #fff0f0;">"30"</span>
<span style="color: #888888;"># --------------</span>
spec:
ingressClassName: azure-application-gateway
rules:
- host: wi-api.aksblue.ch-wi-dev-euw-001.net
http:
paths:
- path: /*
pathType: Prefix
backend:
service:
name: wi-api-clusterip
port:
number: 8091
</pre></div>
<p>When our application pods are running the containers are injected with below shown environemnt variables. This allows our application to authnticate via the <a href="https://azure.github.io/azure-workload-identity/docs/concepts.html" target="_blank">user assigned managed identity as explained in here</a>..</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhhyMmIeu9C-VoNRJmHC5v9hJWCiAixzYhhmx7ajBI8SDhrzGNpytpMQooPbztIuOQPeoWV_ox7h41H0YLMhEVIIWkDd2Ja7aSPCOm2Zu5ep8Z1OAeS7mH87QRhdtzP_rEs86JQa3a7Z7AteMlktULo1tIhQ6VkCanIDniXJu_iI-3JTz7eO73VXDzRCIXw" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="704" data-original-width="1231" height="357" src="https://blogger.googleusercontent.com/img/a/AVvXsEhhyMmIeu9C-VoNRJmHC5v9hJWCiAixzYhhmx7ajBI8SDhrzGNpytpMQooPbztIuOQPeoWV_ox7h41H0YLMhEVIIWkDd2Ja7aSPCOm2Zu5ep8Z1OAeS7mH87QRhdtzP_rEs86JQa3a7Z7AteMlktULo1tIhQ6VkCanIDniXJu_iI-3JTz7eO73VXDzRCIXw=w622-h357" width="622" /></a></div><br />So with this setup we can use code such as below to load app configuration with keyvault acces to our apps running in AKS using workload identity. app configuration endpoint would be just only the endpoint without any secret or connection information, for example <span style="font-family: courier;">https://ch-wi-dev-euw-001-appconfig-ac.azconfig.io </span>is enough to enable access to app config as we are using default credntials now.<p></p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"><span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Identity</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Azure.Security.KeyVault.Secrets</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Microsoft.Extensions.Configuration</span>;
<span style="color: #008800; font-weight: bold;">using</span> <span style="color: #0e84b5; font-weight: bold;">Microsoft.Extensions.Configuration.AzureAppConfiguration</span>;
<span style="color: #008800; font-weight: bold;">namespace</span> <span style="color: #0e84b5; font-weight: bold;">common.lib.Configs</span>
{
<span style="color: #008800; font-weight: bold;">public</span> <span style="color: #008800; font-weight: bold;">class</span> <span style="color: #bb0066; font-weight: bold;">ConfigLoader</span>
{
<span style="color: #008800; font-weight: bold;">public</span> <span style="color: #008800; font-weight: bold;">static</span> <span style="color: #008800; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">LoadConfiguration</span>(IConfigurationBuilder configBuilder)
{
configBuilder.AddJsonFile(Environment.GetEnvironmentVariable(<span style="background-color: #fff0f0;">"CH_WIDEMO_CONFIG"</span>));
<span style="color: #333399; font-weight: bold;">var</span> config = configBuilder.Build();
<span style="color: #333399; font-weight: bold;">string?</span> appConfigEndpont = config.GetSection(<span style="background-color: #fff0f0;">"AppConfigEndpoint"</span>).Value;
<span style="color: #333399; font-weight: bold;">string?</span> appConfigLabel = config.GetSection(<span style="background-color: #fff0f0;">"AppConfigLabel"</span>).Value;
<span style="color: #333399; font-weight: bold;">string?</span> sharedAppConfiglabel = config.GetSection(<span style="background-color: #fff0f0;">"SharedAppConfiglabel"</span>).Value;
<span style="color: #333399; font-weight: bold;">string?</span> keyVaultName = config.GetSection(<span style="background-color: #fff0f0;">"KeyVaultName"</span>).Value;
<span style="color: #333399; font-weight: bold;">string?</span> aadTenantId = config.GetSection(<span style="background-color: #fff0f0;">"AadTenantId"</span>).Value;
<span style="color: #888888;">//Load configuration from Azure App Configuration</span>
configBuilder.AddAzureAppConfiguration(options =>
{
DefaultAzureCredential azureCredentials = <span style="color: #008800; font-weight: bold;">new</span>();
options.Connect(
<span style="color: #008800; font-weight: bold;">new</span> <span style="color: #0066bb; font-weight: bold;">Uri</span>(appConfigEndpont),
azureCredentials);
options
.Select(KeyFilter.Any, sharedAppConfiglabel)
.Select(KeyFilter.Any, appConfigLabel);
SecretClient secretClient = <span style="color: #008800; font-weight: bold;">new</span>(
<span style="color: #008800; font-weight: bold;">new</span> <span style="color: #0066bb; font-weight: bold;">Uri</span>(<span style="background-color: #ffaaaa; color: red;">$</span><span style="background-color: #fff0f0;">"https://{keyVaultName}.vault.azure.net/"</span>),
azureCredentials);
options.ConfigureKeyVault(kv =>
kv.Register(secretClient));
});
configBuilder.Build();
}
}
}
</pre></div>
<p><br /></p><p>This is possible because in terraform we can grant the necessary permision to the user assigned managed identity, as shown below.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">resource "azurerm_key_vault" "instancekeyvault" {
name = "${var.PREFIX}-${var.PROJECT}-${var.ENVNAME}-kv"
location = azurerm_resource_group.instancerg.location
resource_group_name = azurerm_resource_group.instancerg.name
tenant_id = data.azurerm_client_config.current.tenant_id
sku_name = "standard"
enabled_for_deployment = false
enabled_for_disk_encryption = false
purge_protection_enabled = false <span style="color: #888888;"># allow purge for drop and create in demos. else this should be set to true</span>
network_acls {
bypass = "AzureServices"
default_action = "Deny"
ip_rules = ["xxx.xxx.xxx.xxx/32", "${chomp(data.http.mytfip.response_body)}/32"]
virtual_network_subnet_ids = [
"${azurerm_subnet.aks.id}"
]
}
# Sub Owners
access_policy {
tenant_id = var.TENANTID
object_id = data.azuread_group.sub_owners.object_id
key_permissions = ["Get", "Purge", "Recover"]
secret_permissions = ["Get", "List", "Set", "Delete", "Purge", "Recover"]
certificate_permissions = ["Create", "Get", "Import", "List", "Update", "Delete", "Purge", "Recover"]
}
# Infra Deployment Service Principal
access_policy {
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
key_permissions = ["Get", "Purge", "Recover"]
secret_permissions = ["Get", "List", "Set", "Delete", "Purge", "Recover"]
certificate_permissions = ["Create", "Get", "Import", "List", "Update", "Delete", "Purge", "Recover"]
}
# Containers in AKS via user assigned identity
access_policy {
tenant_id = var.TENANTID
object_id = azurerm_user_assigned_identity.aks.principal_id <span style="color: #888888;"># principal_id is the object id of the user assigned identity</span>
secret_permissions = ["Get", "List", ]
}
tags = merge(tomap({
Service = "key_vault",
}), local.tags)
}
</pre></div>
<p><br /></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># AKS user assigned identity as a reader</span></div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_role_assignment"</span> <span style="color: #4fc1ff;">"appconf_datareader_aks"</span> {</div><div> <span style="color: #9cdcfe;">scope</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_app_configuration</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">appconf</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">role_definition_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"App Configuration Data Reader"</span></div><div> <span style="color: #9cdcfe;">principal_id</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_user_assigned_identity</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">aks</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">principal_id</span></div><div>}</div></div><p>We can access storage blobs via default credntials as well as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> <span style="color: #569cd6;">private</span> <span style="color: #569cd6;">static</span> <span style="color: #4ec9b0;">BlobServiceClient</span> <span style="color: #dcdcaa;">GetBlobServiceClient</span>(<span style="color: #569cd6;">string</span> <span style="color: #9cdcfe;">accountName</span>)</div><div> {</div><div> <span style="color: #c586c0;">return</span> <span style="color: #569cd6;">new</span>(<span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">Uri</span>(<span style="color: #ce9178;">$"</span><span style="color: #ce9178;">https://</span><span style="color: #ce9178;">{</span><span style="color: #9cdcfe;">accountName</span><span style="color: #ce9178;">}</span><span style="color: #ce9178;">.blob.core.windows.net</span><span style="color: #ce9178;">"</span>),</div><div> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">DefaultAzureCredential</span>());</div><div> }</div></div><p>Or storage queue as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">QueueClient</span> <span style="color: #9cdcfe;">queueClient</span> <span style="color: #d4d4d4;">=</span> <span style="color: #569cd6;">new</span>(</div><div><span style="color: #569cd6;"><span> </span>new</span> <span style="color: #4ec9b0;">Uri</span>(<span style="color: #ce9178;">$"</span><span style="color: #ce9178;">https://</span><span style="color: #ce9178;">{</span><span style="color: #4fc1ff;">QueueStorageName</span><span style="color: #ce9178;">}</span><span style="color: #ce9178;">.queue.core.windows.net/</span><span style="color: #ce9178;">{</span><span style="color: #4fc1ff;">QueueName</span><span style="color: #ce9178;">}</span><span style="color: #ce9178;">"</span>),</div><div> <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">DefaultAzureCredential</span>());</div></div><p><br /></p><p>Above are only few examples. With workload identity enabled, your containers deployed to AKS can access any Azure resource with <a href="https://learn.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?wt.mc_id=DT-MVP-5000590&view=azure-dotnet#defaultazurecredential" target="_blank">DefaultAzureCredential</a> securely using a managed identity. This far better secure approach than having to store connection strings, secrets etc. for your application usage purpose and having to pass those secret information around in your application components.</p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-62915351654452243162023-12-09T04:59:00.000-08:002024-01-21T05:41:09.375-08:00Setting Up Helm in WSL<p> Kubernetes applications can be deployed easily with helm. Meny useful tools such as KEDA (Kubernetes Event Driven Autoscaler) deployments can be done with helm, using few simple steps. In this post let's look at how to setup helm in WSL so that we can use it to setup applications using helm charts.<span></span></p><a name='more'></a><p></p><p>As the first step of setting up helm in WSL we have to run the below command.</p><p><span style="font-family: courier;">curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjQXv9X7GP8cTUMRBOx-KcLkiMvSJdkz6clVUnMLxSZ_6i-I5gjb2XOf6G6Gvfbc5wZXRY1SqzGjuL_--shjxloSQUdX7GEU2SB4IJt3XqvViKKka7t9x4KIcBavh_0BfuIn-KQDBBawLlbzi-ItbXnLjQW6KVcCNIu3k7YTa6fvTj674y3uZVrJiF3cYhL" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="193" data-original-width="2267" height="71" src="https://blogger.googleusercontent.com/img/a/AVvXsEjQXv9X7GP8cTUMRBOx-KcLkiMvSJdkz6clVUnMLxSZ_6i-I5gjb2XOf6G6Gvfbc5wZXRY1SqzGjuL_--shjxloSQUdX7GEU2SB4IJt3XqvViKKka7t9x4KIcBavh_0BfuIn-KQDBBawLlbzi-ItbXnLjQW6KVcCNIu3k7YTa6fvTj674y3uZVrJiF3cYhL=w839-h71" width="839" /></a></div><br />Next run command below.<p></p><p><span style="font-family: courier;">sudo apt-get install apt-transport-https --yes</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhgoHTlcTs3tWzdf8ftXxR1GnqHpVwdY8nFNr_2U6TFQn9RErWr0q_ggAxTJreSgGXS-IqPLEoKwWWk0enlXhDyNKtLPJYAcQjnLRWZJTB0fspc650QVL-q-N9hyLBSCfKJUfoLYn-l0tohUw0Dyss92FSFD640THTDliU_I7I1kj9l7IK1TWckqG8CaNjw" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="245" data-original-width="1313" height="122" src="https://blogger.googleusercontent.com/img/a/AVvXsEhgoHTlcTs3tWzdf8ftXxR1GnqHpVwdY8nFNr_2U6TFQn9RErWr0q_ggAxTJreSgGXS-IqPLEoKwWWk0enlXhDyNKtLPJYAcQjnLRWZJTB0fspc650QVL-q-N9hyLBSCfKJUfoLYn-l0tohUw0Dyss92FSFD640THTDliU_I7I1kj9l7IK1TWckqG8CaNjw=w650-h122" width="650" /></a></div><br /><p></p><p>Run below command as the nxt step.</p><p><span style="font-family: courier;">echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEicLssnyehdeM2l1wFSdWrQ-30YA7YA5KJoEu3gXF2-ydTXxErvDOoFEm26lQPW8LEjFplW8oZSpX0sJDu5RmmvxidNKrBzWaJe5dAoV4vdoNenHsiJQJqjyGvTHY0IWsd0l6J-BX16Nc5_AcYLa1-njO2F3UufNG-b1Zi4aEFcpUNBf-rW-oruRj3v8ESt" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="195" data-original-width="1816" height="68" src="https://blogger.googleusercontent.com/img/a/AVvXsEicLssnyehdeM2l1wFSdWrQ-30YA7YA5KJoEu3gXF2-ydTXxErvDOoFEm26lQPW8LEjFplW8oZSpX0sJDu5RmmvxidNKrBzWaJe5dAoV4vdoNenHsiJQJqjyGvTHY0IWsd0l6J-BX16Nc5_AcYLa1-njO2F3UufNG-b1Zi4aEFcpUNBf-rW-oruRj3v8ESt=w642-h68" width="642" /></a></div><br />Next update packages with below.<p></p><p><span style="font-family: courier;">sudo apt-get update</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgrTVQgLE_CVC68tlqXh85mNBS-xOo7HZKG-sX03ZAQn8ZzyDvwTGffYjVBnGkJvhkSxXvYIo62k-_ye658wXSXZB83YmN6TWHteVZp-HLX8opOCtDNoqULa81AGsnzVKrOYWg7oKqVxHXextA9c4J3YgPgrlQN8tScYRm2uaM67hQ0x0zHTrX7fNWz6kMd" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1541" data-original-width="1451" height="632" src="https://blogger.googleusercontent.com/img/a/AVvXsEgrTVQgLE_CVC68tlqXh85mNBS-xOo7HZKG-sX03ZAQn8ZzyDvwTGffYjVBnGkJvhkSxXvYIo62k-_ye658wXSXZB83YmN6TWHteVZp-HLX8opOCtDNoqULa81AGsnzVKrOYWg7oKqVxHXextA9c4J3YgPgrlQN8tScYRm2uaM67hQ0x0zHTrX7fNWz6kMd=w594-h632" width="594" /></a></div><br />We can now setup helm using below command.<p></p><p><span style="font-family: courier;">sudo apt-get install helm</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjJ1xG2GLSG8jBpvepqjEhySdZ_RdJMpeyqXU6X3mhmV_3YBGG6cDfB6Z45NXHHhY8jOdZXvP37kFh4_mVFPEelttDTpjH1-WR_JMrqoi5lNJ5hIIUau4UQgVtQ_xx_u1q3jtOCLugLR7Bt043v3EMmKQKWndRqmihaBozv60rXRzLORufjqi-RQEl5QMWP" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="545" data-original-width="1385" height="226" src="https://blogger.googleusercontent.com/img/a/AVvXsEjJ1xG2GLSG8jBpvepqjEhySdZ_RdJMpeyqXU6X3mhmV_3YBGG6cDfB6Z45NXHHhY8jOdZXvP37kFh4_mVFPEelttDTpjH1-WR_JMrqoi5lNJ5hIIUau4UQgVtQ_xx_u1q3jtOCLugLR7Bt043v3EMmKQKWndRqmihaBozv60rXRzLORufjqi-RQEl5QMWP=w575-h226" width="575" /></a></div><br />We can verify helm setup correctly by running <span style="font-family: courier;">helm version</span> or <span style="font-family: courier;">helm help</span>.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgSpLvpGvJpjR59-O4bme4rpKk3opYazrBtqvcDEGs6kv_xTljEqi9W094Ul3m5nwV4TarUinpQ7cpjjcn6HGY8ylYqrDxi6xSbd31KfCf6sR4k5OlmTy5ZZdRzVcBTsfD_EGyZGN1xBKoCAv5nTk6DOIsKXCnkxOrDcsDxAcVkwaZOzTyW1nnvDzY2gZKh" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="737" data-original-width="2096" height="216" src="https://blogger.googleusercontent.com/img/a/AVvXsEgSpLvpGvJpjR59-O4bme4rpKk3opYazrBtqvcDEGs6kv_xTljEqi9W094Ul3m5nwV4TarUinpQ7cpjjcn6HGY8ylYqrDxi6xSbd31KfCf6sR4k5OlmTy5ZZdRzVcBTsfD_EGyZGN1xBKoCAv5nTk6DOIsKXCnkxOrDcsDxAcVkwaZOzTyW1nnvDzY2gZKh=w612-h216" width="612" /></a></div><br />Now that we have helm ready with WSL we can go ahead and setup tools such as KEDA easily which we wll discuss in a next post.<p></p><p><br /></p><p><br /></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-37162239391519138642023-11-15T07:25:00.000-08:002024-03-10T09:47:11.299-07:00Installing .NET 8 Runtime on Ubuntu 22.04 Docker Image<p> .NET 8 was release on November 14. There are docker container images for .NET 8 available for dotnet runtime and can be found with tag list here<a href="https://mcr.microsoft.com/v2/dotnet/runtime/tags/list" target="_blank"> https://mcr.microsoft.com/v2/dotnet/runtime/tags/list </a>. However, if you want to setup .NET 8 runtime on another specific Linux docker image for example on <span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">ubuntu:jammy</span> , <span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">amd64/ubuntu:22.04 </span>or with a special image such as ffmpeg Linux server image <span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">linuxserver/ffmpeg:amd64-version-6.0-cli</span>, where you might want to run your .NET app to use ffmpeg, In such cases, where you you might have to setup .NET 8 runtime on a specific docker image, with your other tools readily available, details mentioned may come in handy. Lets, see how we can install .NET 8 runtime on base Ubuntu 22.04 images, using a docker file.<span></span></p><a name='more'></a><p></p><p>What you would need is below. This will install required dependencis as well as .NET 8 runtime on any of the above mentioned docker images.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhX8hoXtZuiYzNZTiBRViPT8qJwyjVQhk3mbJ4BoaNBe_3GaDct_dbf1A4XB8fNt5ZPgtpgOLM74CSInDHZiahZINnEBfURKOEehiYR4GV3m4cU4zB5aUqRXD0pk-Xe_LDsH_puX9nkrLpVJ-dulS5x9L51pyQj0RmhkaEQv2IOyO5faeJuCOX1yoZ1HoJt" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="497" data-original-width="1256" height="257" src="https://blogger.googleusercontent.com/img/a/AVvXsEhX8hoXtZuiYzNZTiBRViPT8qJwyjVQhk3mbJ4BoaNBe_3GaDct_dbf1A4XB8fNt5ZPgtpgOLM74CSInDHZiahZINnEBfURKOEehiYR4GV3m4cU4zB5aUqRXD0pk-Xe_LDsH_puX9nkrLpVJ-dulS5x9L51pyQj0RmhkaEQv2IOyO5faeJuCOX1yoZ1HoJt=w647-h257" width="647" /></a></div><br />You can copy code from below.<p></p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">FROM ubuntu:jammy AS base
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
curl \
ca-certificates \
libc6 \
libgcc1 \
libgcc-s1 \
libgssapi-krb5-2 \
libicu70 \
liblttng-ust1 \
libssl3 \
libstdc++6 \
libunwind8 \
zlib1g \
&& rm -rf /var/lib/apt/lists/*
RUN curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin -Channel 8.0 -Runtime dotnet -InstallDir /usr/share/dotnet
RUN ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet
</pre></div>
<p>Instead of using the dotnet-install.sh hopefully we will be able to use below when the packages are avaialble.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><br /><div><span style="color: #569cd6;">RUN</span> apt-get install -y dotnet-runtime-8.0 # not available yet<br /></div><br /></div><p> With this a dotnet app can be run on any of the above mentioned docker images for esample below console app code runs fine in <span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">linuxserver/ffmpeg:amd64-version-6.0-cli</span> image and provide output as shown. This enables the .NET app to use ffmpeg in docker container without having us to set up ffmpeg by ourselves on a dotnet base docker image.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">namespace</span> <span style="color: #4ec9b0;">videoprocessor</span><span style="color: #d4d4d4;">.</span><span style="color: #4ec9b0;">xabe</span>;</div><br /><div><span style="color: #569cd6;">class</span> <span style="color: #4ec9b0;">Program</span></div><div>{</div><div> <span style="color: #569cd6;">static</span> <span style="color: #569cd6;">void</span> <span style="color: #dcdcaa;">Main</span>(<span style="color: #569cd6;">string</span>[] <span style="color: #9cdcfe;">args</span>)</div><div> {</div><div> <span style="color: #569cd6;">string</span><span style="color: #d4d4d4;">?</span> <span style="color: #9cdcfe;">mediaPath</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">Environment</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">GetEnvironmentVariable</span>(<span style="color: #ce9178;">"MEDIA_PATH"</span>);</div><br /><div> <span style="color: #c586c0;">if</span> (<span style="color: #9cdcfe;">mediaPath</span> <span style="color: #569cd6;">is</span> <span style="color: #569cd6;">not</span> <span style="color: #569cd6;">null</span>)</div><div> {</div><div> <span style="color: #569cd6;">string</span>[] <span style="color: #9cdcfe;">files</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">Directory</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">GetFiles</span>(<span style="color: #9cdcfe;">mediaPath</span>);</div><br /><br /><div> <span style="color: #9cdcfe;">Console</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">WriteLine</span>(<span style="color: #ce9178;">$"</span><span style="color: #ce9178;">Media path is: </span><span style="color: #ce9178;">{</span><span style="color: #9cdcfe;">mediaPath</span><span style="color: #ce9178;">}</span><span style="color: #ce9178;">"</span>);</div><div> <span style="color: #9cdcfe;">Console</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">WriteLine</span>(<span style="color: #ce9178;">$"</span><span style="color: #ce9178;">Files: </span><span style="color: #ce9178;">{</span><span style="color: #9cdcfe;">files</span><span style="color: #ce9178;">[</span><span style="color: #b5cea8;">0</span><span style="color: #ce9178;">]}</span><span style="color: #ce9178;">"</span>);</div><div> }</div><div> }</div><div>}</div><br /></div><p><br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiJ-agcgheTjHR7y98DnWYgMyLb7vxSRF_okwFmftmFessJD8cNMprNQWkmLt37x9RoSlFU58RSjkAanaAHpa__1tjz4iXMru0G_WAK7ZcOOxt2GrkpRN4mcv0GbZExttdcNCuOKwXSWjDdtAXm4UdCDHbtCIw7OdwmsVnMzpdslkHBN48DlCjdrCI2Qzgj" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="77" data-original-width="418" height="109" src="https://blogger.googleusercontent.com/img/a/AVvXsEiJ-agcgheTjHR7y98DnWYgMyLb7vxSRF_okwFmftmFessJD8cNMprNQWkmLt37x9RoSlFU58RSjkAanaAHpa__1tjz4iXMru0G_WAK7ZcOOxt2GrkpRN4mcv0GbZExttdcNCuOKwXSWjDdtAXm4UdCDHbtCIw7OdwmsVnMzpdslkHBN48DlCjdrCI2Qzgj=w596-h109" width="596" /></a></div><br /><br /></div><p></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-58130778366185784382023-11-15T06:29:00.000-08:002023-11-15T06:29:47.637-08:00Installing .NET 8.0 SDK on WSL<p> WSL (Windows subsystem for Linux) is a great way to work with Linux on Windows. .NET 8 is released on November 14th, and let's see how we can get .NET 8 SDK setup on WSL to build and test our .NET 8 apps on Linux on a windows machine.</p><p>What we want is when we do a dotnet --list-sdks to see the .NET 8 available in WSL. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhj2PGGbeTod92Ofq3zpe0sIjQ1ny1GHUQxll7jyhgk08KKrq8c9oD2bT4VprKup40GqRSbrM6XjZ8J42bry2qVyPJglUFVhgVgjyZ8RGMfjQ_iywZcP1_slM9KsWVM6Ed2AACHAooMhn-f5-y--kFispRcxoRTSN2naB5nNkXGBlUfKackkbAyu5KTwt_F" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="151" data-original-width="504" height="174" src="https://blogger.googleusercontent.com/img/a/AVvXsEhj2PGGbeTod92Ofq3zpe0sIjQ1ny1GHUQxll7jyhgk08KKrq8c9oD2bT4VprKup40GqRSbrM6XjZ8J42bry2qVyPJglUFVhgVgjyZ8RGMfjQ_iywZcP1_slM9KsWVM6Ed2AACHAooMhn-f5-y--kFispRcxoRTSN2naB5nNkXGBlUfKackkbAyu5KTwt_F=w581-h174" width="581" /></a></div><br /><span><a name='more'></a></span>To get .NET 8 SDK installed on WSL simple run below command.. Once complted the .NET 8 SDK is ready to go on WSL.<p></p><p><span style="font-family: courier;">sudo apt-get update && </span><span style="font-family: courier;">sudo apt-get install -y dotnet-sdk-8.0</span></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-61766940865819714912023-11-11T05:17:00.003-08:002023-11-11T05:17:53.071-08:00Terraform vs Azure Portal Defaults for Azure Storage Soft Delete<p> Azure storage support soft deletion of blobs, blob containers and file shares. When we create a storage account usng Azure portal, by default soft deletion will be enabled with 7 day retention.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgw5xkqi6jArnyW_GPh55Q5pHHMs3K_PPPdRVm6Ntx2_BWPLTC9soXWZaguZgS2NF8uQDhEToDZrgawIKVLCAG7QQLd_htkhGLPkNqq9V4vrpOrRZAdq1NLcZ0zMVpC9yilJCjizDqju4r6s-v4fm0OJDgvPENecWlPD63Sf7iozYoF_HT_5fAHiIbEvcsY" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1203" data-original-width="1540" height="346" src="https://blogger.googleusercontent.com/img/a/AVvXsEgw5xkqi6jArnyW_GPh55Q5pHHMs3K_PPPdRVm6Ntx2_BWPLTC9soXWZaguZgS2NF8uQDhEToDZrgawIKVLCAG7QQLd_htkhGLPkNqq9V4vrpOrRZAdq1NLcZ0zMVpC9yilJCjizDqju4r6s-v4fm0OJDgvPENecWlPD63Sf7iozYoF_HT_5fAHiIbEvcsY=w442-h346" width="442" /></a></div><br /><span><a name='more'></a></span>However when we are setting up an storage account with terraform, for example consider below terraform code. <p></p><div style="background-color: #1f1f1f; line-height: 19px;"><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_account"</span> <span style="color: #4fc1ff;">"queue"</span> {</div><div><span style="color: #cccccc; font-family: Consolas, Courier New, monospace;"><span style="font-size: 14px; white-space: pre;"> </span></span><span style="color: #9cdcfe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">name</span><span style="color: #9cdcfe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> </span><span style="color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">=</span><span style="color: #9cdcfe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> </span><span style="color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">"</span><span style="color: #ce9178; font-family: Consolas, Courier New, monospace;"><span style="font-size: 14px; white-space: pre;">chvideodeveuw001queuest</span></span><span style="color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">account_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Standard"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">account_replication_type</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"LRS"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">account_kind</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"StorageV2"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">access_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Hot"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">allow_nested_items_to_be_public</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">min_tls_version</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"TLS1_2"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">cross_tenant_replication_enabled</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">}</div><br /><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_queue"</span> <span style="color: #4fc1ff;">"video"</span> {</div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"demovideoqueue"</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> <span style="color: #9cdcfe;">storage_account_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_storage_account</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">queue</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">}</div></div><p>The storage account shows it is created with soft deletion disabled by default.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhTVm693xPufuGV7tHlqvDVeOoYyv6K1tuCec6DjnXRGSxSmMIZpDC8K3xp-BuWtg1fTd-ZLntbXWWbyJECXw5Ko1nIVCb18kk7K7JPfmQKUw0xkOkSBKzFgtEqEzUyDkKTn5zsRxJEUHLduVwJDAelFF0E7KqURfmWjGyh5N-uZu68dxjQRBYzkNqIJHV8" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1301" data-original-width="1744" height="422" src="https://blogger.googleusercontent.com/img/a/AVvXsEhTVm693xPufuGV7tHlqvDVeOoYyv6K1tuCec6DjnXRGSxSmMIZpDC8K3xp-BuWtg1fTd-ZLntbXWWbyJECXw5Ko1nIVCb18kk7K7JPfmQKUw0xkOkSBKzFgtEqEzUyDkKTn5zsRxJEUHLduVwJDAelFF0E7KqURfmWjGyh5N-uZu68dxjQRBYzkNqIJHV8=w565-h422" width="565" /></a></div><br />However for the file shares by default soft deletion is enabled even with terraform.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhLcfeG5uldB-7CRLeu0mlzvUiPA-eBfu1QFlGx1A-ql8gmh-AT23jK1EpwUI4TsTTkGTCYXVjsLT1QDBet7vcQp4_kZSHTMa69k0yq7OE70uS4dK4XzS8q76w6keCZ4LZ8PkYoZezBtypd48WadzKRJx4meofL11UJIM7_hDXApAg2QsdVG6vx3p4Y5uDy" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="834" data-original-width="2983" height="178" src="https://blogger.googleusercontent.com/img/a/AVvXsEhLcfeG5uldB-7CRLeu0mlzvUiPA-eBfu1QFlGx1A-ql8gmh-AT23jK1EpwUI4TsTTkGTCYXVjsLT1QDBet7vcQp4_kZSHTMa69k0yq7OE70uS4dK4XzS8q76w6keCZ4LZ8PkYoZezBtypd48WadzKRJx4meofL11UJIM7_hDXApAg2QsdVG6vx3p4Y5uDy=w640-h178" width="640" /></a></div><br />With terraform we cannot explicitly set soft deletion to be disabled for blobs or blob containers. Terrrraform sets by defaut disabled for soft deletion for blobs and blob containers as shown above. But, if we are to specify a soft deletion setting in terraform, it must be set with soft deletion for 1 day at least.<p></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_storage_account"</span> <span style="color: #4fc1ff;">"queue"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}${</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}${</span><span style="color: #dcdcaa;">replace</span><span style="color: #ce9178;">(</span><span style="color: #9cdcfe;">var</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">ENVNAME</span><span style="color: #ce9178;">, "-", "")</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">queuest"</span></div><div> <span style="color: #9cdcfe;">resource_group_name</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">location</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">instancerg</span><span style="color: #d4d4d4;">.</span><span style="color: #9cdcfe;">location</span></div><div> <span style="color: #9cdcfe;">account_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Standard"</span></div><div> <span style="color: #9cdcfe;">account_replication_type</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"LRS"</span></div><div> <span style="color: #9cdcfe;">account_kind</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"StorageV2"</span></div><div> <span style="color: #9cdcfe;">access_tier</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"Hot"</span></div><div> <span style="color: #9cdcfe;">allow_nested_items_to_be_public</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><div> <span style="color: #9cdcfe;">min_tls_version</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"TLS1_2"</span></div><div> <span style="color: #9cdcfe;">cross_tenant_replication_enabled</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #c586c0;">false</span></div><br /><div> <span style="color: #4ec9b0;">blob_properties</span> {</div><div> <span style="color: #4ec9b0;">delete_retention_policy</span> {</div><div> <span style="color: #9cdcfe;">days</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #b5cea8;">1</span></div><div> }</div><div> <span style="color: #4ec9b0;">container_delete_retention_policy</span> {</div><div> <span style="color: #9cdcfe;">days</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #b5cea8;">1</span></div><div> }</div><div> }</div><br /><div> <span style="color: #4ec9b0;">share_properties</span> {</div><div> <span style="color: #4ec9b0;">retention_policy</span> {</div><div> <span style="color: #9cdcfe;">days</span><span style="color: #9cdcfe;"> </span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;"> </span><span style="color: #b5cea8;">1</span></div><div> }</div><div> }</div><div> </div><div>}</div></div><p>With terraform documentation it seems not possible to disable file share soft delete. Looks like ifneed to disable file share soft deletion only via Azure portal. Or we can use Azure Bicep for IaC with more capablitites to handle Azure resources than terraform, which has properties to <a href="https://learn.microsoft.com/en-us/azure/templates/microsoft.storage/storageaccounts/fileservices?wt.mc_id=DT-MVP-5000590&pivots=deployment-language-bicep" target="_blank">enable/disable soft deletion for fileshares as well</a>.</p><span><!--more--></span><span><!--more--></span><span><!--more--></span>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-14950577503193121122023-10-27T02:39:00.001-07:002023-10-27T02:39:27.044-07:00Install Kubectl on WSL (Windows Subsystem for Linux)<p> Kubectl is the command line which will help to do everything with AKS or any other kubernetes set up. To setup or update kubectl on WSL follow the steps below. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjm0jEzxg1U-naiuWHq94BWogJ21gwRui6ftxR-3bmMbN83hNSPTweYW91IU1qAcGzcZFXoAKquanwAUbPbGXyuVXcC-kWJ7MldPFnRImJe7I_9jNAAXp5EJPUJMvQHWDISp3QL-X_6fw4cxWGiAmAnqTBpOt1L81cLvcKM1gbZjvaX_QLqpjB8et99lLRk" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="207" data-original-width="1021" height="136" src="https://blogger.googleusercontent.com/img/a/AVvXsEjm0jEzxg1U-naiuWHq94BWogJ21gwRui6ftxR-3bmMbN83hNSPTweYW91IU1qAcGzcZFXoAKquanwAUbPbGXyuVXcC-kWJ7MldPFnRImJe7I_9jNAAXp5EJPUJMvQHWDISp3QL-X_6fw4cxWGiAmAnqTBpOt1L81cLvcKM1gbZjvaX_QLqpjB8et99lLRk=w668-h136" width="668" /></a><span><a name='more'></a></span></div><p></p><ul class="ak-ul"><li><p>Open WSL in Windows or using VS Code Terminal and execute below steps in WSL</p></li><li><p>Download kubectl specific version say 1.27.3 </p><ul class="ak-ul"><li><p><span class="code" spellcheck="false"><span style="font-family: courier;">curl -LO "https://dl.k8s.io/release/v1.27.3/bin/linux/amd64/kubectl"</span></span></p></li></ul></li><li><p>Once download completes run below command to install kubectl in WSL.</p><ul class="ak-ul"><li><p><span class="code" spellcheck="false"><span style="font-family: courier;">sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl</span></span></p></li></ul></li><li><p>Check the installed version</p><ul class="ak-ul"><li><p><span class="code" spellcheck="false"><span style="font-family: courier;">kubectl version --short</span></span></p></li></ul></li></ul>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-45115106659292479832023-10-19T07:08:00.005-07:002023-10-19T07:08:45.971-07:00Transform json Files in Azure Pipelines<p> In .NET aplication we use the appsettings.json to keep values such as connection string etc for local development mostly. In the context of Azure app servies or containeried envronments etc, we configure mostly app config service (with key vault to manage secrets) to keep our connection information, and any other config values. Howver, sometimes legecy deployments which are still done targeting on premise servers etc might need us to update actual appsetting.josn files or web config files etc. Let's look at how to transform josn files in an Azure pipeline.<span></span></p><a name='more'></a><p></p><p><b><u>The expected outcome</u></b></p><p>The expectation is transforming the appsettings json file value based on a variable value defined in a release pipeline. We can of cource do the same in YAML pipline as well.</p><p>The pipline should substitute the values in json as shown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjFec7ijb0AsGcjWsdaGw5SNNqSNcpr8IDF8OvYL_6vqSBszr4frcy5yKn0DXahVe86vWnloIx6Cbsu9UyxHDtKRPP-tUei0ZCYUdv8DYYVBjdJsfenrSru_zdrvd-axXc8Te5tTTKn0YDt3iuY3IKXbBFv3Bc78N3wTKfI997OLdCz5mtQnPA7YmyTK2yF" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="690" data-original-width="2115" height="218" src="https://blogger.googleusercontent.com/img/a/AVvXsEjFec7ijb0AsGcjWsdaGw5SNNqSNcpr8IDF8OvYL_6vqSBszr4frcy5yKn0DXahVe86vWnloIx6Cbsu9UyxHDtKRPP-tUei0ZCYUdv8DYYVBjdJsfenrSru_zdrvd-axXc8Te5tTTKn0YDt3iuY3IKXbBFv3Bc78N3wTKfI997OLdCz5mtQnPA7YmyTK2yF=w672-h218" width="672" /></a></div><br /><b><u> How to do</u></b><p></p><p>Assume a josn file is containing below details.</p><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 12px; line-height: 16px; white-space: pre;"><div>{</div><div> <span style="color: #a31515;">"Logging"</span>: {</div><div> <span style="color: #a31515;">"LogLevel"</span>: {</div><div> <span style="color: #a31515;">"Default"</span>: <span style="color: #0451a5;">"Information"</span>,</div><div> <span style="color: #a31515;">"Microsoft.AspNetCore"</span>: <span style="color: #0451a5;">"Warning"</span></div><div> }</div><div> },</div><div> <span style="color: #a31515;">"ConnectionStrings"</span>: {</div><div> <span style="color: #a31515;">"UseDB"</span>: <span style="color: #0451a5;">"true"</span>,</div><div> <span style="color: #a31515;">"ConnectionStringName"</span>: <span style="color: #0451a5;">"DefaultConnection"</span>,</div><div> <span style="color: #a31515;">"DefaultConnection"</span>: <span style="color: #0451a5;">"Data Source=MYSVR\\SQL2019ENT;Initial Catalog=MYDB;Trusted_Connection=True;"</span></div><div> },</div><div> <span style="color: #a31515;">"mysettings"</span>: {</div><div> <span style="color: #a31515;">"id"</span>: <span style="color: #0451a5;">"1111111"</span>,</div><div> <span style="color: #a31515;">"name"</span>: <span style="color: #0451a5;">"TEST - My AB"</span>,</div><div> <span style="color: #a31515;">"userName"</span>: <span style="color: #0451a5;">"admin@111111"</span>,</div><div> <span style="color: #a31515;">"integrationName"</span>: <span style="color: #0451a5;">"Chaminda"</span>,</div><div> <span style="color: #a31515;">"clientId"</span>: <span style="color: #0451a5;">"CHsecretid"</span>,</div><div> <span style="color: #a31515;">"clientSecret"</span>: <span style="color: #0451a5;">"ChSecret"</span></div><div> },</div><div> <span style="color: #a31515;">"AllowedHosts"</span>: <span style="color: #0451a5;">"*"</span></div><div>}</div></div><p>And we just want to replace the default connection string, using a variable value. In a variable group we can define a variable with below name.</p><p><span style="background-color: #fffffe; color: #a31515; font-family: Consolas, "Courier New", monospace; font-size: 12px; white-space: pre;">ConnectionStrings.</span><span style="background-color: #fffffe; color: #a31515; font-family: Consolas, "Courier New", monospace; font-size: 12px; white-space: pre;">DefaultConnection</span></p><p>Then we can use a trasform file task as shown below.</p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">- task: FileTransform@2
displayName: <span style="background-color: #fff0f0;">'File</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Transform:</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">'</span>
inputs:
folderPath: <span style="background-color: #fff0f0;">'$(System.DefaultWorkingDirectory)/my_api/v1_API/my_api'</span>
jsonTargetFiles: appsettings.json
</pre></div>
<p>For the folder path you can provide a web deployment package zip name as below.</p><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;">- task: FileTransform@2
displayName: <span style="background-color: #fff0f0;">'File</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Transform:</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">'</span>
inputs:
folderPath: <span style="background-color: #fff0f0;">'$(System.DefaultWorkingDirectory)/my_api/v1_API/my_api/webapi.zip'</span>
jsonTargetFiles: appsettings.json
</pre></div><p>I</p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-33927223979041372942023-10-18T08:17:00.007-07:002023-10-18T08:18:48.713-07:00Code Unit Test Coverage with Azure Pipelines<p> Unit tests are essntial to ensure the code we develop is working as intended. Running the unit tests Azure pipelines is really helpful to not to miss the unit test failures. However, to give afurther assuarance we need to check the coverage of code with unit tests in our projects. Let's look at steps required to obtain a code unit tests coverage report in Azure piplines in <a href="https://cobertura.github.io/cobertura/" target="_blank">cobertura</a> format.</p><p><b><u>The expected outcome</u></b></p><p>The coverage here shows the first library, all code lines (100%) are covered with tests and the the other library has only 22.2% code coverage. </p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhpxPCeYQZq4-DeW4Ue6y5acTuBYD_UUTzr5KMGMGHlVsNnobtgXYkcQy3YmPjbg3-0bBDq7PZnvlM9u2Ec4rgwce_TR3ZudLrdSJf2xxCTr8R00xH3A_9lwbbUNH9_rO1gP9uDE9iwVPlIPn1qIRwwFUl8glQH5LALubDWmOU5x0SuctwraF3grbS_xAB_" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="831" data-original-width="1601" height="349" src="https://blogger.googleusercontent.com/img/a/AVvXsEhpxPCeYQZq4-DeW4Ue6y5acTuBYD_UUTzr5KMGMGHlVsNnobtgXYkcQy3YmPjbg3-0bBDq7PZnvlM9u2Ec4rgwce_TR3ZudLrdSJf2xxCTr8R00xH3A_9lwbbUNH9_rO1gP9uDE9iwVPlIPn1qIRwwFUl8glQH5LALubDWmOU5x0SuctwraF3grbS_xAB_=w672-h349" width="672" /></a></div><br /><p></p><p><span></span></p><a name='more'></a>We can dig deeper into the report by clicking on the code file name and find which lines are not covered.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjDghRDMSnrcA1A59WXNqR4-i3Q0yZm08XRgkJW1MZFH7yOQFk5yWdHV7W86m2s4hKdTYHRs525Lvh2QumuQjWpVYo4WIiazxQ8aU3K7HmaMB4a9DFXB9JX_w970fTKpphNRaGp-1cnVog3NaHiNEVE7b4EkRtZFp6O4df7NF1YKQ1VciZn8SzfRnqe3U2w" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="699" data-original-width="1449" height="292" src="https://blogger.googleusercontent.com/img/a/AVvXsEjDghRDMSnrcA1A59WXNqR4-i3Q0yZm08XRgkJW1MZFH7yOQFk5yWdHV7W86m2s4hKdTYHRs525Lvh2QumuQjWpVYo4WIiazxQ8aU3K7HmaMB4a9DFXB9JX_w970fTKpphNRaGp-1cnVog3NaHiNEVE7b4EkRtZFp6O4df7NF1YKQ1VciZn8SzfRnqe3U2w=w607-h292" width="607" /></a></div><br /><b><u>How to do</u></b><p></p><p>As the first step we need to add <a href="https://www.nuget.org/packages/coverlet.collector" target="_blank">coverlet.collector </a>NuGet package to all our test projects as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: grey;"><</span><span style="color: #569cd6;">PackageReference</span> <span style="color: #9cdcfe;">Include</span>=<span style="color: #ce9178;">"coverlet.collector"</span> <span style="color: #9cdcfe;">Version</span>=<span style="color: #ce9178;">"6.0.0"</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">PrivateAssets</span><span style="color: grey;">></span>all<span style="color: grey;"></</span><span style="color: #569cd6;">PrivateAssets</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">IncludeAssets</span><span style="color: grey;">></span>runtime; build; native; contentfiles; analyzers; buildtransitive<span style="color: grey;"></</span><span style="color: #569cd6;">IncludeAssets</span><span style="color: grey;">></span></div><div> <span style="color: grey;"></</span><span style="color: #569cd6;">PackageReference</span><span style="color: grey;">></span></div></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi2ZQmVaKxw78z2oRUZRf3ysioQ0qvO2J-72i685onr65e-WoiayAc3SR_Iqw1Wd6kY5GyFIVwgseeDqJ8JPE5FdE1UqrICyZHosyM4Y1wQmCO14VDHRuIGbpaBNVx1UK6PUrKjQYsrxIJdB58zn7z4eyIwxkfcwSPCBrgIEh80Obybaz8w4eECh8SjVQX9" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="544" data-original-width="848" height="417" src="https://blogger.googleusercontent.com/img/a/AVvXsEi2ZQmVaKxw78z2oRUZRf3ysioQ0qvO2J-72i685onr65e-WoiayAc3SR_Iqw1Wd6kY5GyFIVwgseeDqJ8JPE5FdE1UqrICyZHosyM4Y1wQmCO14VDHRuIGbpaBNVx1UK6PUrKjQYsrxIJdB58zn7z4eyIwxkfcwSPCBrgIEh80Obybaz8w4eECh8SjVQX9=w652-h417" width="652" /></a></div><br />Then we can define a <span style="font-family: courier;">coverage.settings</span> file (this is essentially a run setting file for unit tests) wit below content.<p></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: grey;"><?</span><span style="color: #569cd6;">xml</span><span style="color: #9cdcfe;"> version</span>=<span style="color: #ce9178;">"1.0"</span><span style="color: #9cdcfe;"> encoding</span>=<span style="color: #ce9178;">"utf-8"</span> <span style="color: grey;">?></span></div><div><span style="color: grey;"><</span><span style="color: #569cd6;">RunSettings</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">DataCollectionRunSettings</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">DataCollectors</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">DataCollector</span> <span style="color: #9cdcfe;">friendlyName</span>=<span style="color: #ce9178;">"XPlat code coverage"</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">Configuration</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">Format</span><span style="color: grey;">></span>Cobertura<span style="color: grey;"></</span><span style="color: #569cd6;">Format</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">ExcludeByFile</span><span style="color: grey;">></span>**/Resources/*.Designer.cs,**/Program.cs,**/Startup.cs<span style="color: grey;"></</span><span style="color: #569cd6;">ExcludeByFile</span><span style="color: grey;">></span></div><div> <span style="color: grey;"><</span><span style="color: #569cd6;">SkipAutoProps</span><span style="color: grey;">></span>true<span style="color: grey;"></</span><span style="color: #569cd6;">SkipAutoProps</span><span style="color: grey;">></span></div><div> <span style="color: grey;"></</span><span style="color: #569cd6;">Configuration</span><span style="color: grey;">></span></div><div> <span style="color: grey;"></</span><span style="color: #569cd6;">DataCollector</span><span style="color: grey;">></span></div><div> <span style="color: grey;"></</span><span style="color: #569cd6;">DataCollectors</span><span style="color: grey;">></span></div><div> <span style="color: grey;"></</span><span style="color: #569cd6;">DataCollectionRunSettings</span><span style="color: grey;">></span></div><div><span style="color: grey;"></</span><span style="color: #569cd6;">RunSettings</span><span style="color: grey;">></span></div></div><p>Using that coverage setting we can generate the <span style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">Cobertura </span>format report in pipeline tasks as shown below.</p><p>The additional argument passed to dotnet test are</p><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">--collect</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">"XPlat</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Code</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Coverage"</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">--settings</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">pipelines/settings/coverage.settings</span></pre><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><span style="background-color: #fff0f0;"><br /></span></pre><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><br /></pre><pre style="line-height: 16.25px; margin-bottom: 0px; margin-top: 0px;"><span style="background-color: #fff0f0;"><br /></span></pre><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">- task: DotNetCoreCLI@2
displayName: <span style="background-color: #fff0f0;">'dotnet</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">test'</span>
inputs:
command: test
projects: <span style="background-color: #fff0f0;">'**/*.tests.csproj'</span>
arguments: <span style="background-color: #fff0f0;">'--configuration</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">$(build_configuration)</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">--collect</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">"XPlat</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Code</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Coverage"</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">--settings</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">pipelines/settings/coverage.settings'</span>
publishTestResults: true
testRunTitle: <span style="background-color: #fff0f0;">'${{</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">parameters.build_os</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">}}</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Unit</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">Tests'</span>
testResultsFiles: <span style="background-color: #fff0f0;">'**/*.trx'</span>
workingDirectory: <span style="background-color: #fff0f0;">'$(build.sourcesdirectory)'</span>
- task: PublishCodeCoverageResults@1
displayName: <span style="background-color: #fff0f0;">'Publish</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">code</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">coverage</span><span style="color: #996633;"> </span><span style="background-color: #fff0f0;">report'</span>
inputs:
codeCoverageTool: <span style="background-color: #fff0f0;">'Cobertura'</span>
summaryFileLocation: <span style="background-color: #fff0f0;">'$(Agent.TempDirectory)/**/coverage.cobertura.xml'</span>
</pre></div>
<p>Once pipline with above task executed the code coverage report as shown in above expected outcome can be obtained.</p><p><br /></p><p><br /></p><p><br /></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-4192177135568510942023-10-13T08:06:00.001-07:002023-11-25T06:12:23.986-08:00Conditionally Passing Different Values to Template Parameters in Azure DevOps Pipelines<p>Parameters Azure pipeline template helps to achieve dynamic behaviours in pipeline templates. Passing different values for template parameters based on conditions, via a single usage of teplate will help to reduce duplication of usage of template in the pipeline. Let's look at how to conditionally pass different values to template paramters with a practical example.</p><p><b><u>The expected behavoir</u></b></p><p>Let's assume there are three stages in the pipeline.</p><p></p><ul style="text-align: left;"><li>Unit test stage - to build and unit test</li><li>Build and push Docker image stage</li><li>Deploy app stage</li></ul><p></p><p>If we are in develop branch (refs/heads/develop) we want to run unit tests, build dcoker image and depending on both stages want to run a deploy.<span></span></p><a name='more'></a><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg4jaofHiOdWF3A795m8I85uFxQgwIJ7pYTXENbCsE0VohXcZu5Cy4pf6T-xrExWsbvKk734Ve-E6UaeuO9HPMR4HrumM49ANEuv_zjP5HTG34p0o7OlnsEh8AXYy0tINJKKPBawlzeksUw62nFgIyVGPoGF1lUiL0JXCx8VHB_sB38ySWWqvxPc5p8kTSw" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="816" data-original-width="1117" height="398" src="https://blogger.googleusercontent.com/img/a/AVvXsEg4jaofHiOdWF3A795m8I85uFxQgwIJ7pYTXENbCsE0VohXcZu5Cy4pf6T-xrExWsbvKk734Ve-E6UaeuO9HPMR4HrumM49ANEuv_zjP5HTG34p0o7OlnsEh8AXYy0tINJKKPBawlzeksUw62nFgIyVGPoGF1lUiL0JXCx8VHB_sB38ySWWqvxPc5p8kTSw=w544-h398" width="544" /></a></div><br /><p></p><p>If any other branch we just want to run build docker image and deploy stages.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh0arlmdsPHcWo3FOOyrsesUYPa3eCFLLZsFj_DsVP1Gt_r0ZYaAqDZvmJQFc76ATQK9Kz5BKJLpzyy2Up3RVD2_oB4HPs4PSAIGbqRaBi4bbvdUDrkY76py38LTZ0j7AwpOYHbVA3_0tHHTmXZBDd3Ms_46hxOubAIB36Q3KJcYKTgrmrddtJZZBGQ76Lr" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="641" data-original-width="1069" height="323" src="https://blogger.googleusercontent.com/img/a/AVvXsEh0arlmdsPHcWo3FOOyrsesUYPa3eCFLLZsFj_DsVP1Gt_r0ZYaAqDZvmJQFc76ATQK9Kz5BKJLpzyy2Up3RVD2_oB4HPs4PSAIGbqRaBi4bbvdUDrkY76py38LTZ0j7AwpOYHbVA3_0tHHTmXZBDd3Ms_46hxOubAIB36Q3KJcYKTgrmrddtJZZBGQ76Lr=w538-h323" width="538" /></a></div><br /><p></p><p><b><u>How to do</u></b></p><p>To achieve above we have to have a conditional stage dependency set depending on the pipeline running branch. The deploy app stage can take an input parameter to get the dependson stages as shown below. The default value is set as both the unit test and docker byuld stages. Instead it is possible to set it as [] to have no depends on stages.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">parameters</span>:</div><div> - <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">dependson</span></div><div> <span style="color: #569cd6;">type</span>: <span style="color: #ce9178;">object</span></div><div> <span style="color: #569cd6;">default</span>: </div><div> - <span style="color: #ce9178;">build_and_unittest</span></div><div> - <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> - <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">appname</span></div><div> <span style="color: #569cd6;">type</span>: <span style="color: #ce9178;">string</span></div><br /><div><span style="color: #569cd6;">stages</span>:</div><div> - <span style="color: #569cd6;">stage</span>: <span style="color: #ce9178;">deploy_app</span></div><div> <span style="color: #569cd6;">dependsOn</span>: <span style="color: #ce9178;">${{ parameters.dependson }}</span></div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Deploy App'</span></div><div> <span style="color: #569cd6;">jobs</span>:</div><div> - <span style="color: #569cd6;">job</span>: <span style="color: #ce9178;">deploy_app</span></div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Deploy App'</span></div><div> <span style="color: #569cd6;">pool</span>:</div><div> <span style="color: #569cd6;">vmImage</span>: <span style="color: #ce9178;">'ubuntu-latest'</span></div><div> <span style="color: #569cd6;">steps</span>:</div><div> - <span style="color: #569cd6;">checkout</span>: <span style="color: #ce9178;">none</span></div><div> </div><div> - <span style="color: #569cd6;">script</span>: <span style="color: #c586c0;">|</span></div><div><span style="color: #ce9178;"> echo "Deploy App ${{ parameters.appname }}"</span></div></div><p>The docker build and unit test stages are somple and defined as shown below. Note in both below stages there is no dependsOn stage set so they run parallelly once the pipeline is started running.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">stages</span>:</div><div> - <span style="color: #569cd6;">stage</span>: <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">dependsOn</span>: []</div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Build and Push Docker Image'</span></div><div> <span style="color: #569cd6;">jobs</span>:</div><div> - <span style="color: #569cd6;">job</span>: <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Build and Push Docker Image'</span></div><div> <span style="color: #569cd6;">pool</span>:</div><div> <span style="color: #569cd6;">vmImage</span>: <span style="color: #ce9178;">'ubuntu-latest'</span></div><div> <span style="color: #569cd6;">steps</span>:</div><div> - <span style="color: #569cd6;">checkout</span>: <span style="color: #ce9178;">none</span></div><div> </div><div> - <span style="color: #569cd6;">script</span>: <span style="color: #c586c0;">|</span></div><div><span style="color: #ce9178;"> echo "Build and Push Docker Image"</span></div></div><p><br /></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">stages</span>:</div><div> - <span style="color: #569cd6;">stage</span>: <span style="color: #ce9178;">build_and_unittest</span></div><div> <span style="color: #569cd6;">dependsOn</span>: []</div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Build and Unit Test'</span></div><div> <span style="color: #569cd6;">jobs</span>:</div><div> - <span style="color: #569cd6;">job</span>: <span style="color: #ce9178;">build_and_unittest</span></div><div> <span style="color: #569cd6;">displayName</span>: <span style="color: #ce9178;">'Build and Unit Test'</span></div><div> <span style="color: #569cd6;">pool</span>:</div><div> <span style="color: #569cd6;">vmImage</span>: <span style="color: #ce9178;">'ubuntu-latest'</span></div><div> <span style="color: #569cd6;">steps</span>:</div><div> - <span style="color: #569cd6;">checkout</span>: <span style="color: #ce9178;">none</span></div><div> </div><div> - <span style="color: #569cd6;">script</span>: <span style="color: #c586c0;">|</span></div><div><span style="color: #ce9178;"> echo "Build and Unit Test"</span></div></div><p><br /></p><p>In the pipeline wich uses these three stages, unit test stage can be set to run only if the branch is develop as shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">stages</span>:</div><div> - <span style="color: #569cd6;">${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}</span>:</div><div> - <span style="color: #569cd6;">template</span>: <span style="color: #ce9178;">templates/stages/build_and_unittest.yml</span></div></div><p>Then when using the deploy app stage the depends on paramter can be set as conditionally as shown below. Note that unlike above condition we should not use a starting - when we set conditions for the parameters. If the branch is dev we pass both stages unit test and docker build as dependencies to the deploy app stage. If the the branch is not develop then just the build docker image passed as the dependson parameter value.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> - <span style="color: #569cd6;">template</span>: <span style="color: #ce9178;">templates/stages/deploy_app.yml</span></div><div> <span style="color: #569cd6;">parameters</span>:</div><div> <span style="color: #569cd6;">${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}</span>:</div><div> <span style="color: #569cd6;">dependson</span>:</div><div> - <span style="color: #ce9178;">build_and_unittest</span></div><div> - <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">${{ else }}</span>:</div><div> <span style="color: #569cd6;">dependson</span>:</div><div> - <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">appname</span>: <span style="color: #ce9178;">'myapp'</span></div></div><p>The pipline code using all stages is as below. This way we can use various different conditions in paramters passing to templates.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">pool</span>:</div><div> <span style="color: #569cd6;">vmimage</span>: <span style="color: #ce9178;">'ubuntu-latest'</span></div><br /><div><span style="color: #569cd6;">stages</span>:</div><div> - <span style="color: #569cd6;">${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}</span>:</div><div> - <span style="color: #569cd6;">template</span>: <span style="color: #ce9178;">templates/stages/build_and_unittest.yml</span></div><div> </div><div> - <span style="color: #569cd6;">template</span>: <span style="color: #ce9178;">templates/stages/build_and_push_docker_image.yml</span></div><div> - <span style="color: #569cd6;">template</span>: <span style="color: #ce9178;">templates/stages/deploy_app.yml</span></div><div> <span style="color: #569cd6;">parameters</span>:</div><div> <span style="color: #569cd6;">${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}</span>:</div><div> <span style="color: #569cd6;">dependson</span>:</div><div> - <span style="color: #ce9178;">build_and_unittest</span></div><div> - <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">${{ else }}</span>:</div><div> <span style="color: #569cd6;">dependson</span>:</div><div> - <span style="color: #ce9178;">build_and_push_docker_image</span></div><div> <span style="color: #569cd6;">appname</span>: <span style="color: #ce9178;">'myapp'</span></div></div><p><br /></p><iframe width="560" height="315" src="https://www.youtube.com/embed/JMgGQdbbVxY?si=BBAp0-AmGXXGpics" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-75659486128703681012023-09-28T05:18:00.010-07:002023-09-28T05:21:57.665-07:00Reducing Log Analytics Cost by Preventing Container Logs Ingession from Azure Kubernetes Services (AKS) <p> Monitoring is an essential part of a deployment of software on aplatform such as AKS. However, once monitoring enabled there could be significant cost involved for monitoring data. When we enable log analytics workspace to ingest monitoring data from AKS, by default AKS will ingest all container logs, except for <span style="font-family: courier;">kube-system </span>and <span style="font-family: courier;">gatekeeper-system </span>namespaces. If our application are having large amout of container logs generated, then the cost will be lot higher for log analytics workspace. In case we are using app insights/ or an alternative system to monitor the application logs, for the applications deployed to AKS, we have enuough information to diagnose issues etc. Therefore, to reduce unnecessary cost log analytics can be done, by preventing AKS ingestion of the container logs to log analytics workspace. Let's explore the steps.</p><p><b><span style="font-size: medium;"><u>Expected outcome</u></span></b></p><p>As shown in the below figure, the log analytics workspace ingestion over time chart, is indicating the container logs data is no longer getting ingested after the change is applied.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgBUUb2WRfBVfTCvy_-51Xs5Ni1RlWLKvHcZ4pMvItuDGPBJCRGYmRPqkM_lfTeniCFlmRxaG9DiWNp4xRaZ7kOCSjv9Cja8y5mNm9_YnhE6L0VKZpIpRx-8Pr_pD04VhYkakG_SBtWTGCwxFG75BEdymGTEpdHggSDSv4UerJldbPtSz8vraEHhGizlHTu" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="598" data-original-width="871" height="408" src="https://blogger.googleusercontent.com/img/a/AVvXsEgBUUb2WRfBVfTCvy_-51Xs5Ni1RlWLKvHcZ4pMvItuDGPBJCRGYmRPqkM_lfTeniCFlmRxaG9DiWNp4xRaZ7kOCSjv9Cja8y5mNm9_YnhE6L0VKZpIpRx-8Pr_pD04VhYkakG_SBtWTGCwxFG75BEdymGTEpdHggSDSv4UerJldbPtSz8vraEHhGizlHTu=w592-h408" width="592" /></a></div><br /> <span><a name='more'></a></span>If we ispect in more detail in log analytics usage table with below query, we can see over a 90 day perioed the highest potion (nearly 52%) of log analytics cost was due to the container logs from AKS.<p></p><div style="background-color: #111217; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>Usage</div><div>| where TimeGenerated > startofday(ago(90d)) </div><div>| where IsBillable == true</div><div>| summarize IngestedGB = sum(Quantity) / 1000 by DataType</div><div>| sort by IngestedGB desc</div><div>| render piechart</div></div><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhxAguWWo730ZDcAzcc809lu8Z29w9Qys0YXwWNeKZRyjcaBkwZ8jP3jy_W_2uZcCLCk0g4tpSTCnumHMirsfUbcT1jfldoKhS9NqYtjDDdo1jPeO9_KcYR_C2zu6H23dJoDj4_IOKnXDVFgZ8pDP8NDqPSjBXXvFG-Da_OrX9XiqLHfrLnTXl2_sZrI--s" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="638" data-original-width="1235" height="327" src="https://blogger.googleusercontent.com/img/a/AVvXsEhxAguWWo730ZDcAzcc809lu8Z29w9Qys0YXwWNeKZRyjcaBkwZ8jP3jy_W_2uZcCLCk0g4tpSTCnumHMirsfUbcT1jfldoKhS9NqYtjDDdo1jPeO9_KcYR_C2zu6H23dJoDj4_IOKnXDVFgZ8pDP8NDqPSjBXXvFG-Da_OrX9XiqLHfrLnTXl2_sZrI--s=w633-h327" width="633" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjjShL9NIvYfGc9rPwRfpGd6gFJ2kVfcy8fzOrah2xeYUZXfsVF9FUv1Y4Y9NGECuxKgXA491UmTQqIH7c_XSOon9XVlT1ODVxQjTDEo5EOwRGz6eYkpXl22uKA72orvm3Uh_vQuhQbZxtHGBA91d3uOQRuuS7tWd6pndLPFbxriedXDrd3kwUn2xTbTcw5" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="521" data-original-width="701" height="443" src="https://blogger.googleusercontent.com/img/a/AVvXsEjjShL9NIvYfGc9rPwRfpGd6gFJ2kVfcy8fzOrah2xeYUZXfsVF9FUv1Y4Y9NGECuxKgXA491UmTQqIH7c_XSOon9XVlT1ODVxQjTDEo5EOwRGz6eYkpXl22uKA72orvm3Uh_vQuhQbZxtHGBA91d3uOQRuuS7tWd6pndLPFbxriedXDrd3kwUn2xTbTcw5=w596-h443" width="596" /></a></div><br /><br /></div><div><p>For the last 24 hours, the container logs is about 6% (0.5GB of data).</p><div style="background-color: #111217; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>Usage</div><div>| where TimeGenerated > ago(24h) </div><div>| where IsBillable == true</div><div>| summarize IngestedGB = sum(Quantity) / 1000 by DataType</div><div>| sort by IngestedGB desc</div><div>| render piechart</div></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjxzFrKbDBuCGCcmPes9se_fXToOsAiNsP321CPQ8PTWZqPo-UZaWkTfn6VCWsJKxLeFXzaCBGvBmxDb8MVePcIoKHFp5CwDdY27eeptwIclkbDGoVQeZUehogiqsy4Mkd-cJ6ajWwhl9p4AnM6gM4dfMMWpwBWaDq7SDofm6YQSKYZ5a_GViERHdGuhQhR" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="644" data-original-width="1220" height="341" src="https://blogger.googleusercontent.com/img/a/AVvXsEjxzFrKbDBuCGCcmPes9se_fXToOsAiNsP321CPQ8PTWZqPo-UZaWkTfn6VCWsJKxLeFXzaCBGvBmxDb8MVePcIoKHFp5CwDdY27eeptwIclkbDGoVQeZUehogiqsy4Mkd-cJ6ajWwhl9p4AnM6gM4dfMMWpwBWaDq7SDofm6YQSKYZ5a_GViERHdGuhQhR=w646-h341" width="646" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgVL8Ble_Q8OLTFNhIBCO_XuV6V8huP79lXXIQ3e-MHy5e6tjRTOVSRDfO9ewQKFJ77fg9eQNOqqjPuVxL5vjJZFU1hOYs6R6wIQm5gO1MHnKKqcXW1Q1HeMLXnhcPQ77g9WRG7Y_HVHuR64Pd6IyLLH0-X3tDfwEuStLUEjM6xjQln_uzKv9WgyPm14Jua" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="653" data-original-width="893" height="449" src="https://blogger.googleusercontent.com/img/a/AVvXsEgVL8Ble_Q8OLTFNhIBCO_XuV6V8huP79lXXIQ3e-MHy5e6tjRTOVSRDfO9ewQKFJ77fg9eQNOqqjPuVxL5vjJZFU1hOYs6R6wIQm5gO1MHnKKqcXW1Q1HeMLXnhcPQ77g9WRG7Y_HVHuR64Pd6IyLLH0-X3tDfwEuStLUEjM6xjQln_uzKv9WgyPm14Jua=w615-h449" width="615" /></a></div><br /><br /><p></p><p>However, after applying change, if we inspect the last 22 hour ingest data usage table for billable data in log analytics workspace, we can hardly see any ingestion of container logs (only few MBs). This is just beasue the change was applied just at the begining of last 22 hour.</p><div style="background-color: #111217; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>Usage</div><div>| where TimeGenerated > ago(22h) </div><div>| where IsBillable == true</div><div>| summarize IngestedGB = sum(Quantity) / 1000 by DataType</div><div>| sort by IngestedGB desc</div><div>| render piechart</div></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgEifGXDFodrgOfKr37Y7CQfoiShGapv6-TMS564-LopaMyF4jbsGovME_hTuzmVEQz5U1i-LHmib9zvAAYy9jfB80qQG78pTwx7VtY0UpUOLAmyvpEkIAnyfW2R8-pQAI7-BGXIZbUgYIuoX_n1FZNGCp3EfLtK9TFyEDSo9FHlWQZ_-fGhVoGDVxC3Wtv" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="634" data-original-width="1217" height="334" src="https://blogger.googleusercontent.com/img/a/AVvXsEgEifGXDFodrgOfKr37Y7CQfoiShGapv6-TMS564-LopaMyF4jbsGovME_hTuzmVEQz5U1i-LHmib9zvAAYy9jfB80qQG78pTwx7VtY0UpUOLAmyvpEkIAnyfW2R8-pQAI7-BGXIZbUgYIuoX_n1FZNGCp3EfLtK9TFyEDSo9FHlWQZ_-fGhVoGDVxC3Wtv=w640-h334" width="640" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiGWGPJBHI22ZQ0KU3KSEzBASrkOxHvAmFkvMDhuSW8AModPmjxemKyBbas1AkU5xPPyS7xXX5xNjp9LfPIVWcEL7VYbvTOQL19fMpoV_xW1q1a3Xq3f54pxT7pUewTfhpc096iHwSjFbumPb3M-Bs_jJaXtskuMrmd64E-YSUq7LppUKtwtP75OHkDRTUN" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="638" data-original-width="1196" height="336" src="https://blogger.googleusercontent.com/img/a/AVvXsEiGWGPJBHI22ZQ0KU3KSEzBASrkOxHvAmFkvMDhuSW8AModPmjxemKyBbas1AkU5xPPyS7xXX5xNjp9LfPIVWcEL7VYbvTOQL19fMpoV_xW1q1a3Xq3f54pxT7pUewTfhpc096iHwSjFbumPb3M-Bs_jJaXtskuMrmd64E-YSUq7LppUKtwtP75OHkDRTUN=w630-h336" width="630" /></a></div><br /><p></p><p>For last 21 hours considered there is no billable usage of container logs ingestion whoch shows, the change is effective and AKS no loger ingest container logs to log analytics workspace.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjZqGgypAEwaD2SiyCfVVmR8yczzRFmMMRRJDNWgarWSoDS19enClSyR6Nfbze41D-W195l5XrcnoJwaK2uosgTa3cZnFjZXF57aVKfpBNne9w_AV5knlgxqhznQhn5Wma3_GxNn9afjbljITtzUj-nOSSxcnttfac3l1t4fuMEJDCjc4t_A5v2qhPhI-ts" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="645" data-original-width="1191" height="348" src="https://blogger.googleusercontent.com/img/a/AVvXsEjZqGgypAEwaD2SiyCfVVmR8yczzRFmMMRRJDNWgarWSoDS19enClSyR6Nfbze41D-W195l5XrcnoJwaK2uosgTa3cZnFjZXF57aVKfpBNne9w_AV5knlgxqhznQhn5Wma3_GxNn9afjbljITtzUj-nOSSxcnttfac3l1t4fuMEJDCjc4t_A5v2qhPhI-ts=w643-h348" width="643" /></a></div><br /><br /><p></p><p><b><span style="font-size: medium;"><u>How to apply the change</u></span></b></p><p>As <a href="https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-agent-config?wt.mc_id=DT-MVP-5000590#data-collection-settings" target="_blank">documented here</a> we can create a kubernetes config map in AKS, for disabling container logs ingest to log analytics workspace. The exclude namespace is effective only when you want to enable container log ingestion to namespaces other than excluded ones.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEit8b5AezdsvsESKFZbovNwNqonZn7BAM4ZObQbWJ356AWHmlJ8ajUnHJSvGX74vzAh15ylSREdGNixJ7kyILL1l5jfGAZmDfVwqstDuhJo5mBs90T_cPIxtU_eZa7zY7H6mZXL0fZZyp6Wh2QRDXJfKr3jT2vmNJ18VL8JdeuA7HbDI7FYOyrOmvGzCqZs" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="724" data-original-width="859" height="516" src="https://blogger.googleusercontent.com/img/a/AVvXsEit8b5AezdsvsESKFZbovNwNqonZn7BAM4ZObQbWJ356AWHmlJ8ajUnHJSvGX74vzAh15ylSREdGNixJ7kyILL1l5jfGAZmDfVwqstDuhJo5mBs90T_cPIxtU_eZa7zY7H6mZXL0fZZyp6Wh2QRDXJfKr3jT2vmNJ18VL8JdeuA7HbDI7FYOyrOmvGzCqZs=w613-h516" width="613" /></a></div><br />The config map yaml below can be used for the purpose. <a href="https://raw.githubusercontent.com/microsoft/Docker-Provider/ci_prod/kubernetes/container-azm-ms-agentconfig.yaml" target="_blank">The full details of the file can be found here</a>.<p></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># Refer below for settings in this config map</span></div><div><span style="color: #6a9955;"># https://raw.githubusercontent.com/microsoft/Docker-Provider/ci_prod/kubernetes/container-azm-ms-agentconfig.yaml</span></div><div><span style="color: #6a9955;"># https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-agent-config#data-collection-settings</span></div><div><span style="color: #569cd6;">kind</span>: <span style="color: #ce9178;">ConfigMap</span></div><div><span style="color: #569cd6;">apiVersion</span>: <span style="color: #ce9178;">v1</span></div><div><span style="color: #569cd6;">data</span>:</div><div> <span style="color: #569cd6;">schema-version</span>:</div><div> <span style="color: #ce9178;">v1</span></div><div> <span style="color: #569cd6;">config-version</span>:</div><div> <span style="color: #ce9178;">ver1</span></div><div> <span style="color: #569cd6;">log-data-collection-settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [log_collection_settings]</span></div><div><span style="color: #ce9178;"> [log_collection_settings.stdout]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div><span style="color: #ce9178;"> exclude_namespaces = ["kube-system","gatekeeper-system"]</span></div><div><span style="color: #ce9178;"> [log_collection_settings.stderr]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div><span style="color: #ce9178;"> exclude_namespaces = ["kube-system","gatekeeper-system"]</span></div><div><span style="color: #ce9178;"> [log_collection_settings.env_var]</span></div><div><span style="color: #ce9178;"> enabled = true</span></div><div><span style="color: #ce9178;"> [log_collection_settings.enrich_container_logs]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div><span style="color: #ce9178;"> [log_collection_settings.collect_all_kube_events]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div> <span style="color: #569cd6;">prometheus-data-collection-settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [prometheus_data_collection_settings.cluster]</span></div><div><span style="color: #ce9178;"> interval = "1m"</span></div><div><span style="color: #ce9178;"> monitor_kubernetes_pods = false</span></div><div><span style="color: #ce9178;"> [prometheus_data_collection_settings.node]</span></div><div><span style="color: #ce9178;"> interval = "1m"</span></div><div> <span style="color: #569cd6;">metric_collection_settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [metric_collection_settings.collect_kube_system_pv_metrics]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div> <span style="color: #569cd6;">alertable-metrics-configuration-settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [alertable_metrics_configuration_settings.container_resource_utilization_thresholds]</span></div><div><span style="color: #ce9178;"> container_cpu_threshold_percentage = 95.0</span></div><div><span style="color: #ce9178;"> container_memory_rss_threshold_percentage = 95.0</span></div><div><span style="color: #ce9178;"> container_memory_working_set_threshold_percentage = 95.0</span></div><div><span style="color: #ce9178;"> [alertable_metrics_configuration_settings.pv_utilization_thresholds]</span></div><div><span style="color: #ce9178;"> pv_usage_threshold_percentage = 60.0</span></div><div><span style="color: #ce9178;"> [alertable_metrics_configuration_settings.job_completion_threshold]</span></div><div><span style="color: #ce9178;"> job_completion_threshold_time_minutes = 360</span></div><div> <span style="color: #569cd6;">integrations</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [integrations.azure_network_policy_manager]</span></div><div><span style="color: #ce9178;"> collect_basic_metrics = false</span></div><div><span style="color: #ce9178;"> collect_advanced_metrics = false</span></div><div><span style="color: #ce9178;"> [integrations.azure_subnet_ip_usage]</span></div><div><span style="color: #ce9178;"> enabled = false</span></div><div> <span style="color: #569cd6;">agent-settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [agent_settings.prometheus_fbit_settings]</span></div><div><span style="color: #ce9178;"> tcp_listener_chunk_size = 10</span></div><div><span style="color: #ce9178;"> tcp_listener_buffer_size = 10</span></div><div><span style="color: #ce9178;"> tcp_listener_mem_buf_limit = 200</span></div><div><span style="color: #569cd6;">metadata</span>:</div><div> <span style="color: #569cd6;">name</span>: <span style="color: #ce9178;">container-azm-ms-agentconfig</span></div><div> <span style="color: #569cd6;">namespace</span>: <span style="color: #ce9178;">kube-system</span></div></div><p>The above config map fill be automatically loaded by the <span style="font-family: courier;">ama-logs</span> and <span style="font-family: courier;">ama-logs-windows </span>(If you have windows containers as well) pods. </p><p><span style="font-family: courier;">kubectl get pods -n kube-system</span></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgBjUHuTifEVwdGbOQesZ7aNuozJGjfc8vGNBu2eRbVn0VlFVr4C_QyVGWA2HHZYmoNw0YqGWHqjM-ZC2kIL504QlD6ZhFWOTyr_y7Msp8vYeUkd7PEfugBxeVFPvF9ko2PqaDP0pcITOZcerTSwH6ZxlFE50Fv0P8Nf9fXuH59LlBtDB3eQe-H7oxZtiwT" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="546" data-original-width="791" height="404" src="https://blogger.googleusercontent.com/img/a/AVvXsEgBjUHuTifEVwdGbOQesZ7aNuozJGjfc8vGNBu2eRbVn0VlFVr4C_QyVGWA2HHZYmoNw0YqGWHqjM-ZC2kIL504QlD6ZhFWOTyr_y7Msp8vYeUkd7PEfugBxeVFPvF9ko2PqaDP0pcITOZcerTSwH6ZxlFE50Fv0P8Nf9fXuH59LlBtDB3eQe-H7oxZtiwT=w585-h404" width="585" /></a></div><br /><span style="font-family: courier;">kubectl logs ama-logs-9t42x -n kube-system --timestamps=true</span><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgY75PabMco89NYdxuVgmXh4Dsz6Rn6y3zYU4IW6mTk7ty1QNDr71FtFg68iRHH9rek9pdcSPokiK4NIKVoRLLQ2eqp5nY7i6yBuYwEEgtJY86ssY4bXxYOQRKFbWnwQp11nWBZKdgSh2U3nIVqSchd7PYYs78HNduiVG6w2lz3AYxI4ZYSLUXYmJurpBR0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="250" data-original-width="1040" height="158" src="https://blogger.googleusercontent.com/img/a/AVvXsEgY75PabMco89NYdxuVgmXh4Dsz6Rn6y3zYU4IW6mTk7ty1QNDr71FtFg68iRHH9rek9pdcSPokiK4NIKVoRLLQ2eqp5nY7i6yBuYwEEgtJY86ssY4bXxYOQRKFbWnwQp11nWBZKdgSh2U3nIVqSchd7PYYs78HNduiVG6w2lz3AYxI4ZYSLUXYmJurpBR0=w654-h158" width="654" /></a></div><br /><br /><p></p><p>You could enable container log ingestion and still disable it for given namespace say "demo" by using settings as below. Container logs will be ingested for namespaces other than excluded onces, which are "kube-system","gatekeeper-system" and "demo". </p><div style="color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"><div style="background-color: #1f1f1f; line-height: 19px;"><div><span style="color: #569cd6;">log-data-collection-settings</span>: <span style="color: #c586c0;">|</span><span style="color: #569cd6;">-</span></div><div><span style="color: #ce9178;"> [log_collection_settings]</span></div><div><span style="color: #ce9178;"> [log_collection_settings.stdout]</span></div><div><span style="color: #ce9178;"> enabled = true</span></div><div><span style="color: #ce9178;"> exclude_namespaces = ["kube-system","gatekeeper-system","demo"]</span></div><div><span style="color: #ce9178;"> [log_collection_settings.stderr]</span></div><div><span style="color: #ce9178;"> enabled = true</span></div><div><span style="color: #ce9178;"> exclude_namespaces = ["kube-system","gatekeeper-system","demo"]</span></div></div></div><p><br /></p></div>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-43840649331793994782023-09-17T05:17:00.002-07:002024-03-10T08:10:26.568-07:00Passing Environment Variables to Reusable Workflows in GitHub Actions<p> We have discueed, that we have to use an environment variable to handle input parameter default values, if we are using trigger for workflow <span style="font-family: courier;">on push</span> in the post "<a href="https://chamindac.blogspot.com/2023/09/setting-workflow-environment-variable.html" target="_blank">Setting Workflow Environment Variable Based on Input Parameter in GitHub Actions - on workflow_dispatch and Use a Default Value on push</a>". If we have to pass on the input paramter value from a workflow to a reusable workflow, it does not work as expected and it is a <a href="https://docs.github.com/en/actions/using-workflows/reusing-workflows?wt.mc_id=DT-MVP-5000590#limitations" target="_blank">limitation of reusable workflows as explained in here</a>. Let's try to understand with an example how to pass an env variable to a reusable workflow.<span></span></p><a name='more'></a><p></p><p>We can define a reusable workflow as shown below. The trigger should be set as <span style="font-family: courier;">workflow_call </span>to make a workflow reusable in another workflow. We have defined an input varaible for the reusable workflow as shown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi2pgFEeElpoFFT2q761mEm0QgY1k_E93G02BfiHvPK4qIogs975mqZ6mSI85am1n6VV3sYESc6i8xMrAZ6TNzL_Xkvz8A9C_pWrKNoPZUnkhiMO3Z1wiwi7mKQRlNIbWwvcXR_w3_2ALI8u3a7fufpYa57gLsNr075KdWs2dVd6qYxYeXLscVEZ4peIFRc" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1028" data-original-width="1872" height="323" src="https://blogger.googleusercontent.com/img/a/AVvXsEi2pgFEeElpoFFT2q761mEm0QgY1k_E93G02BfiHvPK4qIogs975mqZ6mSI85am1n6VV3sYESc6i8xMrAZ6TNzL_Xkvz8A9C_pWrKNoPZUnkhiMO3Z1wiwi7mKQRlNIbWwvcXR_w3_2ALI8u3a7fufpYa57gLsNr075KdWs2dVd6qYxYeXLscVEZ4peIFRc=w587-h323" width="587" /></a></div><br /><br /><p></p><p>Then before we attempt to use the reusable workflow above let's try to pass an input variable to workflow and make it work wiith an envronment variable to ensure it is available to use, when we use <span style="font-family: courier;">on push </span>trigger or <span style="font-family: courier;">on workflow_dispatch </span>trigger as described in the post "<a href="https://chamindac.blogspot.com/2023/09/setting-workflow-environment-variable.html" target="_blank">Setting Workflow Environment Variable Based on Input Parameter in GitHub Actions - on workflow_dispatch and Use a Default Value on push</a>".</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEips9VLIe11bzt13AAcR5ZeOEC0ruIaLjkkAUwbPwWNaM-SgUJpb3kaKy7Od9CzpAiHj13Y0QaG-oAWUU4RfbxzWaji5HWlsRIUr-XUCgvqXfEaL6LTNghhHe1cIZd0st4W9l1mccn4DRKUf70DNS6o_AMY6uEpYbAkrDsveC39AiKoI3pcnan5h1hBZ2aT" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1197" data-original-width="2047" height="352" src="https://blogger.googleusercontent.com/img/a/AVvXsEips9VLIe11bzt13AAcR5ZeOEC0ruIaLjkkAUwbPwWNaM-SgUJpb3kaKy7Od9CzpAiHj13Y0QaG-oAWUU4RfbxzWaji5HWlsRIUr-XUCgvqXfEaL6LTNghhHe1cIZd0st4W9l1mccn4DRKUf70DNS6o_AMY6uEpYbAkrDsveC39AiKoI3pcnan5h1hBZ2aT=w602-h352" width="602" /></a></div><br />When we run the above workflow it shows that <span style="background-color: white; color: #222222; font-family: courier; font-size: 13.2px;">${{ env.apps }}</span> has a value.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhd8nsJ9Rsz8VRbvIsrcSMJ5_CcIlxZMVO78tA4iEUia84tWeIFqayQ1JmRGbj7LH9whbRqpxCvzlojOwBhsSsuqai5603skoy8sJFNqOdUyYY5Om3rXWBlr_6VpzuSZCqR90nRXrroFR302XuUGf0r2j8q_4avRxQyKW-QktLVHXTIVKA7QDJa7hqfIM6H" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="819" data-original-width="1684" height="290" src="https://blogger.googleusercontent.com/img/a/AVvXsEhd8nsJ9Rsz8VRbvIsrcSMJ5_CcIlxZMVO78tA4iEUia84tWeIFqayQ1JmRGbj7LH9whbRqpxCvzlojOwBhsSsuqai5603skoy8sJFNqOdUyYY5Om3rXWBlr_6VpzuSZCqR90nRXrroFR302XuUGf0r2j8q_4avRxQyKW-QktLVHXTIVKA7QDJa7hqfIM6H=w595-h290" width="595" /></a></div><br /><p></p><p>Let's try to pass the same to the reusable workflow and see what happens. As you can see below we can refer to the reusable workflow on the same repo as shown below..</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgapYyT-ZVWHGnQ2AvcTu6FgKwmm-5giZ9d99Yt3xvxuSC7RZnXutDXVLEPfRvJpXDbp9X0giqtfiNeeuwVRbGRZqRej-HNCzkZe81L7I-7FQxA7HJsA9kymJMJMhMkpHymVQXRt-1k2udziimfzIKvIFnFcGyLYux0epcKBE4DezvrIIIXbyizUhN1tH6T" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1388" data-original-width="2063" height="414" src="https://blogger.googleusercontent.com/img/a/AVvXsEgapYyT-ZVWHGnQ2AvcTu6FgKwmm-5giZ9d99Yt3xvxuSC7RZnXutDXVLEPfRvJpXDbp9X0giqtfiNeeuwVRbGRZqRej-HNCzkZe81L7I-7FQxA7HJsA9kymJMJMhMkpHymVQXRt-1k2udziimfzIKvIFnFcGyLYux0epcKBE4DezvrIIIXbyizUhN1tH6T=w617-h414" width="617" /></a></div><br />However, when we try to pass env variable to the reusable workflow, the workflow shows an error message <span face="ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace" style="background-color: white; color: #656d76; font-size: 12px; white-space-collapse: preserve;">The workflow is not valid. .github/workflows/all_apps_cicd.yml (Line: 26, Col: 13): Unrecognized named-value: 'env'. Located at position 1 within expression: env.apps</span>. This is due to the <a href="https://docs.github.com/en/actions/using-workflows/reusing-workflows?wt.mc_id=DT-MVP-5000590#limitations" target="_blank">limitations described here in GitHub documentation</a> regarding env variables usage in reusable workflows.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEikPttOFnfd1v9euikxRrnX0cPeZ7bptJQkoHhaqcGeR9mVZH_iOuxNo33N5qVo0fry016id0f-PfN8udUvYyQYzSsyZ0aIfQx5CmHy_Ot5-rckLua8vldRvVWhlHXj9HZzUm8BWoW0fAHoONtFeOfkVWFxgyowjccVp8GU6lOpQdgHWipG8ZiXzhK9Q12N" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1587" data-original-width="2744" height="359" src="https://blogger.googleusercontent.com/img/a/AVvXsEikPttOFnfd1v9euikxRrnX0cPeZ7bptJQkoHhaqcGeR9mVZH_iOuxNo33N5qVo0fry016id0f-PfN8udUvYyQYzSsyZ0aIfQx5CmHy_Ot5-rckLua8vldRvVWhlHXj9HZzUm8BWoW0fAHoONtFeOfkVWFxgyowjccVp8GU6lOpQdgHWipG8ZiXzhK9Q12N=w621-h359" width="621" /></a></div><br />To overcome this issue, as a workaround, we have to use env variable in an additional job, which runs prior to all reusable workflows. In that prerequisite job we have to output the env variable value and use that output variable in the reusable workflow call input, as shown below. The reusable workflow call set to be dependent (using needs job) on the prerequisite <span style="font-family: courier;">env_vars </span>job which is exposing the env variable as output variable. Then we can pass the output using <span style="font-family: courier;">needs.env_vars.outputs.apps</span> to the reusable workflow.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEidFXfwIGgKF0t_nndKYR8sTA2sxd4BEFwRZrKcTs2sGMeBNaOXKOMPDzspMSMkxIHr7X7sxoZVlUb3Qu9ee4pHj9Z5idRWRpCUTzEOK4ozl6F9-IGlcEERpko5sJtfp5iRmj54U81gsKDCD1qJ6d5Ix-qywO3kZT9aJIJJ78OfYVRuWSC-9yVoleG1VlJ2" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1522" data-original-width="2059" height="472" src="https://blogger.googleusercontent.com/img/a/AVvXsEidFXfwIGgKF0t_nndKYR8sTA2sxd4BEFwRZrKcTs2sGMeBNaOXKOMPDzspMSMkxIHr7X7sxoZVlUb3Qu9ee4pHj9Z5idRWRpCUTzEOK4ozl6F9-IGlcEERpko5sJtfp5iRmj54U81gsKDCD1qJ6d5Ix-qywO3kZT9aJIJJ78OfYVRuWSC-9yVoleG1VlJ2=w637-h472" width="637" /></a></div><br /><br /><p></p><p>The reusable workflow runs after the <span style="font-family: courier;">env_vars </span>job.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEj2KhgQuudH8uAHXELgtK-6gSwY7dHBZJQwLj0AptCZ_J2xXUubbgtvttyv9EfXdsoJ6IRzkCbmQt3mlSS2spVek0FdDvQE0Ba_ceLAN-LQhcrjLquBsWR2uwPWyey_25SRs0VEacO0PvwJv7nIxxIHt8TKNHs_vOqUn59LmzTbZawq4FGhseGomi2QrTou" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="948" data-original-width="1921" height="289" src="https://blogger.googleusercontent.com/img/a/AVvXsEj2KhgQuudH8uAHXELgtK-6gSwY7dHBZJQwLj0AptCZ_J2xXUubbgtvttyv9EfXdsoJ6IRzkCbmQt3mlSS2spVek0FdDvQE0Ba_ceLAN-LQhcrjLquBsWR2uwPWyey_25SRs0VEacO0PvwJv7nIxxIHt8TKNHs_vOqUn59LmzTbZawq4FGhseGomi2QrTou=w586-h289" width="586" /></a></div><br />With this workaround we can get the env variable passed to the reusable workflow. <a href="https://github.com/chamindac/aks_eventhubs_blue_green/tree/b347ae03d81c6a33dd1fd0258f99555ee2cc151a/.github/workflows" target="_blank">The full example code used in this post is available in GitHub here</a>.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhl9aafUUKzEFn5zSZRNc-CUviTB5rxL9yyh8INxks1yguHaSyRZz-Y-ohlILn1UFVQZqC71jlgA5QzTiuNbRHkE1Rb4q4tt1KmYeLheH4G8uRkbAUoha7QRDsM4Stix1MIG0syhO1IjcrFFnm-JoD8X29bVyCW_7wqYjfa2TITF3nMG5_mMCWeAHHE6nUM" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="962" data-original-width="1560" height="349" src="https://blogger.googleusercontent.com/img/a/AVvXsEhl9aafUUKzEFn5zSZRNc-CUviTB5rxL9yyh8INxks1yguHaSyRZz-Y-ohlILn1UFVQZqC71jlgA5QzTiuNbRHkE1Rb4q4tt1KmYeLheH4G8uRkbAUoha7QRDsM4Stix1MIG0syhO1IjcrFFnm-JoD8X29bVyCW_7wqYjfa2TITF3nMG5_mMCWeAHHE6nUM=w567-h349" width="567" /></a></div><br /><br /><p></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-19654052669206160742023-09-14T00:49:00.003-07:002024-03-10T08:07:04.107-07:00Setting Workflow Environment Variable Based on Input Parameter in GitHub Actions - on workflow_dispatch and Use a Default Value on push<p> GitHub actions workflows support input only on manual trigger <span style="font-family: courier;">workflow_dispatch</span>. What if we need to use default value in other triggers such as on <span style="font-family: courier;">push </span>and use input parameter in case of manual trigger <span style="font-family: courier;">workflow_dispatch</span>? Let's explore our options with an example.<span></span></p><a name='more'></a><p></p><p>Consider we want to pass list of app names we want to build and deploy in the pipeline as an input parameter. If we want to omit app we should be able to do so while triggering the workflow manually. If pipeline is triggered automatically all apps listed in paramter as default value should be deployed.</p><p>We can define input parameter for the pipeline as shwon below. Why below app list defined as a json array syntax? We can discuss the reason in another post. For now lets try to assume the value of all apps should be listed this way.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjL4VE6aZ-qFhm1Epzx7BsUkRk83N3hM2720gP9YNv91DugJqPbvW4PQ06U_CMEqp6uoLoZ1cMPnCPqbNWPoWz5jbyVjAcYgA145PHGuy6kw-8zQGo_oNYQqws9P5wLCjVTlwD-bnY16UaGSmLABDwYLrpE1oO4GYgvyu7-XRLAgz4rZNKDb1xJXXIEkJbB" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="205" data-original-width="441" height="283" src="https://blogger.googleusercontent.com/img/a/AVvXsEjL4VE6aZ-qFhm1Epzx7BsUkRk83N3hM2720gP9YNv91DugJqPbvW4PQ06U_CMEqp6uoLoZ1cMPnCPqbNWPoWz5jbyVjAcYgA145PHGuy6kw-8zQGo_oNYQqws9P5wLCjVTlwD-bnY16UaGSmLABDwYLrpE1oO4GYgvyu7-XRLAgz4rZNKDb1xJXXIEkJbB=w607-h283" width="607" /></a></div><br /><br /><p></p><p>Then if we try to print this input in our workflow job as shown below it should print the default value.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEijtT9tv7ytH_-Lxm9Nj_njWHyqdBVYdSYm7dMGFQHnj_cQJP9wjDXIbtfacXbSFKiaHDoiLVnBBNgHFDJQhKSnZmpP5ylQBXrqGau_ORYthVtcFGhZGYZNLnF00ZWzZ1TneF_kqi7kFbmlHxUvTGiXQijqAYImQZ15U1Q_X3l9-plmzv8utBWIwZ7fIh3a" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="157" data-original-width="367" height="262" src="https://blogger.googleusercontent.com/img/a/AVvXsEijtT9tv7ytH_-Lxm9Nj_njWHyqdBVYdSYm7dMGFQHnj_cQJP9wjDXIbtfacXbSFKiaHDoiLVnBBNgHFDJQhKSnZmpP5ylQBXrqGau_ORYthVtcFGhZGYZNLnF00ZWzZ1TneF_kqi7kFbmlHxUvTGiXQijqAYImQZ15U1Q_X3l9-plmzv8utBWIwZ7fIh3a=w613-h262" width="613" /></a></div><br /><br /><p></p><p>So what happens if we trigger workflow manually? We can change the default value of the apps parameter.. Notice we have changed invoice-api to order-api (Actual requirement is listing all apps and when we trigger removing names of app we want to omit building and deploying).</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhV9W3R0NaR9j8P7yNLlXjdWh3rYyK3GezzL65Z0MvRxJNch4sj0OimP_pU9bsvnWgEdSh-tgLWmwYduJowp0rjNVB9NRXJQrA777N38QJ5bxr6ZuEqqrhe9m0McwFfMkDW3M4UZNfmU5qHBhRGqYh1uVp5m9p364A83NPl9PuxcOfE9jGg5YTFrIm_yEzp" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="242" data-original-width="324" height="337" src="https://blogger.googleusercontent.com/img/a/AVvXsEhV9W3R0NaR9j8P7yNLlXjdWh3rYyK3GezzL65Z0MvRxJNch4sj0OimP_pU9bsvnWgEdSh-tgLWmwYduJowp0rjNVB9NRXJQrA777N38QJ5bxr6ZuEqqrhe9m0McwFfMkDW3M4UZNfmU5qHBhRGqYh1uVp5m9p364A83NPl9PuxcOfE9jGg5YTFrIm_yEzp=w451-h337" width="451" /></a></div><br />In the workflow run we can see the change to input parameter is effective and it can be obtained with <span style="font-family: courier;">${{ inputs.apps }}</span><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjCdzXG-sAMZGYIakhrZy7FqjPcalFwPk5_0kD5xwEIS1h3ePPk2m3lfO7YBxMxuTeSwA4i1Pu812mYBL52UTbz9FHEBRKifdnIgpGuV2q49SSG9mFA3liiDSTIroEtQGInzX2CUYXk5UqbZE7zX8v_664uIKvwsJGajkvwb2eUhbqYPUhY-3sKEj9tRlO5" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="353" data-original-width="811" height="241" src="https://blogger.googleusercontent.com/img/a/AVvXsEjCdzXG-sAMZGYIakhrZy7FqjPcalFwPk5_0kD5xwEIS1h3ePPk2m3lfO7YBxMxuTeSwA4i1Pu812mYBL52UTbz9FHEBRKifdnIgpGuV2q49SSG9mFA3liiDSTIroEtQGInzX2CUYXk5UqbZE7zX8v_664uIKvwsJGajkvwb2eUhbqYPUhY-3sKEj9tRlO5=w554-h241" width="554" /></a></div><br /><br /></div><div>If we trigger without changing the input parameter value the defualt value is available with </div><div><span style="font-family: courier;">${{ inputs.apps }}</span></div><div><span style="font-family: courier;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg2T6MfJVx1BNDqOwakKIK3EVtdeoUI9OMECLYzzBEki5GH3GG3-B3zJKZLCpYxUd7wL3vaQhX5iXke2R-0bNqHpfVOo0Wad-R5OHYp8GYMXzCWv-wFpc2e-8hQmPZG1Uwetib0ZC5Pi3w3JXPNKpywQTuxh2k4uWqcDb3ztVMSa2NgvJUwnY5TBXwGXiBK" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="476" data-original-width="863" height="338" src="https://blogger.googleusercontent.com/img/a/AVvXsEg2T6MfJVx1BNDqOwakKIK3EVtdeoUI9OMECLYzzBEki5GH3GG3-B3zJKZLCpYxUd7wL3vaQhX5iXke2R-0bNqHpfVOo0Wad-R5OHYp8GYMXzCWv-wFpc2e-8hQmPZG1Uwetib0ZC5Pi3w3JXPNKpywQTuxh2k4uWqcDb3ztVMSa2NgvJUwnY5TBXwGXiBK=w612-h338" width="612" /></a></div><br /><br /></span></div><div><span style="font-family: courier;"><br /></span></div><div><br /></div><div>All looks good above,<span style="font-size: large;"> <b>so what's the problem?</b></span></div><div><br /></div><div>Let's see what happens when the workflow run on a push. We can see on push we do not have a value for the expression <span style="font-family: courier;">${{ inputs.apps }}</span></div><div><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi4JiE4kP3oM8QT0T9Y2fOZYxlW28XhEopIWDoqcd5hhGGFVpmyrdp7XsTCH7cswscgpf8chslCYtA95kco8GUyKpxnMYst-57bzpM-zVgFISoMCE2UTicfEUzQEnKcMrcxW91OjU6Qa5VDOG7zd_w45c8pCS8JkcuCFXXZxx7JXrqOCMurhLAzIMfkaGlE" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="419" data-original-width="741" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEi4JiE4kP3oM8QT0T9Y2fOZYxlW28XhEopIWDoqcd5hhGGFVpmyrdp7XsTCH7cswscgpf8chslCYtA95kco8GUyKpxnMYst-57bzpM-zVgFISoMCE2UTicfEUzQEnKcMrcxW91OjU6Qa5VDOG7zd_w45c8pCS8JkcuCFXXZxx7JXrqOCMurhLAzIMfkaGlE=w635-h360" width="635" /></a></div><br /><div style="text-align: left;"><span style="font-size: large; font-weight: bold;">How to make it work </span>for both <span style="font-family: courier;">push</span> and <span style="font-family: courier;">workflow_dispatch</span><span style="font-family: courier;">?</span></div></div><div><br /></div><div>We can use an environment variable at the workflow level and assign it via an expression. Only ceveat is we have to specify the default value for the parameter twice. <a href="https://docs.github.com/en/actions/learn-github-actions/expressions?wt.mc_id=DT-MVP-5000590#example" target="_blank">GitHub workflow supports bit weired syntax to assign a variable with a condistion, via expression as documented here</a>.</div><div><br /></div><div><div> env:</div><div> myvar: ${{ condition && true?value || false?value }}</div></div><div><br /></div><div>For the above issue we can create a env variable <span style="font-family: courier;">apps</span> as shown below. We check if the trigger is push then we assign the default value. If it is triggered manually we assign the input parameter value to ensure any changes to input paramter value is taken into consideration. <a href="https://github.com/chamindac/aks_eventhubs_blue_green/blob/3af2a987f371d67f4964a2de95e5583ca13c6782/.github/workflows/all_apps_cicd.yml" target="_blank">The full workflow code is available in GitHub here</a>.</div><div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiixMEDGAz0aF_eaCsPtNrYYzh4f-5eAlv55YHxBAOTPMEVyW-WfE0QvXR1MOTvpTFgUaiaSct_UkxfKW2f-EunL1doJHbsmGIubMGuHWWaPs1rIKfF-M2kj-Xggqro-kN0xurK9t9B3thqsiEN6j_LDm-gIrgIz_swx1mss2IJ-B6kT85jV_1v8TI5dzm0" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="618" data-original-width="835" height="469" src="https://blogger.googleusercontent.com/img/a/AVvXsEiixMEDGAz0aF_eaCsPtNrYYzh4f-5eAlv55YHxBAOTPMEVyW-WfE0QvXR1MOTvpTFgUaiaSct_UkxfKW2f-EunL1doJHbsmGIubMGuHWWaPs1rIKfF-M2kj-Xggqro-kN0xurK9t9B3thqsiEN6j_LDm-gIrgIz_swx1mss2IJ-B6kT85jV_1v8TI5dzm0=w633-h469" width="633" /></a></div><br />Now, when change is pushed and workflow is run for on push event, the env variable app containes the defalut value, even though the input apps value is empty.</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEij9L4dsXjW-0Bhh8PoLKQDLhJWM1f-nxNZNMjisZXLFQZqGCb-BSe7SpDVYvMDFaj833J1A29cOG6vlqZqxyBGzWfkbXFyLc8DF_ZfWhkdpvqQArC9dctft2rOw1YVzYgW_doEXwbFF1WdMKkw8B3zSwgDQf8V2mpLrTbMD1fbZijNqIcVQtAGSnojFxJ_" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="523" data-original-width="958" height="335" src="https://blogger.googleusercontent.com/img/a/AVvXsEij9L4dsXjW-0Bhh8PoLKQDLhJWM1f-nxNZNMjisZXLFQZqGCb-BSe7SpDVYvMDFaj833J1A29cOG6vlqZqxyBGzWfkbXFyLc8DF_ZfWhkdpvqQArC9dctft2rOw1YVzYgW_doEXwbFF1WdMKkw8B3zSwgDQf8V2mpLrTbMD1fbZijNqIcVQtAGSnojFxJ_=w613-h335" width="613" /></a></div><br /><br />When we trigger the workflow manually with a change to input it is available to env variable apps. Therefore we can now rely on the env variable to obtain the value for all steps of the workflow.'</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEipkOimvL7n7bPlaYFf1tL8sGkEtp04UK0_n3P10N9F4fEvb2PQJ9P7oXSbJu536OrWKTeDDDtcb0jfFsi_mi79XV7Q6ZDPIVui4hwQ3SFv8507xtqNvI8sV2dMxJ1PRjJRD9TL8jaxzPvmbaHllzNO7DA7zg0bt6RS4PKeQ-ZIM2AaMxgzYhQkTyPxi4CK" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="231" data-original-width="413" height="241" src="https://blogger.googleusercontent.com/img/a/AVvXsEipkOimvL7n7bPlaYFf1tL8sGkEtp04UK0_n3P10N9F4fEvb2PQJ9P7oXSbJu536OrWKTeDDDtcb0jfFsi_mi79XV7Q6ZDPIVui4hwQ3SFv8507xtqNvI8sV2dMxJ1PRjJRD9TL8jaxzPvmbaHllzNO7DA7zg0bt6RS4PKeQ-ZIM2AaMxgzYhQkTyPxi4CK=w430-h241" width="430" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg8SGmT0SNixndnGd8ny-ymzffhsXgOdMVES-T1qMLc7vl8CPxq1hJ2RcharvBKiDI_ntZ6pUhiq3yVWZH7xBOIzzoRJvumr_ybW9Hz9ukYPC_lRePERWuNucBtEvO9YYbmYdKfiyAcqSKaisWZPgN9wqQOi4zDkuofv2RgFa7KoJv-HSh9Twb70vRHWVnP" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="521" data-original-width="1167" height="288" src="https://blogger.googleusercontent.com/img/a/AVvXsEg8SGmT0SNixndnGd8ny-ymzffhsXgOdMVES-T1qMLc7vl8CPxq1hJ2RcharvBKiDI_ntZ6pUhiq3yVWZH7xBOIzzoRJvumr_ybW9Hz9ukYPC_lRePERWuNucBtEvO9YYbmYdKfiyAcqSKaisWZPgN9wqQOi4zDkuofv2RgFa7KoJv-HSh9Twb70vRHWVnP=w644-h288" width="644" /></a></div><br /><br /></div><div><br /></div><div><br /></div>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-83886608862239230332023-09-07T06:59:00.003-07:002024-03-10T08:01:06.854-07:00Avoid GitHub Action Workflow Code Duplication - Using Composite Actions to Create Reusable Templates<p> <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?wt.mc_id=DT-MVP-5000590&view=azure-devops&pivots=templates-includes" target="_blank">Azure DevOps yaml files for piplines can be created as templates</a> to avoid duplicating same set of tasks in multiple areas of the pipelines. For GitHub actions workflow, we have seen here in "<a href="https://chamindac.blogspot.com/2023/08/build-and-unit-tests-for-net-apps-with.html" target="_blank">Build and Unit Tests for .NET Apps with GitHub Actions</a>" the same steps are repeated in both Windows and Linux, build and unit test run. The solution to avoid duplicating same steps in multiple jobs in a GitHub actions workflow is the usage of composite actions. Let's modify our workflow implemented in "<a href="https://chamindac.blogspot.com/2023/08/build-and-unit-tests-for-net-apps-with.html" target="_blank">Build and Unit Tests for .NET Apps with GitHub Actions</a>" to use composite action to build and unit test, and reuse that in both Windows and Linux job.<span></span></p><a name='more'></a><p></p><p><b>Defining the composite action</b></p><p>Composite action must be created in a yaml file named as <span style="font-family: courier;">action.yml</span>. The file should be located in a path such as below.</p><p><span style="font-family: courier;">.github/actions/action_name_folder/action.yml</span></p><p>For our build and unit test action let's use path as <span style="font-family: courier;">.github/actions/build_and_unittest/action.yml</span> Lets define the build, unit test and publish test result steps in our action. But first we need one input paramter defined, to take input whether we are building on Linux or Windows platform to publish our test results accordingly. We can also define outputs from composite actions which we can dicuss in a next post. </p><p>The input can be defined as shown below for a composite action. Here we define <span style="font-family: courier;">build_os</span> as our input to the composite action. Then we have to specify the action should run using composite. Then we can define the steps we need to execute in the action. <a href="https://github.com/chamindac/aks_eventhubs_blue_green/blob/06fffd1c8517c0db80fdc0476506d9da7a5633a4/.github/actions/build_and_unittest/action.yml" target="_blank">The full composite action yaml for build and unit test can be found here in GitHub</a>. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgHNLW43fp3ATj-ay69UBnH9YxMXtRUgGGXVz99dq3nVl_oQ6zNEf9YY7Xbkpj8hDKKOQhlnoowzN4BeZ0bQru-J__LMZPRyImpdAXX9yVGXIoYECmOBOYN4YQF-YgnqZgnsopLsq_qO9z0C_dz5xapEruuc8jUgNRqFlh8YJ0E22ucZcIxTowEioeL7uWg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="374" data-original-width="566" height="372" src="https://blogger.googleusercontent.com/img/a/AVvXsEgHNLW43fp3ATj-ay69UBnH9YxMXtRUgGGXVz99dq3nVl_oQ6zNEf9YY7Xbkpj8hDKKOQhlnoowzN4BeZ0bQru-J__LMZPRyImpdAXX9yVGXIoYECmOBOYN4YQF-YgnqZgnsopLsq_qO9z0C_dz5xapEruuc8jUgNRqFlh8YJ0E22ucZcIxTowEioeL7uWg=w563-h372" width="563" /></a></div><br />The input can be consumed in steps as shown below. Syntax is somewhat similar to <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/process/template-parameters?wt.mc_id=DT-MVP-5000590&view=azure-devops" target="_blank">Azure DevOps pipelines yaml template parameter usage</a>.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhPR0LOcJ3bQidlWD391LWR2kVwdwhea2m5sQLkovWVGdhWOwVdxtI8e7YmH2-ZhGJ731avrU9c8QxaCZaIZJq3EVjwLfpkGegMU6cPd93b23xBw2oqhiJcVGxgvpRm8q5Cu_XgYFgIh-QlbLZ_HkMNM3soE5QOaHJR88zwpUqsk8LZiN2WGUwCOY2Vryy3" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="369" data-original-width="794" height="274" src="https://blogger.googleusercontent.com/img/a/AVvXsEhPR0LOcJ3bQidlWD391LWR2kVwdwhea2m5sQLkovWVGdhWOwVdxtI8e7YmH2-ZhGJ731avrU9c8QxaCZaIZJq3EVjwLfpkGegMU6cPd93b23xBw2oqhiJcVGxgvpRm8q5Cu_XgYFgIh-QlbLZ_HkMNM3soE5QOaHJR88zwpUqsk8LZiN2WGUwCOY2Vryy3=w589-h274" width="589" /></a></div><p><b>Using the composite action</b></p>We can then consume the composite action we created in the GitHub action wokflow as sown below in multiple jobs. The build and unit test composite action is used in both Windows and Linux jobs, and notice how the input parameter is passed from the job to the composite action. <a href="https://github.com/chamindac/aks_eventhubs_blue_green/blob/06fffd1c8517c0db80fdc0476506d9da7a5633a4/.github/workflows/all_apps_cicd.yml" target="_blank">The full workflow code utilizing the composite action is available in here in GitHub</a>.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh0o82JhY2R4CkTRZdgarD6uGDO6AZN3KJTauhsWCYV4iNF4jUfV40FaqEMz1a08EDKvdiqBCSp8_MY96PEGKrPfuH_5jC0D61dfwcCSqP6vA8tNYvPT-WWpY2ER1-1mPCINYkxi1ZwPTvYKTQ_QE9noBz2BvPeqx7M7te8K5NbeV-Xbk3aSTdUoJpRXFHt" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="760" data-original-width="542" height="810" src="https://blogger.googleusercontent.com/img/a/AVvXsEh0o82JhY2R4CkTRZdgarD6uGDO6AZN3KJTauhsWCYV4iNF4jUfV40FaqEMz1a08EDKvdiqBCSp8_MY96PEGKrPfuH_5jC0D61dfwcCSqP6vA8tNYvPT-WWpY2ER1-1mPCINYkxi1ZwPTvYKTQ_QE9noBz2BvPeqx7M7te8K5NbeV-Xbk3aSTdUoJpRXFHt=w577-h810" width="577" /></a></div><br />.<p></p><p><br /><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-13570795848209447342023-09-02T03:17:00.002-07:002023-09-06T07:34:34.264-07:00Running Pod Counts for All Services Summary - Grafana Chart with Azure Monitor for AKSMonitoring the actual running pod count, against the desired pod count by each horizontal pod autoscaler, for your application is useful to get a summarize view, that will help to understand ow the pods are really scaling based on scale out demand of each individual service. We can enable <a href="https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-enable-aks?WT.mc_id=AZ-MVP-5000590&tabs=portal-azure-monitor#existing-aks-cluster" target="_blank">monitoring AKS cluster with Managed Grafana in Azure with Azure Monitor and Log Analytics Workspace</a>. Let's write a query to and create Grafana chart for this purpose.<span></span><p><b>Expected Outcome</b></p><p>Each service actual scaling over time is shown as summary. This indicate based on load how each service in your application scale out based on the demanded by horizontal pod autoscaler.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhEsz3YncXu6v_WSo-96HHR4TMGiaDdpJNs8Tns1tbB-utyIH2-e9SS1-7GmCQ6KzcaWdvE9v96R8P-gEt-mVma89AHYjw5RfpPE7TdQhhgErJrHqD0xy5aH1FMeOzu43XTKJ0t8QCTjkbzq59S_tlR_otlP9XbqnZxpSJh18H-I_X2SEO3P1P1HNtftg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="681" data-original-width="2865" height="160" src="https://blogger.googleusercontent.com/img/a/AVvXsEhEsz3YncXu6v_WSo-96HHR4TMGiaDdpJNs8Tns1tbB-utyIH2-e9SS1-7GmCQ6KzcaWdvE9v96R8P-gEt-mVma89AHYjw5RfpPE7TdQhhgErJrHqD0xy5aH1FMeOzu43XTKJ0t8QCTjkbzq59S_tlR_otlP9XbqnZxpSJh18H-I_X2SEO3P1P1HNtftg=w674-h160" width="674" /></a></div><br /><br /><span><a name='more'></a></span>This is useful when we compare the actual running pods with <a href="https://chamindac.blogspot.com/2023/06/hpa-desired-pod-counts-for-all-services.html" target="_blank">desired pod counts</a> as shown below.<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhDyh1ng8e2n-7xOd-fEH2An5OcSZrIKkxgs-M7tMcDKwlAIeIjnmj0teYtQzy9Sh-UUhazX7lFQ0EEPpB3boDoOnZM-EnVPIlIrDZLRVHPgU8pv63Qh2bl0hUHMmVrDVjxAnaZ4AhMCa0Q97jA-J2dTX-0yHwirh6y9i1atanrYgPR-s9DAHqTnQil3g" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1406" data-original-width="2876" height="313" src="https://blogger.googleusercontent.com/img/a/AVvXsEhDyh1ng8e2n-7xOd-fEH2An5OcSZrIKkxgs-M7tMcDKwlAIeIjnmj0teYtQzy9Sh-UUhazX7lFQ0EEPpB3boDoOnZM-EnVPIlIrDZLRVHPgU8pv63Qh2bl0hUHMmVrDVjxAnaZ4AhMCa0Q97jA-J2dTX-0yHwirh6y9i1atanrYgPR-s9DAHqTnQil3g=w643-h313" width="643" /></a></div><br /><p></p><p>We can use the query below to create the running pod count summary panel in Grafana with Azure monitor.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>KubePodInventory</div><div>// | where <span style="color: #569cd6;">$_</span>_timeFilter(TimeGenerated) // use only in grafana</div><div>| where ClusterName == "aks-chdemo-dev04"</div><div>| where Namespace in('mydemo') and ControllerKind == 'ReplicaSet'</div><div>| extend pod_label = todynamic(PodLabel)</div><div>| extend app_name = todynamic(pod_label[0].app)</div><div>| summarize running_pods = sumif(1, PodStatus == 'Running' ) by TimeGenerated, tostring(app_name)</div><div>| order by TimeGenerated asc</div><div>| project TimeGenerated, app_name, running_pods</div></div><p>The <a href="https://github.com/chamindac/aks-monitoring/tree/master/running_pod_count_summary" target="_blank">full json for Grafana panel is available in GitHub here</a>. You can replace the id of the panel and the subscription, log analytics workspace name, resource group name etc.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiu4e4JZ4ZiY5M7tm_wdBbIPe0XMErcMTNu3MrD1gkqqaPrlWdT04bGVqf_HWcgy6xMBa5I4lqF3FRY5wUpNY1sFbRKE_lw_IuAWmJFAhffIwmrbRCtdoEMVllFaAnlUmYYXffbQi-dVEbsuZYc0dbb57qv5JKwz8g82FGbhF-KSONCa7aJHl8rYkTbwQ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1082" data-original-width="2047" height="315" src="https://blogger.googleusercontent.com/img/a/AVvXsEiu4e4JZ4ZiY5M7tm_wdBbIPe0XMErcMTNu3MrD1gkqqaPrlWdT04bGVqf_HWcgy6xMBa5I4lqF3FRY5wUpNY1sFbRKE_lw_IuAWmJFAhffIwmrbRCtdoEMVllFaAnlUmYYXffbQi-dVEbsuZYc0dbb57qv5JKwz8g82FGbhF-KSONCa7aJHl8rYkTbwQ=w597-h315" width="597" /></a></div><br />If you use the json in GitHub (Modify with your panel id, subscription id etc.) and use query inspector of the panel to replace json for panel the required settings will be automatically applied.<p></p><p><br /></p><p></p><p><br /><br /></p><span><!--more--></span><span><!--more--></span>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-29863746391380399132023-08-23T07:10:00.008-07:002023-09-06T03:15:45.530-07:00Build and Unit Tests for .NET Apps with GitHub Actions<p> Executing unit tests and viewing results in unit tests in a build pipeline is essential to keep high quality in an application deployment via any pipeline system. GitHub actions are the way forward to implement pipelines with repositories in GitHub. Let's explore how to run unit tests and view results in GitHub actions workflow.<span></span></p><a name='more'></a><p></p><p><b>Expected outcome</b></p><p>We need a test run report as shown below, showing summary of each project unit test results as well as details of each project test results.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiBbcqE-Xov27azk5o7UwPdkh5OUXkM2lWuEolgfGswBI6TsAvKe2kQV8yJQSepFWPfZ3L902Ic6gJsfFXQamEBqYR4BavhMcRbD2kdSIP8ZMzJLRHEquEEnCbZiIlMYLSdsmSmpcv4Iddn12en2_8FWjccUUnIyGjrXVQeksLQrBnf6kQXBKpb0OT2Oi2p" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="792" data-original-width="845" height="590" src="https://blogger.googleusercontent.com/img/a/AVvXsEiBbcqE-Xov27azk5o7UwPdkh5OUXkM2lWuEolgfGswBI6TsAvKe2kQV8yJQSepFWPfZ3L902Ic6gJsfFXQamEBqYR4BavhMcRbD2kdSIP8ZMzJLRHEquEEnCbZiIlMYLSdsmSmpcv4Iddn12en2_8FWjccUUnIyGjrXVQeksLQrBnf6kQXBKpb0OT2Oi2p=w630-h590" width="630" /></a></div><br /><p></p><p>We can run tests for Windows as well and get a Windows test results report as shown below.</p><p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg5cejaJwJOLpwKylojQkCWroA8CCMFeTOPNeGyRRyrDWSdLsvK3YQ_tOxN-hJrxM7rLw8J442kN-462UJH9dsfAmxA0muidGnPicMrY90-AtbRqrjUeT_x1l08MBKjIwLJUbEtb3WCdkdP20d4lkArQ0zFirnnGYBj_UD98iEXvu_rBKlpSlxzQjByiNxs" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="795" data-original-width="838" height="605" src="https://blogger.googleusercontent.com/img/a/AVvXsEg5cejaJwJOLpwKylojQkCWroA8CCMFeTOPNeGyRRyrDWSdLsvK3YQ_tOxN-hJrxM7rLw8J442kN-462UJH9dsfAmxA0muidGnPicMrY90-AtbRqrjUeT_x1l08MBKjIwLJUbEtb3WCdkdP20d4lkArQ0zFirnnGYBj_UD98iEXvu_rBKlpSlxzQjByiNxs=w638-h605" width="638" /></a></div><br /><p></p><p>The report should give us any failed test details as whown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhTCHX01ALDnIP0JUfR2N61DbIqrvQRqsAzBA8DjoRtE4ntTjVXYf84vZ9G2uww8Z-5aLlQbGr3m0PlvsBzSo8tNRCsQsC7oY6wh7m_LQJg3fRColcrleKFWrFkSlSs15_fhmXd8genJk5ZuaPUM37FbgnHooJeD9G6j4u4Jcnfe68p4fcoH1nTIYyGFsYo" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="504" data-original-width="824" height="360" src="https://blogger.googleusercontent.com/img/a/AVvXsEhTCHX01ALDnIP0JUfR2N61DbIqrvQRqsAzBA8DjoRtE4ntTjVXYf84vZ9G2uww8Z-5aLlQbGr3m0PlvsBzSo8tNRCsQsC7oY6wh7m_LQJg3fRColcrleKFWrFkSlSs15_fhmXd8genJk5ZuaPUM37FbgnHooJeD9G6j4u4Jcnfe68p4fcoH1nTIYyGFsYo=w588-h360" width="588" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhedPWy6McxNokAJKV0GNTPDefChzdTvfdBD9UdMoAn1KWPepl80per2-LbtXHQn8Dbzjy843rfFCVoAjjv8QrQSH51GYfWLBUuohdBfeOTyb_4XZa8siKj88Ee2vHzfUJxxWWZJ-w_F_PPQK8VEbXBTp9LbSjwwx0Mv-dcouzwGEvp0yRhibGz1dWRQgRr" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="335" data-original-width="477" height="386" src="https://blogger.googleusercontent.com/img/a/AVvXsEhedPWy6McxNokAJKV0GNTPDefChzdTvfdBD9UdMoAn1KWPepl80per2-LbtXHQn8Dbzjy843rfFCVoAjjv8QrQSH51GYfWLBUuohdBfeOTyb_4XZa8siKj88Ee2vHzfUJxxWWZJ-w_F_PPQK8VEbXBTp9LbSjwwx0Mv-dcouzwGEvp0yRhibGz1dWRQgRr=w549-h386" width="549" /></a></div><br /><b>How to do</b><p></p><p>Let's look at step by step how to implment a github action workflow to build and run uni tests for a .NET applications, then generate a results report as shown above.</p><p><a href="https://github.com/chamindac/aks_eventhubs_blue_green/blob/405a32ba6d9e2103d37db5a3f384b0c6484c6b4d/.github/workflows/all_apps_cicd.yml" target="_blank">Full code example for the workflow can be found here in GitHub</a>.</p><p>Let's look at how to run the build and unit test steps for Linux first.</p><p>Here we allow to build on push as well as with a manual trigger.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEimmm2_24EbNoMn0TFZhp9pSqI4nZi68IzOAkzEwIhZl-YR1r9EZ9wPcV_IWexIn2CRuDtNtB0kfarFzTt6By-7wfM9zvzZEqxCzv7_bc4KNR9M9I-C4Eu2PRY8cgD5FTA7cpx--YRcI6xEA-CslrDHO0n2g8FcZqW1ng_-Lu4aMZ-gs2qcl2JO-pDVJgz4" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="53" data-original-width="262" height="123" src="https://blogger.googleusercontent.com/img/a/AVvXsEimmm2_24EbNoMn0TFZhp9pSqI4nZi68IzOAkzEwIhZl-YR1r9EZ9wPcV_IWexIn2CRuDtNtB0kfarFzTt6By-7wfM9zvzZEqxCzv7_bc4KNR9M9I-C4Eu2PRY8cgD5FTA7cpx--YRcI6xEA-CslrDHO0n2g8FcZqW1ng_-Lu4aMZ-gs2qcl2JO-pDVJgz4=w607-h123" width="607" /></a></div>Next we need to create a job for Linux. Here we use ubuntu_latest runner. The permission setting is required to enable the test results trx files to be read for reporting.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhE8_teqXiKWaeMLbrBFq1cw1faf2ZS1CeUzbWpMhvVpK-WUpOxBkrabeuR25h_LjyTQ5VG9jMubDw5n2wrlxHp-cj3x8ALgvV8dt6b0rJbAHZBlie0thBOHRLoOM2yei4LIwJiXHykt1aF2y_OGIdYswA2CMkyNr2Nf6QvZ3uqBMfXbvNTyP5NLj2b9EXE" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="145" data-original-width="237" height="279" src="https://blogger.googleusercontent.com/img/a/AVvXsEhE8_teqXiKWaeMLbrBFq1cw1faf2ZS1CeUzbWpMhvVpK-WUpOxBkrabeuR25h_LjyTQ5VG9jMubDw5n2wrlxHp-cj3x8ALgvV8dt6b0rJbAHZBlie0thBOHRLoOM2yei4LIwJiXHykt1aF2y_OGIdYswA2CMkyNr2Nf6QvZ3uqBMfXbvNTyP5NLj2b9EXE=w456-h279" width="456" /></a></div>The next step is to restore NuGet packages and build the projects. Then run unit tests for each test project (*.Tests.csproj). Each test project generates results as a .trx. Full job code is shown below.<p></p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"> build_and_unittest_linux:
runs-on: <span style="background-color: #fff0f0;">"ubuntu-latest"</span>
permissions:
id-token: write
contents: read
checks: write
steps:
- uses: actions/checkout@v3
- uses: actions/setup-dotnet@v3
with:
dotnet-version: <span style="background-color: #fff0f0;">'6.0.x'</span>
- name: Install dependencies
run: dotnet restore
- name: Build all projects
run: dotnet build --no-restore
- name: Run unit tests
uses: Amadevus/pwsh-script@v2.0.3
id: dotnet_test
with:
script: |
<span style="color: #003366; font-weight: bold;">dir "*.Tests.csproj" -Recurse | %{dotnet test $PSItem.FullName --logger ("trx;LogFileName=" + ($PSItem.Name -replace ".csproj", ".trx")) --no-restore --no-build}</span>
<span style="color: #003366; font-weight: bold;">dir "*.Tests.trx" -Recurse | %{ copy-item -Path $PSItem.FullName };</span>
- run: echo '${{ steps.dotnet_test.outputs.result }}'
- name: Create test report
uses: dorny/test-reporter@v1.6.0
if: success() || failure() <span style="color: #888888;"># run this step even if previous step failed</span>
with:
name: Test Results Linux <span style="color: #888888;"># Name of the check run which will be created</span>
path: ./*.Tests.trx <span style="color: #888888;"># Path to test results</span>
reporter: dotnet-trx <span style="color: #888888;"># Format of test results</span>
</pre></div>
<p></p><div class="separator" style="clear: both; text-align: center;"><br /></div>The report is generated using the .trx files by the <a href="https://github.com/dorny/test-reporter" target="_blank">dorny/test-reporter</a>@v1.6.0 action. Similar to above Linux job we can run same actions in a Windows runner as well as shown below.<!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;"> build_and_unittest_windows:
runs-on: <span style="background-color: #fff0f0;">"windows-latest"</span>
permissions:
id-token: write
contents: read
checks: write
steps:
- uses: actions/checkout@v3
- uses: actions/setup-dotnet@v3
with:
dotnet-version: <span style="background-color: #fff0f0;">'6.0.x'</span>
- name: Install dependencies
run: dotnet restore
- name: Build all projects
run: dotnet build --no-restore
- name: Run unit tests
uses: Amadevus/pwsh-script@v2.0.3
id: dotnet_test
with:
script: |
<span style="color: #003366; font-weight: bold;">dir "*.Tests.csproj" -Recurse | %{dotnet test $PSItem.FullName --logger ("trx;LogFileName=" + ($PSItem.Name -replace ".csproj", ".trx")) --no-restore --no-build}</span>
<span style="color: #003366; font-weight: bold;">dir "*.Tests.trx" -Recurse | %{ copy-item -Path $PSItem.FullName };</span>
- run: echo '${{ steps.dotnet_test.outputs.result }}'
- name: Create test report
uses: dorny/test-reporter@v1.6.0
if: success() || failure() <span style="color: #888888;"># run this step even if previous step failed</span>
with:
name: Test Results Windows <span style="color: #888888;"># Name of the check run which will be created</span>
path: ./*.Tests.trx <span style="color: #888888;"># Path to test results</span>
reporter: dotnet-trx <span style="color: #888888;"># Format of test results</span>
</pre></div><p>We can see some code duplication in the above two jobs. In a next post lest look at how to make it refactored to avoid duplication.</p>
Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-75828456759124039532023-08-14T02:35:00.003-07:002023-08-20T07:14:08.973-07:00Importing a Git Repo with LFS (Large Files) to Azure GIt Repos<p> Importing a git repo to Azure DevOps Git is straight forward using the Azure DevOps portal. However, if the importing repo has large files with git lfs then the cloning of the repo after import fails due to the large files are not imported. Let's see how to get the large files fixed for the imported repo.<span></span></p><a name='more'></a><p></p><p>An example of large file import error is shown below.</p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>Downloading somepath/AdobeDNGConverter.exe (<span style="color: #569cd6;">64</span> MB)</div><div><span style="color: #ce9178; font-weight: bold;">Error</span> downloading object: somepath/AdobeDNGConverter.exe (<span style="color: #569cd6;">cd06721</span>): </div><div>Smudge <span style="color: #ce9178; font-weight: bold;">error:</span> <span style="color: #ce9178; font-weight: bold;">Error</span> downloading somepath/AdobeDNGConverter.exe </div><div>(cd0672hash): [cd067hash] LFS object not found: [<span style="color: #569cd6;">404</span>] LFS object not found</div><br /><div>Errors logged to <span style="color: #ce9178;">'C:\repos\myrepo\.git\lfs\logs\20230814T102105.1576211.log'</span>.</div><div>Use `git lfs logs last` to view the log.</div><div><span style="color: #ce9178; font-weight: bold;">error:</span> external filter <span style="color: #ce9178;">'git-lfs filter-process'</span> failed</div><div>fatal: somepath/AdobeDNGConverter.exe: smudge filter lfs failed</div><div><span style="color: #ce9178;">warning:</span> Clone succeeded, but checkout failed.</div><div>You can inspect what was checked out with <span style="color: #ce9178;">'git status'</span></div><div>and retry with <span style="color: #ce9178;">'git restore --source=HEAD :/'</span></div></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjbWfbXoXaifKNi1VwPrvT0IckBb3UGG2L1yFaxCf7SYmmWE4xdwiGEro7bZ6EdGmEU5vtHSyzzt4kAv5I_odXv3PeH0L-9dwLbWor8SNYv34APuhHNcTfiY8g-OeDAjHUofP969GglnFM1cPRrkEaXtqpjtPnKc0ovKmNbJAWSl2xJkpbY25nI958AFbln" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="191" data-original-width="836" height="145" src="https://blogger.googleusercontent.com/img/a/AVvXsEjbWfbXoXaifKNi1VwPrvT0IckBb3UGG2L1yFaxCf7SYmmWE4xdwiGEro7bZ6EdGmEU5vtHSyzzt4kAv5I_odXv3PeH0L-9dwLbWor8SNYv34APuhHNcTfiY8g-OeDAjHUofP969GglnFM1cPRrkEaXtqpjtPnKc0ovKmNbJAWSl2xJkpbY25nI958AFbln=w637-h145" width="637" /></a></div><br /><br /><p></p><p>When verified with the imported repo in the Azure DevOps portal the large files shown as nt imported but the file hash information is avalable.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiXyRHy4E6NohP33muFyUiX7Xw2rnSJnX-Ffk3H-98eVKhGV3oj8LFG_tcRVaI1DoyUWX43av4B8oyj5iNkPW04HMQMhTksA8X8Rr3FQjQ5omOUaFnktjHIv7NeYCaKs7Xr5Bv3-GXyPRSwWCdB56p8V58CWn50awMpjkF5b6pBAOgsYKA0C0rsVquelvHh" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="226" data-original-width="733" height="187" src="https://blogger.googleusercontent.com/img/a/AVvXsEiXyRHy4E6NohP33muFyUiX7Xw2rnSJnX-Ffk3H-98eVKhGV3oj8LFG_tcRVaI1DoyUWX43av4B8oyj5iNkPW04HMQMhTksA8X8Rr3FQjQ5omOUaFnktjHIv7NeYCaKs7Xr5Bv3-GXyPRSwWCdB56p8V58CWn50awMpjkF5b6pBAOgsYKA0C0rsVquelvHh=w604-h187" width="604" /></a></div><br /><br />To fix the issue we can execute the below steps.<div><br /></div><div>Open Git bash and change directory to the locally unsuccessfull cloned repo. Then set the remote source as the original source repo where you have imported the repo from, using the command below.<br /><p></p><p> git remote add source <orginal source repo clone url></p><p>Then execute a git lfs fetch from source which will download all the missing large files to your local repo.</p><p> git lfs fetch source --all</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEggDMsv6NlnS4P5-wGTd6rboK9aupqEBq0E7avKoGJIuPsnjhMJFzXBi1SS_b0nP9j8k0MUK2jYdhWQVJb6gx-v9vXBFw3aDkcLiWO8B0KTpPNY6oILZmmZvxo_o3W6vbDcIZwsxLrXeQqJmdEVG0hf7WbIyfK4BuE1OruJC_ju2NdSYgTB5K0tOZvQ8n1z" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="61" data-original-width="414" height="90" src="https://blogger.googleusercontent.com/img/a/AVvXsEggDMsv6NlnS4P5-wGTd6rboK9aupqEBq0E7avKoGJIuPsnjhMJFzXBi1SS_b0nP9j8k0MUK2jYdhWQVJb6gx-v9vXBFw3aDkcLiWO8B0KTpPNY6oILZmmZvxo_o3W6vbDcIZwsxLrXeQqJmdEVG0hf7WbIyfK4BuE1OruJC_ju2NdSYgTB5K0tOZvQ8n1z=w613-h90" width="613" /></a></div><br />Then make sure to add the remote target as the newly imported Azure git repo clone url.</div><div>git remote add target <imported azure devops git repo clone url></div><div><br /></div><div>Executing command below will push all the missing files to the imported repo in Azure DevOps.<br /><p></p><p>git lfs push target --all</p><div>Once completed in Azure git repo you can find the file is now available to download.</div><div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEiw5myNPvqSt_cOR50r3N3APpcKAHSX-FAFVxlWdEFJkofv10xbav0Ukplp9cfIyk3Wh39k8wyLlK_Y_s-7UtHgPLqza4kvTeaZaJ5Vmd_HGMbvDh2XaknPRuAQ5eXuiTwhNCiQKbiPowlAZmc_nsTDR8xGPAiyoTDG8A0BIiCQdGOL9h1VDc79C2HBIFYz" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="201" data-original-width="795" height="161" src="https://blogger.googleusercontent.com/img/a/AVvXsEiw5myNPvqSt_cOR50r3N3APpcKAHSX-FAFVxlWdEFJkofv10xbav0Ukplp9cfIyk3Wh39k8wyLlK_Y_s-7UtHgPLqza4kvTeaZaJ5Vmd_HGMbvDh2XaknPRuAQ5eXuiTwhNCiQKbiPowlAZmc_nsTDR8xGPAiyoTDG8A0BIiCQdGOL9h1VDc79C2HBIFYz=w636-h161" width="636" /></a></div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><br /></div><p><br /></p><p><br /><br /></p><p><br /></p><p><br /></p><p><br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><br /></div><br /><br /><p></p></div>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-84133346424743184812023-07-29T11:29:00.002-07:002023-07-30T01:51:44.320-07:00Using Object Type Azure DevOps YAML Pipeline Parameter in PowerShell Task<p> Using a YAML pipeline paramter in a PowerShell task is straight forward for types such as <span style="font-family: courier;">string</span>. For example, if there is a YAML pipeline parameter named <span style="background-color: #fffffe; font-size: 14px; white-space: pre;"><span style="font-family: courier;">env</span></span><span style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> of type </span><span style="background-color: #fffffe; font-size: 14px; white-space: pre;"><span style="font-family: courier;">string</span></span><span style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">, </span>we can read it in PowerShell task with <span style="background-color: #fffffe; font-size: 14px; white-space: pre;"><span style="font-family: courier;">$envName = '${{ parameters.env }};</span></span><span style="background-color: #fffffe; color: #0451a5; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;"> </span>without any issue. However, if the parameter is type of <span style="font-family: courier;">object </span>we cannot read, the paramter the same way we do with string parameters. If we try to read <span style="font-family: courier;">object</span> parameter named <span style="font-family: courier;">apps</span>, <span style="background-color: #fffffe; font-family: courier; font-size: 14px; white-space: pre;">$apps = '${{ parameters.apps}};</span> , there will be an YAML validation error staring the pipeline such as below. </p><p><span face=""Segoe UI VSS (Regular)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", Helvetica, Ubuntu, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"" style="background-color: #f9ebeb; font-size: 14px; white-space-collapse: preserve;">/azure-pipelines.yml (Line: 57, Col: 23): Unable to convert from Array to String. Value: Array</span></p><p>Even if we try to use below it will be same issue.</p><p><span style="background-color: #fffffe; font-family: courier; font-size: 14px; white-space: pre;">[String[]]$apps = '${{ parameters.apps}};</span></p><p><span style="background-color: #fffffe; font-family: courier; font-size: 14px; white-space: pre;">[PSObject[]]</span><span style="background-color: #fffffe; font-family: courier; font-size: 14px; white-space: pre;">$apps = '${{ parameters.apps}};</span></p><p>Let's explore the issueand solution in detail.<span></span></p><a name='more'></a><p></p><p><b><u>The issue in detail</u></b></p><p>Consider below two parameters defined in a YAML pipeline.</p><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: teal;">parameters</span>:</div><div>- <span style="color: teal;">name</span>: <span style="color: #0451a5;">env</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Select environment'</span></div><div> <span style="color: teal;">type</span>: <span style="color: #0451a5;">string</span></div><div> <span style="color: teal;">default</span>: <span style="color: #0451a5;">dev</span></div><div> <span style="color: teal;">values</span>:</div><div> - <span style="color: #0451a5;">dev</span></div><div> - <span style="color: #0451a5;">qa</span></div><div> - <span style="color: #0451a5;">prod</span></div><div>- <span style="color: teal;">name</span>: <span style="color: #0451a5;">apps</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Apps to deploy'</span></div><div> <span style="color: teal;">type</span>: <span style="color: #0451a5;">object</span></div><div> <span style="color: teal;">default</span>: [</div><div> <span style="color: #0451a5;">'customer_api'</span>,</div><div> <span style="color: #0451a5;">'invoice_api'</span>,</div><div> <span style="color: #0451a5;">'order_api'</span>,</div><div> <span style="color: #0451a5;">'payment_api'</span>,</div><div> <span style="color: #0451a5;">'order_eventhandler'</span>,</div><div> <span style="color: #0451a5;">'invoice_eventhandler'</span>,</div><div> <span style="color: #0451a5;">'customer_eventhandler'</span></div><div> ]</div></div><p>The parameter env is type of string and it can be used in a PowerShell task without any issue as shwon below,</p><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 2'</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div></div><p>However, if we try to use the apps parameter, which is type of object we will get the error below, when starting the pipeline.</p><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 2'</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div><div><span style="color: #0451a5;"> $apps = '${{ parameters.apps}}';</span></div></div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhHk82jjs2gwQzCSX15xWpziveWRyqdw9heVzbR_MYhyVIyyvfiqxDHl2sMa7af94Z3BukYXezxJMsroOHNzwBxkoul7Ia-6SU_ZhiryDe_HrYejbOeESLAgaM5a880t0Pz_P_ES3W3vCwglGnpHprABCH-cVqql903zmPhj2NCxS_R1Oko0Z9qrpbItElh" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1599" data-original-width="918" height="978" src="https://blogger.googleusercontent.com/img/a/AVvXsEhHk82jjs2gwQzCSX15xWpziveWRyqdw9heVzbR_MYhyVIyyvfiqxDHl2sMa7af94Z3BukYXezxJMsroOHNzwBxkoul7Ia-6SU_ZhiryDe_HrYejbOeESLAgaM5a880t0Pz_P_ES3W3vCwglGnpHprABCH-cVqql903zmPhj2NCxS_R1Oko0Z9qrpbItElh=w563-h978" width="563" /></a></div><br /><br /><b><u>The Solution</u></b><div><br /></div><div>There are two ways to fix this issue. Once is to use an env variable setup and join object type contents with a seperator such as ;. Then in PowereShell we can split it and obtain an array. This works fine for the parmeter apps we have used in this example. </div><div><br /></div><div><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 1'</span></div><div> <span style="color: teal;">env</span>:</div><div> <span style="color: teal;">APP_NAMES</span>: <span style="color: #0451a5;">${{ join(';', parameters.apps) }}</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div><br /><div><span style="color: #0451a5;"> $apps = $env:APP_NAMES -split ";"</span></div><br /><div><span style="color: #0451a5;"> foreach($app in $apps)</span></div><div><span style="color: #0451a5;"> {</span></div><div> <span style="color: #0451a5;">Write-Host ($app);</span></div><div> <span style="color: #0451a5;">}</span></div></div></div><div><br /></div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh6QE5fyjdmwWlPeWdGmExpCgT4YA7xjF6M_w4rBzWCX_dj2HibHjJprSKpt3nmxV7uOMXhDLVrMxG_6tJh1YpMXrur-ng36BR3CU76uUYc90_2cOqWF75VYnbI3rBjo6M1oYizFJJMWJxv6chgvg6JqS6A3fOF9Sd0qISqck-JHuMobqIxfXeQ5LokrLWH" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="831" data-original-width="1320" height="361" src="https://blogger.googleusercontent.com/img/a/AVvXsEh6QE5fyjdmwWlPeWdGmExpCgT4YA7xjF6M_w4rBzWCX_dj2HibHjJprSKpt3nmxV7uOMXhDLVrMxG_6tJh1YpMXrur-ng36BR3CU76uUYc90_2cOqWF75VYnbI3rBjo6M1oYizFJJMWJxv6chgvg6JqS6A3fOF9Sd0qISqck-JHuMobqIxfXeQ5LokrLWH=w575-h361" width="575" /></a></div><br /><br /></div><div>However, if the parameter is more complex object such as below the above solutioon will fail.</div><div><br /></div><div><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;">parameters:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> - name:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;"> </span><span class="hljs-string" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #a31515; font-size: 14px; outline-color: inherit; white-space: pre;">complexparam</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> type:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;"> </span><span class="hljs-string" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #a31515; font-size: 14px; outline-color: inherit; white-space: pre;">object</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> default:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> this_is:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> a_complex:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;"> </span><span class="hljs-string" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #a31515; font-size: 14px; outline-color: inherit; white-space: pre;">object</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-attr" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0451a5; font-size: 14px; outline-color: inherit; white-space: pre;"> with:</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-bullet" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0064c8; font-size: 14px; outline-color: inherit; white-space: pre;"> -</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;"> </span><span class="hljs-string" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #a31515; font-size: 14px; outline-color: inherit; white-space: pre;">one</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;">
</span><span class="hljs-bullet" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #0064c8; font-size: 14px; outline-color: inherit; white-space: pre;"> -</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; color: #161616; font-size: 14px; white-space: pre;"> </span><span class="hljs-string" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace" style="background-color: #f2f2f2; box-sizing: inherit; color: #a31515; font-size: 14px; outline-color: inherit; white-space: pre;">two</span></div><div><br /></div><div><br /></div><div><br /></div><div>To obtain a complex object paramter to a PSObject, we can use syntax below, in a PowerShell task.</div><div><span style="background-color: #fffffe; color: #0451a5; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">$apps = '${{ convertToJson(parameters.apps) }}' | ConvertFrom-Json; </span>converts complex YAML parameter to json representation and the convert it from json to PSObject, which we can use in PowerShell to read through the complex object.</div><div><br /></div><div><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 2'</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div><br /><div><span style="color: #0451a5;"> $apps = '${{ convertToJson(parameters.apps) }}' | ConvertFrom-Json;</span></div><br /><div><span style="color: #0451a5;"> foreach($app in $apps)</span></div><div><span style="color: #0451a5;"> {</span></div><div> <span style="color: #0451a5;">Write-Host ($app);</span></div><div> <span style="color: #0451a5;">}</span></div></div></div><div><br /><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh9Ev3QT2io3mNYp_Jr-fF7Jp_Cj1SaXVRkAGeTpG4nSx5qBq3lpElwJmkl78rikukx9pI1WIjsqrkkFYJwd_7_5cTWXRfefcHU2zA7QQ_s1ChY6Svfg3ha0umdqsXup-bTtntD9Z4lWE722aNLc7yjxfxSRdOPilDioWwgLrIbk8alKIWCbHQF3NM0DVBT" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1027" data-original-width="1413" height="426" src="https://blogger.googleusercontent.com/img/a/AVvXsEh9Ev3QT2io3mNYp_Jr-fF7Jp_Cj1SaXVRkAGeTpG4nSx5qBq3lpElwJmkl78rikukx9pI1WIjsqrkkFYJwd_7_5cTWXRfefcHU2zA7QQ_s1ChY6Svfg3ha0umdqsXup-bTtntD9Z4lWE722aNLc7yjxfxSRdOPilDioWwgLrIbk8alKIWCbHQF3NM0DVBT=w585-h426" width="585" /></a></div><br /><br /><p></p><p>The full test pipline YAML code is as below.</p><p><br /></p><div style="background-color: #fffffe; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: teal;">parameters</span>:</div><div>- <span style="color: teal;">name</span>: <span style="color: #0451a5;">env</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Select environment'</span></div><div> <span style="color: teal;">type</span>: <span style="color: #0451a5;">string</span></div><div> <span style="color: teal;">default</span>: <span style="color: #0451a5;">dev</span></div><div> <span style="color: teal;">values</span>:</div><div> - <span style="color: #0451a5;">dev</span></div><div> - <span style="color: #0451a5;">qa</span></div><div> - <span style="color: #0451a5;">prod</span></div><div>- <span style="color: teal;">name</span>: <span style="color: #0451a5;">apps</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Apps to deploy'</span></div><div> <span style="color: teal;">type</span>: <span style="color: #0451a5;">object</span></div><div> <span style="color: teal;">default</span>: [</div><div> <span style="color: #0451a5;">'customer_api'</span>,</div><div> <span style="color: #0451a5;">'invoice_api'</span>,</div><div> <span style="color: #0451a5;">'order_api'</span>,</div><div> <span style="color: #0451a5;">'payment_api'</span>,</div><div> <span style="color: #0451a5;">'order_eventhandler'</span>,</div><div> <span style="color: #0451a5;">'invoice_eventhandler'</span>,</div><div> <span style="color: #0451a5;">'customer_eventhandler'</span></div><div> ]</div><br /><div><span style="color: teal;">trigger</span>: <span style="color: #0451a5;">none</span></div><br /><div><span style="color: teal;">stages</span>:</div><div> - <span style="color: teal;">stage</span>: <span style="color: #0451a5;">YAML_Params</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">"YAML Params"</span></div><div> <span style="color: teal;">jobs</span>:</div><div> - <span style="color: teal;">job</span>: <span style="color: #0451a5;">YAML_Params</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">YAML Params</span></div><div> <span style="color: teal;">pool</span>:</div><div> <span style="color: teal;">vmImage</span>: <span style="color: #0451a5;">ubuntu-latest</span></div><div> <span style="color: teal;">steps</span>:</div><div> - <span style="color: teal;">checkout</span>: <span style="color: #0451a5;">none</span></div><div> </div><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 1'</span></div><div> <span style="color: teal;">env</span>:</div><div> <span style="color: teal;">APP_NAMES</span>: <span style="color: #0451a5;">${{ join(';', parameters.apps) }}</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div><br /><div><span style="color: #0451a5;"> $apps = $env:APP_NAMES -split ";"</span></div><br /><div><span style="color: #0451a5;"> foreach($app in $apps)</span></div><div><span style="color: #0451a5;"> {</span></div><div> <span style="color: #0451a5;">Write-Host ($app);</span></div><div> <span style="color: #0451a5;">}</span></div><div> </div><div> - <span style="color: teal;">task</span>: <span style="color: #0451a5;">PowerShell@2</span></div><div> <span style="color: teal;">displayName</span>: <span style="color: #0451a5;">'Read parameter 2'</span></div><div> <span style="color: teal;">inputs</span>:</div><div> <span style="color: teal;">targetType</span>: <span style="color: #0451a5;">'inline'</span></div><div> <span style="color: teal;">script</span>: |</div><div><span style="color: #0451a5;"> $envName = '${{ parameters.env }}';</span></div><div><span style="color: #0451a5;"> Write-Host ($envName);</span></div><br /><div><span style="color: #0451a5;"> $apps = '${{ convertToJson(parameters.apps) }}' | ConvertFrom-Json;</span></div><br /><div><span style="color: #0451a5;"> foreach($app in $apps)</span></div><div><span style="color: #0451a5;"> {</span></div><div> <span style="color: #0451a5;">Write-Host ($app);</span></div><div> <span style="color: #0451a5;">}</span></div></div></div>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-48356321970396468762023-07-22T09:12:00.001-07:002023-07-23T02:31:55.321-07:00Mapping SQL Database SKU Bicep Specs with Available SKUs in a Region<p> How to fnd SKU information for SQL databses <a href="https://learn.microsoft.com/en-us/azure/templates/microsoft.sql/servers/databases??WT.mc_id=AZ-MVP-5000590&pivots=deployment-language-bicep" target="_blank">is documented here</a>. The command suggested to use is <span style="font-family: courier;">az sql db list-editions -l region -o table</span> . This would provide available list of SQL database SKU optons to chose form for a given region. However, the headings and parameters required in bicep is bit confusing to figure out intially. Let's look at how to map values for available SKUs provided by <span style="font-family: courier;">az sql db list-editions -l region -o table</span> command, and parmeters in Bicep SKU for SQL database.<span></span></p><a name='more'></a><p></p><p>Consider the Bicep code below for creating a SQL database.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqlserverName</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseName</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseSKUCapacity</span> <span style="color: #9cdcfe;">int</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseSKUFamily</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseSKUName</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseSKUSize</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseSKUTier</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseBackupStorageRedudancy</span> <span style="color: #9cdcfe;">string</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">sqldatabaseMaxSizeBytes</span> <span style="color: #9cdcfe;">int</span></div><div><span style="color: #c586c0;">param</span> <span style="color: #9cdcfe;">location</span> <span style="color: #9cdcfe;">string</span></div><br /><div><span style="color: #c586c0;">resource</span> <span style="color: #9cdcfe;">sqlServerDatabase</span> <span style="color: #ce9178;">'Microsoft.Sql/servers/databases@2022-11-01-preview'</span> = {</div><div> <span style="color: #9cdcfe;">name</span>: <span style="color: #ce9178;">'</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">sqlserverName</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">/</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">sqldatabaseName</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">'</span></div><div> <span style="color: #9cdcfe;">location</span>: <span style="color: #9cdcfe;">location</span></div><br /><div> <span style="color: #9cdcfe;">sku</span>: {</div><div> <span style="color: #9cdcfe;">capacity</span>: <span style="color: #9cdcfe;">sqldatabaseSKUCapacity</span></div><div> <span style="color: #9cdcfe;">family</span>: <span style="color: #9cdcfe;">sqldatabaseSKUFamily</span></div><div> <span style="color: #9cdcfe;">name</span>: <span style="color: #9cdcfe;">sqldatabaseSKUName</span></div><div> <span style="color: #9cdcfe;">size</span>: <span style="color: #9cdcfe;">sqldatabaseSKUSize</span></div><div> <span style="color: #9cdcfe;">tier</span>: <span style="color: #9cdcfe;">sqldatabaseSKUTier</span></div><div> }</div><br /><div> <span style="color: #9cdcfe;">properties</span>: {</div><div> <span style="color: #9cdcfe;">requestedBackupStorageRedundancy</span>:<span style="color: #9cdcfe;">sqldatabaseBackupStorageRedudancy</span></div><div> <span style="color: #9cdcfe;">collation</span>: <span style="color: #ce9178;">'SQL_Latin1_General_CP1_CI_AS'</span></div><div> <span style="color: #9cdcfe;">maxSizeBytes</span>: <span style="color: #9cdcfe;">sqldatabaseMaxSizeBytes</span></div><div> }</div><div>}</div><br /></div><p>The paramters can be mapped as shown in below. The example uses eastus region SKUs.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEh3T2V2c3lyKap0wGZSLv0RG8oHDHZqor7Dw79OUP0TIDAHidde3DTKg2BjGNhuH1r5s7OgwNGWklC8MShqRzDMJ2llN33fu9sZaLp80rHV6VL-Grr7DCaUZJ3TM2BMVGcIl-UKvJ3LgLSVcnv17BKkrNXCmWK_c3Eq_YZQ3zrT1JtIt296RJaEV1taQ77P" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1743" data-original-width="2700" height="422" src="https://blogger.googleusercontent.com/img/a/AVvXsEh3T2V2c3lyKap0wGZSLv0RG8oHDHZqor7Dw79OUP0TIDAHidde3DTKg2BjGNhuH1r5s7OgwNGWklC8MShqRzDMJ2llN33fu9sZaLp80rHV6VL-Grr7DCaUZJ3TM2BMVGcIl-UKvJ3LgLSVcnv17BKkrNXCmWK_c3Eq_YZQ3zrT1JtIt296RJaEV1taQ77P=w652-h422" width="652" /></a></div><br /><p></p><p>The mapping can be summaruzed below. </p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><br /><div> <span style="color: #4ec9b0;">sku</span>: {</div><div> <span style="color: #4ec9b0;">capacity</span>: <span style="color: #9cdcfe;">sqldatabaseSKUCapacity</span> <span style="color: #6a9955;">// Maps to Capacity</span></div><div> <span style="color: #4ec9b0;">family</span>: <span style="color: #9cdcfe;">sqldatabaseSKUFamily</span> <span style="color: #6a9955;">// Maps to Family</span></div><div> <span style="color: #4ec9b0;">name</span>: <span style="color: #9cdcfe;">sqldatabaseSKUName</span> <span style="color: #6a9955;">// Maps to sku</span></div><div> <span style="color: #4ec9b0;">size</span>: <span style="color: #9cdcfe;">sqldatabaseSKUSize</span> <span style="color: #6a9955;">// Maps to ServiceObjective</span></div><div> <span style="color: #4ec9b0;">tier</span>: <span style="color: #9cdcfe;">sqldatabaseSKUTier</span> <span style="color: #6a9955;">// Maps to Edition</span></div><div> }</div><br /></div><p>For standard DTU 50 it would be below in after deployed in ARM.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhsoSYekeV2MStHmg0hWObwduZE_gqs99OQkbh11cNs_dimdoGPdQYW5myO8jcqIn7oH_DsKLYyvJfP7fY0Ak3IZVqfnPyBzVAsUzxWlMAogWGjDD5yjngr44YtnPeZfZhHtagaCcW4_6MmRCcDOA4J4s1adawHmcMyea5aVJVcg6BSlMuLJinNSsNb5nwx" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="549" data-original-width="1082" height="284" src="https://blogger.googleusercontent.com/img/a/AVvXsEhsoSYekeV2MStHmg0hWObwduZE_gqs99OQkbh11cNs_dimdoGPdQYW5myO8jcqIn7oH_DsKLYyvJfP7fY0Ak3IZVqfnPyBzVAsUzxWlMAogWGjDD5yjngr44YtnPeZfZhHtagaCcW4_6MmRCcDOA4J4s1adawHmcMyea5aVJVcg6BSlMuLJinNSsNb5nwx=w561-h284" width="561" /></a></div><br />So full bicep parameters for SKU and size would be as below (This is from Azure DevOps variable group).<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEi424rLu5kTuB6dlmDOJdzHD-lANyTmre1HAVigkEfnY_57powGakKuuxY26sLdZKYjbIulUW3q4hI_mIWY7HTVbZHndSx8PAGOtoK9GA98Ei2xfq-8DdHxgJiNNpAOkhaQgiBn0Qutb58t6cNQzl2CZ3NTlLWs6jab9Z7KFmUrHHpuoScMefQ2EruJhHi2" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="430" data-original-width="991" height="233" src="https://blogger.googleusercontent.com/img/a/AVvXsEi424rLu5kTuB6dlmDOJdzHD-lANyTmre1HAVigkEfnY_57powGakKuuxY26sLdZKYjbIulUW3q4hI_mIWY7HTVbZHndSx8PAGOtoK9GA98Ei2xfq-8DdHxgJiNNpAOkhaQgiBn0Qutb58t6cNQzl2CZ3NTlLWs6jab9Z7KFmUrHHpuoScMefQ2EruJhHi2=w536-h233" width="536" /></a></div><br /><br /><p></p><p>For general purpose example below</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjLGsjMNZ8mi2kKuytm0DBw8nlIsIv9hZvAtyaJ3JSydEgVpAOjutDObA2-iOPRUGeRHjuOqgeJR2sWCtjn9o8oaZDhDlDLL8hdEfXt2srRsCTZ3dosnNd_u7Q9fuby2GcGVN2vOp1cLwvka_-MhM7Ax2dowgJyTopknkdCFrrjrY5x8yO_x2NzoBj1yaQv" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1229" data-original-width="2348" height="328" src="https://blogger.googleusercontent.com/img/a/AVvXsEjLGsjMNZ8mi2kKuytm0DBw8nlIsIv9hZvAtyaJ3JSydEgVpAOjutDObA2-iOPRUGeRHjuOqgeJR2sWCtjn9o8oaZDhDlDLL8hdEfXt2srRsCTZ3dosnNd_u7Q9fuby2GcGVN2vOp1cLwvka_-MhM7Ax2dowgJyTopknkdCFrrjrY5x8yO_x2NzoBj1yaQv=w628-h328" width="628" /></a></div><br />Settings goes as below in ARM template.<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgRDQE_YR6C_aA9V5FaG8zZk8OSYjFWg3Qj_gV_sd-Atg4DaOup3dBS4m4X4GC8jT8JInzAMeporZux60KKEK-ieZMLNHh3HtNI7PsFybadaNKAyb0BaO73PyUHOgqijKr5iADF4BEiTD3CIVRsaXktBYfPyqqdaRBsgQfxWu5hKfiB5Y5KOcgBb7sbJJeM" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="432" data-original-width="799" height="307" src="https://blogger.googleusercontent.com/img/a/AVvXsEgRDQE_YR6C_aA9V5FaG8zZk8OSYjFWg3Qj_gV_sd-Atg4DaOup3dBS4m4X4GC8jT8JInzAMeporZux60KKEK-ieZMLNHh3HtNI7PsFybadaNKAyb0BaO73PyUHOgqijKr5iADF4BEiTD3CIVRsaXktBYfPyqqdaRBsgQfxWu5hKfiB5Y5KOcgBb7sbJJeM=w568-h307" width="568" /></a></div><br /><p></p><p>Parameters for bicep in Azure DevOps variable group as below.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhT-WYJlyqvgWR9qZEhdQs_dcvPtKX6H-ZSzties1UPNnQrNzLvxv1L26-XcYEmjYyQTUrA9uXl1ofPKJV8ljFP-OPMCLQOlpQnh0UVatQi0dj-3wsVke0VvMUHMP2t85_Rf7gBEPe3tib8A8QYVstr3mHxFsVJ5J4hOiB7rpjhIyOQIfHJu7x5XKj9moOl" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="452" data-original-width="1083" height="265" src="https://blogger.googleusercontent.com/img/a/AVvXsEhT-WYJlyqvgWR9qZEhdQs_dcvPtKX6H-ZSzties1UPNnQrNzLvxv1L26-XcYEmjYyQTUrA9uXl1ofPKJV8ljFP-OPMCLQOlpQnh0UVatQi0dj-3wsVke0VvMUHMP2t85_Rf7gBEPe3tib8A8QYVstr3mHxFsVJ5J4hOiB7rpjhIyOQIfHJu7x5XKj9moOl=w633-h265" width="633" /></a></div><br /><p></p><p><br /></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-23432522803987959962023-07-15T01:00:00.000-07:002024-03-10T07:04:30.957-07:00Ensure Azure App Config Refresh for Keyvault Secret Updates in Terraform<p>Keyvault secrets can be used in Azure app conciguratoins and can be setup with terraform. However, if the secret is modified then modified secret reference is not get updated to the app configuration. There are two ways to fix this issue in terraform. Let's explore them.<span></span></p><a name='more'></a><p></p><p>First let's look at the problem.</p><p>Consider below secrets defined in terraform for keyvault.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># Secrets </span></div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_key_vault_secret"</span> <span style="color: #4fc1ff;">"secret"</span> {</div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">AzureWebJobsStorage</span> = <span style="color: #9cdcfe;">azurerm_storage_account</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">primary_connection_string</span></div><div> <span style="color: #9cdcfe;">SqlDBPwd</span> <span> </span>= <span style="color: #9cdcfe;">azurerm_mssql_server</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">administrator_login_password</span></div><div> }</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">value</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">key_vault_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span>.<span style="color: #9cdcfe;">id</span></div><br /><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span></div><div> ]</div><div>}<span style="background-color: transparent;"> </span></div></div><p>To refer these in the app configuration we can defined them as below.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_app_configuration_key"</span> <span style="color: #4fc1ff;">"config_vault"</span> {</div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">"AzureWebJobsStorage"</span> = azurerm_key_vault_secret.secret[<span style="color: #ce9178;">"AzureWebJobsStorage"</span>].id</div><div> <span style="color: #9cdcfe;">"</span><span style="color: #9cdcfe;">SqlDBPwd</span><span style="color: #9cdcfe;">"</span> = azurerm_key_vault_secret.secret[<span style="color: #ce9178;">"</span><span style="color: #ce9178;">SqlDBPwd</span><span style="color: #ce9178;">"</span>].id</div><div> }</div><div> <span style="color: #9cdcfe;">configuration_store_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_app_configuration</span>.<span style="color: #9cdcfe;">appconf</span>.<span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">key</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">type</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"vault"</span> <span style="color: #6a9955;"># keyvault reference</span></div><div> <span style="color: #9cdcfe;">label</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span>.<span style="color: #9cdcfe;">instancerg</span>.<span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">vault_key_reference</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_role_assignment</span>.<span style="color: #9cdcfe;">appconf_dataowner</span></div><div> ]</div><div>}</div></div><p><b>The problem of this appraoch</b>: As explained above the <span style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">azurerm_key_vault_secret.secret[</span><span style="background-color: #1e1e1e; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">"</span><span style="background-color: #1e1e1e; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">SqlDBPwd</span><span style="background-color: #1e1e1e; color: #ce9178; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">"</span><span style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">].id</span> is giving the keyvalt reference with the version id. So that the app config will store the value in below format. The Guid represents the version of the keyvault secret.</p><p><span style="font-family: courier;">https://keyvaultname.vault.azure.net/secrets/SqlDBPwd/15061a22c4284082bf0c6e8f3f7cb501</span></p><p>When the the secret value is changed for SqlDBPwd the terraform for keyvault will udate the secret vaule in keyvault which will have a diffrent GUID. However, this is not getting updated to the app config if we use the above terraform code to set the app configuration values.</p><p><b>The Solution - Option One - Not so good</b></p><p>One option to fix the issue is refer the keyvault secret as data in terraform as shown below and get it applied to the app configuration.This will ensure the new version changes in keyvault secret for SqlDBPwd is always get applied to app configuration. </p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># Secrets </span></div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_key_vault_secret"</span> <span style="color: #4fc1ff;">"secret"</span> {</div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">AzureWebJobsStorage</span> = <span style="color: #9cdcfe;">azurerm_storage_account</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">primary_connection_string</span></div><div> <span style="color: #9cdcfe;">SqlDBPwd</span> = <span style="color: #9cdcfe;">azurerm_mssql_server</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">administrator_login_password</span></div><div> }</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">value</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">key_vault_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span>.<span style="color: #9cdcfe;">id</span></div><br /><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span></div><div> ]</div><div>}<span style="background-color: transparent;"> </span></div><div><span style="background-color: transparent;"><br /></span></div><div><div style="line-height: 19px;"><div><span style="color: #6a9955;"># Refer SqlDBPwd as data to get updated value</span></div><div><span style="color: #4ec9b0;">data</span> <span style="color: #4fc1ff;">"azurerm_key_vault_secret"</span> <span style="color: #4fc1ff;">"sqldbpwd"</span> {</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #ce9178;">SqlDBPwd</span><span style="color: #ce9178;">"</span></div><div> <span style="color: #9cdcfe;">key_vault_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span>.<span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_key_vault_secret</span>.<span style="color: #9cdcfe;">secret</span></div><div> ]</div><div>}</div><div><br /></div><div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_app_configuration_key"</span> <span style="color: #4fc1ff;">"config_vault"</span> {</div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">"AzureWebJobsStorage"</span> = azurerm_key_vault_secret.secret[<span style="color: #ce9178;">"AzureWebJobsStorage"</span>].id</div><div> <span style="color: #9cdcfe;">"</span><span style="color: #9cdcfe;">SqlDBPwd</span><span style="color: #9cdcfe;">"</span> = data.azurerm_key_vault_secret.sqldbpwd.id</div><div> }</div><div> <span style="color: #9cdcfe;">configuration_store_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_app_configuration</span>.<span style="color: #9cdcfe;">appconf</span>.<span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">key</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">type</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"vault"</span> <span style="color: #6a9955;"># keyvault reference</span></div><div> <span style="color: #9cdcfe;">label</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span>.<span style="color: #9cdcfe;">instancerg</span>.<span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">vault_key_reference</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_role_assignment</span>.<span style="color: #9cdcfe;">appconf_dataowner</span></div><div> ]</div><div>}</div></div><div><br /></div></div></div></div><p>If we need to have same for other secrets we need to define data blocks for each secrets. This would mean lot of unneccessary complexity to code. </p><p><b>The Solution - Option Two</b></p><p><b></b></p><div class="separator" style="clear: both; text-align: center;"><b><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjt_NbPXhjPfQ6mbeLoWMVTmZdISammFbJDf-enZXGG_DQXQyrqPLK7FVxyQ2CT182R1mshOAmlQVXmogTXKyTwE_tsSESs3hWd2MUZa4IrDFuIUdCbQzjNKQhU7Jd6p8M1kvNr_vMSS0JHtQmQ0Cu5iSK3o-xtqNRps1fDF6y6rO9p80RB9QasHeqA5fu9" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="791" data-original-width="2189" height="231" src="https://blogger.googleusercontent.com/img/a/AVvXsEjt_NbPXhjPfQ6mbeLoWMVTmZdISammFbJDf-enZXGG_DQXQyrqPLK7FVxyQ2CT182R1mshOAmlQVXmogTXKyTwE_tsSESs3hWd2MUZa4IrDFuIUdCbQzjNKQhU7Jd6p8M1kvNr_vMSS0JHtQmQ0Cu5iSK3o-xtqNRps1fDF6y6rO9p80RB9QasHeqA5fu9=w637-h231" width="637" /></a></b></div><b><br /></b>Use of <span style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">versionless_id</span> is the best option. Then each of the secret of keyvault will be referred without version in below format.<p></p><p><span style="font-family: courier;">https://keyvaultname.vault.azure.net/secrets/SqlDBPwd</span></p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># Secrets </span></div><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_key_vault_secret"</span> <span style="color: #4fc1ff;">"secret"</span> {</div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">AzureWebJobsStorage</span> = <span style="color: #9cdcfe;">azurerm_storage_account</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">primary_connection_string</span></div><div> <span style="color: #9cdcfe;">SqlDBPwd</span> = <span style="color: #9cdcfe;">azurerm_mssql_server</span>.<span style="color: #9cdcfe;">demo</span>.<span style="color: #9cdcfe;">administrator_login_password</span></div><div> }</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">value</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">key_vault_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span>.<span style="color: #9cdcfe;">id</span></div><br /><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_key_vault</span>.<span style="color: #9cdcfe;">instancekeyvault</span></div><div> ]</div><div>}<span style="background-color: transparent;"> </span></div><div><span style="background-color: transparent;"><br /></span></div><div><div style="line-height: 19px;"><div><span style="color: #4ec9b0;">resource</span> <span style="color: #4fc1ff;">"azurerm_app_configuration_key"</span> <span style="color: #4fc1ff;">"config_vault"</span> {</div><div><div> <span style="color: #9cdcfe;">for_each</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>{</div><div> <span style="color: #9cdcfe;">"AzureWebJobsStorage"</span> = azurerm_key_vault_secret.secret[<span style="color: #ce9178;">"AzureWebJobsStorage"</span>].versionless_id</div><div> <span style="color: #9cdcfe;">"</span><span style="color: #9cdcfe;">SqlDBPwd</span><span style="color: #9cdcfe;">"</span> = data.azurerm_key_vault_secret.sqldbpwd.versionless_id</div><div> }</div><div> <span style="color: #9cdcfe;">configuration_store_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_app_configuration</span>.<span style="color: #9cdcfe;">appconf</span>.<span style="color: #9cdcfe;">id</span></div><div> <span style="color: #9cdcfe;">key</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">key</span></div><div> <span style="color: #9cdcfe;">type</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"vault"</span> <span style="color: #6a9955;"># keyvault reference</span></div><div> <span style="color: #9cdcfe;">label</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">azurerm_resource_group</span>.<span style="color: #9cdcfe;">instancerg</span>.<span style="color: #9cdcfe;">name</span></div><div> <span style="color: #9cdcfe;">vault_key_reference</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">each</span>.<span style="color: #9cdcfe;">value</span></div><div> <span style="color: #9cdcfe;">depends_on</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span>[</div><div> <span style="color: #9cdcfe;">azurerm_role_assignment</span>.<span style="color: #9cdcfe;">appconf_dataowner</span></div><div> ]</div><div>}</div></div><div><br /></div></div></div></div><p>This will ensure latest version of keyvault secret is always referred by the app configuration so that your application will always obtain the latest version of secret value.</p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-46978534685299661672023-07-13T11:24:00.001-07:002023-07-13T11:24:06.729-07:00Fix Terraform Azure AD App Registration (SPN) Read Permssions Running with Azure DevOps Pipelines<p> Azure DevOps use service principals (SPN or Azure AD app registration) to make a service connection to Azure to be able to run Terraform or other IaC based resource deployments targeting Azure. You may run into issue while trying to read another Azure AD app registration information, within terraform. For example consider below code segment.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #6a9955;"># aks kv app</span></div><div><span style="color: #4ec9b0;">data</span> <span style="color: #4fc1ff;">"azuread_application"</span> <span style="color: #4fc1ff;">"akskv"</span> {</div><div> <span style="color: #9cdcfe;">display_name</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span>.<span style="color: #9cdcfe;">PREFIX</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-</span><span style="color: #569cd6;">${</span><span style="color: #9cdcfe;">var</span>.<span style="color: #9cdcfe;">PROJECT</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">-aks-kv-app"</span></div><div>}</div><br /><div><span style="color: #4ec9b0;">data</span> <span style="color: #4fc1ff;">"azuread_service_principal"</span> <span style="color: #4fc1ff;">"akskv"</span> {</div><div> <span style="color: #9cdcfe;">application_id</span><span style="color: #9cdcfe;"> </span>=<span style="color: #9cdcfe;"> </span><span style="color: #9cdcfe;">data</span>.<span style="color: #9cdcfe;">azuread_application</span>.<span style="color: #9cdcfe;">akskv</span>.<span style="color: #9cdcfe;">application_id</span></div><div>}</div></div><p></p><p></p><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span></span></p><a name='more'></a>You may encounter the issue shown below.<p></p><p><span style="font-family: courier;">##[error]Terraform command 'plan' failed with exit code '1'.</span><span style="font-family: courier;">##[error]╷</span><span style="font-family: courier;"> Error: Listing applications for filter "displayName eq 'ch-demo-aks-kv-app'"</span></p><p><span style="font-family: courier;">with data.azuread_application.akskv, </span><span style="font-family: courier;">on rbac.tf line 8, in data "azuread_application" "akskv":</span><span style="font-family: courier;"> data "azuread_application" "akskv"</span></p><p><span style="font-family: courier;">ApplicationsClient.BaseClient.Get(): unexpected status 403 with OData</span><span style="font-family: courier;"> error: Authorization_RequestDenied: Insufficient privileges to complete the </span><span style="font-family: courier;">operation.</span></p><div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEjGL0vESZ4ksiKL_Xamph4rZY7yFnF_zTpuMVP0osGjRberEaME19pgdPkoO5-U7eyGpzKToEEYBgw032cYBPp2NPUJIv8ynUw-Xyqq-a9dhj0COc3YE1-fmYEoW7U-iIAPeqXvWMjc7n73yGl3u6aqc-GH5n3bZOXG08Bt-9giaosfY0UHilZLYYXHaHkS" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="367" data-original-width="673" height="346" src="https://blogger.googleusercontent.com/img/a/AVvXsEjGL0vESZ4ksiKL_Xamph4rZY7yFnF_zTpuMVP0osGjRberEaME19pgdPkoO5-U7eyGpzKToEEYBgw032cYBPp2NPUJIv8ynUw-Xyqq-a9dhj0COc3YE1-fmYEoW7U-iIAPeqXvWMjc7n73yGl3u6aqc-GH5n3bZOXG08Bt-9giaosfY0UHilZLYYXHaHkS=w632-h346" width="632" /></a></div><br /></div>This is due to the service connection SPN is not having permissions to read app registrations in AD. As you can see the SPN is only allowed with AD group read.<br /><div><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhTft00hgksBMP_CjSV5TgvZOs4apcdCjdLDc_rnracxkWt3ttrhLJdW4gFYqyhjpsV48E_np_mKzZxxg-3hbtc6kpJhyfjMA-wkmF7sDW1uslVk9a2XK7SdXo45mrp-KWRZ2lLCHaDhUniurlz53lskLYCVWUX1JbGG3TG3IFjhp60lh_FWm1uYJCVFIA_" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="648" data-original-width="1551" height="255" src="https://blogger.googleusercontent.com/img/a/AVvXsEhTft00hgksBMP_CjSV5TgvZOs4apcdCjdLDc_rnracxkWt3ttrhLJdW4gFYqyhjpsV48E_np_mKzZxxg-3hbtc6kpJhyfjMA-wkmF7sDW1uslVk9a2XK7SdXo45mrp-KWRZ2lLCHaDhUniurlz53lskLYCVWUX1JbGG3TG3IFjhp60lh_FWm1uYJCVFIA_=w610-h255" width="610" /></a></div><div><br /></div>We need to add Application.ReadAll permissions to the SPN which is executing the terraform, so that it is able to read other SPNs (Azure AD apps).<br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhizRr1WA-b1pLRKmq97FCXNz-5j1-Af5cIaWvdapk67cKQZYGQsrbW8SZNZ_2t_7zk0lGGBGfsMIGmMzkG0CLs27NL3bOmGnskrf8Xe4lO4a5IxzqrFv0AsJgSAS8IEfxRjOi49cWAJUTMiQr1HHnMyGqOWh__laMDoxQ32-yTuG6Ck6LPLhaYuJKC4M5F" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="829" data-original-width="1042" height="485" src="https://blogger.googleusercontent.com/img/a/AVvXsEhizRr1WA-b1pLRKmq97FCXNz-5j1-Af5cIaWvdapk67cKQZYGQsrbW8SZNZ_2t_7zk0lGGBGfsMIGmMzkG0CLs27NL3bOmGnskrf8Xe4lO4a5IxzqrFv0AsJgSAS8IEfxRjOi49cWAJUTMiQr1HHnMyGqOWh__laMDoxQ32-yTuG6Ck6LPLhaYuJKC4M5F=w611-h485" width="611" /></a></div><br /><p></p><p>Once permissions added we need to grant concent.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhOH-CWe-xYaSj5IFHz0_grp6KAqQUMrye6ZGRHrjwymj9HFye7ZxNXYrG8k4KrCXwGmtPj4I1mrDRWx3uo_46_HPO4xAVgbkPfRlJ4qG3toa4zunTI-mY7DiWERL7DzLdzmuych7x2Lk5Ho2QmoQ0t0e-c4KiFs8TEaoxFvjZMkVw9HjNt63UuluDtnV-e" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="621" data-original-width="1284" height="303" src="https://blogger.googleusercontent.com/img/a/AVvXsEhOH-CWe-xYaSj5IFHz0_grp6KAqQUMrye6ZGRHrjwymj9HFye7ZxNXYrG8k4KrCXwGmtPj4I1mrDRWx3uo_46_HPO4xAVgbkPfRlJ4qG3toa4zunTI-mY7DiWERL7DzLdzmuych7x2Lk5Ho2QmoQ0t0e-c4KiFs8TEaoxFvjZMkVw9HjNt63UuluDtnV-e=w626-h303" width="626" /></a></div><p>With below permissions set in the SPN the Terraform code shown above in the post can execute in Azure DevOps pipline without any issues.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhjIdLu7TPT-A4-YPaEMMP8sSyvhg2Pbm_doa_dMi8oxwTLwapKbtx5UHDziU8ulr7Fhstz6RbOC3w5SKjQcBBLYU3ZJ9G392MYndNRNmrWiZ0yDxJPuB9wqcTbJOvj8TKk3LblzvUriiJzEfRbdEDmvPXhrvfSApI6UiBgxE9AYUM7Gq88Ed40JojF6Tpp" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="769" data-original-width="1561" height="320" src="https://blogger.googleusercontent.com/img/a/AVvXsEhjIdLu7TPT-A4-YPaEMMP8sSyvhg2Pbm_doa_dMi8oxwTLwapKbtx5UHDziU8ulr7Fhstz6RbOC3w5SKjQcBBLYU3ZJ9G392MYndNRNmrWiZ0yDxJPuB9wqcTbJOvj8TKk3LblzvUriiJzEfRbdEDmvPXhrvfSApI6UiBgxE9AYUM7Gq88Ed40JojF6Tpp=w647-h320" width="647" /></a></div><p><br /></p></div>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0tag:blogger.com,1999:blog-4684874465987951601.post-77150182991235391112023-07-11T02:25:00.001-07:002023-07-11T10:12:52.157-07:00Generate KVSet json Format Using appsettings json for Updating Azure App Configurations<p> To apply Azure App Configurations including key vault secrets reference key values and normal key values, using a single file requires to use KVSet file as <a href="https://learn.microsoft.com/en-us/azure/azure-app-configuration/concept-config-file?WT.mc_id=AZ-MVP-5000590" target="_blank">described in the Article here</a>. With Azure pipelines updating the app config values require, two seperate files to be used in default mode to update, app configs for non secrets and secrets. However, developers of .NET applications would prefer to keep the appsettings file for keeping configurations for development purpose, rather than keeping seperate file to keep references to secret key values. Therefore, in Azure pipline implmentation, it would be required to generate a KVSet file using an app setting file.<span></span></p><a name='more'></a><p></p><p>For example consider the below appsettings file.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>{</div><div> <span style="color: #9cdcfe;">"AppDomainName"</span>: <span style="color: #ce9178;">"fakeorg.com"</span>,</div><div> <span style="color: #9cdcfe;">"Swagger"</span>: {</div><div> <span style="color: #9cdcfe;">"AllowAnonymousSwagger"</span>: <span style="color: #569cd6;">true</span></div><div> },</div><div> <span style="color: #9cdcfe;">"AzureAdB2C"</span>: {</div><div> <span style="color: #9cdcfe;">"Instance"</span>: <span style="color: #ce9178;">"https://fakeorg.b2clogin.com"</span>,</div><div> <span style="color: #9cdcfe;">"Tenant"</span>: <span style="color: #ce9178;">"fakeorg.onmicrosoft.com"</span>,</div><div> <span style="color: #9cdcfe;">"TenantId"</span>: <span style="color: #ce9178;">"fake-tenatid"</span>,</div><div> <span style="color: #9cdcfe;">"ClientId"</span>: <span style="color: #ce9178;">"fake-clientid"</span>,</div><div> <span style="color: #9cdcfe;">"Domain"</span>: <span style="color: #ce9178;">"fakeorg.onmicrosoft.com"</span>,</div><div> <span style="color: #9cdcfe;">"SignedOutCallbackPath"</span>: <span style="color: #ce9178;">"/signout/B2C_1A_AD_SIGNUP_SIGNIN"</span>,</div><div> <span style="color: #9cdcfe;">"SignUpSignInPolicyId"</span>: <span style="color: #ce9178;">"B2C_1A_AD_SIGNUP_SIGNIN"</span>,</div><div> <span style="color: #9cdcfe;">"ClientSecret"</span>: <span style="color: #ce9178;">"fakeclientsecret"</span>,</div><div> <span style="color: #9cdcfe;">"Scope"</span>: <span style="color: #ce9178;">"https://fakeorg.onmicrosoft.com/fake-id/access_as_user"</span>,</div><div> <span style="color: #9cdcfe;">"ShouldCreateNewAzureB2CUserWhenCreatingNewApplicationUser"</span>: <span style="color: #569cd6;">true</span></div><div> },</div><div> <span style="color: #9cdcfe;">"DocuSignClmConfiguration"</span>: {</div><div> <span style="color: #9cdcfe;">"BaseUrl"</span>: <span style="color: #ce9178;">"https://fakeapi.springcm.com/v2"</span>,</div><div> <span style="color: #9cdcfe;">"UploadBaseUrl"</span>: <span style="color: #ce9178;">"https://fakeuploadapi.springcm.com"</span>,</div><div> <span style="color: #9cdcfe;">"Authentication"</span>: {</div><div> <span style="color: #9cdcfe;">"Audience"</span>: <span style="color: #ce9178;">"account-d.docusign.com"</span>,</div><div> <span style="color: #9cdcfe;">"ClientId"</span>: <span style="color: #ce9178;">"fake-clientid"</span></div><div> }</div><div> }</div><div>}</div></div><p>In above appsettings below should be considered as secrets.</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div>AzureAdB<span style="color: #b5cea8;">2</span>C.ClientId</div><div>AzureAdB<span style="color: #b5cea8;">2</span>C.ClientSecret</div><div>DocuSignClmConfiguration.Authentication.ClientId</div></div><p>.NET application should be able to use appsettings file for local runs and should load configurations from Azure App Configuration service for deployed app in Azure.</p><p>To generate the KVSet file as specified <a href="https://learn.microsoft.com/en-us/azure/azure-app-configuration/concept-config-file?WT.mc_id=AZ-MVP-5000590" target="_blank">in the Article here</a> we can write a PowerShell script as show below. We can use the script in Azure DevOps pipelines to apply the app configs to Azure App Configuration service, which we are going to discuss in the next post.</p><p>The script requires appsetting json (hierarchy of app setting defined sperated with dots) and a variable group defined with secret list .</p><p>For example, above <span style="background-color: #1e1e1e; color: #9cdcfe; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">ClientId </span>of <span style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">AzureAdB2C</span> represnted in variable group with value for it as <span style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; white-space: pre;">AzureAdB2C.ClientId</span>. </p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgiOl6TvpaXnhDNOer5FmdGAA7mKobgW41Ppn4w8K0wE_HVrLkEzwpkwZCDTdw27n2L99FFuJqCwlmcSgyFr_dgKeLS-ss4aHOyO45MrxaEwrl8hUYfNVMu8xvoC46DhLusPTxri16z5E6L8wPbMLTF1vjkbmAQ1_b9XQMBtpoFDG3kfYNk_V7CwJgMqA" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1594" data-original-width="1888" height="472" src="https://blogger.googleusercontent.com/img/a/AVvXsEgiOl6TvpaXnhDNOer5FmdGAA7mKobgW41Ppn4w8K0wE_HVrLkEzwpkwZCDTdw27n2L99FFuJqCwlmcSgyFr_dgKeLS-ss4aHOyO45MrxaEwrl8hUYfNVMu8xvoC46DhLusPTxri16z5E6L8wPbMLTF1vjkbmAQ1_b9XQMBtpoFDG3kfYNk_V7CwJgMqA=w559-h472" width="559" /></a></div><br />Same name <span style="background-color: #1e1e1e;"><span style="color: #d4d4d4; font-family: Consolas, Courier New, monospace;"><span style="font-size: 14px; white-space: pre;">AzureAdB2C.ClientId </span></span></span>is used in secret list variable group as well to define as a secret.<p></p><p>If we have no secrets we can create secret list with just a dummy variable name.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhaH3a92a5Nks1INgQ0lTPiBndXkW9EOIdda3ud3AXlXg1wMssV6jyOBf0eY5fVj6CkXCbijoSF2RWZs0p9SlSmXBk90zD473wzGdGQ4mm-L19AQvbgDn5lUk1PKfp9oUszMpgSiYYUiSQLrEBFTT00h8UeCrPNFYEUVVHaIzIHGTPspzpkDaD3fTY5Nw" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1184" data-original-width="1317" height="446" src="https://blogger.googleusercontent.com/img/a/AVvXsEhaH3a92a5Nks1INgQ0lTPiBndXkW9EOIdda3ud3AXlXg1wMssV6jyOBf0eY5fVj6CkXCbijoSF2RWZs0p9SlSmXBk90zD473wzGdGQ4mm-L19AQvbgDn5lUk1PKfp9oUszMpgSiYYUiSQLrEBFTT00h8UeCrPNFYEUVVHaIzIHGTPspzpkDaD3fTY5Nw=w496-h446" width="496" /></a></div><br />For our case we can define three secret variable names.<p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEj9UwiNyvKjw75TVeEaEtVMkHWjgc2asSd473aZtgkC7uI8C7Ysqnw0AlK9-CwlgHgg6qgMcpJ8zmARg2M33uC3KtpFUElefHqUs-aDyO0QGisvVNrahqAFJ9TlgiqpgQsYIPATzpmpVCUCQriGhUBrAJV74vMScW-3aHxrEfmGXI1M3BdoZ63oNMDQHA" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1383" data-original-width="1355" height="497" src="https://blogger.googleusercontent.com/img/a/AVvXsEj9UwiNyvKjw75TVeEaEtVMkHWjgc2asSd473aZtgkC7uI8C7Ysqnw0AlK9-CwlgHgg6qgMcpJ8zmARg2M33uC3KtpFUElefHqUs-aDyO0QGisvVNrahqAFJ9TlgiqpgQsYIPATzpmpVCUCQriGhUBrAJV74vMScW-3aHxrEfmGXI1M3BdoZ63oNMDQHA=w487-h497" width="487" /></a></div><br />Script will identyfy secrets based on secret-list variable group (in below case variable group with id 27) file and will generate the KVSet file with the correct content type.<p></p><!--HTML generated using hilite.me--><div style="background: rgb(255, 255, 255); border-color: gray; border-image: initial; border-style: solid; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;"><pre style="line-height: 125%; margin: 0px;">$appSettingsFile = 'C:\temp\appsettingstokvset\appsettings.json';
$kvSecretsFile = 'C:\temp\appsettingstokvset\secretlist.txt';
$configLabel = 'demo001';
$token = 'pattoken';
$azdoAccoutName= 'chamindac';
$teamProjectName='YAML_CICD';
$User='';
$secretListVarGroupId='27';
function Get-PropValue {
param (
$props,
$propNamePrefix,
$KVSetItems
)
foreach($prop in $props)
{
if($prop.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject')
{
$originalPropNamePrefix = $propNamePrefix;
$propNamePrefix = $propNamePrefix + '.' + $prop.Name;
$KVSetItems = Get-PropValue -props $prop.Value.psObject.Properties -propNamePrefix $propNamePrefix -KVSetItems $KVSetItems
$propNamePrefix = $originalPropNamePrefix;
}
else
{
[string]$fullPropName = ($propNamePrefix + '.' + $prop.Name).Substring(1)
#$fullPropName
#$prop.Value
if ($kvSecrets.Contains($fullPropName))
{
#Write-Host 'secret'
$KVSetItem ='
{
"key": "' + $fullPropName.Replace('.',':') + '",
"value": "{\"uri\":\"' + $prop.Value + '\"}",
"label": "' + $configLabel +'",
"content_type": "application/vnd.microsoft.appconfig.keyvaultref+json;charset=utf-8",
"tags": {}
}';
}
else
{
$KVSetitem ='
{
"key": "' + $fullPropName.Replace('.',':') + '",
"value": "' + $prop.Value + '",
"label": "' + $configLabel +'",
"content_type": null,
"tags": {}
}';
}
$KVSetItems = $KVSetItems + ',' + $KVSetitem;
#Write-Host '------------------'
#$KVSetItem
#Write-Host '++++++++++++++++++'
}
}
return $KVSetItems;
}
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $User,$token)));
$header = @{Authorization=("Basic {0}" -f $base64AuthInfo)};
$Uri = 'https://dev.azure.com/' + $azdoAccoutName + '/' + $teamProjectName + '/_apis/distributedtask/variablegroups/' + $secretListVarGroupId + '?api-version=7.1-preview.2'
$kvSecrets = (Invoke-RestMethod -Method Get -ContentType application/json -Uri $Uri -Headers $header).variables.psObject.Properties.Name
Write-Host '------------------'
Write-Host 'secret-list'
Write-Host '------------------'
$kvSecrets
Write-Host '------------------'
$appSettings = Get-Content -Path $appSettingsFile | ConvertFrom-Json
$KVSetItems = '';
$KVSetItems = Get-PropValue -props $appSettings.psObject.Properties -propNamePrefix '' -KVSetItems $KVSetItems
$KVSet = '{
"items": ['+ $KVSetItems.Substring(1) +'
]
}
';
$KVSet
$KVSet | Out-File -FilePath 'C:\temp\kvset.json'
</pre></div>
<p>Once the script is executed it will generate the KVSet file content as shown below.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEhzxQMEhrQUPDTCt8G1JfFrdm33OU6xL3714QnT334vwEf-H1LJRJvi-U86K8AOuK7usuc6yDe7i0HbO4V93UuwmxAmtHiwMBcBJRjOd8FJzjORd7RnzGyMA75KqO8_FWcfBnWmvpS3Z_GbXHVeetWQNeuVuDtmu_iEFKZJzM8bp1ylvkUzZb3QJ6eHiQ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="3127" data-original-width="1577" height="1289" src="https://blogger.googleusercontent.com/img/a/AVvXsEhzxQMEhrQUPDTCt8G1JfFrdm33OU6xL3714QnT334vwEf-H1LJRJvi-U86K8AOuK7usuc6yDe7i0HbO4V93UuwmxAmtHiwMBcBJRjOd8FJzjORd7RnzGyMA75KqO8_FWcfBnWmvpS3Z_GbXHVeetWQNeuVuDtmu_iEFKZJzM8bp1ylvkUzZb3QJ6eHiQ=w649-h1289" width="649" /></a></div><br /><br /><p></p>Chaminda Chandrasekarahttp://www.blogger.com/profile/10785099125374685765noreply@blogger.com0