Friday, 5 April 2024

Loop Jobs based on Parameter Value in Azure DevOps Pipelines

 Consider a situation where we want to perform same set of steps in a pipeline, multiple times. A good example would be building or deloying multiple apps, using same set of steps. Let's explore this example to understand how we can loop through set of pipeline steps, to build multiple apps using a list of app names provided as a parameter in the pipeline.

The app name list can be provided as a parameter as shown below.

parameters:
  - name: apps
    displayName: 'Apps to deploy'
    type: object
    default: [
      'customer_api',
      'invoice_api',
      'order_api',
      'payment_api',
      'order_eventhandler',
      'invoice_eventhandler',
      'customer_eventhandler'
      ]

We can then call a template for build and push docker images for the apps. Here, the each app in the parameter allows to loop through and create job for each app.

      # Build and push docker images for each app
      - ${{ each app in parameters.apps }}:
        - template: ../jobs/build_and_push_app.yml
          parameters:
            app: ${{ app }}


The template for job is as below.

parameters:
  - name: app
    type: string

jobs:
  - job: build_and_push_${{ parameters.app }}
    workspace:
      clean: all
    displayName: Build and push ${{ parameters.app }}
    variables:
      # Get app variables for the app
      - template: ../vars/app_vars.yml
        parameters:
          app: ${{ parameters.app }}
      # Get app OS specific variables from os_vars for the app, based on app buildos received 
# from app_vars
      - template: ../vars/os_vars.yml
        parameters:
          buildos: ${{ variables.buildos }}
    # Set agent VM image based on app OS received from os_vars
    pool:
      vmImage: ${{ variables.agentvmimage }}
    steps:
        # - script: |
        #     echo $(agentvmimage)
        #     echo $(buildos)
        #     echo $(aksappname)

        - checkout: self
          fetchDepth: 1
          lfs: true
          clean: true
          submodules: true
          persistCredentials: true

        - task: Docker@2
          displayName: 'Build docker image'
          inputs:
            containerRegistry: '$(aks_shared_container_registry)'
            repository: 'demo/$(aksappname)'
            command: build
            Dockerfile: '**/$(projectname)/Dockerfile'
            buildContext: '$(Build.Repository.LocalPath)'
            tags: '$(Build.BuildId)'
            arguments: '--no-cache'
          retryCountOnTaskFailure: 2
        
        - task: Docker@2
          displayName: 'Push docker image'
          inputs:
            containerRegistry: '$(aks_shared_container_registry)'
            repository: 'demo/$(aksappname)'
            command: push
            tags: '$(Build.BuildId)'
          retryCountOnTaskFailure: 3

Variables for apps are setup as below.

parameters:
  - name: app
    type: string
    
variables:
  - name: aksappname
    value: $[replace('${{ parameters.app }}', '_', '-')]

  - ${{ if eq(parameters.app, 'customer_api') }}:
    - name: projectname
      value: Customer.API
    - name: buildos
      value: windows
    - name: aksappcontainerrequestscpu
      value: 500m
    - name: aksappcontainermemorylimit
      value: 1Gi
    - name: aksapppriorityclassname
      value: demo-medium-priority-win
    - name: deploymentyaml
      value: k8s_api.yaml
  - ${{ elseif eq(parameters.app, 'invoice_api') }}:
    - name: projectname
      value: Invoice.API
    - name: buildos
      value: linux
    - name: aksappcontainerrequestscpu
      value: 200m
    - name: aksappcontainermemorylimit
      value: 512Mi
    - name: aksapppriorityclassname
      value: demo-highest-priority-linux
    - name: deploymentyaml
      value: k8s_api.yaml
  - ${{ elseif eq(parameters.app, 'order_api') }}:
    - name: projectname
      value: Order.API
    - name: buildos
      value: windows
    - name: aksappcontainerrequestscpu
      value: 500m
    - name: aksappcontainermemorylimit
      value: 1Gi
    - name: aksapppriorityclassname
      value: demo-highest-priority-win
    - name: deploymentyaml
      value: k8s_api.yaml
  - ${{ elseif eq(parameters.app, 'payment_api') }}:
    - name: projectname
      value: Payment.API
    - name: buildos
      value: linux
    - name: aksappcontainerrequestscpu
      value: 200m
    - name: aksappcontainermemorylimit
      value: 512Mi
    - name: aksapppriorityclassname
      value: demo-highest-priority-linux
    - name: deploymentyaml
      value: k8s_api.yaml
  - ${{ elseif eq(parameters.app, 'order_eventhandler') }}:
    - name: projectname
      value: Order.EventHandler
    - name: buildos
      value: linux
    - name: aksappcontainerrequestscpu
      value: 200m
    - name: aksappcontainermemorylimit
      value: 512Mi
    - name: aksapppriorityclassname
      value: demo-medium-priority-linux
    - name: deploymentyaml
      value: k8s_eh.yaml
  - ${{ elseif eq(parameters.app, 'invoice_eventhandler') }}:
    - name: projectname
      value: Invoice.EventHandler
    - name: buildos
      value: windows
    - name: aksappcontainerrequestscpu
      value: 500m
    - name: aksappcontainermemorylimit
      value: 1Gi
    - name: aksapppriorityclassname
      value: demo-medium-priority-win
    - name: deploymentyaml
      value: k8s_eh.yaml
  - ${{ elseif eq(parameters.app, 'customer_eventhandler') }}:
    - name: projectname
      value: Customer.EventHandler
    - name: buildos
      value: linux
    - name: aksappcontainerrequestscpu
      value: 200m
    - name: aksappcontainermemorylimit
      value: 512Mi
    - name: aksapppriorityclassname
      value: demo-medium-priority-linux
    - name: deploymentyaml
      value: k8s_eh.yaml

The OS varables are as in below template.

parameters:
  - name: buildos
    type: string
    # default: linux
    # values:
    #   - linux
    #   - windows
    
variables:
  - ${{ if eq(parameters.buildos, 'windows') }}:
    - name: containersleepcommand
      value: '"powershell.exe","-c","sleep"'
    - name: agentvmimage
      value: windows-2022
  - ${{ else }}:
    - name: containersleepcommand
      value: '"sleep"'
    - name: agentvmimage
      value: ubuntu-22.04

The expectation is to create multiple build and push jobs as shown below. The full pipline code is avaialble in the public repo in Azure DevOps here.





No comments:

Popular Posts