Create Multi Stage Pipeline Structure with GitHub Actions - The Layout - Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 2

 In "Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 1" we have discussed the limitation of GitHub action workflows compared to Azure DevOps pipelines. Further, we have compared the features and possible mappings. In this post, let's try how we can create GitHub workflow structure to make it similar to Azure DevOps pipeline shown in "Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 1".

The code for workflows can be structured as shown below for the piline requirement discussed in "Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 1". Note that we have seperated folder structure for job actions and step actions to organize the code properly in a manageable way. However, both are really composite actions in GitHub. Each action name is setup using a folder name as it must be action.yml or action.yaml for actions.

The main workflow is defined as eck_on_aks.yml. All others are reusable workflows to form the structure of the workflow. However, it is not possible to organize them into seprate stage folder etc. as organized in Azure pipeline templates due to limitation of GitHub reusable workflows must be in .github\workflows path. The above GitHub worflow structure represents below, pipeline structure of Azure pipelines (of course with limitations of GitHub actions), to achive pipeline as shown in "Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 1".

Lets now look at the a the outcome of GitHub workflow setup, and see how it maps to Azure pipeline structure above creating the below pipeline stages.

Azure pipeline

GitHub Workflow

Note that unlike above fully functional Azure pipeline here in GitHub workflow is only having the structure created only. No approval gates setup or actual steps of execution implemented. But structure is setup keeping the real working workflow in mind. The intialize stage in Azure pipeline publishes AKS manifest files as artifacts in same pipeline, to be used in pepeline jobs, without full repo checkout. However, this is intentionally skipped in GitHub workflow as it seems checkout in every job is required to get the actions and jobs to be executed in GitHub workflow.

Unlike Azure pipelione env and varaible group independance, GitHub variables are bound to env or repository or at organization level. Therefore, with limitations we discussed in "Mapping Azure DevOps Multi-Stage Pipelines to GitHub Actions - Part 1" we have to split the GitHub workflow to have 2 parts in deploy and destroy phase, to get the approval gates applied, since  we do not have a manual intervention task as in Azure pipelines. Only env can provide an approval gate. Even though unnnecessary we will have to approve even the health check stage as well in GitHub workflow, since we want to keep environment related variables bound to a given environment, and use them in the given stage of workflow. Then to see the actual jobs happen in each phase/stage we have to look inside the workflow stage as below. For example look at Deploy phase plan IaC here, Compared Azure pipeline and GitHub wortkflow. Hilighted are the two jobs.

Since, we want to get an approval to proceed deploy stage Deploy IaC is in the next stage as you can see below. See below the deploy stage jobs for deploying and validating is mapped in GitHub workflow deploy/deploy_Iac stage.

Similar way, we have other stages compared as shown below.

In the destroy stage we have skipped the AKS node pool validation job. The same template used with successful skip. In Azure piplines we can conditionally decide the steps or jobs on templates, but in GitHub workflows, to do it properly avoiding duplication of code, multiple unnecessary approvals, ensure env varaible usage etc, is to use a skip job, which we can discuss in the next post.

The structure workflow and actions code is available here in GitHub (Note that this code will evetually evolve into a fully working workflow setup). In the next post let's discuss the structure implementation in detail with the workflows and the actions code setup.