GitOps CI/CD workflows with GitHub Actions

The current state of GitHub Actions is promising but still has some rough edges when it comes to configuring complete GitOps CI/CD workflows. Compared to other systems such as GitLab-CI, it can be a pain to achieve the pipelines you're after without violating DRY principles.

However, after skimming the web for answers and a lot of trials and errors, I've managed to find some tricks to get the job done. Let's get into it!

Dependent and conditional jobs#

Suppose you need a complete GitOps CI/CD pipeline for your next JAMStack application, with continuous integration on every push, preview deployment on the develop branch and production deployment only on the master branch.

The workflow usually goes as follows:

1. Testing stage (tests, lint check, vulnerabilities check)

2. Building stage

3. Depends on the testing stage

4. Build for preview if on develop branch

5. Build for production if on master branch

6. Deploying stage

7. Deploy for preview if on develop branch, depends on preview build

8. Deploy for production if on master branch, depends on production build

The tricky part with GitHub Action workflows is to define conditions to trigger certain jobs on certain conditions. For this, you need to rely on the needs statement, the if statement and context and expression syntax:

build-preview:
  name: Build preview
  needs: test
  if: contains(github.ref, 'develop')

A complete template for this kind of workflow would be:

name: CI/CD

on: [push]

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      ...

  build-preview:
    name: Build Preview
    runs-on: ubuntu-latest
    needs: test
    if: contains(github.ref, 'develop')

    steps:
      - uses: actions/checkout@v2
      ...
      - name: Upload build artifacts
        uses: actions/upload-artifact@v1
        with:
          name: build-preview
          path: dist

  build-production:
    name: Build Production
    runs-on: ubuntu-latest
    needs: test
    if: contains(github.ref, 'master')

    steps:
      - uses: actions/checkout@v2
      ...
      - name: Upload build artifacts
        uses: actions/upload-artifact@v1
        with:
          name: build-production
          path: dist

  deploy-preview:
    name: Deploy Preview
    runs-on: ubuntu-latest
    needs: build-preview
    if: contains(github.ref, 'develop')

    steps:
      - uses: actions/checkout@v2
      - name: Download build artifacts
        uses: actions/download-artifact@v1
        with:
          name: build-preview
          path: dist
      ...

  deploy-production:
    name: Deploy Production
    runs-on: ubuntu-latest
    needs: build-production
    if: contains(github.ref, 'master')

    steps:
      - uses: actions/checkout@v2
      - name: Download build artifacts
        uses: actions/download-artifact@v1
        with:
          name: build-production
          path: dist
      ...

And before you ask, no, you cannot store the results of contains(...) in environment variables to be able to re-use them if job's if conditions. According to the documentation, "you can only use the env context in the value of the with and name keys, or in a step's if conditional".

Git reference retrieval#

Sometimes, you need to dynamically retrieve some information from the current Git reference. A common use-case is to tag a Docker image using the Git reference information (branch name, commit SHA1, tag name, etc.).

One way of doing it is to use shell parameter expansion:

jobs:
  docker-build:
    name: Docker build
    runs-on: ubuntu-latest

    env:
      IMAGE_NAME: mynamespace/myimage
      IMAGE_TAG: ${GITHUB_REF##*/}

    steps:
      - uses: actions/checkout@v2
      - name: Build Docker image (${{ env.IMAGE_TAG }})
        run: docker build -t $IMAGE_NAME:$IMAGE_TAG .
      ...

If the tag value might contain a / character, it might be better to use the replace syntax:

env:
  IMAGE_TAG: ${GITHUB_REF/refs\/tags\//}

Dynamic environment variables#

The current implementation of GitHub Action workflows does not support YAML anchors, strongly limiting DRY (Don't Repeat Yourself) principles. This is a shame as this is usually the secret weapon of complex and reusable CI/CD pipeline configurations!

Nevertheless, GitHub Action workflow have the hability to dynamically set environment variables which can allow some code factoring. To set the value of an environment variable dynamically, the set-env statement can be used:

steps:
  - name: Set an env var only for tags
    if: contains(github.ref, 'tags')
    run: echo "::set-env name=VAR_NAME::value"

and the new value for VAR_NAME will be available for any subsequent actions in a job, but not the current action which sets it.

Going back to building a Docker image, a common need is to tag the image with latest most of the time, and tag it with the version number (from the Git tag) for a release build. It could go as follows:

build-docker:
  name: Docker build
  runs-on: ubuntu-latest
  needs: test
  if: contains(github.ref, 'master') || contains(github.ref, 'tags')

  env:
    IMAGE_NAME: mynamespace/myimage
    IMAGE_TAG: latest

  steps:
    - uses: actions/checkout@v2
    - name: Select Docker image tag (production only)
      if: contains(github.ref, 'tags')
      run: echo "::set-env name=IMAGE_TAG::${GITHUB_REF##*/}"
    - name: Build Docker image (${{ env.IMAGE_TAG }})
      run: |
        docker pull $IMAGE_NAME:latest
        docker build -t $IMAGE_NAME:$IMAGE_TAG .

Not completely DRY but at least we did not have to write two separate jobs for different build configurations depending on the Git reference, so that's a already a win!

Conclusion#

The GitHub Actions platform allows more than just CI/CD, being built on a marketplace community, being integrated with non-Git events (issues, etc.) and supporting not just Linux-based builds with Windows and MacOS virtual machines.

However, compared to other CI/CD platforms such as GitLab-CI, defining complete GitOps CI/CD pipelines is not that easy and error-prone. While searching for the current best-practices, it seems I am clearly not alone facing those issues and hopefully the GitHub team will fix all those quirks in the next updates.