whoami

Márk Sági-Kazár

Head of Open Source @ OpenMeter

CNCF Ambassador


YAML engineer for life 😭



@sagikazarmark

https://sagikazarmark.hu

hello@sagikazarmark.hu

CI/CD today

…aka the pain we all know

Day 1…

…how it starts 😃

name: CI

on:
  push:
    branches: [main]

jobs:
  build:
    name: build the software
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: |
          build
          test
          lint

Day 101…

…760 lines of YAML & Bash 😟

# Foobar pipeline
# Include the Common CI pipeline parameters.
include:
  - project: 'foo/bar/foobarproject/cicdtemplate'
    file: '/Common.gitlab-ci.yml'
  #- /force-app/ui-tests/pipeline.yml

stages:
  - build-metadata-package
  - run-js-tests
  - validate-package
  - deploy-package
  - run-unit-tests
  - run-api-tests
  - run-ui-tests
  - integration

####################################################
# Builds the Metadata Package, builds the Package
# files and Destructive changes
####################################################
build-metadata-package:
  stage: build-metadata-package
  except:
    ...

Push and pray 🙏

Developer environment

brew install go
brew install golangci-lint

make build
make test
make lint
  • brew vs nix vs golang.org
  • versions?
  • BSD vs GNU (make, sed, etc)

CI/CD environment

- uses: actions/setup-go@v4
  with:
    go-version: "1.21.3"

- run: go build .
- run: go test -race -v ./…

- uses: golangci/golangci-lint-action@v3
  with:
    version: "v1.54.2"- uses: actions/setup-go@v4
  with:
    go-version: "1.21.3"

- run: go build .
- run: go test -race -v ./…

- uses: golangci/golangci-lint-action@v3
  with:
    version: "v1.54.2"

We are still dealing with drift in 2024?!

  • custom, vendor-specific format

All is well until you need to make changes

  • Upgrades (eg. new version of Go)
  • New tooling (eg. new security scanner)
  • Moving to a new platform
  • company gets acquired and needs to move to the new parent company’s platform

Lessons learned

YAML…

  • is not reusable 1
  • is not composable
  • provides poor flow control
  • is not portable

YAML is a poor choice of “language” for software development workflows.

  1. Anchors don’t count

CI/CD is just an interface

  • Developer and CI environments overlap
  • Drift between them is annoying
  • Time wasted due to “push and pray”
  • CI/CD is no longer a dev or platform only domain

Dev + CI = Software development workflows

  • even if they don’t overlap, they should -> run locally

No more YAML

  • Use a general-purpose programming language
  • One that developers/platform engineers already know
  • Reuseable components

Workflows that run anywhere

  • Run CI locally
  • No more push and pray
  • No more drift between dev and CI
  • Bridge between developers and platform engineers
  • maximize reuse between dev and CI

Show me the code!

func (m *Ci) Test() *Container {
    return dag.Container().
        From("golang").
        WithWorkdir("/src").
        WithMountedDirectory("/src", dag.Directory()).
        WithExec([]string{"go", "test", "-v", "./..."})
}

More code!

func (m *Ci) Test() *Container {
    return dag.Go().
        WithSource(m.Source).
        Exec([]string{"go", "test", "-v", "./..."})
}

Running locally

dagger call test

Running in CI

Jenkins

pipeline { agent { label 'dagger' }
  stages {
    stage("dagger") {
      steps {
        sh '''
          dagger call <build/test/deploy>
        '''
      }
    }
  }
}

GitHub Actions

name: CI
on:
  push:
    branches: [main]

jobs:
  dagger:
    name: dagger
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: dagger call <build/test/deploy>

MOAR CODE!!!

Demo content

https://github.com/sagikazarmark/demo-kcd-romania-2024

Thank you

Any questions?



@sagikazarmark

https://sagikazarmark.hu

hello@sagikazarmark.hu


  1. Slides

  2. Tools

  3. Close
  • Slide 1
  • whoami
  • CI/CD today
  • Day 1…
  • Day 101…
  • Push and pray 🙏...
  • Developer environment
  • CI/CD environment
  • All is well until you need to make changes
  • Lessons learned
  • YAML…
  • CI/CD is just an interface
  • Slide 13
  • No more YAML
  • Workflows that run anywhere
  • Show me the code!
  • More code!
  • Running locally
  • Running in CI
  • MOAR CODE!!!...
  • Demo content
  • Thank you
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • ? Keyboard Help