dagger/docs/learn/107-kubernetes.md

slug
/learn/107-kubernetes

Dagger 107: deploy to Kubernetes

This tutorial illustrates how to use dagger to build, push and deploy Docker images to Kubernetes.

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Prerequisites

For this tutorial, you will need a Kubernetes cluster.

<Tabs defaultValue="kind" groupId="provider" values={[ {label: 'kind', value: 'kind'}, {label: 'GKE', value: 'gke'}, {label: 'EKS', value: 'eks'}, ]}>

Kind is a tool for running local Kubernetes clusters using Docker.

1. Install kind

Follow these instructions to install kind.

Alternatively, on macOS using homebrew:

brew install kind

2. Start a local registry

docker run -d -p 5000:5000 --name registry registry:2

3. Create a cluster with the local registry enabled in containerd

cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
    endpoint = ["http://registry:5000"]
EOF

4. Connect the registry to the cluster network

docker network connect kind registry

This tutorial can be run against a GCP GKE cluster and GCR

This tutorial can be run against a AWS EKS cluster and ECR

Initialize a Dagger Workspace and Environment

(optional) Setup example app

You will need the local copy of the Dagger examples repository used in previous guides

git clone https://github.com/dagger/examples

Make sure that all commands are run from the todoapp directory:

cd examples/todoapp

(optional) Initialize a Cue module

In this guide we will use the same directory as the root of the Dagger workspace and the root of the Cue module; but you can create your Cue module anywhere inside the Dagger workspace.

cue mod init

Organize your package

Let's create a new directory for our Cue package:

mkdir kube

Create a basic plan

Create a file named manifest.cue and add the following configuration to it.

package kube

// inlined kubernetes manifest as a string
manifest: """
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx
    labels:
      app: nginx
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: nginx
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
          - name: nginx
            image: nginx:1.14.2
            ports:
              - containerPort: 80
  """

This will define a manifest variable containing the inlined Kubernetes YAML used to create a nginx deployment.

Next, create source.cue.

<Tabs defaultValue="kind" groupId="provider" values={[ {label: 'kind', value: 'kind'}, {label: 'GKE', value: 'gke'}, {label: 'EKS', value: 'eks'}, ]}>

package kube

import (
  "dagger.io/dagger"
  "dagger.io/kubernetes"
)

// input: ~/.kube/config file used for deployment
// set with `dagger input secret kubeconfig -f ~/.kube/config`
kubeconfig: dagger.#Secret @dagger(input)

// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
  // reference the `kubeconfig` input above
  "kubeconfig": kubeconfig

  // reference to the manifest defined in `manifest.cue`
  "manifest": manifest
}

package kube

import (
  "dagger.io/kubernetes"
  "dagger.io/gcp/gke"
)

// gkeConfig used for deployment
gkeConfig: gke.#KubeConfig @dagger(input)

kubeconfig: gkeConfig.kubeconfig

// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
  // reference the `kubeconfig` input above
  "kubeconfig": kubeconfig

  // reference to the manifest defined in `manifest.cue`
  "manifest": manifest
}

package kube

import (
  "dagger.io/kubernetes"
  "dagger.io/aws/eks"
)

// eksConfig used for deployment
eksConfig: eks.#KubeConfig @dagger(input)

kubeconfig: eksConfig.kubeconfig

// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
  // reference the `kubeconfig` input above
  "kubeconfig": kubeconfig

  // reference to the manifest defined in `manifest.cue`
  "manifest": manifest
}

This defines:

• kubeconfig a string input: kubernetes configuration (~/.kube/config) used for kubectl
• deploy: Deployment step using the package dagger.io/kubernetes. It takes the manifest defined earlier and deploys it to the Kubernetes cluster specified in kubeconfig.

Setup the environment

Create a new environment

Now that your Cue package is ready, let's create an environment to run it:

dagger new 'kube'

Load the plan into the environment

Now let's configure the new environment to use our package as its plan:

cp kube/*.cue .dagger/env/kube/plan/

Note: you need to copy the files from your package into the environment, as shown above. If you make more changes to your package, you will need to copy the new version, or it will not be used. In the future, we will add the ability to reference your Cue package directory, making this manual copy unnecessary.

Configure the environment

Before we can bring up the deployment, we need to provide the kubeconfig input declared in the configuration. Otherwise, dagger will complain about a missing input:

$ dagger up
6:53PM ERR system | required input is missing    input=kubeconfig

You can inspect the list of inputs (both required and optional) using dagger input list:

<Tabs defaultValue="kind" groupId="provider" values={[ {label: 'kind', value: 'kind'}, {label: 'GKE', value: 'gke'}, {label: 'EKS', value: 'eks'}, ]}>

$ dagger input list
Input             Type              Description
kubeconfig        string            ~/.kube/config file used for deployment
deploy.namespace  string            Kubernetes Namespace to deploy to

$ dagger input list
Input                        Type              Description
deploy.namespace             string            Kubernetes Namespace to deploy to
gkeConfig.config.region      string            GCP region
gkeConfig.config.project     string            GCP project
gkeConfig.config.serviceKey  dagger.#Secret    GCP service key
gkeConfig.clusterName        string            GKE cluster name

$ dagger input list
Input                       Type              Description
deploy.namespace            string            Kubernetes Namespace to deploy to
eksConfig.config.region     string            AWS region
eksConfig.config.accessKey  dagger.#Secret    AWS access key
eksConfig.config.secretKey  dagger.#Secret    AWS secret key
eksConfig.clusterName       string            EKS cluster name

Let's provide the missing inputs:

<Tabs defaultValue="kind" groupId="provider" values={[ {label: 'kind', value: 'kind'}, {label: 'GKE', value: 'gke'}, {label: 'EKS', value: 'eks'}, ]}>

# we'll use the ~/.kube/config created by `kind`
dagger input text kubeconfig -f ~/.kube/config

dagger input text gkeConfig.config.project <PROJECT>
dagger input text gkeConfig.config.region <REGION>
dagger input text gkeConfig.clusterName <GKE CLUSTER NAME>
dagger input secret gkeConfig.config.serviceKey -f <PATH TO THE SERVICEKEY.json>

dagger input text eksConfig.config.region <REGION>
dagger input text eksConfig.clusterName <EKS CLUSTER NAME>
dagger input secret eksConfig.config.accessKey <ACCESS KEY>
dagger input secret eksConfig.config.secretKey <SECRET KEY>

Deploying

Now is time to deploy to kubernetes.

$ dagger up
deploy | computing
deploy | #26 0.700 deployment.apps/nginx created
deploy | completed    duration=900ms

Let's verify the deployment worked:

$ kubectl get deployments
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           1m

CUE Kubernetes manifests

In this section we will convert the inlined YAML manifest to CUE to take advantage of the language features.

For a more advanced example, see the official CUE Kubernetes tutorial

First, let's replace manifest.cue with the following configuration. This is a straightforward one-to-one conversion from YAML to CUE, only the syntax has changed.

package kube

import (
  "encoding/yaml"
)

nginx: {
  apiVersion: "apps/v1"
  kind:       "Deployment"
  metadata: {
    "name": "nginx"
    labels: app: "nginx"
  }
  spec: {
    replicas: 1
    selector: matchLabels: app: "nginx"
    template: {
      metadata: labels: app: "nginx"
      spec: containers: [{
        "name":  "nginx"
        "image": image
        ports: [{
          containerPort: port
        }]
      }]
    }
  }
}

manifest: yaml.Marshal(nginx)

We're using the built-in yaml.Marshal function to convert CUE back to YAML so Kubernetes still receives the same manifest.

You need to copy the changes to the plan in order for Dagger to reference them

cp kube/*.cue .dagger/env/kube/plan/

You can inspect the configuration using dagger query to verify it produces the same manifest:

$ dagger query manifest -f text
apiVersion: apps/v1
kind: Deployment
...

Now that the manifest is defined in CUE, we can take advantage of the language to remove a lot of boilerplate and repetition.

Let's define a re-usable #Deployment definition in todoapp/kube/deployment.cue":

package kube

// Deployment template containing all the common boilerplate shared by
// deployments of this application.
#Deployment: {
  // name of the deployment. This will be used to automatically label resouces
  // and generate selectors.
  name: string

  // container image
  image: string

  // 80 is the default port
  port: *80 | int

  // 1 is the default, but we allow any number
  replicas: *1 | int

  // Deployment manifest. Uses the name, image, port and replicas above to
  // generate the resource manifest.
  manifest: {
    apiVersion: "apps/v1"
    kind:       "Deployment"
    metadata: {
      "name": name
      labels: app: name
    }
    spec: {
      "replicas": replicas
      selector: matchLabels: app: name
      template: {
        metadata: labels: app: name
        spec: containers: [{
          "name":  name
          "image": image
          ports: [{
            containerPort: port
          }]
        }]
      }
    }
  }
}

manifest.cue can be rewritten as follows:

import (
  "encoding/yaml"
)

nginx: #Deployment & {
  name:  "nginx"
  image: "nginx:1.14.2"
}

manifest: yaml.Marshal(nginx.manifest)

Update the plan

cp kube/*.cue .dagger/env/kube/plan/

Let's make sure it yields the same result:

$ dagger query deploy.manifest -f text
apiVersion: apps/v1
kind: Deployment
...

And we can now deploy it:

$ dagger up
deploy | computing
deploy | #26 0.700 deployment.apps/nginx unchanged
deploy | completed    duration=900ms

Building, pushing and deploying Docker images

Rather than deploying an existing (nginx) image, we're going to build a Docker image from source, push it to a registry and update the kubernetes configuration.

Update the plan

The following configuration will:

• Declare a repository input as a dagger.#Artifact. This will be mapped to the source code directory.
• Declare a registry input. This is the address used for docker push
• Use dagger.io/docker to build and push the image
• Use the registry image reference (push.ref) as the image for the deployment.

package kube

import (
  "encoding/yaml"

  "dagger.io/dagger"
  "dagger.io/docker"
)

// input: source code repository, must contain a Dockerfile
// set with `dagger input dir repository ./app`
repository: dagger.#Artifact @dagger(input)

// registry to push images to
registry: string @dagger(input)

// docker build the `repository` directory
image: docker.#Build & {
    source: repository
}

// push the `image` to the `registry`
push: docker.#Push & {
  source: image
  ref: registry
}

// use the `#Deployment` template to generate the kubernetes manifest
app: #Deployment & {
  name: "test"

  // use the reference of the image we just pushed
  // this creates a dependency: `app` will only be deployed after the image is
  // built and pushed.
  "image": push.ref
}

manifest: yaml.Marshal(app.manifest)

Update the plan

cp kube/*.cue .dagger/env/kube/plan/

Connect the Inputs

Next, we'll provide the two new inputs, repository and registry.

For the purpose of this tutorial we'll be using hello-go as example source code.

$ git clone https://github.com/aluzzardi/hello-go.git
dagger input dir repository ./hello-go
dagger input text registry "localhost:5000/image"

Bring up the changes

$ dagger up
repository | computing
repository | completed    duration=0s
image | computing
image | completed    duration=1s
deploy | computing
deploy | #26 0.709 deployment.apps/hello created
deploy | completed    duration=900ms

Let's verify the deployment worked:

$ kubectl get deployments
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           1m
hello   1/1     1            1           1m

Next Steps

Deploy on a hosted Kubernetes cluster:

• GKE
• EKS

Authenticate to a remote registry:

• ECR
• GCR

Integrate kubernetes tools with Dagger:

• Helm
• Kustomize