From 4221011064abf020999b2465c870cd12d0038993 Mon Sep 17 00:00:00 2001 From: Andrea Luzzardi Date: Thu, 3 Jun 2021 18:18:32 -0700 Subject: [PATCH] docs: kubernetes tutorial Signed-off-by: Andrea Luzzardi --- docs/tutorials/_category_.json | 4 + docs/tutorials/kubernetes.md | 423 +++++++++++++++++++++++++++++++++ 2 files changed, 427 insertions(+) create mode 100644 docs/tutorials/_category_.json create mode 100644 docs/tutorials/kubernetes.md diff --git a/docs/tutorials/_category_.json b/docs/tutorials/_category_.json new file mode 100644 index 00000000..ae26fa51 --- /dev/null +++ b/docs/tutorials/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Tutorials", + "position": 5 +} diff --git a/docs/tutorials/kubernetes.md b/docs/tutorials/kubernetes.md new file mode 100644 index 00000000..e71c2238 --- /dev/null +++ b/docs/tutorials/kubernetes.md @@ -0,0 +1,423 @@ +# Kubernetes + +This example illustrates how to use `dagger` to build, push and deploy Docker +images to Kubernetes. + +## Prerequisites + +### Setup a local Kubernetes cluster + +While dagger supports GKE and EKS, for the purpose of this example, we'll be +using [kind](https://kind.sigs.k8s.io/) to install a local Kubernetes cluster +in addition to a local container registry, no cloud account required. + +1\. Install kind + +Follow [these instructions](https://kind.sigs.k8s.io/docs/user/quick-start) to +install kind. + +Alternatively, on macOS using [homebrew](https://brew.sh/): + +```shell +brew install kind +``` + +2\. Start a local registry + +```shell +docker run -d -p 5000:5000 --name registry registry:2 +``` + +3\. Create a cluster with the local registry enabled in containerd + +```bash +cat < + +Create a file named `.dagger/env/default/plan/manifest.cue` and add the +following configuration to it. + +```cue title=".dagger/env/default/plan/manifest.cue" +package main + +// inlined kubernetes manifest as a string +manifest: """ + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + labels: + app: nginx + spec: + replicas: 1 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + """ +``` + +This will define a `manifest` variable containing the inlined Kubernetes YAML +used to create a _nginx_ deployment. + +Next, create `.dagger/env/default/plan/main.cue`. + +```cue title=".dagger/env/default/plan/main.cue" +package main + +import ( + "dagger.io/kubernetes" +) + +// input: ~/.kube/config file used for deployment +// set with `dagger input text kubeconfig -f ~/.kube/config` +kubeconfig: string @dagger(input) + +// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a +// Kubernetes cluster. +deploy: kubernetes.#Apply & { + // reference the `kubeconfig` input above + "kubeconfig": kubeconfig + + // reference to the manifest defined in `manifest.cue` + "manifest": manifest +} +``` + +This defines: + +- `kubeconfig` a _string_ **input**: kubernetes configuration (`~/.kube/config`) + used for `kubectl` +- `deploy`: Deployment step using the package `dagger.io/kubernetes`. It takes + the `manifest` defined earlier and deploys it to the Kubernetes cluster specified in `kubeconfig`. + +### Configure the environment + +Before we can bring up the deployment, we need to provide the `kubeconfig` input +declared in the configuration. Otherwise, dagger will complain about a missing input: + +```shell +$ dagger up +6:53PM ERR system | required input is missing input=kubeconfig +``` + +You can inspect the list of inputs (both required and optional) using `dagger input list`: + +```shell +$ dagger input list +Input Type Description +kubeconfig string ~/.kube/config file used for deployment +deploy.namespace string Kubernetes Namespace to deploy to +``` + +Let's provide the missing input: + +```shell +# we'll use the ~/.kube/config created by `kind` +dagger input text kubeconfig -f ~/.kube/config +``` + +### Deploying + +Now is time to deploy to kubernetes. + +```shell +$ dagger up +deploy | computing +deploy | #26 0.700 deployment.apps/nginx created +deploy | completed duration=900ms +``` + +Let's verify the deployment worked: + +```shell +$ kubectl get deployments +NAME READY UP-TO-DATE AVAILABLE AGE +nginx 1/1 1 1 1m +``` + +## CUE Kubernetes manifests + +In this section we will convert the inlined YAML manifest to CUE to take advantage of the language features. + +For a more advanced example, see the +[official CUE Kubernetes tutorial](https://github.com/cuelang/cue/blob/v0.4.0/doc/tutorial/kubernetes/README.md) + +First, let's replace `manifest.cue` with the following configuration. This is a +straightforward one-to-one conversion from YAML to CUE, only the syntax has changed. + +```cue title=".dagger/env/default/plan/manifest.cue" +package main + +import ( + "encoding/yaml" +) + +nginx: { + apiVersion: "apps/v1" + kind: "Deployment" + metadata: { + "name": "nginx" + labels: app: "nginx" + } + spec: { + replicas: 1 + selector: matchLabels: app: "nginx" + template: { + metadata: labels: app: "nginx" + spec: containers: [{ + "name": "nginx" + "image": image + ports: [{ + containerPort: port + }] + }] + } + } +} + +manifest: yaml.Marshal(nginx) +``` + +We're using the built-in `yaml.Marshal` function to convert CUE back to YAML so +Kubernetes still receives the same manifest. + +You can inspect the configuration using `dagger query` to verify it produces the +same manifest: + +```shell +$ dagger query manifest -f text +apiVersion: apps/v1 +kind: Deployment +... +``` + +Now that the manifest is defined in CUE, we can take advantage of the language +to remove a lot of boilerplate and repetition. + +Let's define a re-usable `#Deployment` definition in `.dagger/env/default/plan/deployment.cue"`: + +```cue title=".dagger/env/default/plan/deployment.cue" +package main + +// Deployment template containing all the common boilerplate shared by +// deployments of this application. +#Deployment: { + // name of the deployment. This will be used to automatically label resouces + // and generate selectors. + name: string + + // container image + image: string + + // 80 is the default port + port: *80 | int + + // 1 is the default, but we allow any number + replicas: *1 | int + + // Deployment manifest. Uses the name, image, port and replicas above to + // generate the resource manifest. + manifest: { + apiVersion: "apps/v1" + kind: "Deployment" + metadata: { + "name": name + labels: app: name + } + spec: { + "replicas": replicas + selector: matchLabels: app: name + template: { + metadata: labels: app: name + spec: containers: [{ + "name": name + "image": image + ports: [{ + containerPort: port + }] + }] + } + } + } +} +``` + +`manifest.cue` can be rewritten as follows: + +```cue title=".dagger/env/default/plan/manifest.cue" +import ( + "encoding/yaml" +) + +nginx: #Deployment & { + name: "nginx" + image: "nginx:1.14.2" +} + +manifest: yaml.Marshal(nginx.manifest) +``` + +Let's make sure it yields the same result: + +```shell +$ dagger query deploy.manifest -f text +apiVersion: apps/v1 +kind: Deployment +... +``` + +And we can now deploy it: + +```shell +$ dagger up +deploy | computing +deploy | #26 0.700 deployment.apps/nginx unchanged +deploy | completed duration=900ms +``` + +## Building, pushing and deploying Docker images + +Rather than deploying an existing (`nginx`) image, we're going to build a Docker +image from source, push it to a registry and update the kubernetes configuration. + +### Update the plan + +The following configuration will: + +- Declare a `repository` input as a `dagger.#Artifact`. This will be mapped to + the source code directory. +- Declare a `registry` input. This is the address used for docker push +- Use `dagger.io/docker` to build and push the image +- Use the registry image reference (`push.ref`) as the image for the deployment. + +```cue title=".dagger/env/default/plan/manifest.cue" +package main + +import ( + "encoding/yaml" + + "dagger.io/docker" +) + +// input: source code repository, must contain a Dockerfile +// set with `dagger input dir repository ./app` +repository: dagger.#Artifact @dagger(input) + +// registry to push images to +registry: string @dagger(input) + +// docker build the `repository` directory +image: docker.#Build & { + source: repository +} + +// push the `image` to the `registry` +push: docker.#Push & { + source: image + ref: registry +} + +// use the `#Deployment` template to generate the kubernetes manifest +app: #Deployment & { + name: "test" + + // use the reference of the image we just pushed + // this creates a dependency: `app` will only be deployed after the image is + // built and pushed. + "image": push.ref +} + +manifest: yaml.Marshal(app.manifest) +``` + +### Connect the Inputs + +Next, we'll provide the two new inputs, `repository` and `registry`. + +For the purpose of this tutorial we'll be using +[hello-go](https://github.com/aluzzardi/hello-go) as example source code. + +```shell +$ git clone https://github.com/aluzzardi/hello-go.git +dagger input dir repository ./hello-go +dagger input text registry "localhost:5000/image" +``` + +### Bring up the changes + +```shell +$ dagger up +repository | computing +repository | completed duration=0s +image | computing +image | completed duration=1s +deploy | computing +deploy | #26 0.709 deployment.apps/hello created +deploy | completed duration=900ms +``` + +Let's verify the deployment worked: + +```shell +$ kubectl get deployments +NAME READY UP-TO-DATE AVAILABLE AGE +nginx 1/1 1 1 1m +hello 1/1 1 1 1m +``` + +## Next Steps + +Deploy on a hosted Kubernetes cluster: + +- [GKE](https://github.com/dagger/dagger/tree/main/stdlib/gcp/gke) +- [EKS](https://github.com/dagger/dagger/tree/main/stdlib/aws/eks) + +Authenticate to a remote registry: + +- [ECR](https://github.com/dagger/dagger/tree/main/stdlib/aws/ecr) +- [GCR](https://github.com/dagger/dagger/tree/main/stdlib/gcp/gcr) + +Integrate kubernetes tools with Dagger: + +- [Helm](https://github.com/dagger/dagger/tree/main/stdlib/kubernetes/helm) +- [Kustomize](https://github.com/dagger/dagger/tree/main/stdlib/kubernetes/kustomize)