kjuulh/dagger
Archived
1
0
Fork 0
Code Issues 1 Pull Requests 1 Projects Releases Wiki Activity

Merge pull request #561 from aluzzardi/docs-kubernetes

Browse Source 
docs: kubernetes tutorial
This commit is contained in:
Sam Alba 2021-06-14 18:35:42 +02:00 committed by GitHub
commit 4c382d55cd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 617 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.markdownlint.yaml
View File

@ -3,3 +3,6 @@ default: true
# MD013/line-length - Line length # MD013/line-length - Line length
MD013: false MD013: false
# MD033 - Inline HTML. Needed for tabs in docusaurus
MD033: false

614
docs/programming/guides/kubernetes.md Normal file
View File

@ -0,0 +1,614 @@
# Kubernetes
This tutorial illustrates how to use dagger to build, push and deploy Docker
images to Kubernetes.
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## Prerequisites
For this tutorial, you will need a Kubernetes cluster.
<Tabs
defaultValue="kind"
groupId="provider"
values={[
{label: 'kind', value: 'kind'},
{label: 'GKE', value: 'gke'},
{label: 'EKS', value: 'eks'},
]}>
<TabItem value="kind">
[Kind](https://kind.sigs.k8s.io/docs/user/quick-start) is a tool for running local Kubernetes clusters using Docker.
1\. Install kind
Follow [these instructions](https://kind.sigs.k8s.io/docs/user/quick-start) to
install kind.
Alternatively, on macOS using [homebrew](https://brew.sh/):
```shell
brew install kind
```
2\. Start a local registry
```shell
docker run -d -p 5000:5000 --name registry registry:2
```
3\. Create a cluster with the local registry enabled in containerd
```bash
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
endpoint = ["http://registry:5000"]
EOF
```
4\. Connect the registry to the cluster network
```shell
docker network connect kind registry
```
</TabItem>
<TabItem value="gke">
This tutorial can be run against a [GCP GKE](https://cloud.google.com/kubernetes-engine) cluster and [GCR](https://cloud.google.com/container-registry)
</TabItem>
<TabItem value="eks">
This tutorial can be run against a [AWS EKS](https://aws.amazon.com/eks/) cluster and [ECR](https://aws.amazon.com/ecr/)
</TabItem>
</Tabs>
## Initialize a Dagger Workspace and Environment
```shell
mkdir dagger-kubernetes && cd dagger-kubernetes
dagger init
dagger new default
```
## Create a basic plan
Create a file named `.dagger/env/default/plan/manifest.cue` and add the
following configuration to it.
```cue title=".dagger/env/default/plan/manifest.cue"
package main
// inlined kubernetes manifest as a string
manifest: """
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
"""
```
This will define a `manifest` variable containing the inlined Kubernetes YAML
used to create a _nginx_ deployment.
Next, create `.dagger/env/default/plan/main.cue`.
<Tabs
defaultValue="kind"
groupId="provider"
values={[
{label: 'kind', value: 'kind'},
{label: 'GKE', value: 'gke'},
{label: 'EKS', value: 'eks'},
]}>
<TabItem value="kind">
```cue title=".dagger/env/default/plan/main.cue"
package main
import (
"dagger.io/dagger"
"dagger.io/kubernetes"
)
// input: ~/.kube/config file used for deployment
// set with `dagger input secret kubeconfig -f ~/.kube/config`
kubeconfig: dagger.#Secret @dagger(input)
// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
// reference the `kubeconfig` input above
"kubeconfig": kubeconfig
// reference to the manifest defined in `manifest.cue`
"manifest": manifest
}
```
</TabItem>
<TabItem value="gke">
```cue title=".dagger/env/default/plan/main.cue"
package main
import (
"dagger.io/kubernetes"
"dagger.io/gcp/gke"
)
// gkeConfig used for deployment
gkeConfig: gke.#KubeConfig @dagger(input)
kubeconfig: gkeConfig.kubeconfig
// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
// reference the `kubeconfig` input above
"kubeconfig": kubeconfig
// reference to the manifest defined in `manifest.cue`
"manifest": manifest
}
```
</TabItem>
<TabItem value="eks">
```cue title=".dagger/env/default/plan/main.cue"
package main
import (
"dagger.io/kubernetes"
"dagger.io/aws/eks"
)
// eksConfig used for deployment
eksConfig: eks.#KubeConfig @dagger(input)
kubeconfig: eksConfig.kubeconfig
// deploy uses the `dagger.io/kubernetes` package to apply a manifest to a
// Kubernetes cluster.
deploy: kubernetes.#Resources & {
// reference the `kubeconfig` input above
"kubeconfig": kubeconfig
// reference to the manifest defined in `manifest.cue`
"manifest": manifest
}
```
</TabItem>
</Tabs>
This defines:
- `kubeconfig` a _string_ **input**: kubernetes configuration (`~/.kube/config`)
used for `kubectl`
- `deploy`: Deployment step using the package `dagger.io/kubernetes`. It takes
the `manifest` defined earlier and deploys it to the Kubernetes cluster specified in `kubeconfig`.
### Configure the environment
Before we can bring up the deployment, we need to provide the `kubeconfig` input
declared in the configuration. Otherwise, dagger will complain about a missing input:
```shell
$ dagger up
6:53PM ERR system | required input is missing input=kubeconfig
```
You can inspect the list of inputs (both required and optional) using `dagger input list`:
<!--
<Tabs
defaultValue="kind"
groupId="provider"
values={[
{label: 'kind', value: 'kind'},
{label: 'GKE', value: 'gke'},
{label: 'EKS', value: 'eks'},
]}>
<TabItem value="kind">
</TabItem>
<TabItem value="gke">
</TabItem>
<TabItem value="eks">
</TabItem>
</Tabs>
-->
<Tabs
defaultValue="kind"
groupId="provider"
values={[
{label: 'kind', value: 'kind'},
{label: 'GKE', value: 'gke'},
{label: 'EKS', value: 'eks'},
]}>
<TabItem value="kind">
```shell
$ dagger input list
Input Type Description
kubeconfig string ~/.kube/config file used for deployment
deploy.namespace string Kubernetes Namespace to deploy to
```
</TabItem>
<TabItem value="gke">
```shell
$ dagger input list
Input Type Description
deploy.namespace string Kubernetes Namespace to deploy to
gkeConfig.config.region string GCP region
gkeConfig.config.project string GCP project
gkeConfig.config.serviceKey dagger.#Secret GCP service key
gkeConfig.clusterName string GKE cluster name
```
</TabItem>
<TabItem value="eks">
```shell
$ dagger input list
Input Type Description
deploy.namespace string Kubernetes Namespace to deploy to
eksConfig.config.region string AWS region
eksConfig.config.accessKey dagger.#Secret AWS access key
eksConfig.config.secretKey dagger.#Secret AWS secret key
eksConfig.clusterName string EKS cluster name
```
</TabItem>
</Tabs>
Let's provide the missing inputs:
<Tabs
defaultValue="kind"
groupId="provider"
values={[
{label: 'kind', value: 'kind'},
{label: 'GKE', value: 'gke'},
{label: 'EKS', value: 'eks'},
]}>
<TabItem value="kind">
```shell
# we'll use the ~/.kube/config created by `kind`
dagger input text kubeconfig -f ~/.kube/config
```
</TabItem>
<TabItem value="gke">
```shell
dagger input text gkeConfig.config.project <PROJECT>
dagger input text gkeConfig.config.region <REGION>
dagger input text gkeConfig.clusterName <GKE CLUSTER NAME>
dagger input secret gkeConfig.config.serviceKey -f <PATH TO THE SERVICEKEY.json>
```
</TabItem>
<TabItem value="eks">
```shell
dagger input text eksConfig.config.region <REGION>
dagger input text eksConfig.clusterName <EKS CLUSTER NAME>
dagger input secret eksConfig.config.accessKey <ACCESS KEY>
dagger input secret eksConfig.config.secretKey <SECRET KEY>
```
</TabItem>
</Tabs>
### Deploying
Now is time to deploy to kubernetes.
```shell
$ dagger up
deploy | computing
deploy | #26 0.700 deployment.apps/nginx created
deploy | completed duration=900ms
```
Let's verify the deployment worked:
```shell
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 1m
```
## CUE Kubernetes manifests
In this section we will convert the inlined YAML manifest to CUE to take advantage of the language features.
For a more advanced example, see the
[official CUE Kubernetes tutorial](https://github.com/cuelang/cue/blob/v0.4.0/doc/tutorial/kubernetes/README.md)
First, let's replace `manifest.cue` with the following configuration. This is a
straightforward one-to-one conversion from YAML to CUE, only the syntax has changed.
```cue title=".dagger/env/default/plan/manifest.cue"
package main
import (
"encoding/yaml"
)
nginx: {
apiVersion: "apps/v1"
kind: "Deployment"
metadata: {
"name": "nginx"
labels: app: "nginx"
}
spec: {
replicas: 1
selector: matchLabels: app: "nginx"
template: {
metadata: labels: app: "nginx"
spec: containers: [{
"name": "nginx"
"image": image
ports: [{
containerPort: port
}]
}]
}
}
}
manifest: yaml.Marshal(nginx)
```
We're using the built-in `yaml.Marshal` function to convert CUE back to YAML so
Kubernetes still receives the same manifest.
You can inspect the configuration using `dagger query` to verify it produces the
same manifest:
```shell
$ dagger query manifest -f text
apiVersion: apps/v1
kind: Deployment
...
```
Now that the manifest is defined in CUE, we can take advantage of the language
to remove a lot of boilerplate and repetition.
Let's define a re-usable `#Deployment` definition in `.dagger/env/default/plan/deployment.cue"`:
```cue title=".dagger/env/default/plan/deployment.cue"
package main
// Deployment template containing all the common boilerplate shared by
// deployments of this application.
#Deployment: {
// name of the deployment. This will be used to automatically label resouces
// and generate selectors.
name: string
// container image
image: string
// 80 is the default port
port: *80 | int
// 1 is the default, but we allow any number
replicas: *1 | int
// Deployment manifest. Uses the name, image, port and replicas above to
// generate the resource manifest.
manifest: {
apiVersion: "apps/v1"
kind: "Deployment"
metadata: {
"name": name
labels: app: name
}
spec: {
"replicas": replicas
selector: matchLabels: app: name
template: {
metadata: labels: app: name
spec: containers: [{
"name": name
"image": image
ports: [{
containerPort: port
}]
}]
}
}
}
}
```
`manifest.cue` can be rewritten as follows:
```cue title=".dagger/env/default/plan/manifest.cue"
import (
"encoding/yaml"
)
nginx: #Deployment & {
name: "nginx"
image: "nginx:1.14.2"
}
manifest: yaml.Marshal(nginx.manifest)
```
Let's make sure it yields the same result:
```shell
$ dagger query deploy.manifest -f text
apiVersion: apps/v1
kind: Deployment
...
```
And we can now deploy it:
```shell
$ dagger up
deploy | computing
deploy | #26 0.700 deployment.apps/nginx unchanged
deploy | completed duration=900ms
```
## Building, pushing and deploying Docker images
Rather than deploying an existing (`nginx`) image, we're going to build a Docker
image from source, push it to a registry and update the kubernetes configuration.
### Update the plan
The following configuration will:
- Declare a `repository` input as a `dagger.#Artifact`. This will be mapped to
the source code directory.
- Declare a `registry` input. This is the address used for docker push
- Use `dagger.io/docker` to build and push the image
- Use the registry image reference (`push.ref`) as the image for the deployment.
```cue title=".dagger/env/default/plan/manifest.cue"
package main
import (
"encoding/yaml"
"dagger.io/dagger"
"dagger.io/docker"
)
// input: source code repository, must contain a Dockerfile
// set with `dagger input dir repository ./app`
repository: dagger.#Artifact @dagger(input)
// registry to push images to
registry: string @dagger(input)
// docker build the `repository` directory
image: docker.#Build & {
source: repository
}
// push the `image` to the `registry`
push: docker.#Push & {
source: image
ref: registry
}
// use the `#Deployment` template to generate the kubernetes manifest
app: #Deployment & {
name: "test"
// use the reference of the image we just pushed
// this creates a dependency: `app` will only be deployed after the image is
// built and pushed.
"image": push.ref
}
manifest: yaml.Marshal(app.manifest)
```
### Connect the Inputs
Next, we'll provide the two new inputs, `repository` and `registry`.
For the purpose of this tutorial we'll be using
[hello-go](https://github.com/aluzzardi/hello-go) as example source code.
```shell
$ git clone https://github.com/aluzzardi/hello-go.git
dagger input dir repository ./hello-go
dagger input text registry "localhost:5000/image"
```
### Bring up the changes
```shell
$ dagger up
repository | computing
repository | completed duration=0s
image | computing
image | completed duration=1s
deploy | computing
deploy | #26 0.709 deployment.apps/hello created
deploy | completed duration=900ms
```
Let's verify the deployment worked:
```shell
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 1m
hello 1/1 1 1 1m
```
## Next Steps
Deploy on a hosted Kubernetes cluster:
- [GKE](https://github.com/dagger/dagger/tree/main/stdlib/gcp/gke)
- [EKS](https://github.com/dagger/dagger/tree/main/stdlib/aws/eks)
Authenticate to a remote registry:
- [ECR](https://github.com/dagger/dagger/tree/main/stdlib/aws/ecr)
- [GCR](https://github.com/dagger/dagger/tree/main/stdlib/gcp/gcr)
Integrate kubernetes tools with Dagger:
- [Helm](https://github.com/dagger/dagger/tree/main/stdlib/kubernetes/helm)
- [Kustomize](https://github.com/dagger/dagger/tree/main/stdlib/kubernetes/kustomize)