kjuulh/dagger
Archived
1
0
Fork 0
Code Issues 1 Pull Requests 1 Projects Releases Wiki Activity

Merge pull request #1860 from gerhard/europa-docs-plan-tanguy-fresh-pair-of-eyes

Browse Source
This commit is contained in:
Solomon Hykes 2022-03-26 12:45:13 -07:00 committed by GitHub
commit 9f042800da
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 107 additions and 66 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

98
docs/core-concepts/1202-plan.md
View File

@ -5,15 +5,19 @@ displayed_sidebar: europa
# It all starts with a plan # It all starts with a plan
A CI/CD pipeline declared in Dagger starts with a plan, specifically `dagger.#Plan` ## Plan structure
This plan is the entrypoint for everything that runs within a pipeline. The simplest plan will usually: A config declared in Dagger starts with a plan, specifically `dagger.#Plan`
- interact with the client filesystem to read (e.g. source code) or write files (e.g. build output) Within this plan we can:
- read environment variables
- declare a few actions, e.g. deps, test & build
This is our **Getting Started** example app plan structure: - interact with the `client` filesystem
- read files, usually the current directory as `.`
- write files, usually the build output as `_build`
- read `env` variables, such as `NETLIFY_TEAM` in our example
- declare a few `actions`, e.g. `deps`, `test` & `build`
This is our **Getting Started** todoapp plan structure:
```cue file=../tests/core-concepts/plan/structure.cue.fragment ```cue file=../tests/core-concepts/plan/structure.cue.fragment
``` ```
@ -28,46 +32,80 @@ When the above plan gets executed via `dagger do build`, it produces the followi
[✔] actions.build.run.script 0.0s [✔] actions.build.run.script 0.0s
[✔] actions.build.run 0.0s [✔] actions.build.run 0.0s
[✔] actions.build.contents 0.0s [✔] actions.build.contents 0.0s
[✔] client.filesystem.build.write 0.1s [✔] client.filesystem."./_build".write 0.1s
``` ```
Since these actions have run before, they are cached and take less than 1 second to complete. Since these actions have run before, they are cached and take less than 2 seconds to complete.
While the names used for the actions above - `deps`, `test` & `build` - are short & descriptive, While the names used for the actions above - `deps`, `test` & `build` - are short and descriptive,
any other names would have worked. Put differently, action naming does not affect plan execution. any other names would have worked. Put differently, action naming does not affect plan execution.
In the example above, the `deps` action is an instance of the docker package build definition. Lastly, notice that even if the `deploy` action is defined, we did not run it.
Similar to Makefile targets, we have the option of running specific actions.
This is written as `deps: docker.#Build` We ran the `dagger do build` command, which only runs the `build` action (and all its dependent actions).
This Dagger property enables us to keep the entire CI/CD config in a single file, while keeping the integration execution separate from the deployment one.
Separating CI & CD concerns becomes essential as our pipelines grow in complexity, and we learn about operational and security constraints specific to our systems.
Default definition configuration - `docker.#Build` in this case - can be modified via curly brackets, e.g. ## Packages & imports
In order to understand the correlation between actions, definitions and packages, let us focus on the following fragment from our **Getting Started** todoapp config:
```cue ```cue
deps: docker.#Build & { package todoapp
import (
"dagger.io/dagger"
"universe.dagger.io/netlify"
)
dagger.#Plan & {
// ...
actions: {
// ...
deploy: netlify.#Deploy & {
// ... // ...
} }
// ...
}
}
```
We start by declaring the package name, `package todoapp` above.
Next, we import the packages that we use in our plan.
The first import is needed for the `dagger.#Plan` definition to be available.
The second import is for `netlify.#Deploy` to work.
:::info
Which other imports we are missing?
Look at all the actions in the plan structure at the top of this page.
Now check all the available packages in [universe.dagger.io](https://github.com/dagger/dagger/tree/v0.2.0/pkg/universe.dagger.io).
:::
We now understand that the `deploy` action is the deploy definition from the netlify package, written as `deploy: netlify.#Deploy`
Each definition has default values that can be modified via curly brackets. This is what that looks like in practice for our deploy action:
```cue
// ...
deploy: netlify.#Deploy & {
contents: build.contents.output
site: client.env.APP_NAME
token: client.env.NETLIFY_TOKEN
team: client.env.NETLIFY_TEAM
}
// ...
``` ```
We can build complex pipelines efficiently by referencing any definition, from any package in our actions. We can build complex pipelines efficiently by referencing any definition, from any package in our actions.
This is one of the fundamental concepts that makes Dagger a powerful language for CI/CD. This is one of the fundamental concepts that makes Dagger a powerful language for building CI/CD pipelines.
Before we can use a package in a plan, we need to declare it at the top of the pipeline configuration, like this:
```cue If you want to learn more packages in the context of CUE, the config language used by Dagger configs, check out the [Packages](1215-what-is-cue.md#packages) section on the **What is CUE?** page.
import (
"universe.dagger.io/docker"
)
```
Since we are using the plan definition from the dagger package - `dagger.#Plan` - we also need to declare it at the top of the pipeline configuration:
```cue
import (
"dagger.io/dagger"
"universe.dagger.io/docker"
)
```
:::tip :::tip
Now that we understand the basics of a Dagger plan, we are ready to learn more about how to interact with the client environment. Now that we understand the basics of a Dagger plan, we are ready to learn more about how to interact with the client environment.
This will enable us to configure plans just-in-time, save build artefacts, and perform other interactions with the environment within which Dagger runs. We can read the env (including secrets), run commands, use local sockets, etc.
::: :::

37
docs/getting-started/1200-local-dev.md
View File

@ -67,20 +67,20 @@ dagger do build
With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 3 minutes: With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 3 minutes:
```shell ```shell
[✔] client.filesystem.".".read 0.1s [✔] client.filesystem."./".read 0.1s
[✔] actions.deps 118.8s [✔] actions.deps 118.8s
[✔] actions.test.script 0.1s [✔] actions.test.script 0.1s
[✔] actions.test 6.3s [✔] actions.test 6.3s
[✔] actions.build.run.script 0.0s [✔] actions.build.run.script 0.0s
[✔] actions.build.run 43.7s [✔] actions.build.run 43.7s
[✔] actions.build.contents 0.4s [✔] actions.build.contents 0.4s
[✔] client.filesystem.build.write 0.1s [✔] client.filesystem."./_build".write 0.1s
``` ```
Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser. Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser.
The last step - `client.filesystem.build.write` - copies the build result into the `build` directory on the host. The last step - `client.filesystem.build.write` - copies the build result into the `_build` directory on the host.
On macOS, we run `open build/index.html` in our terminal and see the following app preview: On macOS, we run `open _build/index.html` in our terminal and see the following app preview:
![todoapp preview](/img/getting-started/todoapp.macos.png) ![todoapp preview](/img/getting-started/todoapp.macos.png)
@ -99,14 +99,14 @@ I change this line to `What must be done today?` and run the build locally again
```shell ```shell
dagger do build dagger do build
[✔] client.filesystem.".".read 0.0s [✔] client.filesystem."./".read 0.0s
[✔] actions.deps 7.5s [✔] actions.deps 7.5s
[✔] actions.test.script 0.0s [✔] actions.test.script 0.0s
[✔] actions.test 6.0s [✔] actions.test 6.0s
[✔] actions.build.run.script 0.0s [✔] actions.build.run.script 0.0s
[✔] actions.build.run 29.2s [✔] actions.build.run 29.2s
[✔] actions.build.contents 0.0s [✔] actions.build.contents 0.0s
[✔] client.filesystem.build.write 0.1s [✔] client.filesystem."./_build".write 0.1s
``` ```
The total `42.8` time is macOS specific, since the Linux alternative is more than 8x quicker. The total `42.8` time is macOS specific, since the Linux alternative is more than 8x quicker.
@ -160,20 +160,20 @@ dagger do build
With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 1 minute: With an empty cache, installing all dependencies, then testing & generating a build for this example app completes in just under 1 minute:
```shell ```shell
[✔] client.filesystem.".".read 0.3s [✔] client.filesystem."./".read 0.3s
[✔] actions.deps 39.7s [✔] actions.deps 39.7s
[✔] actions.test.script 0.2s [✔] actions.test.script 0.2s
[✔] actions.test 1.9s [✔] actions.test 1.9s
[✔] actions.build.run.script 0.1s [✔] actions.build.run.script 0.1s
[✔] actions.build.run 10.0s [✔] actions.build.run 10.0s
[✔] actions.build.contents 0.6s [✔] actions.build.contents 0.6s
[✔] client.filesystem.build.write 0.1s [✔] client.filesystem."./_build".write 0.1s
``` ```
Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser. Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser.
The last step - `client.filesystem.build.write` - copies the build result into the `build` directory on the host. The last step - `client.filesystem.build.write` - copies the build result into the `_build` directory on the host.
On Linux, we run `xdg-open build/index.html` in our terminal and see the following app preview: On Linux, we run `xdg-open _build/index.html` in our terminal and see the following app preview:
![todoapp preview](/img/getting-started/todoapp.linux.png) ![todoapp preview](/img/getting-started/todoapp.linux.png)
@ -192,14 +192,14 @@ I change this line to `What must be done today?` and run the build locally again
```shell ```shell
dagger do build dagger do build
[✔] client.filesystem.".".read 0.0s [✔] client.filesystem."./".read 0.0s
[✔] actions.deps 1.1s [✔] actions.deps 1.1s
[✔] actions.test.script 0.0s [✔] actions.test.script 0.0s
[✔] actions.test 0.0s [✔] actions.test 0.0s
[✔] actions.build.run.script 0.8s [✔] actions.build.run.script 0.8s
[✔] actions.build.run 2.9s [✔] actions.build.run 2.9s
[✔] actions.build.contents 0.0s [✔] actions.build.contents 0.0s
[✔] client.filesystem.build.write 0.0s [✔] client.filesystem."./_build".write 0.0s
``` ```
Being able to re-run the test & build loop locally in `4.8s`, at the same speed as running `yarn` scripts locally and without adding any extra dependencies to our host, is likely to change our approach to iterating on changes. Being able to re-run the test & build loop locally in `4.8s`, at the same speed as running `yarn` scripts locally and without adding any extra dependencies to our host, is likely to change our approach to iterating on changes.
@ -249,16 +249,15 @@ With an empty cache, installing all dependencies, then testing & generating a bu
[✔] actions.deps 62.1s [✔] actions.deps 62.1s
[✔] actions.build.run.script 0.4s [✔] actions.build.run.script 0.4s
[✔] actions.test.script 0.5s [✔] actions.test.script 0.5s
[✔] client.filesystem.".".read 0.6s [✔] client.filesystem."./".read 0.6s
[✔] actions.test 2.0s [✔] actions.test 2.0s
[✔] actions.build.run 12.4s [✔] actions.build.run 12.4s
[✔] actions.build.contents 0.1s [✔] actions.build.contents 0.1s
[✔] client.filesystem.build.write 0.2s [✔] client.filesystem."./_build".write 0.2s
[✔] client.filesystem.".".read 0.2s
``` ```
Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser. Since this is a static application, we can open the files which are generated in `actions.build.contents` in a browser.
The last step - `client.filesystem.build.write` - copies the build result into the `build` directory on the host. The last step - `client.filesystem.build.write` - copies the build result into the `_build` directory on the host.
On Windows, we run `start build/index.html` in our `Command Prompt` terminal and see the following app preview: On Windows, we run `start build/index.html` in our `Command Prompt` terminal and see the following app preview:
@ -278,16 +277,14 @@ I change this line to `What must be done today?` and run the build locally again
```shell ```shell
dagger do build dagger do build
INF upgrading buildkit have host network=true version=v0.10.0
[✔] actions.build.run.script 0.0s [✔] actions.build.run.script 0.0s
[✔] actions.deps 3.4s [✔] actions.deps 3.4s
[✔] client.filesystem.".".read 0.2s [✔] client.filesystem."./".read 0.1s
[✔] actions.test.script 0.0s [✔] actions.test.script 0.0s
[✔] actions.test 1.8s [✔] actions.test 1.8s
[✔] actions.build.run 7.7s [✔] actions.build.run 7.7s
[✔] actions.build.contents 0.2s [✔] actions.build.contents 0.2s
[✔] client.filesystem.build.write 0.2s [✔] client.filesystem."./_build".write 0.2s
[✔] client.filesystem.".".read 0.1s
``` ```
Being able to re-run the test & build loop locally in `13.6s`, without adding any extra dependencies to our host, is likely to change our approach to iterating on changes. Being able to re-run the test & build loop locally in `13.6s`, without adding any extra dependencies to our host, is likely to change our approach to iterating on changes.

8
docs/tests/core-concepts/plan/structure.cue.fragment
View File

@ -1,3 +1,8 @@
// ...
// A plan has pre-requisited that we cover below.
// For now we focus on the dagger.#Plan structure.
// ...
dagger.#Plan & { dagger.#Plan & {
client: { client: {
filesystem: { filesystem: {
@ -22,5 +27,8 @@ dagger.#Plan & {
// ... // ...
} }
} }
deploy: netlify.#Deploy & {
// ...
}
} }
} }

2
pkg/universe.dagger.io/examples/todoapp/.gitignore vendored
View File

@ -1 +1 @@
build _build

14
pkg/universe.dagger.io/examples/todoapp/netlify.cue → pkg/universe.dagger.io/examples/todoapp/todoapp.cue
View File

@ -1,4 +1,4 @@
package netlify package todoapp
import ( import (
"dagger.io/dagger" "dagger.io/dagger"
@ -19,16 +19,16 @@ dagger.#Plan & {
} }
client: { client: {
filesystem: { filesystem: {
".": read: { "./": read: {
contents: dagger.#FS contents: dagger.#FS
exclude: [ exclude: [
"README.md", "README.md",
"build", "_build",
"netlify.cue", "todoapp.cue",
"node_modules", "node_modules",
] ]
} }
build: write: contents: actions.build.contents.output "./_build": write: contents: actions.build.contents.output
} }
env: { env: {
APP_NAME: string APP_NAME: string
@ -47,11 +47,9 @@ dagger.#Plan & {
} }
}, },
docker.#Copy & { docker.#Copy & {
contents: client.filesystem.".".read.contents contents: client.filesystem."./".read.contents
dest: "/src" dest: "/src"
}, },
// bash.#Run is a superset of docker.#Run
// install yarn dependencies
bash.#Run & { bash.#Run & {
workdir: "/src" workdir: "/src"
mounts: { mounts: {