Archived

This repository has been archived on 2024-04-08. You can view files and clone it, but cannot push or open issues or pull requests.

Solomon Hykes 06f04be214 First pass at programming guide aka "Dagger 102"

Signed-off-by: Solomon Hykes <solomon@dagger.io>

2021-06-15 12:01:06 +02:00

9.8 KiB

Raw Blame History

sidebar_position	slug
1	/programming

Dagger 102: learn to program Dagger

Overview

In this guide you will create your first Dagger environment from scratch, and use it to deploy a React application to 2 locations in parallel: a dedicated Amazon S3 bucket, and a Netlify site.

Anatomy of a Dagger environment

A Dagger environment contains all the code and data necessary to delivery a particular application in a particular way. For example the same application might be delivered to a production and staging environment, each with their own configuration.

An environment is made of 3 parts:

A plan, authored by the environment's developer, using the Cue language. Learn more about plans
Inputs, supplied by the environment's user via the dagger input command, and written to a special file. Inputs may be configuration values, artifacts, or encrypted secrets. Learn more about inputs
Outputs, computed by the Dagger engine via the dagger up command, and recorded to a special directory. Learn more about outputs

We will first develop our environment's plan, then configure its initial inputs, then finally run it to verify that it works.

Developing your plan

Anatomy of a plan

A plan specifies, in code, how to deliver a particular application in a particular way. It is your environment's source code.

Unlike regular imperative programs which specify a sequence of instructions to execute, a Dagger plan is declarative: it lays out your application's supply chain as a graph of interconnected nodes.

Each node in the graph represents a component of the supply chain, for example:

Development tools: source control, CI, build systems, testing systems
Hosting infrastructure: compute, storage, networking, databases, CDN..
Software dependencies: operating systems, languages, libraries, frameworks, etc.

Each link in the graph represents a flow of data between nodes. For example:

source code flows from a git repository to a build system;
system dependencies are combined in a docker image, then uploaded to a registry;
configuration files are generated then sent to a compute cluster or load balancer;
etc.

Introduction to Cue development

Dagger delivery plans are developed in Cue. Cue is a powerful declarative language by the creator of GQL, the language used to deploy all applications at Google. It is a superset of JSON, with additional features to make declarative, data-driven programming as pleasant and productive as regular imperative programming.

If you are new to Cue development, don't worry: this tutorial will walk you through the basic steps to get started, and give you resources to learn more.

In technical terms, our plan is a Cue Package. In this tutorial we will develop a new Cue package from scratch for our plan; but you can use any Cue package as a plan.

Install Cue

Although not strictly necessary, for an optimal development experience we recommend installing a recent version of Cue.

(optional) Prepare Cue learning resources

If you are new to Cue, we recommend keeping the following resources in browser tabs:

The unofficial but excellent Cuetorials in a browser tab, to look up Cue concepts as they appear.
The official Cue interactive sandbox for easy experimentation.

Setup example app

You will need a local copy of the Dagger examples repository.

git clone https://github.com/dagger/examples

Make sure that all commands are run from the voteapp directory:

cd examples/voteapp

Initialize a Cue module

Developing for Dagger takes place in a [https://cuelang.org/docs/concepts/packages/#modules](Cue module). If you are familiar with Go, Cue modules are directly inspired by Go modules. Otherwise, don't worry: a Cue module is simply a directory with one or more Cue packages in it. A Cue module has a cue.mod directory at its root.

In this guide we will use the same directory as the root of the Dagger workspace and the root of the Cue module; but you can create your Cue module anywhere inside the Dagger workspace.

cue mod init

Organize your package

Now we start developing our Cue package at the root of our Cue module.

In this guide we will split our package in multiple files, one per component. But you can organize your package any way you want: the Cue evaluator simply merges together all files from the same package, as long as they are in the same directory and start with the same package clause. It is common for a Cue package to have only one file. See the Cue documentation for more details.

We will call our package multibucket because it sounds badass and vaguely explains what it does. But you can call your packages anything you want.

Let's layout the structure of our package by creating all the files in advance:

pkg=multibucket; touch $pkg-source.cue $pkg-yarn.cue $pkg-netlify.cue

Component 1: app source code

The first component of our plan is the source code of our React application.

In Dagger terms, this component has 2 important properties:

It is an artifact: something that can be represented as a directory.
It is an input: something that is provided by the end user.

Let's write the corresponding Cue code to multibucket-source.cue:

package multibucket

import (
    "dagger.io/dagger"
)

// Source code of the sample application
src: dagger.#Artifact @dagger(input)

This defines a component at the key src, of type dagger.#Artifact, annotated as an user input.

Component 2: yarn package

The second component of our plan is the Yarn package built from the source code.

Let's write it to multibucket-yarn.cue:

package multibucket

import (
    "dagger.io/js/yarn"
)

// Build the source code using Yarn
app: yarn.#Package & {
    source: src
}

Let's break it down:

package multibucket: this file is part of the multibucket package
import ( "dagger.io/js/yarn" ): import a package from the Dagger Universe.
app: yarn.#Package: apply the #Package definition at the key app
&: also merge the following values at the same key...
{ source: src }: set the key app.source to the value of src. This connects our 2 components, forming the first link in our DAG.

Component 3: dedicated S3 bucket

FIXME: this section is not yet available, because the Amazon S3 package does not yet support bucket creation. We welcome external contributions :)

Component 4: deploy to Netlify

The third component of our plan is the Netlify site to which the app will be deployed.

Let's write it to multibucket-netlify.cue:

package multibucket

import (
    "dagger.io/netlify"
)

// Netlify site
site: "netlify": netlify.#Site & {
    contents: app.build
}

This is very similar to the previous component:

We use the same package name as the other files
We import another package from the Dagger Universe.
site: "netlify": site.#Netlify: apply the #Site definition at the key site.netlify. Note the use of quotes to protect the key from name conflict.
&: also merge the following values at the same key...
{ contents: app.build }: set the key site.netlify.contents to the value of app.build. This connects our components 2 and 3, forming the second link in our DAG.

Exploring a package documentation

But wait: how did we know what fields were available in yarn.#Package and netlify.#Site? Answer: thanks to the dagger doc command, which prints the documentation of any package from Dagger Universe.

dagger doc dagger.io/netlify
dagger doc dagger.io/js/yarn

You can also browse the API reference section of the documentation.

Setup the environment

Create a new environment

Now that your Cue package is ready, let's create an environment to run it,

dagger new 'multibucket'

Load the plan into the environment

Now let's configure the new environment to use our package as its plan:

cp multibucket-*.cue .dagger/env/multibucket/plan/

Note: you need to copy the files from your package into the environment's plan directory, as shown above. This means that, if you make more changes to your package, you will need to copy the new version into the plan directory, or it will not be used. If you prefer, you can also edit the cue files directly in the plan directory, but we don't recommend it. In the future, we will probably add the ability to reference your package to make the manual copy unnecessary.

9.8 KiB Raw Blame History