Merge pull request #748 from grouville/docs_explicit_env_cloudformation

docs: specify env when using dagger + fix docs_101 issues
This commit is contained in:
Sam Alba 2021-06-30 12:58:22 +02:00 committed by GitHub
commit 0b0d17efb3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 100 additions and 98 deletions

View File

@ -5,20 +5,20 @@ slug: /learn/101-basics
# Dagger 101: basic usage
In this guide, you will learn the basics of Dagger by interacting with a pre-configured environment.
Then you will move on to creating your own environment from scratch.
Then you will move on to creating your environment from scratch.
Our pre-configured environment deploys a simple [React](https://reactjs.org/)
application to a special hosting environment created and managed by us, the Dagger team, for the purpose of this tutorial.
This will allow you to deploy something "real" right away, without having to configure your own infrastructure first.
application to a unique hosting environment created and managed by us, the Dagger team, for this tutorial.
This will allow you to deploy something "real" right away without configuring your infrastructure first.
In later guides, you will learn how to configure Dagger to deploy to your own infrastructure. And, for advanced users,
how to share access to your infrastructure in the same way that we are sharing access to ours now.
In later guides, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users,
how to share access to your infrastructure in the same way that we share access to ours now.
## Initial setup
### Install Dagger
First, make sure [you have installed dagger on your local machine](/install).
First, make sure [you have installed Dagger on your local machine](/install).
### Setup example app
@ -38,15 +38,15 @@ cd examples/todoapp
### Import the tutorial key
Dagger natively supports encrypted secrets: when a user inputs a value marked as secret
(for example a password, API token or ssh key) it is automatically encrypted with that user's key,
(for example, a password, API token, or ssh key) it is automatically encrypted with that user's key,
and no other user can access that value unless they are explicitly given access.
In the interest of security, Dagger has no way _not_ to encrypt a secret value.
But this causes a dilemma for this tutorial: how do we give unrestricted public access to our
(carefully sandboxed) infrastructure, so that anyone can deploy to it?
But this causes a dilemma for this tutorial: how do we give unrestricted, public access to our
(carefully sandboxed) infrastructure so that anyone can deploy to it?
To solve this dilemma, we included the private key used to encrypt the tutorial's secret inputs.
Simply import the key to your Dagger installation, and you're good to go:
Import the key to your Dagger installation, and you're good to go:
```shell
./import-tutorial-key.sh
@ -54,7 +54,7 @@ Simply import the key to your Dagger installation, and you're good to go:
## First deployment
Now that your environment is setup, you are ready to deploy:
Now that your environment is set up, you are ready to deploy:
```shell
dagger up
@ -75,7 +75,7 @@ NOTE: you don't have to commit your changes to the git repository before deployi
## Under the hood
This example showed you how to deploy and develop on an application that is already configured with dagger. Now, let's learn a few concepts to help you understand how this was put together.
This example showed you how to deploy and develop an application that is already configured with Dagger. Now, let's learn a few concepts to help you understand how this was put together.
### The Environment
@ -87,13 +87,13 @@ You can list existing environment from the `./todoapp` directory:
dagger list
```
You should see an environment named `s3`. You can have many environments within your app. For instance one for `staging`, one for `dev`, etc...
You should see an environment named `s3`. You can have many environments within your app. For instance, one for `staging`, one for `dev`, etc...
Each environment can have different kind of deployment code. For example, a `dev` environment can deploy locally, a `staging` environment can deploy to a remote infrastructure, and so on.
Each environment can have a different kind of deployment code. For example, a `dev` environment can deploy locally; a `staging` environment can deploy to a remote infrastructure, and so on.
### The plan
The plan is the deployment code, that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan:
The plan is the deployment code that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan:
```shell
ls -l .dagger/env/s3/plan/
@ -103,7 +103,7 @@ Any code change to the plan will be applied during the next `dagger up`.
### The inputs
The plan can define one or several `inputs` in order to take some information from the user. Here is how to list the current inputs:
The plan can define one or several `inputs`. Inputs may be configuration values, artifacts, or encrypted secrets provided by the user. Here is how to list the current inputs:
```shell
dagger input list
@ -113,7 +113,7 @@ The inputs are persisted inside the `.dagger` directory and pushed to your git r
### The outputs
The plan defines one or several `outputs`. They can show useful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all inputs:
The plan defines one or several `outputs`. They can show helpful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all inputs:
```shell
dagger output list
@ -121,4 +121,4 @@ dagger output list
## What's next?
At this point, you have deployed your first application using dagger and learned some dagger commands. You are now ready to [learn more about how to program dagger](/learn/102-dev).
At this point, you have deployed your first application using Dagger and learned some dagger commands. You are now ready to [learn more about how to program Dagger](/learn/102-dev).

View File

@ -116,7 +116,7 @@ But you can call your packages anything you want.
Let's create a new directory for our Cue package:
```shell
mkdir multibucket
mkdir cue.mod/multibucket
```
### Component 1: app source code
@ -130,7 +130,7 @@ In Dagger terms, this component has two essential properties:
Let's write the corresponding Cue code to a new file in our package:
```cue title="todoapp/multibucket/source.cue"
```cue title="todoapp/cue.mod/multibucket/source.cue"
package multibucket
import (
@ -147,7 +147,7 @@ This code defines a component at the key `src`, and specifies that it is both an
The second component of our plan is the Yarn package built from the app source code:
```cue title="todoapp/multibucket/yarn.cue"
```cue title="todoapp/cue.mod/multibucket/yarn.cue"
package multibucket
import (
@ -176,7 +176,7 @@ _FIXME_: this section is not yet available because the [Amazon S3 package](https
The third component of our plan is the Netlify site to which the app will be deployed:
```cue title="todoapp/multibucket/netlify.cue"
```cue title="todoapp/cue.mod/multibucket/netlify.cue"
package multibucket
import (
@ -216,27 +216,56 @@ You can also browse the [Dagger Universe](../reference/universe/README.md) refer
Now that your Cue package is ready, let's create an environment to run it:
```shell
dagger new 'multibucket'
dagger new 'multibucket' -m cue.mod/multibucket
```
### Load the plan into the environment
Now let's configure the new environment to use our package as its plan:
```shell
cp multibucket/*.cue .dagger/env/multibucket/plan/
```
Note: you need to copy the files from your package into the environment, as shown above.
If you make more changes to your package, you will need to copy the new version, or it will not be used.
In the future, we will add the ability to reference your Cue package directory, making this manual copy unnecessary.
### Configure user inputs
[This section is not yet written](https://github.com/dagger/dagger/blob/main/CONTRIBUTING.md)
You can inspect the list of inputs (both required and optional) using dagger input list:
```shell
dagger input list -e multibucket
# Input Value Set by user Description
# site.netlify.account.name *"" | string false Use this Netlify account name (also referred to as "team" in the Netlify docs)
# site.netlify.account.token dagger.#Secret false Netlify authentication token
# site.netlify.name string false Deploy to this Netlify site
# site.netlify.create *true | bool false Create the Netlify site if it doesn't exist?
# src dagger.#Artifact false Source code of the sample application
# app.cwd *"." | string false working directory to use
# app.writeEnvFile *"" | string false Write the contents of `environment` to this file, in the "envfile" format
# app.buildDir *"build" | string false Read build output from this directory (path must be relative to working directory)
# app.script *"build" | string false Run this yarn script
# app.args *[] | [] false Optional arguments for the script
```
All the values without default values (without `*`) have to be specified by the user. Here, required fields are:
- `site.netlify.account.token`, your access token
- `site.netlify.name`, name of the published website
- `src`, source code of the app
Please note the type of the user inputs: a string, a #Secret and an artifact. Let's see how to input them:
```shell
# As a string input is expected for `site.netlify.name`, we set a `text` input
dagger input text site.netlify.name <GLOBALLY-UNIQUE-NAME> -e multibucket
# As a secret input is expected for `site.netlify.account.token`, we set a `secret` input
dagger input secret site.netlify.account.token <PERSONAL-ACCESS-TOKEN> -e multibucket
# As an Artifact is exepected for `src`, we set a `dir` input (dagger input list for alternatives)
dagger input dir src . -e multibucket
```
### Deploy
Now that everything is properly set, let's deploy on Netlify:
```shell
dagger up -e multibucket
```
[This section is not yet written](https://github.com/dagger/dagger/blob/main/CONTRIBUTING.md)
### Using the environment

View File

@ -49,7 +49,7 @@ cue mod init
Let's create a new directory for our Cue package:
```shell
mkdir cloudformation
mkdir cue.mod/cloudformation
```
## Create a basic plan
@ -66,7 +66,7 @@ The idea here is to follow best practices in [S3 buckets](https://docs.aws.amazo
Create a file named `template.cue` and add the following configuration to it.
```cue title="todoapp/cloudformation/template.cue"
```cue title="todoapp/cue.mod/cloudformation/template.cue"
package cloudformation
// inlined s3 cloudformation template as a string
@ -164,7 +164,7 @@ The config values are all part of the `aws` relay. Regarding this package, as yo
Let's implement the first step, use the `aws.#Config` relay, and request its first inputs: the region to deploy and the AWS credentials.
```cue title="todoapp/cloudformation/source.cue"
```cue title="todoapp/cue.mod/cloudformation/source.cue"
package cloudformation
import (
@ -186,32 +186,24 @@ This defines:
Now that the Cue package is ready, let's create an environment to run it:
```shell
dagger new 'cloudformation'
dagger new 'cloudformation' -m cue.mod/cloudformation
```
##### 2. Load the plan into the environment
Now let's configure the new environment to use our package as its plan:
```shell
cp cloudformation/*.cue .dagger/env/cloudformation/plan/
```
##### 3. Check plan
##### 2. Check plan
_Pro tips_: To check whether it worked or not, these three commands might help
```shell
dagger input list # List our personal plan's inputs
dagger input list -e cloudformation # List our personal plan's inputs
# Input Value Set by user Description
# awsConfig.region string false AWS region
# awsConfig.accessKey dagger.#Secret false AWS access key
# awsConfig.secretKey dagger.#Secret false AWS secret key
dagger query # Query values / inspect default values (Instrumental in case of conflict)
dagger query -e cloudformation # Query values / inspect default values (Instrumental in case of conflict)
# {}
dagger up # Try to run the plan. As expected, we encounter a failure because some user inputs haven't been set
dagger up -e cloudformation # Try to run the plan. As expected, we encounter a failure because some user inputs haven't been set
# 4:11PM ERR system | required input is missing input=awsConfig.region
# 4:11PM ERR system | required input is missing input=awsConfig.accessKey
# 4:11PM ERR system | required input is missing input=awsConfig.secretKey
@ -222,7 +214,7 @@ dagger up # Try to run the plan. As expected, we encounter a failure because som
Now that we have the `config` definition properly configured, let's modify the Cloudformation one:
```cue title="todoapp/cloudformation/source.cue"
```cue title="todoapp/cue.mod/cloudformation/source.cue"
package cloudformation
import (
@ -257,18 +249,12 @@ This defines:
- `cfnStackName`: Name of the stack, either a default value `stack-suffix` or user input
- `cfnStack`: Cloudformation relay with `AWS config`, `stackName` and `JSON template` as inputs
You need to copy the changes to the plan for Dagger to reference them
```shell
cp cloudformation/*.cue .dagger/env/cloudformation/plan/
```
### Configure the environment
Before bringing up the deployment, we need to provide the `cfnStack` inputs declared in the configuration. Otherwise, Dagger will complain about missing inputs.
```shell
dagger up
dagger up -e cloudformation
# 3:34PM ERR system | required input is missing input=awsConfig.region
# 3:34PM ERR system | required input is missing input=awsConfig.accessKey
# 3:34PM ERR system | required input is missing input=awsConfig.secretKey
@ -278,7 +264,7 @@ dagger up
You can inspect the list of inputs (both required and optional) using dagger input list:
```shell
dagger input list
dagger input list -e cloudformation
# Input Value Set by user Description
# awsConfig.region string false AWS region
# awsConfig.accessKey dagger.#Secret false AWS access key
@ -292,9 +278,9 @@ dagger input list
Let's provide the missing inputs:
```shell
dagger input text awsConfig.region us-east-2
dagger input secret awsConfig.accessKey yourAccessKey
dagger input secret awsConfig.secretKey yourSecretKey
dagger input text awsConfig.region us-east-2 -e cloudformation
dagger input secret awsConfig.accessKey yourAccessKey -e cloudformation
dagger input secret awsConfig.secretKey yourSecretKey -e cloudformation
```
### Deploying
@ -310,7 +296,7 @@ Finally ! We now have a working template ready to be used to provision S3 infras
<TabItem value="nd">
```shell
dagger up
dagger up -e cloudformation
#2:22PM INF suffix.out | computing
#2:22PM INF suffix.out | completed duration=200ms
#2:22PM INF cfnStack.outputs | computing
@ -322,7 +308,7 @@ dagger up
#2:22PM INF cfnStack.outputs | #15 2.948 }
#2:22PM INF cfnStack.outputs | completed duration=35s
dagger output list
dagger output list -e cloudformation
# Output Value Description
# suffix.out "emktqcfwksng" generated random string
# cfnStack.outputs.Name "arn:aws:s3:::stack-emktqcfwksng-s3bucket-9eiowjs1jab4" -
@ -332,7 +318,7 @@ dagger output list
<TabItem value="dd">
```shell
dagger up -l debug
dagger up -l debug -e cloudformation
#Output:
# 3:50PM DBG system | detected buildkit version version=v0.8.3
# 3:50PM DBG system | spawning buildkit job localdirs={
@ -344,7 +330,7 @@ dagger up -l debug
# suffix.out "abnyiemsoqbm" generated random string
# cfnStack.outputs.Name "arn:aws:s3:::stack-abnyiemsoqbm-s3bucket-9eiowjs1jab4" -
dagger output list
dagger output list -e cloudformation
# Output Value Description
# suffix.out "abnyiemsoqbm" generated random string
# cfnStack.outputs.Name "arn:aws:s3:::stack-abnyiemsoqbm-s3bucket-9eiowjs1jab4" -
@ -358,7 +344,7 @@ The deployment went well!
In case of a failure, the `Debug deploy` tab shows the command to get more information.
The name of the provisioned S3 instance lies in the `cfnStack.outputs.Name` output key, without `arn:aws:s3:::`
> With this provisioning infrastructure, your dev team will easily be able to instantiate aws infrastructures: all they need to know is `dagger input list` and `dagger up,` isn't that awesome? :-D
> With this provisioning infrastructure, your dev team will easily be able to instantiate aws infrastructures: all they need to know is `dagger input list -e cloudformation` and `dagger up -e cloudformation` isn't that awesome? :-D
## Cue Cloudformation template
@ -382,7 +368,7 @@ import TabItem from "@theme/TabItem";
}>
<TabItem value="sv">
```cue title="todoapp/cloudformation/convert.cue"
```cue title="todoapp/cue.mod/cloudformation/convert.cue"
package cloudformation
import "encoding/json"
@ -392,7 +378,7 @@ s3Template: json.Unmarshal(template)
</TabItem>
<TabItem value="yv">
```cue title="todoapp/cloudformation/convert.cue"
```cue title="todoapp/cue.mod/cloudformation/convert.cue"
package cloudformation
import "encoding/yaml"
@ -409,8 +395,7 @@ This defines:
You need to empty the plan and copy the `convert.cue` file to the plan for Dagger to reference it
```shell
rm .dagger/env/cloudformation/plan/*
cp cloudformation/template.cue cloudformation/convert.cue .dagger/env/cloudformation/plan/
rm cue.mod/cloudformation/source.cue
```
### 2. Retrieve the Unmarshalled JSON
@ -418,7 +403,7 @@ cp cloudformation/template.cue cloudformation/convert.cue .dagger/env/cloudforma
Then, still in the same folder, query the `s3Template` value to retrieve the Unmarshalled result of `s3`:
```shell
dagger query s3Template > cloudformation/template.cue
dagger query s3Template -e cloudformation
# {
# "AWSTemplateFormatVersion": "2010-09-09",
# "Outputs": {
@ -431,19 +416,19 @@ dagger query s3Template > cloudformation/template.cue
# ...
```
The commented output above is the cue version of the JSON Template
The commented output above is the cue version of the JSON Template, copy it
### 3. Remove convert.cue
```shell
rm cloudformation/convert.cue .dagger/env/cloudformation/plan/convert.cue
rm cue.mod/cloudformation/convert.cue
```
### 4. Store the output
Open `cloudformation/template.cue` and append below elements to this exported Cue definition of the JSON:
Open `cloudformation/template.cue` and append below elements with copied Cue definition of the JSON:
```cue title="todoapp/cloudformation/template.cue"
```cue title="todoapp/cue.mod/cloudformation/template.cue"
// Add this line, to make it part to the cloudformation template
package cloudformation
import "encoding/json"
@ -515,23 +500,17 @@ template: json.Marshal(s3)
We're using the built-in `json.Marshal` function to convert CUE back to JSON, so Cloudformation still receives the same template.
You need to copy the changes to the plan for Dagger to reference them
You can inspect the configuration using `dagger query -e cloudformation` to verify it produces the same manifest:
```shell
cp cloudformation/*.cue .dagger/env/cloudformation/plan/
```
You can inspect the configuration using `dagger query` to verify it produces the same manifest:
```shell
dagger query template -f text
dagger query template -f text -e cloudformation
```
Now that the template is defined in CUE, we can use the language to add more flexibility to our template.
Let's define a re-usable `#Deployment` definition in `todoapp/cloudformation/deployment.cue`:
```cue title="todoapp/cloudformation/deployment.cue"
```cue title="todoapp/cue.mod/cloudformation/deployment.cue"
package cloudformation
#Deployment: {
@ -619,7 +598,7 @@ package cloudformation
`template.cue` can be rewritten as follows:
```cue title="todoapp/cloudformation/template.cue"
```cue title="todoapp/cue.mod/cloudformation/template.cue"
package cloudformation
import "encoding/json"
@ -636,19 +615,13 @@ Verify template
Double-checks at the template level can be done with manual uploads on Cloudformation's web interface or by executing the below command locally:
```shell
tmpfile=$(mktemp ./tmp.XXXXXX) && dagger query template -f text > "$tmpfile" && aws cloudformation validate-template --template-body file://"$tmpfile" ; rm "$tmpfile"
```
Update the plan
```shell
cp cloudformation/*.cue .dagger/env/cloudformation/plan/
tmpfile=$(mktemp ./tmp.XXXXXX) && dagger query template -f text -e cloudformation > "$tmpfile" && aws cloudformation validate-template --template-body file://"$tmpfile" ; rm "$tmpfile"
```
Let's make sure it yields the same result:
```shell
dagger query template -f text
dagger query template -f text -e cloudformation
# {
# "description": "Name S3 Bucket",
# "indexDocument": "index.html",
@ -667,7 +640,7 @@ dagger query template -f text
And we can now deploy it:
```shell
dagger up
dagger up -e cloudformation
#2:22PM INF suffix.out | computing
#2:22PM INF suffix.out | completed duration=200ms
#2:22PM INF cfnStack.outputs | computing
@ -683,7 +656,7 @@ dagger up
Name of the deployed bucket:
```shell
dagger output list
dagger output list -e cloudformation
# Output Value Description
# suffix.out "ucwcecwwshdl" generated random string
# cfnStack.outputs.Name "arn:aws:s3:::stack-ucwcecwwshdl-s3bucket-gaqmj8rzsl08" -