From 9171912918894f20e4991d28b00ed795f26b76aa Mon Sep 17 00:00:00 2001 From: Guillaume de Rouville Date: Mon, 28 Jun 2021 19:38:29 +0200 Subject: [PATCH] docs: specify env when using dagger + fix docs_101 ambiguous naming and english mistakes + change method, now cue.mod Signed-off-by: Guillaume de Rouville --- docs/learn/101-use.md | 36 ++++++------ docs/learn/102-dev.md | 65 +++++++++++++++------ docs/learn/108-cloudformation.md | 97 ++++++++++++-------------------- 3 files changed, 100 insertions(+), 98 deletions(-) diff --git a/docs/learn/101-use.md b/docs/learn/101-use.md index 94cdcede..c0225b07 100644 --- a/docs/learn/101-use.md +++ b/docs/learn/101-use.md @@ -5,20 +5,20 @@ slug: /learn/101-basics # Dagger 101: basic usage In this guide, you will learn the basics of Dagger by interacting with a pre-configured environment. -Then you will move on to creating your own environment from scratch. +Then you will move on to creating your environment from scratch. Our pre-configured environment deploys a simple [React](https://reactjs.org/) -application to a special hosting environment created and managed by us, the Dagger team, for the purpose of this tutorial. -This will allow you to deploy something "real" right away, without having to configure your own infrastructure first. +application to a unique hosting environment created and managed by us, the Dagger team, for this tutorial. +This will allow you to deploy something "real" right away without configuring your infrastructure first. -In later guides, you will learn how to configure Dagger to deploy to your own infrastructure. And, for advanced users, -how to share access to your infrastructure in the same way that we are sharing access to ours now. +In later guides, you will learn how to configure Dagger to deploy to your infrastructure. And, for advanced users, +how to share access to your infrastructure in the same way that we share access to ours now. ## Initial setup ### Install Dagger -First, make sure [you have installed dagger on your local machine](/install). +First, make sure [you have installed Dagger on your local machine](/install). ### Setup example app @@ -38,15 +38,15 @@ cd examples/todoapp ### Import the tutorial key Dagger natively supports encrypted secrets: when a user inputs a value marked as secret -(for example a password, API token or ssh key) it is automatically encrypted with that user's key, +(for example, a password, API token, or ssh key) it is automatically encrypted with that user's key, and no other user can access that value unless they are explicitly given access. In the interest of security, Dagger has no way _not_ to encrypt a secret value. -But this causes a dilemma for this tutorial: how do we give unrestricted public access to our -(carefully sandboxed) infrastructure, so that anyone can deploy to it? +But this causes a dilemma for this tutorial: how do we give unrestricted, public access to our +(carefully sandboxed) infrastructure so that anyone can deploy to it? To solve this dilemma, we included the private key used to encrypt the tutorial's secret inputs. -Simply import the key to your Dagger installation, and you're good to go: +Import the key to your Dagger installation, and you're good to go: ```shell ./import-tutorial-key.sh @@ -54,7 +54,7 @@ Simply import the key to your Dagger installation, and you're good to go: ## First deployment -Now that your environment is setup, you are ready to deploy: +Now that your environment is set up, you are ready to deploy: ```shell dagger up @@ -75,7 +75,7 @@ NOTE: you don't have to commit your changes to the git repository before deployi ## Under the hood -This example showed you how to deploy and develop on an application that is already configured with dagger. Now, let's learn a few concepts to help you understand how this was put together. +This example showed you how to deploy and develop an application that is already configured with Dagger. Now, let's learn a few concepts to help you understand how this was put together. ### The Environment @@ -87,13 +87,13 @@ You can list existing environment from the `./todoapp` directory: dagger list ``` -You should see an environment named `s3`. You can have many environments within your app. For instance one for `staging`, one for `dev`, etc... +You should see an environment named `s3`. You can have many environments within your app. For instance, one for `staging`, one for `dev`, etc... -Each environment can have different kind of deployment code. For example, a `dev` environment can deploy locally, a `staging` environment can deploy to a remote infrastructure, and so on. +Each environment can have a different kind of deployment code. For example, a `dev` environment can deploy locally; a `staging` environment can deploy to a remote infrastructure, and so on. ### The plan -The plan is the deployment code, that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan: +The plan is the deployment code that includes the logic to deploy the local application to an AWS S3 bucket. From the `todoapp` directory, you can list the code of the plan: ```shell ls -l .dagger/env/s3/plan/ @@ -103,7 +103,7 @@ Any code change to the plan will be applied during the next `dagger up`. ### The inputs -The plan can define one or several `inputs` in order to take some information from the user. Here is how to list the current inputs: +The plan can define one or several `inputs`. Inputs may be configuration values, artifacts, or encrypted secrets provided by the user. Here is how to list the current inputs: ```shell dagger input list @@ -113,7 +113,7 @@ The inputs are persisted inside the `.dagger` directory and pushed to your git r ### The outputs -The plan defines one or several `outputs`. They can show useful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all inputs: +The plan defines one or several `outputs`. They can show helpful information at the end of the deployment. That's how we read the deploy `url` at the end of the deployment. Here is the command to list all inputs: ```shell dagger output list @@ -121,4 +121,4 @@ dagger output list ## What's next? -At this point, you have deployed your first application using dagger and learned some dagger commands. You are now ready to [learn more about how to program dagger](/learn/102-dev). +At this point, you have deployed your first application using Dagger and learned some dagger commands. You are now ready to [learn more about how to program Dagger](/learn/102-dev). diff --git a/docs/learn/102-dev.md b/docs/learn/102-dev.md index 76adfe5d..da901325 100644 --- a/docs/learn/102-dev.md +++ b/docs/learn/102-dev.md @@ -116,7 +116,7 @@ But you can call your packages anything you want. Let's create a new directory for our Cue package: ```shell -mkdir multibucket +mkdir cue.mod/multibucket ``` ### Component 1: app source code @@ -130,7 +130,7 @@ In Dagger terms, this component has two essential properties: Let's write the corresponding Cue code to a new file in our package: -```cue title="todoapp/multibucket/source.cue" +```cue title="todoapp/cue.mod/multibucket/source.cue" package multibucket import ( @@ -147,7 +147,7 @@ This code defines a component at the key `src`, and specifies that it is both an The second component of our plan is the Yarn package built from the app source code: -```cue title="todoapp/multibucket/yarn.cue" +```cue title="todoapp/cue.mod/multibucket/yarn.cue" package multibucket import ( @@ -176,7 +176,7 @@ _FIXME_: this section is not yet available because the [Amazon S3 package](https The third component of our plan is the Netlify site to which the app will be deployed: -```cue title="todoapp/multibucket/netlify.cue" +```cue title="todoapp/cue.mod/multibucket/netlify.cue" package multibucket import ( @@ -216,27 +216,56 @@ You can also browse the [Dagger Universe](../reference/universe/README.md) refer Now that your Cue package is ready, let's create an environment to run it: ```shell -dagger new 'multibucket' +dagger new 'multibucket' -m cue.mod/multibucket ``` -### Load the plan into the environment - -Now let's configure the new environment to use our package as its plan: - -```shell -cp multibucket/*.cue .dagger/env/multibucket/plan/ -``` - -Note: you need to copy the files from your package into the environment, as shown above. -If you make more changes to your package, you will need to copy the new version, or it will not be used. -In the future, we will add the ability to reference your Cue package directory, making this manual copy unnecessary. - ### Configure user inputs -[This section is not yet written](https://github.com/dagger/dagger/blob/main/CONTRIBUTING.md) +You can inspect the list of inputs (both required and optional) using dagger input list: + +```shell +dagger input list -e multibucket +# Input Value Set by user Description +# site.netlify.account.name *"" | string false Use this Netlify account name (also referred to as "team" in the Netlify docs) +# site.netlify.account.token dagger.#Secret false Netlify authentication token +# site.netlify.name string false Deploy to this Netlify site +# site.netlify.create *true | bool false Create the Netlify site if it doesn't exist? +# src dagger.#Artifact false Source code of the sample application +# app.cwd *"." | string false working directory to use +# app.writeEnvFile *"" | string false Write the contents of `environment` to this file, in the "envfile" format +# app.buildDir *"build" | string false Read build output from this directory (path must be relative to working directory) +# app.script *"build" | string false Run this yarn script +# app.args *[] | [] false Optional arguments for the script +``` + +All the values without default values (without `*`) have to be specified by the user. Here, required fields are: + +- `site.netlify.account.token`, your access token +- `site.netlify.name`, name of the published website +- `src`, source code of the app + +Please note the type of the user inputs: a string, a #Secret and an artifact. Let's see how to input them: + +```shell +# As a string input is expected for `site.netlify.name`, we set a `text` input +dagger input text site.netlify.name -e multibucket + +# As a secret input is expected for `site.netlify.account.token`, we set a `secret` input +dagger input secret site.netlify.account.token -e multibucket + +# As an Artifact is exepected for `src`, we set a `dir` input (dagger input list for alternatives) +dagger input dir src . -e multibucket + +``` ### Deploy +Now that everything is properly set, let's deploy on Netlify: + +```shell +dagger up -e multibucket +``` + [This section is not yet written](https://github.com/dagger/dagger/blob/main/CONTRIBUTING.md) ### Using the environment diff --git a/docs/learn/108-cloudformation.md b/docs/learn/108-cloudformation.md index b5cf74b4..b7f0ba3b 100644 --- a/docs/learn/108-cloudformation.md +++ b/docs/learn/108-cloudformation.md @@ -49,7 +49,7 @@ cue mod init Let's create a new directory for our Cue package: ```shell -mkdir cloudformation +mkdir cue.mod/cloudformation ``` ## Create a basic plan @@ -66,7 +66,7 @@ The idea here is to follow best practices in [S3 buckets](https://docs.aws.amazo Create a file named `template.cue` and add the following configuration to it. -```cue title="todoapp/cloudformation/template.cue" +```cue title="todoapp/cue.mod/cloudformation/template.cue" package cloudformation // inlined s3 cloudformation template as a string @@ -164,7 +164,7 @@ The config values are all part of the `aws` relay. Regarding this package, as yo Let's implement the first step, use the `aws.#Config` relay, and request its first inputs: the region to deploy and the AWS credentials. -```cue title="todoapp/cloudformation/source.cue" +```cue title="todoapp/cue.mod/cloudformation/source.cue" package cloudformation import ( @@ -186,32 +186,24 @@ This defines: Now that the Cue package is ready, let's create an environment to run it: ```shell -dagger new 'cloudformation' +dagger new 'cloudformation' -m cue.mod/cloudformation ``` -##### 2. Load the plan into the environment - -Now let's configure the new environment to use our package as its plan: - -```shell -cp cloudformation/*.cue .dagger/env/cloudformation/plan/ -``` - -##### 3. Check plan +##### 2. Check plan _Pro tips_: To check whether it worked or not, these three commands might help ```shell -dagger input list # List our personal plan's inputs +dagger input list -e cloudformation # List our personal plan's inputs # Input Value Set by user Description # awsConfig.region string false AWS region # awsConfig.accessKey dagger.#Secret false AWS access key # awsConfig.secretKey dagger.#Secret false AWS secret key -dagger query # Query values / inspect default values (Instrumental in case of conflict) +dagger query -e cloudformation # Query values / inspect default values (Instrumental in case of conflict) # {} -dagger up # Try to run the plan. As expected, we encounter a failure because some user inputs haven't been set +dagger up -e cloudformation # Try to run the plan. As expected, we encounter a failure because some user inputs haven't been set # 4:11PM ERR system | required input is missing input=awsConfig.region # 4:11PM ERR system | required input is missing input=awsConfig.accessKey # 4:11PM ERR system | required input is missing input=awsConfig.secretKey @@ -222,7 +214,7 @@ dagger up # Try to run the plan. As expected, we encounter a failure because som Now that we have the `config` definition properly configured, let's modify the Cloudformation one: -```cue title="todoapp/cloudformation/source.cue" +```cue title="todoapp/cue.mod/cloudformation/source.cue" package cloudformation import ( @@ -257,18 +249,12 @@ This defines: - `cfnStackName`: Name of the stack, either a default value `stack-suffix` or user input - `cfnStack`: Cloudformation relay with `AWS config`, `stackName` and `JSON template` as inputs -You need to copy the changes to the plan for Dagger to reference them - -```shell -cp cloudformation/*.cue .dagger/env/cloudformation/plan/ -``` - ### Configure the environment Before bringing up the deployment, we need to provide the `cfnStack` inputs declared in the configuration. Otherwise, Dagger will complain about missing inputs. ```shell -dagger up +dagger up -e cloudformation # 3:34PM ERR system | required input is missing input=awsConfig.region # 3:34PM ERR system | required input is missing input=awsConfig.accessKey # 3:34PM ERR system | required input is missing input=awsConfig.secretKey @@ -278,7 +264,7 @@ dagger up You can inspect the list of inputs (both required and optional) using dagger input list: ```shell -dagger input list +dagger input list -e cloudformation # Input Value Set by user Description # awsConfig.region string false AWS region # awsConfig.accessKey dagger.#Secret false AWS access key @@ -292,9 +278,9 @@ dagger input list Let's provide the missing inputs: ```shell -dagger input text awsConfig.region us-east-2 -dagger input secret awsConfig.accessKey yourAccessKey -dagger input secret awsConfig.secretKey yourSecretKey +dagger input text awsConfig.region us-east-2 -e cloudformation +dagger input secret awsConfig.accessKey yourAccessKey -e cloudformation +dagger input secret awsConfig.secretKey yourSecretKey -e cloudformation ``` ### Deploying @@ -310,7 +296,7 @@ Finally ! We now have a working template ready to be used to provision S3 infras ```shell -dagger up +dagger up -e cloudformation #2:22PM INF suffix.out | computing #2:22PM INF suffix.out | completed duration=200ms #2:22PM INF cfnStack.outputs | computing @@ -322,7 +308,7 @@ dagger up #2:22PM INF cfnStack.outputs | #15 2.948 } #2:22PM INF cfnStack.outputs | completed duration=35s -dagger output list +dagger output list -e cloudformation # Output Value Description # suffix.out "emktqcfwksng" generated random string # cfnStack.outputs.Name "arn:aws:s3:::stack-emktqcfwksng-s3bucket-9eiowjs1jab4" - @@ -332,7 +318,7 @@ dagger output list ```shell -dagger up -l debug +dagger up -l debug -e cloudformation #Output: # 3:50PM DBG system | detected buildkit version version=v0.8.3 # 3:50PM DBG system | spawning buildkit job localdirs={ @@ -344,7 +330,7 @@ dagger up -l debug # suffix.out "abnyiemsoqbm" generated random string # cfnStack.outputs.Name "arn:aws:s3:::stack-abnyiemsoqbm-s3bucket-9eiowjs1jab4" - -dagger output list +dagger output list -e cloudformation # Output Value Description # suffix.out "abnyiemsoqbm" generated random string # cfnStack.outputs.Name "arn:aws:s3:::stack-abnyiemsoqbm-s3bucket-9eiowjs1jab4" - @@ -358,7 +344,7 @@ The deployment went well! In case of a failure, the `Debug deploy` tab shows the command to get more information. The name of the provisioned S3 instance lies in the `cfnStack.outputs.Name` output key, without `arn:aws:s3:::` -> With this provisioning infrastructure, your dev team will easily be able to instantiate aws infrastructures: all they need to know is `dagger input list` and `dagger up,` isn't that awesome? :-D +> With this provisioning infrastructure, your dev team will easily be able to instantiate aws infrastructures: all they need to know is `dagger input list -e cloudformation` and `dagger up -e cloudformation` isn't that awesome? :-D ## Cue Cloudformation template @@ -382,7 +368,7 @@ import TabItem from "@theme/TabItem"; }> -```cue title="todoapp/cloudformation/convert.cue" +```cue title="todoapp/cue.mod/cloudformation/convert.cue" package cloudformation import "encoding/json" @@ -392,7 +378,7 @@ s3Template: json.Unmarshal(template) -```cue title="todoapp/cloudformation/convert.cue" +```cue title="todoapp/cue.mod/cloudformation/convert.cue" package cloudformation import "encoding/yaml" @@ -409,8 +395,7 @@ This defines: You need to empty the plan and copy the `convert.cue` file to the plan for Dagger to reference it ```shell -rm .dagger/env/cloudformation/plan/* -cp cloudformation/template.cue cloudformation/convert.cue .dagger/env/cloudformation/plan/ +rm cue.mod/cloudformation/source.cue ``` ### 2. Retrieve the Unmarshalled JSON @@ -418,7 +403,7 @@ cp cloudformation/template.cue cloudformation/convert.cue .dagger/env/cloudforma Then, still in the same folder, query the `s3Template` value to retrieve the Unmarshalled result of `s3`: ```shell -dagger query s3Template > cloudformation/template.cue +dagger query s3Template -e cloudformation # { # "AWSTemplateFormatVersion": "2010-09-09", # "Outputs": { @@ -431,19 +416,19 @@ dagger query s3Template > cloudformation/template.cue # ... ``` -The commented output above is the cue version of the JSON Template +The commented output above is the cue version of the JSON Template, copy it ### 3. Remove convert.cue ```shell -rm cloudformation/convert.cue .dagger/env/cloudformation/plan/convert.cue +rm cue.mod/cloudformation/convert.cue ``` ### 4. Store the output -Open `cloudformation/template.cue` and append below elements to this exported Cue definition of the JSON: +Open `cloudformation/template.cue` and append below elements with copied Cue definition of the JSON: -```cue title="todoapp/cloudformation/template.cue" +```cue title="todoapp/cue.mod/cloudformation/template.cue" // Add this line, to make it part to the cloudformation template package cloudformation import "encoding/json" @@ -515,23 +500,17 @@ template: json.Marshal(s3) We're using the built-in `json.Marshal` function to convert CUE back to JSON, so Cloudformation still receives the same template. -You need to copy the changes to the plan for Dagger to reference them +You can inspect the configuration using `dagger query -e cloudformation` to verify it produces the same manifest: ```shell -cp cloudformation/*.cue .dagger/env/cloudformation/plan/ -``` - -You can inspect the configuration using `dagger query` to verify it produces the same manifest: - -```shell -dagger query template -f text +dagger query template -f text -e cloudformation ``` Now that the template is defined in CUE, we can use the language to add more flexibility to our template. Let's define a re-usable `#Deployment` definition in `todoapp/cloudformation/deployment.cue`: -```cue title="todoapp/cloudformation/deployment.cue" +```cue title="todoapp/cue.mod/cloudformation/deployment.cue" package cloudformation #Deployment: { @@ -619,7 +598,7 @@ package cloudformation `template.cue` can be rewritten as follows: -```cue title="todoapp/cloudformation/template.cue" +```cue title="todoapp/cue.mod/cloudformation/template.cue" package cloudformation import "encoding/json" @@ -636,19 +615,13 @@ Verify template Double-checks at the template level can be done with manual uploads on Cloudformation's web interface or by executing the below command locally: ```shell -tmpfile=$(mktemp ./tmp.XXXXXX) && dagger query template -f text > "$tmpfile" && aws cloudformation validate-template --template-body file://"$tmpfile" ; rm "$tmpfile" -``` - -Update the plan - -```shell -cp cloudformation/*.cue .dagger/env/cloudformation/plan/ +tmpfile=$(mktemp ./tmp.XXXXXX) && dagger query template -f text -e cloudformation > "$tmpfile" && aws cloudformation validate-template --template-body file://"$tmpfile" ; rm "$tmpfile" ``` Let's make sure it yields the same result: ```shell -dagger query template -f text +dagger query template -f text -e cloudformation # { # "description": "Name S3 Bucket", # "indexDocument": "index.html", @@ -667,7 +640,7 @@ dagger query template -f text And we can now deploy it: ```shell -dagger up +dagger up -e cloudformation #2:22PM INF suffix.out | computing #2:22PM INF suffix.out | completed duration=200ms #2:22PM INF cfnStack.outputs | computing @@ -683,7 +656,7 @@ dagger up Name of the deployed bucket: ```shell -dagger output list +dagger output list -e cloudformation # Output Value Description # suffix.out "ucwcecwwshdl" generated random string # cfnStack.outputs.Name "arn:aws:s3:::stack-ucwcecwwshdl-s3bucket-gaqmj8rzsl08" -