From a407019349c0c056c1d4abac5733f8968df83320 Mon Sep 17 00:00:00 2001 From: Solomon Hykes Date: Thu, 18 Mar 2021 05:44:50 +0000 Subject: [PATCH] Expand README content, split into multiple files Signed-off-by: Solomon Hykes --- README.md | 112 ++++----------------------- doc/install.md | 14 ++++ doc/operator.md | 43 ++++++++++ ARCHITECTURE.md => doc/programmer.md | 14 +++- doc/tutorials.md | 15 ++++ doc/what_is_adc.md | 25 ++++++ 6 files changed, 125 insertions(+), 98 deletions(-) create mode 100644 doc/install.md create mode 100644 doc/operator.md rename ARCHITECTURE.md => doc/programmer.md (91%) create mode 100644 doc/tutorials.md create mode 100644 doc/what_is_adc.md diff --git a/README.md b/README.md index bd7437b7..e14c56bb 100644 --- a/README.md +++ b/README.md @@ -1,109 +1,29 @@ # Dagger -*WARNING: Dagger is alpha-quality software. It has many bugs, the user interface is minimal, and it may change in incompatible ways at any time. If you are still willing to try it, thank you! We appreciate your help and encourage you to ask questions and report issues.* - Dagger is an *Application Delivery as Code* (ADC) platform. -Using Dagger, software teams can replace their artisanal deployment scripts with a state-of-the-art deployment pipeline, tailor-made for their application and infrastructure, in just a few lines of code. -- Do you spend too much time getting your home-made deployment scripts to work? -- Do you wish you could just use a PaaS like Heroku or Firebase but can’t because your stack is too custom? -- Do you put off infrastructure and workflow improvements because it would slow down development? -- Do you wish you could visualize your entire application delivery workflow as code, all in one place? +Using Dagger, software teams can create state-of-the-art deployment pipelines, tailor-made for their application and infrastructure, in just a few lines of code. -If you answered “yes” to any of the above, Dagger might be a good fit for you. +Dagger might be a good fit for you if: -## The problem with PaaS +* You spend too much time fixing your artisanal deployment scripts +* You wish you could use Heroku or Firebase, but your stack is too custom +* You put off important infrastructure changes because they would slow down development +* You wish you could visualize your whole application's supply chain as one big graph -A PaaS, or platform-as-a-service, is the glue between an application and the cloud infrastructure running it. It automates the deployment process and provides a simplified view of the underlying infrastructure, which makes developers more productive. +## Alpha disclaimer -However, despite the undeniable productivity benefits of using a PaaS, most applications today do not. Why? Because it's not flexible enough: each PaaS only supports certain types of application stacks and infrastructure. Applications that cannot adapt to the platform are simply not supported, and instead are deployed by a mosaic of specialized tools, usually glued together in an artisanal shell script or equivalent. +Dagger is *alpha-quality software*. It has many bugs, the user interface is minimal, and it may change in incompatible ways at any time. If you are still willing to try it, thank you! We appreciate your help and encourage you to ask questions and report issues. -But what if we didn’t have to choose between productivity and flexibility? What if we could have both? That’s where Dagger comes in. With Dagger, *each application defines its own PaaS*, perfectly tailored to its existing stack and infrastructure. The platform adapts to the application, instead of the other way around. - -And because it’s defined *as code*, this custom PaaS can easily be changed over time, as the application stack and infrastructure evolves. +## Examples -## Getting started +## Learn More - -### Installing the dagger command-line - -1. Build the `dagger` command-line tool. You will need [Go](https://golang.org) version 1.16 or later. - -``` -$ make -``` - -2. Copy the `dagger` tool to a location listed in your `$PATH`. For example, to copy it to `/usr/local/bin`: - -``` -$ cp ./cmd/dagger/dagger /usr/local/bin -``` - -3. Compute a test configuration - -Currently `dagger` can only do one thing: compute a configuration with optional inputs, and print the result. - -If you are confused by how to use this for your application, that is normal: `dagger compute` is a low-level command -which exposes the naked plumbing of the Dagger engine. In future versions, more user-friendly commands will be available -which hide the complexity of `dagger compute` (but it will always be available to power users, of course!). - -Here is an example command, using an example configuration: - -``` -$ dagger compute ./examples/simple --input-string www.host=mysuperapp.com --input-dir www.source=. -``` - -### Custom buildkit setup - -Dagger can be configured to use an existing buildkit daemon, running either locally or remotely. This can be done using two environment variables: `BUILDKIT_HOST` and `DOCKER_HOST`. - -To use a buildkit daemon listening on TCP port `1234` on localhost: - -``` -$ export BUILDKIT_HOST=tcp://localhost:1234 -``` - -To use a buildkit daemon running in a container named "super-buildkit" on the local docker host: - -``` -$ export BUILDKIT_HOST=docker-container://super-buildkit -``` - -To use a buildkit daemon running on a remote docker host (be careful to properly secure remotely accessible docker hosts!) - -``` -$ export BUILDKIT_HOST=docker-container://super-buildkit -$ export DOCKER_HOST=tcp://my-remote-docker-host:2376 -``` - -### OpenTracing Support - -Both Dagger and buildkit support opentracing. To capture traces to -[Jaeger](https://github.com/jaegertracing/jaeger), ), set the `JAEGER_TRACE` environment variable to the collection address. - -A `docker-compose` file is available to help bootstrap the tracing environment: - -```sh -docker-compose -f ./tracing.compose.yaml up -d -export JAEGER_TRACE=localhost:6831 -export BUILDKIT_HOST=docker-container://dagger-buildkitd-jaeger - -dagger compute ... -``` - -You can then go to [http://localhost:16686/](http://localhost:16686/) in your browser to see the traces. - - -## The Dagger programming model - -*FIXME*. - -## Dagger and your application - -*FIXME*. - -## Dagger and your infrastructure - -*FIXME*. +* [What is Application Delivery as Code?](doc/what_is_adc.md) +* [Installing Dagger](doc/install.md) +* [Tutorials](doc/tutorials.md) +* [Adding Dagger to your project](doc/adding_dagger.md) +* [Dagger Programmer Manual](doc/programmer.md) +* [Dagger Operator Manual](doc/operator.md) diff --git a/doc/install.md b/doc/install.md new file mode 100644 index 00000000..edd484a8 --- /dev/null +++ b/doc/install.md @@ -0,0 +1,14 @@ +# Installing Dagger + +1. Build the `dagger` command-line tool. You will need [Go](https://golang.org) version 1.16 or later. + +``` +$ make +``` + +2. Copy the `dagger` tool to a location listed in your `$PATH`. For example, to copy it to `/usr/local/bin`: + +``` +$ cp ./cmd/dagger/dagger /usr/local/bin +``` + diff --git a/doc/operator.md b/doc/operator.md new file mode 100644 index 00000000..2b67539f --- /dev/null +++ b/doc/operator.md @@ -0,0 +1,43 @@ +# Dagger Operator Manual + +## Custom buildkit setup + +Dagger can be configured to use an existing buildkit daemon, running either locally or remotely. This can be done using two environment variables: `BUILDKIT_HOST` and `DOCKER_HOST`. + +To use a buildkit daemon listening on TCP port `1234` on localhost: + +``` +$ export BUILDKIT_HOST=tcp://localhost:1234 +``` + +To use a buildkit daemon running in a container named "super-buildkit" on the local docker host: + +``` +$ export BUILDKIT_HOST=docker-container://super-buildkit +``` + +To use a buildkit daemon running on a remote docker host (be careful to properly secure remotely accessible docker hosts!) + +``` +$ export BUILDKIT_HOST=docker-container://super-buildkit +$ export DOCKER_HOST=tcp://my-remote-docker-host:2376 +``` + + +## OpenTracing Support + +Both Dagger and buildkit support opentracing. To capture traces to +[Jaeger](https://github.com/jaegertracing/jaeger), ), set the `JAEGER_TRACE` environment variable to the collection address. + +A `docker-compose` file is available to help bootstrap the tracing environment: + +```sh +docker-compose -f ./tracing.compose.yaml up -d +export JAEGER_TRACE=localhost:6831 +export BUILDKIT_HOST=docker-container://dagger-buildkitd-jaeger + +dagger compute ... +``` + +You can then go to [http://localhost:16686/](http://localhost:16686/) in your browser to see the traces. + diff --git a/ARCHITECTURE.md b/doc/programmer.md similarity index 91% rename from ARCHITECTURE.md rename to doc/programmer.md index 93b9ed89..3b0717c9 100644 --- a/ARCHITECTURE.md +++ b/doc/programmer.md @@ -1,6 +1,8 @@ -# The Dagger architecture +# Dagger Programmer Guide -This document provides details on the internals of Dagger, key design decisions and the rationale behind them. +## The Dagger programming model + +*FIXME* ## What is a DAG? @@ -25,3 +27,11 @@ The DAG architecture has many benefits: To execute a DAG, the dagger runtime JIT-compiles it to a low-level format called llb, and executes it with buildkit. Think of buildkit as a specialized VM for running compute graphs; and dagger as a complete programming environment for that VM. The tradeoff for all those wonderful features is that a DAG architecture cannot be used for all software: only software than can be run as a pipeline. + +## The CUE language + +*FIXME* + +## The Buildkit runtime environment + +*FIXME* diff --git a/doc/tutorials.md b/doc/tutorials.md new file mode 100644 index 00000000..43e727e0 --- /dev/null +++ b/doc/tutorials.md @@ -0,0 +1,15 @@ +# Dagger Tutorials + +## Compute a test configuration + +Currently `dagger` can only do one thing: compute a configuration with optional inputs, and print the result. + +If you are confused by how to use this for your application, that is normal: `dagger compute` is a low-level command +which exposes the naked plumbing of the Dagger engine. In future versions, more user-friendly commands will be available +which hide the complexity of `dagger compute` (but it will always be available to power users, of course!). + +Here is an example command, using an example configuration: + +``` +$ dagger compute ./examples/simple --input-string www.host=mysuperapp.com --input-dir www.source=. +``` diff --git a/doc/what_is_adc.md b/doc/what_is_adc.md new file mode 100644 index 00000000..c473fe7f --- /dev/null +++ b/doc/what_is_adc.md @@ -0,0 +1,25 @@ +# What is Application Delivery as Code? + +## The problem with PaaS + +A PaaS, or platform-as-a-service, is the glue between an application and the cloud infrastructure running it. It automates the deployment process and provides a simplified view of the underlying infrastructure, which makes developers more productive. + +However, despite the undeniable productivity benefits of using a PaaS, most applications today do not. Why? Because it's not flexible enough: each PaaS only supports certain types of application stacks and infrastructure. Applications that cannot adapt to the platform are simply not supported. + + +## The problem with artisanal deploy scripts + +Most applications don't fit in any major PaaS. Instead they are deployed by a patchwork of specialized tools, usually glued together by an artisanal shell script or equivalent. + +*FIXME: example of specialized tools* + +Most teams are unhappy with their deploy script. They are high maintenance, tend to break at the worst possible time, and are less convenient to use than a PaaS. But when you need control of your stack, what other choice is there? + + +## The best of both worlds: Application Delivery as Code + +Application Delivery as Code (ADC) is an alternative to PaaS and artisanal deploy scripts, which combines the best of each. + +Simply put, with ADC each application gets its own single-purpose PaaS. The platform adapts to the application, not the other way around. And because it’s defined *as code*, this custom PaaS can easily be changed over time, as the application stack and infrastructure evolves. + +Because doing this with a general programming language, like Go, would be prohibitively expensive, a good ADC implementation requires a specialized language and runtime environment. For example, Dagger uses [Cue](https://cuelang.org) as its language, and [Buildkit](https://github.com/moby/buildkit) as its runtime environment.