kjuulh/dagger
Archived
1
0
Fork 0
Code Issues 1 Pull Requests 1 Projects Releases Wiki Activity
772 Commits 3 Branches 0 Tags 47 MiB
6ddb973f03
Commit Graph

190 Commits

Author SHA1 Message Date
Andrea Luzzardi
33d5add54b deployment API cleanup 
The API was a bit confusing with code mixing the usage of Deployment and
DeploymentState.

This change marks a clear separation:

- Client is the only piece of code manipulating *Deployment
- CLI commands can manipulate DeploymentState objects and pass them to
  Client

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-02 15:30:38 -07:00
Andrea Luzzardi
2657a516b3 fix lint error 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-02 14:29:17 -07:00
Andrea Luzzardi
5ae5b1bb50 fix dagger input dir 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-01 23:17:38 -07:00
Andrea Luzzardi
526d3af364 state: store in ~/.dagger 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-01 23:00:30 -07:00
Andrea Luzzardi
d848afe7a9 store: fix root directory 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-01 22:16:29 -07:00
Sam Alba
fb9aae38ef cmd/new: added support for plan-git option 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-04-01 17:46:42 -07:00
Andrea Luzzardi
429820036f tests: add basic cli tests 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-04-01 16:46:05 -07:00
Sam Alba
c583dc20ef implemented ClientDoFunc callback to make the client.Up code reusable for other client actions 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-31 16:32:15 -07:00
Andrea Luzzardi
b9bd250634 client: fix cue error reporting 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-31 14:20:02 -07:00
Andrea Luzzardi
993b96dcb4 cue: simplify lookup wrappers 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-31 13:40:46 -07:00
Andrea Luzzardi
a8b41c06b7 cue: Use FillPath everywhere since Fill is now deprecated 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-31 13:14:36 -07:00
Andrea Luzzardi
e6674b7b7e terminology: rename layout -> plan 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-30 19:45:18 -07:00
Andrea Luzzardi
f901918266 terminology: rename route -> deployment 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-30 19:33:21 -07:00
Andrea Luzzardi
0f09ba5e87 minor error messages and linter cleanups 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-30 18:53:09 -07:00
Andrea Luzzardi
a45f3447b7 env -> route cleanup 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-26 16:44:13 -07:00
Andrea Luzzardi
3bfa056b9b tests: fix input test 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-26 16:24:22 -07:00
Andrea Luzzardi
ef84d2d431 inputs: use a struct rather than an interface 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-26 14:11:56 -07:00
Andrea Luzzardi
e08e64b311 store: keep an in-memory index of routes, support lookup by path 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 19:08:52 -07:00
Andrea Luzzardi
a3b84386bc store routes as directories 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 15:50:19 -07:00
Andrea Luzzardi
0e21144529 tests: fix input tests 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 15:46:37 -07:00
Andrea Luzzardi
9fec69f3a0 make store a struct and add tests 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 22:29:11 +00:00
Andrea Luzzardi
524f77df65 tests: fix unit tests 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 22:29:11 +00:00
Sam Alba
7ad541feb1 implemented new, up, list 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-25 22:29:11 +00:00
Andrea Luzzardi
43956e38cc separate Store from State 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 22:29:11 +00:00
Andrea Luzzardi
cba524eb0f merge Route into Env, rebase dagger compute to use the new API 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-25 22:29:11 +00:00
Sam Alba
43b3af6fff init local route dir 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-25 22:29:11 +00:00
Solomon Hykes
7bf05d3cc9 CLI backend to manage layout and inputs 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-03-25 22:29:11 +00:00
Solomon Hykes
37bf20e24b Expand mockup backend for CLI 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-03-25 22:29:11 +00:00
Sam Alba
84188535f3 implemented basic commands: up, down, new, query, list 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-25 22:29:11 +00:00
Solomon Hykes
c33dc5138f Mockup CLI backend API 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-03-25 22:29:10 +00:00
Andrea Luzzardi
9c4cb1b15a pipeline: Fix panic for empty pipelines 
`Solve() of `llb.Scratch()` returns a `nil` reference. This causes
Pipelines with no operations to panic when interacted with, because of
`nil` pointer de-reference.

Mitigates #189 and #184

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-22 16:30:17 -07:00
Andrea Luzzardi
b8de4e5049 cache fix: stable ordering of maps (Env, Mount, ...) 
Maps were causing the same Pipeline to randomly produce slightly different LLB
on each run (because they are represented as an array in LLB, wheras
they're a map in CueLLB).

This forces every Cue field iteration (env, mount, etc) to be
predictable by using stable sorting.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-19 15:07:01 -07:00
Andrea Luzzardi
e6bb3f12f4 inputs decryption using SOPS 
- Add support for --input-json and --input-yaml (plaintext)
- Input files can optionally be encrypted using sops, with transparent
  decryption by dagger

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-18 17:43:13 -07:00
Andrea Luzzardi
5b1e7eeacf pipeline: FetchGit: make a copy of the content to avoid caching issues 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-18 11:18:06 -07:00
Tony Worm
3dab86694e impl test, log patterns when .daggerignore found, lint, fix stdlib/file test imports 
Signed-off-by: Tony Worm <tony@hofstadter.io>
 2021-03-18 12:38:50 -04:00
Tony Worm
a1ba7aa59c implement .daggerignore 
Signed-off-by: Tony Worm <tony@hofstadter.io>
 2021-03-17 19:55:28 -04:00
Andrea Luzzardi
b15c841a24 pipeline: Use result rather than source LLB of sub-pipelines 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-12 13:41:24 -08:00
Andrea Luzzardi
c923e5042b cleanup: solver/fs 
- Solver: Encapsulates all access to Buildkit. Can solve plain LLB, invoke external frontends (for DockerBuild) and export (for ContainerPush)
- FS (now BuildkitFS) implements the standard Go 1.16 io/fs.FS interface and provides a read-only filesystem on top of a buildkit result. It can be used with built-ins such as fs.WalkDir (no need to have our own Walk functions anymore)
- Moved CueBuild into compiler.Build since it no longer depends on Buildkit. Instead it relies on the io/fs.FS interface, which is used both for the base config and the stdlib (go:embed also uses io/fs.FS). Overlaying base and the stdlib is now done by the same code.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-12 13:38:49 -08:00
Andrea Luzzardi
c35eca99e1 push container support 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-11 16:41:19 -08:00
Andrea Luzzardi
3628dbda84 flow cleanup: separate TaskFunc from RunnerFunc 
Extracted from #140

This will help be more flexible in what gets executed and how (e.g. for
running tests for instance)

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-10 15:01:37 -08:00
Solomon Hykes
7a77466a66 DX: change "#dagger: compute: " to "#compute: " 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-03-06 19:39:22 -08:00
Sam Alba
3f445f2421
Merge pull request #158 from dagger/fix-mount-tmpfs 
Fix mount tmpfs
 2021-03-04 18:32:28 -08:00
Sam Alba
514dde4e6d renamed mount arg tmp to tmpfs 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-04 14:03:50 -08:00
Andrea Luzzardi
256f373a52
Merge pull request #154 from dagger/opentracing 
opentracing support
 2021-03-04 11:04:42 -08:00
Andrea Luzzardi
f35df0475b opentracing support 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-04 10:45:20 -08:00
Sam Alba
5fed74bc55
Merge pull request #153 from dagger/example-aws-monitoring 
AWS Cloudformation + HTTP monitoring example
 2021-03-03 18:36:30 -08:00
Sam Alba
ee7bb3c3f5 implemented "write-file" and "mkdir" ops 
Signed-off-by: Sam Alba <sam.alba@gmail.com>
 2021-03-03 17:24:21 -08:00
Andrea Luzzardi
14ff14ea4f automatically start/upgrade buildkitd 
- Automatically start a buildkit daemon if no BUILDKIT_HOST is
  provided (and if not already started)
- Customization of BUILDKIT_HOST is still possible, just like before
- Automatically upgrade the managed daemon to the version used by dagger
  if necessary
- Add CI test to make sure the managed buildkit and the vendored
  buildkit versions match

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-03-02 18:37:30 -08:00
Andrea Luzzardi
4bd0ffd3fc FetchContainer: Inject Docker image metadata into LLB 
Fixes #130

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-25 14:47:46 -08:00
Andrea Luzzardi
074e47d9c0 logs: clean up logs for canceled tasks 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-24 18:48:55 -08:00
Andrea Luzzardi
a8df284e9b local: improve log message 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-24 17:29:51 -08:00
Andrea Luzzardi
b2bc801cd0 fix caching issues with Local 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-24 15:38:49 -08:00
Andrea Luzzardi
1fcb36113f cleanup log messages 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-23 16:37:45 -08:00
Andrea Luzzardi
79b66e82fc logs: extract cue path from buildkit solve status 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-23 15:39:52 -08:00
Andrea Luzzardi
2ae395843e encode cue paths in LLB ops 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-23 15:39:52 -08:00
Andrea Luzzardi
bc2dae7e32 docker build support 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-22 14:14:51 -08:00
Andrea Luzzardi
890fdb4176 performance: reduce the number of fills 
- Remove unnecessary Fill() in Export()
- Change `set()` and the way we store outputs so we don't fill
  intermediaries as much
- WIP: Scan the tree only once. Changed LocalDirs to use cueflow rather than
  doing our own Walk. In a follow up we should use the same flow
  instance.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-19 14:04:40 -08:00
Andrea Luzzardi
3ddd967291 local: fix when include is not specified 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 17:19:49 -08:00
Andrea Luzzardi
845fe7eca4 fix error checking 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 15:27:40 -08:00
Andrea Luzzardi
fef148f6c0 rename go package dagger.io/go 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 13:13:39 -08:00
Andrea Luzzardi
9e86b011d5 cleanup: wrap errors using go 1.13 built-ins 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 10:30:51 -08:00
Andrea Luzzardi
74e084944e compiler cleanup 
Now using the same pattern as Go's http package.

- the `compiler.Compiler` struct can be used directly (and tests to do
  to avoid messing with the global version)
- `compiler.DefaultCompiler` contains a public default Compiler instance
- `compiler` exposes proxy functions (e.g. Compile) back to the
  DefaultCompiler

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 10:26:38 -08:00
Andrea Luzzardi
5ba6dd1617 embedded stdlib 
This PR embeds the stdlib into the dagger binary itself for convenience.
Long term we will want to source the stdlib directly from git.

- Updated go to 1.16 to use the new built-in embedding functionality
- The `stdlib` go package now contains an embed of the stdlib

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-17 10:08:56 -08:00
Andrea Luzzardi
31f1cd9f1d
Merge pull request #119 from dagger/small-cleanup 
Small cleanup
 2021-02-16 14:41:18 -08:00
Solomon Hykes
be8f600c59 dagger/cc: remove locking methods from public API 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-13 17:40:21 +00:00
Solomon Hykes
bff4186bf4 dagger compute auto-fetches standard library from external repo 
- Standard library is hosted at github.com/blocklayerhq/dagger-stdlib
- When dagger repo is made public, stdlib can be hosted from same repo
- Vendored cue.mod is merged with stdlib at file granularity
- When developing dagger, set DAGGER_DEV_STDLIB to a local stdlib
  directory

Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-12 23:31:13 +00:00
Solomon Hykes
e8527ddcf5 No more runtime spec validation 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-12 23:30:46 +00:00
Solomon Hykes
acba8b3988 Simplify runtime code by removing layers of abstraction 
- Remove intermediary types `Component`, `Script`, `Op`, `mount`: just use
  `cc.Value` directly
- Remove `Executable` interface.
- Execute llb code with a simple concrete type `Pipeline`
- Analyze llb code with a simple utility `Analyze`

Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-12 22:20:20 +00:00
Andrea Luzzardi
3718749db5 re-enable spec generation 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-10 18:43:07 -08:00
Andrea Luzzardi
7cbce4eec1
Merge pull request #108 from verdverm/mount-path 
support path option in script/component mounts
 2021-02-09 12:51:14 -08:00
Tony Worm
8de7a47b8d support path option in script/component mounts 
Signed-off-by: Tony Worm <tony@hofstadter.io>
 2021-02-09 15:00:23 -05:00
Solomon Hykes
781b89e41c Simplify op.go by removing unnecessary cue validations 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-08 15:10:32 -08:00
Solomon Hykes
612a25fb9f Simplify code by moving polyfill cue compiler to sub-package cc 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-08 15:10:32 -08:00
Solomon Hykes
622de21883 {do:"subdir"} to select a subdirectory in a script 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-05 15:27:37 -08:00
Solomon Hykes
0202f4447b Don't instantiate a global shared Env in cmd/compute.go 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-05 15:27:19 -08:00
Solomon Hykes
0fe2ef4fe4 Simplify Env API: Compute/Components instead of Compute/Walk 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-05 15:27:19 -08:00
Andrea Luzzardi
5b7bc78cf1
Merge pull request #104 from blocklayerhq/mnt-cache 
mount cache support
 2021-02-05 15:18:32 -08:00
Andrea Luzzardi
b0294e79d1 solver: trace LLB 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-05 15:14:45 -08:00
Andrea Luzzardi
f336164699 mount cache support 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-05 15:08:44 -08:00
Solomon Hykes
b30febc7c2 Fix out-of-the-blue linter errors 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-02-03 12:39:25 -08:00
Andrea Luzzardi
c6e010d4f0 Misc Export fixes 
- Temporarily disable export of base.cue as it causes merge errors at
  the end of compute
- Fixes for JSON export for Scalar and Lists
- Add YAML export
- Removed BOOL and NUMBER support, using JSON for now
- Re-enabled all export tests

Fixes #36

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 16:19:46 -08:00
Andrea Luzzardi
86aa031f92 fix concurrency issues in Compiler.Compile and Value.Source 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 13:52:52 -08:00
Andrea Luzzardi
cd0f21dbd2 client: simplify Compute logic 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 12:32:22 -08:00
Andrea Luzzardi
c9e0d0854d client: remove unused printer 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 11:12:24 -08:00
Andrea Luzzardi
167edf4a76 simplify: remove compute.go 
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 11:03:12 -08:00
Andrea Luzzardi
6e31193d64
Merge pull request #87 from blocklayerhq/input-ux 
Better ux for inputs
 2021-02-02 10:45:48 -08:00
Andrea Luzzardi
78601fefd6 fix dependencies between tasks 
due to using an old snapshot of cue.Value, components relying on
dependent tasks were failing because of non-concretness.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-02-02 10:31:52 -08:00
Solomon Hykes
1954c3f731 dagger compute [--input-dir TARGET=DIR] [--input-string TARGET=STRING] [--input-cue CUE] [--input-git REMOTE#REF] 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-31 22:48:30 -08:00
Dan POP
23a1211cdf Update gen.sh to remove redundant error check 
cue eval spec.cue >/dev/null removed (redunant per @shykes)

Signed-off-by: Dan POP <dan.papandrea@sysdig.com>
 2021-01-28 22:38:38 -08:00
Solomon Hykes
c4e55a6915 Refactor how client prepares environment. Inputs may now reference local directories 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-28 19:29:57 -08:00
Solomon Hykes
a26b767780 Fix bug in localdir (optional fields in spec not supported) 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-28 19:29:57 -08:00
Solomon Hykes
3e5063828a Fix bug in exec/mount 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-28 19:29:57 -08:00
Andrea Luzzardi
cc5a48d39b
Merge pull request #76 from blocklayerhq/output-no-merge 
output: merge base/input/output into a single state.cue
 2021-01-26 16:53:32 -08:00
Solomon Hykes
f933278d43 Fix spec validation & merge so that default values are correctly applied 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-26 16:52:14 -08:00
Andrea Luzzardi
04526b5b78 output: merge base/input/output into a single state.cue 
Exporting base/input/output separately causes merge errors.
For instance, `foo: string | *"default foo"` gets serialized as
`{"foo":"default foo"}`, which will fail to merge if output contains
a different definition of `foo`.

This change temporarily merges the 3 streams together before
serializing.

Fixes #51

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-01-26 12:12:42 -08:00
Andrea Luzzardi
75fbddf6be op: exec: fix dir 
Fixes #30

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
 2021-01-25 16:19:20 -08:00
Solomon Hykes
e10025d688 Refactor op/script/component loading and spec validation 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-25 11:36:18 -08:00
Solomon Hykes
bbe16283ab Incremental cleanup: wrap env instance in a Value 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-25 11:36:18 -08:00
Solomon Hykes
795b7f585c Clean up client config. Preparation for fixing local dir issues 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-22 17:15:22 -08:00
Solomon Hykes
eab6028b70 Replace Fillable interface with a concrete wrapper 
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
 2021-01-21 23:46:58 -08:00