Commit Graph

33 Commits

Author SHA1 Message Date
Andrea Luzzardi
e5c571b8a9 op.#SaveImage: fix data race
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-12 16:05:43 -07:00
Andrea Luzzardi
50d55a8885 op.#SaveImage: export the image ID
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-12 15:39:04 -07:00
Andrea Luzzardi
5a1d4bff62 Support loading artifacts into a Docker Engine
This adds support to loading artifacts (e.g. docker.#Build,
os.#Container, ...) into any arbitrary docker engine (through a
dagger.#Stream for UNIX sockets or SSH for a remote engine)

Implementation:
- Add op.#SaveImage which serializes an artifact into an arbitrary path
  (docker tarball format)
- Add docker.#Load which uses op.#SaveImage to serialize to disk and
  executes `docker load` to load it back

Caveats: Because we're doing this in userspace rather than letting
dagger itself load the image, the performance is pretty bad.

The buildkit API is meant for streaming (get a stream of a docker image
pipe it into docker load). Because of userspace, we have to load the
entire docker image into memory, then serialize it in a single WriteFile
LLB operation.

Example:

```cue
package main

import (
	"alpha.dagger.io/dagger"
	"alpha.dagger.io/docker"
)

source: dagger.#Input & dagger.#Artifact

dockersocket: dagger.#Input & dagger.#Stream

build: docker.#Build & {
	"source": source
}

load: docker.#Load & {
	source: build
	tag:    "testimage"
	socket: dockersocket
}
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-12 14:16:01 -07:00
Andrea Luzzardi
dd1bf18ec2 logger: TTY logs support (live update)
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-05 15:38:34 -07:00
Kevin Poirot
51f96172bd Add #Secret type to Docker #Build args
Signed-off-by: Kevin Poirot <kevin@vazee.fr>
2021-09-23 11:13:59 +02:00
Andrea Luzzardi
0f4a55a482 socket -> stream
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 15:19:15 -07:00
Andrea Luzzardi
5480fb991d dagger.#Socket support
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 11:53:09 -07:00
Tom Chauveau
cf13257b10
Improve SecretStore integration with new method
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-09-01 21:46:56 +02:00
Tom Chauveau
eabf1b52e5
Update docker push tests
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-31 14:44:06 +02:00
Tom Chauveau
a9fd97d7fe
Handle secrets in DockerLogin operation
Before, secret was a plain text string, but it could lead to security issue
so we are now handling secrets as `dagger.#Secret` or string.
I've add a new struct SecretStore that expose the inputStore to easily
retrieve secret value.

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-31 13:04:16 +02:00
Guillaume de Rouville
08f395b70d Add auth to Git.#Repository
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-27 15:58:55 +02:00
Tom Chauveau
d7194f161d Add secret management to op.#FetchGit operation to fetch private repository
- Update `op.cue` to support secrets
- Update `pipeline.go` to use authTokenSecret & authHeaderSecret

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-27 14:41:31 +02:00
Andrea Luzzardi
b8dcc02bb8 performance: compile CUE client side
Restructured the compile logic to happen on the CLI instead of the
BuildKit frontend.

- Avoid uploading the entire workspace to BuildKit on every compilation
- Let the CUE loader scan the files instead of going through the
  BuildKit filesystem gRPC APIs.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-08-20 16:34:49 +02:00
Guillaume de Rouville
9f90aca2b7 op.#Exec hack: improve performance by 100x on big configs
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-07-29 16:37:47 +02:00
Alvise
99d2514aaa Move to OpenTelemetry
Signed-off-by: Alvise <vitalvise@gmail.com>
2021-07-09 09:47:36 +02:00
Andrea Luzzardi
f58ee5811b universe vendoring
Rather than injecting universe at runtime, this change will vendor
alpha.dagger.io in `cue.mod` directly.

Fixes #700

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-08 17:52:21 +02:00
Andrea Luzzardi
f8531fdb0b
Merge pull request #772 from aluzzardi/default-plan-dir
plan: default cue module to .
2021-07-07 17:34:23 +02:00
Andrea Luzzardi
6e215b194e use the workspace as the plan module
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-07 13:12:10 +02:00
Edgar Lee
0501ae91a5 Adds support for op.#FetchHTTP
Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
2021-07-06 14:37:52 -07:00
Sam Alba
efed4f73ea input/dir: ignore .dagger/ directory by default
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-21 11:37:48 +02:00
Sam Alba
7d9ff46ebd revert input dir management to llb.Copy to fix cache for yarn source pkg
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-20 08:32:37 +02:00
Andrea Luzzardi
f39a88e644 cue native: environments can reference a module instead of embedding
one.

Fixes #631

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-16 18:58:56 +02:00
Andrea Luzzardi
5f849bb13e cmd: dagger input edit -> dagger edit
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-16 18:58:56 +02:00
Sam Alba
09617b89da core: replaced persistent cache mounts with directory
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-09 15:22:06 +02:00
Sam Alba
7b4421b9a0 cmd/doc: boiler plate and for inputs / outputs scanning
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-04 09:01:32 +02:00
Andrea Luzzardi
c95b5d2bae docs: deploy using dagger
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-02 15:45:51 -07:00
Sam Alba
06a515f496 environments only ignore the compute layer if it is empty instead of logging a fatal error
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-01 14:14:45 +02:00
Sam Alba
efb4ee209e cmd/up: disable output list when not a tty + better error handling
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-31 18:25:58 +02:00
Sam Alba
6e3ec02ceb added support for output scanning
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-31 15:24:42 +02:00
Andrea Luzzardi
b627b4bc88 add support for excludes in input dir
This adds support for `--include` and `--exclude` for directory inputs.

For instance, this is what you would want to use when passing dagger
repository as an input:

```
inputs:
    repository:
        dir:
            path: .
            exclude:
                - '**/node_modules'
                - cmd/dagger/dagger
                - cmd/dagger/dagger-debug
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-28 16:22:32 -07:00
Andrea Luzzardi
9c0e2d1d95 buildkit secrets support
- Secrets are never exposed in plaintext in the Cue tree. `dagger query`
  won't dump secrets anymore, Cue errors won't contain them either.
- BuildKit-native secrets support through a new `mount` type. This
  ensures secrets will never be part of containerd layers, buildkit
  cache and generally speaking will never be saved to disk in plaintext.
- Updated netlify as an example
- Added tests
- Changed the Cue definition of a secret to:

```
	@dagger(secret)

	id: string
}
```

This is to ensure both that setting the wrong input type on a secret
(e.g. `dagger input text`) will fail, and attempting to misuse the
secret (e.g. interpolating, passing as an env variable, etc) will also
fail properly.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-27 19:14:08 -07:00
Sam Alba
eba35e285b environment inputs scanning can now scan including the user inputs with the plan
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-27 18:45:24 +02:00
Andrea Luzzardi
af776b8abe cleanup: move packages to top level, change vanity URL
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-25 16:54:00 -07:00