Commit Graph

55 Commits

Author SHA1 Message Date
Andrea Luzzardi
5a1d4bff62 Support loading artifacts into a Docker Engine
This adds support to loading artifacts (e.g. docker.#Build,
os.#Container, ...) into any arbitrary docker engine (through a
dagger.#Stream for UNIX sockets or SSH for a remote engine)

Implementation:
- Add op.#SaveImage which serializes an artifact into an arbitrary path
  (docker tarball format)
- Add docker.#Load which uses op.#SaveImage to serialize to disk and
  executes `docker load` to load it back

Caveats: Because we're doing this in userspace rather than letting
dagger itself load the image, the performance is pretty bad.

The buildkit API is meant for streaming (get a stream of a docker image
pipe it into docker load). Because of userspace, we have to load the
entire docker image into memory, then serialize it in a single WriteFile
LLB operation.

Example:

```cue
package main

import (
	"alpha.dagger.io/dagger"
	"alpha.dagger.io/docker"
)

source: dagger.#Input & dagger.#Artifact

dockersocket: dagger.#Input & dagger.#Stream

build: docker.#Build & {
	"source": source
}

load: docker.#Load & {
	source: build
	tag:    "testimage"
	socket: dockersocket
}
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-12 14:16:01 -07:00
Kevin Poirot
51f96172bd Add #Secret type to Docker #Build args
Signed-off-by: Kevin Poirot <kevin@vazee.fr>
2021-09-23 11:13:59 +02:00
Andrea Luzzardi
0f4a55a482 socket -> stream
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 15:19:15 -07:00
Andrea Luzzardi
5480fb991d dagger.#Socket support
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 11:53:09 -07:00
Tom Chauveau
a9fd97d7fe
Handle secrets in DockerLogin operation
Before, secret was a plain text string, but it could lead to security issue
so we are now handling secrets as `dagger.#Secret` or string.
I've add a new struct SecretStore that expose the inputStore to easily
retrieve secret value.

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-31 13:04:16 +02:00
Guillaume de Rouville
08f395b70d Add auth to Git.#Repository
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-27 15:58:55 +02:00
Guillaume de Rouville
3b0e3f6919 Move private repository tests + implement env helper function
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-27 14:41:31 +02:00
Tom Chauveau
a987d1dc34 Add test on op.#FetchGit for private repository
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-27 14:41:31 +02:00
Tom Chauveau
d7194f161d Add secret management to op.#FetchGit operation to fetch private repository
- Update `op.cue` to support secrets
- Update `pipeline.go` to use authTokenSecret & authHeaderSecret

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-27 14:41:31 +02:00
Guillaume de Rouville
9f90aca2b7 op.#Exec hack: improve performance by 100x on big configs
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-07-29 16:37:47 +02:00
Edgar Lee
0501ae91a5 Adds support for op.#FetchHTTP
Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
2021-07-06 14:37:52 -07:00
Tom Chauveau
2f710b0ce4 Fix login miss behavior and update op
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-07-01 14:08:49 +02:00
Solomon Hykes
d7a805f42b stdlib: move all imports to alpha.dagger.io
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-25 10:31:22 +00:00
Solomon Hykes
9d85bab9e7 os.#File: simplify API
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-23 12:56:27 +02:00
Sam Alba
af82c2c910 docs: adds package description to the stdlib Index
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-23 11:05:58 +02:00
Solomon Hykes
53f4ef049a stdlib: dagger.#Output
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-21 14:47:45 +02:00
Solomon Hykes
8383efdba6 stdlib: dagger.#Input
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-21 14:47:45 +02:00
Sam Alba
9281967e16 stdlib: added package doc strings
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-15 10:46:56 +02:00
Andrea Luzzardi
940d9ccfee docs: fix non-escaped URLs
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-08 18:40:08 -07:00
Andrea Luzzardi
c95b5d2bae docs: deploy using dagger
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-02 15:45:51 -07:00
Andrea Luzzardi
7ec90a6155
Merge pull request #497 from myitcv/put_fullop_behind_build_attribute
stdlib: put full definition of #Op behind a build attribute
2021-05-28 23:48:39 -07:00
Andrea Luzzardi
b627b4bc88 add support for excludes in input dir
This adds support for `--include` and `--exclude` for directory inputs.

For instance, this is what you would want to use when passing dagger
repository as an input:

```
inputs:
    repository:
        dir:
            path: .
            exclude:
                - '**/node_modules'
                - cmd/dagger/dagger
                - cmd/dagger/dagger-debug
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-28 16:22:32 -07:00
Andrea Luzzardi
9c0e2d1d95 buildkit secrets support
- Secrets are never exposed in plaintext in the Cue tree. `dagger query`
  won't dump secrets anymore, Cue errors won't contain them either.
- BuildKit-native secrets support through a new `mount` type. This
  ensures secrets will never be part of containerd layers, buildkit
  cache and generally speaking will never be saved to disk in plaintext.
- Updated netlify as an example
- Added tests
- Changed the Cue definition of a secret to:

```
	@dagger(secret)

	id: string
}
```

This is to ensure both that setting the wrong input type on a secret
(e.g. `dagger input text`) will fail, and attempting to misuse the
secret (e.g. interpolating, passing as an env variable, etc) will also
fail properly.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-27 19:14:08 -07:00
Paul Jolly
cd19a4228a
stdlib: put full definition of #Op behind a build attribute
This allows dagger.io/dagger/op.#Op to be maintained in regular
uncommented CUE, but not participate by default in normal dagger
evaluation (given the performance problems it currently introduces).

The "full" #Op can be enabled by passing the "-t fullop" flag:

    cue eval -t fullop ./examples/react

(which currently triggers the performance issue).

Signed-off-by: Paul Jolly <paul@myitcv.io>
2021-05-26 10:31:45 +01:00
Sam Alba
7484df45cd clean up comments
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-21 16:56:30 -07:00
Sam Alba
85cc9abb93 stdlib: adapted lib to new input spec
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-21 16:04:02 -07:00
Solomon Hykes
b526093929 stdlib: simpler and cleaner performance workaround
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-05-13 22:36:06 -07:00
Sam Alba
471b578671 stdlib: #FetchGit extra args
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-13 10:35:22 -07:00
dubo-dubon-duponey
ca6930bd02 Implement user & hosts for Exec
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-12 14:53:52 -07:00
Solomon Hykes
98b3951c73 stdlib: os package
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-05-11 17:32:50 -07:00
dubo-dubon-duponey
53611be733
Add add-hosts to DockerBuild
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-10 15:06:50 -07:00
dubo-dubon-duponey
38200f556c
Add target opt to DockerBuild
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-09 21:09:13 -07:00
Sam Alba
4df8b3e087 stdlib/dagger/op: Added #DockerLogin
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-04-26 17:41:04 -07:00
Andrea Luzzardi
2b7dfcae29 Revert "support registry auth"
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-23 16:59:22 -07:00
Andrea Luzzardi
692bd72095 support registry auth
HACK: the way buildkit works, we can only supply an Auth Provider for the entirety of the build session (`dagger up`). Therefore, we start by scanning all auth in the entire Cue tree and supply an auth provider for all of them.

Drawbacks:
- As soon as you add `auth` in a Pipeline for a registry, all other Pipelines have access to the same registry
- You can't use different credentials for the same registry

Fixes #301

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-20 15:28:33 -07:00
Sam Alba
468ac1220a added comment about always flag + cue fmt
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-04-15 12:34:15 -07:00
Frederick F. Kautz IV
0458c0a838 * add support for --input-file in dagger compute
* secrets now supports bytes
* error reporting for unhandled content data types in WriteFile

Signed-off-by: Frederick F. Kautz IV <fkautz@alumni.cmu.edu>
2021-04-11 16:18:59 -07:00
Solomon Hykes
ee621590ff stdlib: dagger.io/js/react
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-04-07 16:28:39 -07:00
Solomon Hykes
6673ae69b3 stdlib: rename dagger.io/llb to dagger.io/dagger/op
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-04-06 10:28:40 -07:00
Andrea Luzzardi
c16455249e cue files: #compute -> #up
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-02 16:08:15 -07:00
Andrea Luzzardi
8a4aa4d42b pipeline: #compute -> #up
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-02 16:08:15 -07:00
Andrea Luzzardi
1a3dc3debc dagger spec: fix Artifact definition
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-03-31 14:20:31 -07:00
Solomon Hykes
54a2fe4393 stdlib package: llb
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-03-15 22:25:18 -07:00
Tony Worm
472070d7f8 add #WriteFile and #Mkdir to #Op
Signed-off-by: Tony Worm <tony@hofstadter.io>
2021-03-15 22:08:54 -07:00
Solomon Hykes
6460a5c9fc Rename dagger.#Dir to dagger.#Artifact
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-03-13 00:11:59 -08:00
Andrea Luzzardi
c35eca99e1 push container support
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-03-11 16:41:19 -08:00
Andrea Luzzardi
db2184262c Add dagger.#Secret to dagger spec.
This currently does nothing and is only useful for flagging secrets.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-03-10 15:14:39 -08:00
Tony Worm
3e5df44a1d cue/fmt: stdlib/dagger
Signed-off-by: Tony Worm <tony@hofstadter.io>
2021-03-08 11:11:29 -05:00
Tony Worm
3a7a21326a fixes perf with dogfood
Signed-off-by: Tony Worm <tony@hofstadter.io>
2021-03-08 10:09:21 -05:00
Solomon Hykes
7a77466a66 DX: change "#dagger: compute: " to "#compute: "
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-03-06 19:39:22 -08:00