Commit Graph

65 Commits

Author SHA1 Message Date
Solomon Hykes
c1c585bcd5 Europa: integrate core packages, separate universe
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-12-15 09:27:58 +00:00
Andrea Luzzardi
c3e9883f57 image spec: fix linter
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-13 16:13:49 +01:00
Andrea Luzzardi
ff6c7d1c1f engine.#Pull implementation
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-13 16:13:49 +01:00
Andrea Luzzardi
89daebbc4c engine spec: move core types to types.cue
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-09 14:14:36 -05:00
Andrea Luzzardi
4b7a143cc6 stdlib: move core definitions to engine package
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-07 16:00:56 -05:00
Andrea Luzzardi
5b7b1cab79 Migrate dagger.#Secret and dagger.#Stream to new format
- Refactored to keep every transformation of built-in types (e.g. FS,
  Secret, etc) to/from CUE in the same place (plancontext)
- dagger.#Service and dagger.#Secret are now following the new FS-like format
  (e.g. `_service: id: string`)
- Backward compatibility
- dagger.#Stream is now an alias for dagger.#Service

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-01 17:33:35 -08:00
Andrea Luzzardi
0aea10d23e dagger.#FS support
- Implement dagger.#FS support
- Migrate `context.imports` to dagger.#FS
- Backward compat: dagger.#FS can be passed in lieu of a
  dagger.#Artifact
- For instance, an import (`dagger.#FS`) can be passed to the current
  `yarn.#Package` implementation

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-30 12:58:50 -08:00
Andrea Luzzardi
2a4db167e4 runtime: new execution engine
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-24 16:51:52 -08:00
Andrea Luzzardi
a61e8dcb62 prepare the transition to #Plan.context
This change helps the transition between `dagger input` and `#Plan.context`.

In summary, the codebase now relies on a *context* for execution with mapping to *IDs*.
In the future, *context* will come from a `#Plan.context`.
In the meantime, a bridge converts `dagger input` to a plan context. This allows both *old* and *new* style configurations to co-exist with the same underlying engine.

- Implement `plancontext`. Context holds the execution context for a plan. Currently this includes the platform, local directories, secrets and services (e.g. unix/npipe).
- Contextual data can be registered at any point. In the future, this will be done by `#Plan.context`
- Migrated the `dagger input` codebase to register inputs in a `plancontext`
- Migrated low-level types/operations to the *Context ID* pattern.
  - `dagger.#Stream` now only includes an `id` (instead of `unix` path)
  - `dagger.#Secret` still includes only an ID, but now it's based off `plancontext`
  - `op.#Local` now only includes an `id` (instead of `path`, `include`, `exclude`.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-19 11:29:38 -08:00
Sam Alba
e6de59a340 added support for stream type npipe (Windows named pipe)
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-11-08 17:16:38 -08:00
Andrea Luzzardi
5a1d4bff62 Support loading artifacts into a Docker Engine
This adds support to loading artifacts (e.g. docker.#Build,
os.#Container, ...) into any arbitrary docker engine (through a
dagger.#Stream for UNIX sockets or SSH for a remote engine)

Implementation:
- Add op.#SaveImage which serializes an artifact into an arbitrary path
  (docker tarball format)
- Add docker.#Load which uses op.#SaveImage to serialize to disk and
  executes `docker load` to load it back

Caveats: Because we're doing this in userspace rather than letting
dagger itself load the image, the performance is pretty bad.

The buildkit API is meant for streaming (get a stream of a docker image
pipe it into docker load). Because of userspace, we have to load the
entire docker image into memory, then serialize it in a single WriteFile
LLB operation.

Example:

```cue
package main

import (
	"alpha.dagger.io/dagger"
	"alpha.dagger.io/docker"
)

source: dagger.#Input & dagger.#Artifact

dockersocket: dagger.#Input & dagger.#Stream

build: docker.#Build & {
	"source": source
}

load: docker.#Load & {
	source: build
	tag:    "testimage"
	socket: dockersocket
}
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-12 14:16:01 -07:00
Kevin Poirot
51f96172bd Add #Secret type to Docker #Build args
Signed-off-by: Kevin Poirot <kevin@vazee.fr>
2021-09-23 11:13:59 +02:00
Andrea Luzzardi
0f4a55a482 socket -> stream
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 15:19:15 -07:00
Andrea Luzzardi
5480fb991d dagger.#Socket support
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 11:53:09 -07:00
Tom Chauveau
a9fd97d7fe
Handle secrets in DockerLogin operation
Before, secret was a plain text string, but it could lead to security issue
so we are now handling secrets as `dagger.#Secret` or string.
I've add a new struct SecretStore that expose the inputStore to easily
retrieve secret value.

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-31 13:04:16 +02:00
Guillaume de Rouville
08f395b70d Add auth to Git.#Repository
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-27 15:58:55 +02:00
Guillaume de Rouville
3b0e3f6919 Move private repository tests + implement env helper function
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-27 14:41:31 +02:00
Tom Chauveau
a987d1dc34 Add test on op.#FetchGit for private repository
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-27 14:41:31 +02:00
Tom Chauveau
d7194f161d Add secret management to op.#FetchGit operation to fetch private repository
- Update `op.cue` to support secrets
- Update `pipeline.go` to use authTokenSecret & authHeaderSecret

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-27 14:41:31 +02:00
Guillaume de Rouville
9f90aca2b7 op.#Exec hack: improve performance by 100x on big configs
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-07-29 16:37:47 +02:00
Edgar Lee
0501ae91a5 Adds support for op.#FetchHTTP
Signed-off-by: Edgar Lee <edgarhinshunlee@gmail.com>
2021-07-06 14:37:52 -07:00
Tom Chauveau
2f710b0ce4 Fix login miss behavior and update op
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-07-01 14:08:49 +02:00
Solomon Hykes
d7a805f42b stdlib: move all imports to alpha.dagger.io
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-25 10:31:22 +00:00
Solomon Hykes
9d85bab9e7 os.#File: simplify API
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-23 12:56:27 +02:00
Sam Alba
af82c2c910 docs: adds package description to the stdlib Index
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-23 11:05:58 +02:00
Solomon Hykes
53f4ef049a stdlib: dagger.#Output
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-21 14:47:45 +02:00
Solomon Hykes
8383efdba6 stdlib: dagger.#Input
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-06-21 14:47:45 +02:00
Sam Alba
9281967e16 stdlib: added package doc strings
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-06-15 10:46:56 +02:00
Andrea Luzzardi
940d9ccfee docs: fix non-escaped URLs
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-08 18:40:08 -07:00
Andrea Luzzardi
c95b5d2bae docs: deploy using dagger
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-02 15:45:51 -07:00
Andrea Luzzardi
7ec90a6155
Merge pull request #497 from myitcv/put_fullop_behind_build_attribute
stdlib: put full definition of #Op behind a build attribute
2021-05-28 23:48:39 -07:00
Andrea Luzzardi
b627b4bc88 add support for excludes in input dir
This adds support for `--include` and `--exclude` for directory inputs.

For instance, this is what you would want to use when passing dagger
repository as an input:

```
inputs:
    repository:
        dir:
            path: .
            exclude:
                - '**/node_modules'
                - cmd/dagger/dagger
                - cmd/dagger/dagger-debug
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-28 16:22:32 -07:00
Andrea Luzzardi
9c0e2d1d95 buildkit secrets support
- Secrets are never exposed in plaintext in the Cue tree. `dagger query`
  won't dump secrets anymore, Cue errors won't contain them either.
- BuildKit-native secrets support through a new `mount` type. This
  ensures secrets will never be part of containerd layers, buildkit
  cache and generally speaking will never be saved to disk in plaintext.
- Updated netlify as an example
- Added tests
- Changed the Cue definition of a secret to:

```
	@dagger(secret)

	id: string
}
```

This is to ensure both that setting the wrong input type on a secret
(e.g. `dagger input text`) will fail, and attempting to misuse the
secret (e.g. interpolating, passing as an env variable, etc) will also
fail properly.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-27 19:14:08 -07:00
Paul Jolly
cd19a4228a
stdlib: put full definition of #Op behind a build attribute
This allows dagger.io/dagger/op.#Op to be maintained in regular
uncommented CUE, but not participate by default in normal dagger
evaluation (given the performance problems it currently introduces).

The "full" #Op can be enabled by passing the "-t fullop" flag:

    cue eval -t fullop ./examples/react

(which currently triggers the performance issue).

Signed-off-by: Paul Jolly <paul@myitcv.io>
2021-05-26 10:31:45 +01:00
Sam Alba
7484df45cd clean up comments
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-21 16:56:30 -07:00
Sam Alba
85cc9abb93 stdlib: adapted lib to new input spec
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-21 16:04:02 -07:00
Solomon Hykes
b526093929 stdlib: simpler and cleaner performance workaround
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-05-13 22:36:06 -07:00
Sam Alba
471b578671 stdlib: #FetchGit extra args
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-13 10:35:22 -07:00
dubo-dubon-duponey
ca6930bd02 Implement user & hosts for Exec
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-12 14:53:52 -07:00
Solomon Hykes
98b3951c73 stdlib: os package
Signed-off-by: Solomon Hykes <solomon@dagger.io>
2021-05-11 17:32:50 -07:00
dubo-dubon-duponey
53611be733
Add add-hosts to DockerBuild
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-10 15:06:50 -07:00
dubo-dubon-duponey
38200f556c
Add target opt to DockerBuild
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2021-05-09 21:09:13 -07:00
Sam Alba
4df8b3e087 stdlib/dagger/op: Added #DockerLogin
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-04-26 17:41:04 -07:00
Andrea Luzzardi
2b7dfcae29 Revert "support registry auth"
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-23 16:59:22 -07:00
Andrea Luzzardi
692bd72095 support registry auth
HACK: the way buildkit works, we can only supply an Auth Provider for the entirety of the build session (`dagger up`). Therefore, we start by scanning all auth in the entire Cue tree and supply an auth provider for all of them.

Drawbacks:
- As soon as you add `auth` in a Pipeline for a registry, all other Pipelines have access to the same registry
- You can't use different credentials for the same registry

Fixes #301

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-20 15:28:33 -07:00
Sam Alba
468ac1220a added comment about always flag + cue fmt
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-04-15 12:34:15 -07:00
Frederick F. Kautz IV
0458c0a838 * add support for --input-file in dagger compute
* secrets now supports bytes
* error reporting for unhandled content data types in WriteFile

Signed-off-by: Frederick F. Kautz IV <fkautz@alumni.cmu.edu>
2021-04-11 16:18:59 -07:00
Solomon Hykes
ee621590ff stdlib: dagger.io/js/react
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-04-07 16:28:39 -07:00
Solomon Hykes
6673ae69b3 stdlib: rename dagger.io/llb to dagger.io/dagger/op
Signed-off-by: Solomon Hykes <sh.github.6811@hykes.org>
2021-04-06 10:28:40 -07:00
Andrea Luzzardi
c16455249e cue files: #compute -> #up
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-04-02 16:08:15 -07:00