Commit Graph

48 Commits

Author SHA1 Message Date
Andrea Luzzardi
fb16ac8b29 vendoring: support multiple modules
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-11 16:51:54 -08:00
Andrea Luzzardi
282759c0e5 cue modules: move stdlib to pkg/alpha.dagger.io
In preparation for Europa, we will vendor multiple CUE modules:

- `pkg/alpha.dagger.io`: legacy non-europa packages
- `pkg/dagger.io`: core Europa packages
- `pkg/universe.dagger.io`: Europa universe

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-11 13:16:37 -08:00
Andrea Luzzardi
25ee7087bf semi-atomic vendoring to improve concurrency
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-23 18:09:54 +01:00
Sam Alba
fdb4b94276 state: moved stdlib lock to upper function to fix race condition
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-21 16:11:07 -08:00
Richard Jones
b5b73a3a8f
removed unnecessary leading new lines
Signed-off-by: Richard Jones <richard@dagger.io>
2021-12-08 16:38:45 -07:00
Richard Jones
83b10d4031 refactored finding of cue.mod parent
Signed-off-by: Richard Jones <richard@dagger.io>
2021-12-08 15:28:33 -07:00
Richard Jones
5c9965f10a passings args to cue loader to mimick cue eval
Signed-off-by: Richard Jones <richard@dagger.io>
2021-12-08 15:28:33 -07:00
Andrea Luzzardi
a58ca16c3d plancontext cleanup
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-07 14:20:08 -05:00
Andrea Luzzardi
337ed0d1cc remove hardcoded stdlib path
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-01 17:33:35 -08:00
Andrea Luzzardi
5b7b1cab79 Migrate dagger.#Secret and dagger.#Stream to new format
- Refactored to keep every transformation of built-in types (e.g. FS,
  Secret, etc) to/from CUE in the same place (plancontext)
- dagger.#Service and dagger.#Secret are now following the new FS-like format
  (e.g. `_service: id: string`)
- Backward compatibility
- dagger.#Stream is now an alias for dagger.#Service

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-01 17:33:35 -08:00
Andrea Luzzardi
0aea10d23e dagger.#FS support
- Implement dagger.#FS support
- Migrate `context.imports` to dagger.#FS
- Backward compat: dagger.#FS can be passed in lieu of a
  dagger.#Artifact
- For instance, an import (`dagger.#FS`) can be passed to the current
  `yarn.#Package` implementation

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-30 12:58:50 -08:00
Andrea Luzzardi
2a4db167e4 runtime: new execution engine
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-24 16:51:52 -08:00
Andrea Luzzardi
a61e8dcb62 prepare the transition to #Plan.context
This change helps the transition between `dagger input` and `#Plan.context`.

In summary, the codebase now relies on a *context* for execution with mapping to *IDs*.
In the future, *context* will come from a `#Plan.context`.
In the meantime, a bridge converts `dagger input` to a plan context. This allows both *old* and *new* style configurations to co-exist with the same underlying engine.

- Implement `plancontext`. Context holds the execution context for a plan. Currently this includes the platform, local directories, secrets and services (e.g. unix/npipe).
- Contextual data can be registered at any point. In the future, this will be done by `#Plan.context`
- Migrated the `dagger input` codebase to register inputs in a `plancontext`
- Migrated low-level types/operations to the *Context ID* pattern.
  - `dagger.#Stream` now only includes an `id` (instead of `unix` path)
  - `dagger.#Secret` still includes only an ID, but now it's based off `plancontext`
  - `op.#Local` now only includes an `id` (instead of `path`, `include`, `exclude`.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-19 11:29:38 -08:00
Sam Alba
6ef5f52db5 project: add dagger.lock to gitignore when pulling an external dep
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-11-17 16:25:49 -08:00
Sam Alba
e6de59a340 added support for stream type npipe (Windows named pipe)
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-11-08 17:16:38 -08:00
Andrea Luzzardi
23d488d462
Merge pull request #1103 from samalba/fix-windows-dir-encoding
state: fixed input dir encoding on windows
2021-11-08 13:20:31 -08:00
Sam Alba
21e1f89f3a state: fixed input dir encoding on windows
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-11-05 17:16:23 -07:00
Tom Chauveau
a63c4e989a
Change "arch" into "platform" because it's more accurate.
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-11-05 16:21:21 +01:00
Sam Alba
954192118e
Merge pull request #1074 from TomChv/feat/pipeline-platform-config
Handle running architecture configuration
2021-11-01 16:14:20 -07:00
Tom Chauveau
4a461a0021
Fix engine to don't write architecture in values.yaml if no one provided.
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-10-30 12:07:33 +02:00
Tom Chauveau
0839cfd1d0
Change default architecture to linux/amd64
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-10-28 16:17:00 +02:00
Sam Alba
994a8034c5 state: force use of stdlib embed - temporarily until universe repo is ready
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-10-22 15:41:14 -07:00
Sam Alba
5e6d1261f7 mod: lock the version of universe for updates of package
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-10-22 15:37:28 -07:00
Tom Chauveau
e9de597654
Update dagger new cmd to specify architecture when creating an environment
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-10-22 21:54:18 +02:00
Tom Chauveau
caa03a37cc
Update dagger engine to uses a given architecture instead of default one.
Check #1071 for more details

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-10-22 21:02:12 +02:00
Sam Alba
779dda1aca mod: added basic logging + context in sub libs
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-10-22 11:02:34 -07:00
Sam Alba
cee8c91e50 mod: added support for version constraint when fetching a remote version
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-10-21 17:50:18 -07:00
Sam Alba
fe83b578f6 state: disabled remote stdlib install (keep vendoring until migration is complete)
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-10-13 15:25:57 -07:00
Sam Alba
e9ca8f38e6 Vendoring improved
* update dagger init with package manager downloading stdlib

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* split mod get and update functions

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* write to package checksum to dagger.sum when installing/updating

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* encure checksums are valid when compiling input

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* remove references to github.com/tjovicic in docs 1010 and 1011

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* refactor mod get command

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* update logic of moving dir when installing packages

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix linting errors

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* revert changes to 1010 docs

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* updating error log line in mod/get

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix ci tests when using vendoring

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* update alpha.dagger.io version to v0.1

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix mod repo test

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* return error if package already installed

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* remove already installed packages when installing

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix issue when vendoring stdlib

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* update mod command with filelock while installing

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix linting errors

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

* fix path of mod lock file

Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

Co-authored-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>

Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-10-13 15:25:30 -07:00
Guillaume de Rouville
a7ca7e11ba Fix/dagger edit: wrong dir Path leads to infinite loop
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-09-26 20:56:22 +02:00
Richard Jones
7061ac816e
changed workspace to project
Signed-off-by: Richard Jones <richard@dagger.io>
2021-09-23 12:02:57 -06:00
Andrea Luzzardi
5480fb991d dagger.#Socket support
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-09-17 11:53:09 -07:00
Richard Jones
defaea5a87
referencing secretInput directly rather than looking up in State by key
Signed-off-by: Richard Jones <richard@dagger.io>
2021-09-16 09:31:41 -06:00
Richard
0666d6f55a added hash of the secret key+value to invalidate buildkit cache when value changes
Signed-off-by: Richard <richard@dagger.io>
2021-09-16 09:23:50 -06:00
Guillaume de Rouville
6fe0ec367a dagger input bool implementation
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-24 16:07:28 +02:00
Andrea Luzzardi
b8dcc02bb8 performance: compile CUE client side
Restructured the compile logic to happen on the CLI instead of the
BuildKit frontend.

- Avoid uploading the entire workspace to BuildKit on every compilation
- Let the CUE loader scan the files instead of going through the
  BuildKit filesystem gRPC APIs.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-08-20 16:34:49 +02:00
Tom Chauveau
c637549b15
Fix input git to correctly handle subdir argument (solve #884)
Previously, the subdir argument wa ignored by dagger input git.
I've fix that behavior and add a test.

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-17 15:06:28 +02:00
Andrea Luzzardi
f58ee5811b universe vendoring
Rather than injecting universe at runtime, this change will vendor
alpha.dagger.io in `cue.mod` directly.

Fixes #700

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-08 17:52:21 +02:00
Andrea Luzzardi
6e215b194e use the workspace as the plan module
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-07 13:12:10 +02:00
Andrea Luzzardi
919d5576d0 plan: default cue module to .
Fixes #699

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-07 13:12:10 +02:00
Andrea Luzzardi
687c0e33a4 keychain: always ensure the default key is generated
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-17 17:23:11 +02:00
Andrea Luzzardi
b393ecc034 cleanup workspace interface
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-16 18:58:56 +02:00
Andrea Luzzardi
f39a88e644 cue native: environments can reference a module instead of embedding
one.

Fixes #631

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-16 18:58:56 +02:00
Andrea Luzzardi
5f849bb13e cmd: dagger input edit -> dagger edit
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-16 18:58:56 +02:00
Andrea Luzzardi
b627b4bc88 add support for excludes in input dir
This adds support for `--include` and `--exclude` for directory inputs.

For instance, this is what you would want to use when passing dagger
repository as an input:

```
inputs:
    repository:
        dir:
            path: .
            exclude:
                - '**/node_modules'
                - cmd/dagger/dagger
                - cmd/dagger/dagger-debug
```

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-28 16:22:32 -07:00
Andrea Luzzardi
9c0e2d1d95 buildkit secrets support
- Secrets are never exposed in plaintext in the Cue tree. `dagger query`
  won't dump secrets anymore, Cue errors won't contain them either.
- BuildKit-native secrets support through a new `mount` type. This
  ensures secrets will never be part of containerd layers, buildkit
  cache and generally speaking will never be saved to disk in plaintext.
- Updated netlify as an example
- Added tests
- Changed the Cue definition of a secret to:

```
	@dagger(secret)

	id: string
}
```

This is to ensure both that setting the wrong input type on a secret
(e.g. `dagger input text`) will fail, and attempting to misuse the
secret (e.g. interpolating, passing as an env variable, etc) will also
fail properly.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-27 19:14:08 -07:00
Sam Alba
1f4bbc53ad cmd/init: print workspace path
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-05-26 16:26:13 +02:00
Andrea Luzzardi
af776b8abe cleanup: move packages to top level, change vanity URL
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-25 16:54:00 -07:00