Commit Graph

30 Commits

Author SHA1 Message Date
Marcos Lilljedahl
d80acf805b Add experimental way to set a target platform when building
Add an --experimental-platform flag to the do command to allow
overriding the default auto-detected build platform until we find the
time to think about the definitive multi-platform builds UX

Signed-off-by: Marcos Lilljedahl <marcosnils@gmail.com>
2022-04-05 18:24:14 -03:00
Marcos Lilljedahl
34c7a2ff12 Automatically set target platform based on client architecture
Set the default platform based on the client's OS and architecture. This
function is the same one that buildkit uses (https://github.com/moby/buildkit/blob/master/frontend/dockerfile/builder/build.go#L100-L102) to set the default build target platform

Signed-off-by: Marcos Lilljedahl <marcosnils@gmail.com>
2022-04-05 15:30:11 -03:00
Tom Chauveau
19c0f999f4
Fix export cache issue
Resolve #1551 and #1020.
We are never returning the result of solved operations so Buildkit could not
cache the layer.
This commit implements a simple system to forward operations' result to the
main build to cache it.

Signed-off-by: Vasek - Tom C <tom.chauveau@epitech.eu>
2022-03-31 20:36:50 +02:00
Andrea Luzzardi
a61291a887 engine: redact dynamic secrets (e.g. #NewSecret)
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-07 12:08:37 -08:00
Andrea Luzzardi
5b7b1cab79 Migrate dagger.#Secret and dagger.#Stream to new format
- Refactored to keep every transformation of built-in types (e.g. FS,
  Secret, etc) to/from CUE in the same place (plancontext)
- dagger.#Service and dagger.#Secret are now following the new FS-like format
  (e.g. `_service: id: string`)
- Backward compatibility
- dagger.#Stream is now an alias for dagger.#Service

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-12-01 17:33:35 -08:00
Andrea Luzzardi
0aea10d23e dagger.#FS support
- Implement dagger.#FS support
- Migrate `context.imports` to dagger.#FS
- Backward compat: dagger.#FS can be passed in lieu of a
  dagger.#Artifact
- For instance, an import (`dagger.#FS`) can be passed to the current
  `yarn.#Package` implementation

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-30 12:58:50 -08:00
Andrea Luzzardi
81ec746d2c core: component -> task
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-24 16:58:24 -08:00
Andrea Luzzardi
cdcb09126b client: take localdirs as argument
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-24 16:13:30 -08:00
Andrea Luzzardi
b92905e175 client: re-enable opentelemetry for buildkit
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-24 16:05:53 -08:00
Andrea Luzzardi
a61e8dcb62 prepare the transition to #Plan.context
This change helps the transition between `dagger input` and `#Plan.context`.

In summary, the codebase now relies on a *context* for execution with mapping to *IDs*.
In the future, *context* will come from a `#Plan.context`.
In the meantime, a bridge converts `dagger input` to a plan context. This allows both *old* and *new* style configurations to co-exist with the same underlying engine.

- Implement `plancontext`. Context holds the execution context for a plan. Currently this includes the platform, local directories, secrets and services (e.g. unix/npipe).
- Contextual data can be registered at any point. In the future, this will be done by `#Plan.context`
- Migrated the `dagger input` codebase to register inputs in a `plancontext`
- Migrated low-level types/operations to the *Context ID* pattern.
  - `dagger.#Stream` now only includes an `id` (instead of `unix` path)
  - `dagger.#Secret` still includes only an ID, but now it's based off `plancontext`
  - `op.#Local` now only includes an `id` (instead of `path`, `include`, `exclude`.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-11-19 11:29:38 -08:00
Tom Chauveau
caa03a37cc
Update dagger engine to uses a given architecture instead of default one.
Check #1071 for more details

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-10-22 21:02:12 +02:00
Andrea Luzzardi
c08f619b02 Fix local directory error handling
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-20 11:38:42 -07:00
Tom Chauveau
a9fd97d7fe
Handle secrets in DockerLogin operation
Before, secret was a plain text string, but it could lead to security issue
so we are now handling secrets as `dagger.#Secret` or string.
I've add a new struct SecretStore that expose the inputStore to easily
retrieve secret value.

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-31 13:04:16 +02:00
Sam Alba
47ef0a4c2a
Merge pull request #862 from tjovicic/package-manager
First version of package manager
2021-08-27 15:25:20 -07:00
Tom Chauveau
3f0350359e
Improve solver channel management according to @aluzzardi comments
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-23 17:22:01 +02:00
Tom Chauveau
95468ce2b3
Solve sub build miss synchronous behavior
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-23 15:33:16 +02:00
Tihomir Jovicic
85e870f8dd Merge branch 'main' into package-manager 2021-08-23 06:37:47 +02:00
Tom Chauveau
39ec4a0a2f
Fix waiting group in client runner
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-20 18:39:12 +02:00
Andrea Luzzardi
b8dcc02bb8 performance: compile CUE client side
Restructured the compile logic to happen on the CLI instead of the
BuildKit frontend.

- Avoid uploading the entire workspace to BuildKit on every compilation
- Let the CUE loader scan the files instead of going through the
  BuildKit filesystem gRPC APIs.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-08-20 16:34:49 +02:00
Tom Chauveau
9adeecd005
Fix data race issue by synchronizing events channel through waiting group
Previously, there was a data race condition error when `op
.#PushContainer` was called. It was because of the `Export` func
in `solver.go` that create a sub build pipeline and forward his
events to the main events channel (that catch all events that we log).
To fix it, channel are now split between build pipeline and
centralize into the old output channel (`ch`).

Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-20 15:33:15 +02:00
Tom Chauveau
22a9445455
Track data race error
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-08-20 14:09:32 +02:00
Tihomir Jovicic
0010609f4d First version of package manager
Signed-off-by: Tihomir Jovicic <tihomir.jovicic.develop@gmail.com>
2021-08-01 21:22:48 +02:00
Andrea Luzzardi
67d5609aee support buildkit cache export/import
- Add support for buildkit cache export/import
- Enable GHA cache for universe test (using buildkit v0.9 GHA support)

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-28 14:28:20 +02:00
Sam Alba
ac32d6f57b buildkit client does not return environment to force the caller to implement its logic in the build callback
Signed-off-by: Sam Alba <sam.alba@gmail.com>
2021-07-13 16:15:34 +02:00
Alvise
99d2514aaa Move to OpenTelemetry
Signed-off-by: Alvise <vitalvise@gmail.com>
2021-07-09 09:47:36 +02:00
Andrea Luzzardi
4d9ef37b48 log stderr output as INFO
Fixes #241

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-22 12:48:42 +02:00
Andrea Luzzardi
ac34df319a docker socket forwarding support
- This PR adds a new mount type: `docker.sock` (in addition to `cache`
  and `tmp`)
- It's then able to mount the LOCAL (as in, from the machine running
  dagger) docker socket inside the container by pretending to be an SSH
  Agent (hijacking the SSH agent forwarding support of buildkit)

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-04 16:14:25 -07:00
Andrea Luzzardi
ebbc9707ec automatically redact secrets from logs
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-29 02:18:01 -07:00
Andrea Luzzardi
9c0e2d1d95 buildkit secrets support
- Secrets are never exposed in plaintext in the Cue tree. `dagger query`
  won't dump secrets anymore, Cue errors won't contain them either.
- BuildKit-native secrets support through a new `mount` type. This
  ensures secrets will never be part of containerd layers, buildkit
  cache and generally speaking will never be saved to disk in plaintext.
- Updated netlify as an example
- Added tests
- Changed the Cue definition of a secret to:

```
	@dagger(secret)

	id: string
}
```

This is to ensure both that setting the wrong input type on a secret
(e.g. `dagger input text`) will fail, and attempting to misuse the
secret (e.g. interpolating, passing as an env variable, etc) will also
fail properly.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-27 19:14:08 -07:00
Andrea Luzzardi
af776b8abe cleanup: move packages to top level, change vanity URL
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-25 16:54:00 -07:00