Commit Graph

121 Commits

Author SHA1 Message Date
Gerhard Lazu
374ac1800a
ci: Remove release.yml workflow
This has not been used in months, all releases run through
auto-release.yml now.

Follow-up to https://github.com/dagger/dagger/pull/2048

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-04-05 19:04:32 +01:00
Andrea Luzzardi
4be71eb578 ci: fix auto-release secrets
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-04-05 10:59:28 -07:00
Andrea Luzzardi
d1d993a79f ci: fix workflow secret names
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-04-05 10:47:12 -07:00
Andrea Luzzardi
93d2224073 ci: move ci infra top-level
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-04-04 18:06:54 -07:00
Andrea Luzzardi
8fe29cdc76 ci: use dagger from source
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-04-04 15:34:02 -07:00
Tom Chauveau
19c0f999f4
Fix export cache issue
Resolve #1551 and #1020.
We are never returning the result of solved operations so Buildkit could not
cache the layer.
This commit implements a simple system to forward operations' result to the
main build to cache it.

Signed-off-by: Vasek - Tom C <tom.chauveau@epitech.eu>
2022-03-31 20:36:50 +02:00
Marcos Lilljedahl
b84a273f45 ci: Fix key copying
Signed-off-by: Marcos Lilljedahl <marcosnils@gmail.com>
2022-03-30 19:18:39 -03:00
Marcos Lilljedahl
e47148ba64 ci: Unify keys and add private key for testing purposes
This commit enables PR's to run inegration tests by relying on keys with
only the necessary permissions to run on CI workloads

Signed-off-by: Marcos Lilljedahl <marcosnils@gmail.com>
2022-03-30 19:18:32 -03:00
guillaume
d5149823cc Exclude docs from universe and integration tests
Any cue file created in the docs triggered these tests. It shouldn't.

Signed-off-by: guillaume <guillaume.derouville@gmail.com>
2022-03-30 18:09:01 +02:00
Andrea Luzzardi
ec95602060 ci: enable website tests on PRs
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-29 11:22:15 -07:00
Marcos Lilljedahl
2290641fa4 Bump golangci-lint to latest
Golint is replaced with revive since the former has been archived and
replaced with the latter.

Signed-off-by: Marcos Lilljedahl <marcosnils@gmail.com>
2022-03-21 17:40:10 -03:00
Andrea Luzzardi
085862200c ci: switch auto release to patch versions
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-14 22:36:32 -07:00
Andrea Luzzardi
71f6f3a958 test: Fix universe tests
- Remove universe.dagger.io/cue.mod from `.gitignore`
- Re-add symlink
- Remove test-docs.yml workflow

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-11 12:41:45 -08:00
Andrea Luzzardi
7b4b59153d
Merge pull request #1769 from aluzzardi/ci-autorelease-patch
ci: auto release: bump patch version
2022-03-10 19:01:54 -08:00
Andrea Luzzardi
2e65145acb ci: auto release: bump patch version
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-10 16:01:00 -08:00
Andrea Luzzardi
f5bad7fb53 ci: fix broken GH yaml
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-10 15:28:04 -08:00
Andrea Luzzardi
3942f6b64c ci: close stale PRs automatically
Label PRs as `stale` after 14 days, close them 7 days after.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-03-10 14:24:39 -08:00
Sam Alba
eec7bc0a95 ci: fixed github action
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2022-03-09 18:14:01 -08:00
Sam Alba
5767721478 ci: added github workflow for running the tests
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2022-03-09 17:30:44 -08:00
Gerhard Lazu
c3f21958d2
Europa docs: From local dev to CI environment doc page
The todoapp example contains a Netlify plan which uses the latest dagger
additions: do & Client API. We are thinking of merging the examples
repository into this one to make working with this easier. This is a
step in that direction.

We are not using the yarn package so that we can revert
https://github.com/dagger/dagger/pull/1673 without breaking this
implementation.

The GitHub Action is WIP, we will continue with that tomorrow:
https://github.com/dagger/dagger-for-github/issues/24

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-03-08 22:17:24 +00:00
dubo-dubon-duponey
0bfb1bd447
Changes to bats files should trigger testing
Signed-off-by: dubo-dubon-duponey <dubodubonduponey+github@pm.me>
2022-02-24 17:17:58 -08:00
Joel Longtine
f939c81faf Use the correct target in the Makefile for Europa
Signed-off-by: Joel Longtine <joel@dagger.io>
2022-02-23 12:20:01 -07:00
Joel Longtine
0705b0eeae Skip non-europa tests
Signed-off-by: Joel Longtine <joel@dagger.io>
2022-02-23 12:20:01 -07:00
Gerhard Lazu
15f156e8c7
Use more descriptive names for GitHub Checks
The existing checks didn't read well in the `gh` cli:

    ❯ gh pr checks
    All checks were successful
    0 failing, 7 successful, 0 skipped, and 0 pending checks

    ✓  DCO                                 https://probot.github.io/apps/dco/
    ✓  Docs                         6m37s  https://github.com/dagger/dagger/runs/5136044870?check_suite_focus=true
    ✓  Everything                   1m43s  https://github.com/dagger/dagger/runs/5136044863?check_suite_focus=true
    ✓  Integration                  4m10s  https://github.com/dagger/dagger/runs/5136044871?check_suite_focus=true
    ✓  Universe                     11m6s  https://github.com/dagger/dagger/runs/5136044860?check_suite_focus=true
    ✓  Universe - Europa            3m18s  https://github.com/dagger/dagger/runs/5136044932?check_suite_focus=true
    ✓  netlify/devel-docs-dagge...         https://app.netlify.com/sites/devel-docs-dagger-io/deploys/620499ee88ee240008b6cabf

This change gives them more descriptive names:

    ❯ gh pr checks
    All checks were successful
    0 failing, 8 successful, 0 skipped, and 0 pending checks

    ✓  DCO                                             https://probot.github.io/apps/dco/
    ✓  Integration Tests                       4m22s   https://github.com/dagger/dagger/runs/5139842686?check_suite_focus=true
    ✓  Lint                                    1m29s   https://github.com/dagger/dagger/runs/5139842774?check_suite_focus=true
    ✓  Test Docs                               7m44s   https://github.com/dagger/dagger/runs/5139842682?check_suite_focus=true
    ✓  Unit Tests                              2m11s   https://github.com/dagger/dagger/runs/5139842684?check_suite_focus=true
    ✓  Universe Tests                          13m14s  https://github.com/dagger/dagger/runs/5139842932?check_suite_focus=true
    ✓  Universe Tests - Europa                 2m36s   https://github.com/dagger/dagger/runs/5139842678?check_suite_focus=true
    ✓  netlify/devel-docs-dagger-io/deploy...          https://deploy-preview-1593--devel-docs-dagger-io.netlify.app

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-02-10 11:31:21 +00:00
Gerhard Lazu
a14c2eaad1
Deploy docs on every commit to the main branch
We want doc updates to flow continuously, not just on a new tag.

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-02-10 10:30:08 +00:00
Gerhard Lazu
bcd175e5eb
Use paths instead of paths-ignore for trigerring actions
Andrea made some great suggestions, thanks!

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-02-04 12:18:21 +00:00
Gerhard Lazu
1b17d3b9e2
Do not run lint & tests on unrelated file changes
These paths shouldn't trigger lint or test runs.

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-02-04 12:00:54 +00:00
Gerhard Lazu
3f5f4af402
Restore auto-release time to the original value
This was set temporarily 4h later than originally intended so that it
would go through the day that we expected it to. Now that we know it
worked, let's restore it back to the time that matches morning in SFO &
early evening in Europe.

I see no reason to create extra PR work, shipping it straight into main.
FTR, this changes a single value, 21 -> 17, and a few comments.

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-02-01 16:18:17 +00:00
Gerhard Lazu
8c009f9d59
Default semver bump type when scheduled
When this workflow is scheduled, instead of triggered manually, we need
a default bump type, otherwise it will fail:
https://github.com/dagger/dagger/runs/4942069177?check_suite_focus=true#step:5:11

If the SEMVER_BUMP environment variable is not set via an input, which
will be the case for all scheduled runs, default to the same 'prerel
alpha..' value.

Modified the auto-trigger to later on today so that we can check the
change in the final setting.

This is a Ship & Show follow-up to
https://github.com/dagger/dagger/pull/1476

What is Ship / Show / Ask?
https://martinfowler.com/articles/ship-show-ask.html

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 19:58:41 +00:00
Gerhard Lazu
fb09e2a1c6
Split CI workflow into individual workflows
I had to re-run all jobs in the CI workflow at least 10 times in the
past 2 days. The problem is that when one jobs fails, all jobs need to
re-run, which sometimes results in different jobs failing. It would be
great if we could only re-run the jobs that failed, rather than all the
jobs in the CI workflow. Going forward, we should focus on improving
flaky tests, and speed the jobs which take the longest, but for now this
is a good start.

Before this change, we were wasting a lot of dev time - 2h in total for
my last PR #1476 - but also wasting CI minutes. Some of us were even
tempted to ignore CI 😱. This is a very slipper slope, and while it may
feel liberating in the short-term, there are many "windmill monsters"
down this path - don't do it.

Have a look at the CI workflow before this change to see how many
failures we had:
https://github.com/dagger/dagger/actions/workflows/ci.yml

Without looking at the jobs that failed, can you guess which areas are
the flakiest and need our attention the most? Integration & Universe are
good guesses, and I wish we could see this without digging into the CI
workflow - this change does that.

There is a lot more that can be improved here, but I didn't want to get
too carried away. The biggest improvement that we can make is switch
this to Dagger, which has some challenges, but I definitely intend to
tackle them because it feels worth it. This is good enough for now.

This is a ship & show PR. If all tests pass, this is a straight merge. I
am keeping it atomic so that we can revert it if we don't like it.

cc @aluzzardi @talentedmrjones @jlongtine @samalba @shykes @grouville

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 18:44:42 +00:00
Gerhard Lazu
8682f63aaa
Unblock auto-releases
Last time this ran, GoReleaser built an artefact with the wrong version
- it didn't bump it correctly. It was meant to build 0.1.0-alpha.33, but
it built 0.1.0-alpha.32 instead:
https://github.com/dagger/dagger/runs/4860126130?check_suite_focus=true#step:7:94

This new approach is a simpler and more explicit tag bump by leveraging
the semver-tool directly. A link to this utility is included in the
comments. We version it in this repository so that it is all
self-contained.

We also use the gh CLI tool directly, instead of a GitHub Action that
hides the implementation detail behind Typescript. We now have two very
simple gh CLI invocations that do all that. While we still use the
https://github.com/lewagon/wait-on-check-action GitHub Action to wait
on running checks, and abort if any check failed, I didn't want to
bundle that improvement into this PR - it's already big enough.

As a meaningful improvement, we should have a Dagger package that bumps
versions.  It would have been so much easier to use that Dagger package.
That implies us switching our GitHub Actions to Dagger, which we should
totally do. Small steps ftw!

Next step: run 0.1.0 release manually
Step 2: run 0.2.0-alpha.1 release manually
Step 3: wait for 0.2.0-alpha.2 to be produced automatically, tomorrow.

Pair: @aluzzardi

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 11:42:46 +00:00
Andrea Luzzardi
5016cf5e30 netlify: Europa port
- Fix netlify.#Deploy (there's still FIXMEs)
- Externalize the `deploy.sh` script
- Add tests
- Misc engine fixes for more explicit error messages

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-20 16:10:14 -08:00
Andrea Luzzardi
568cd194a4 tests: Basic test infrastructure for Europa Universe
- Each package can include its own bats file (e.g. `universe.dagger.io/yarn/test/yarn.bats`)
- universe.dagger.io includes a common bash helper file
- bats is installed/launched through yarn with minimal setup
- shellcheck is done across the entire repo
- Integrated into our CI

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-19 12:39:39 -08:00
Gerhard Lazu
c5c4fdac29
Fix running-workflow-name in bump_version-tag-release job
This will prevent the job waiting on itself to complete (which will not
happen until GitHub Actions kills the run).

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-18 22:02:28 +00:00
Gerhard Lazu
59e9fae189
Also release in the Trigger Release workflow
The first implementation of the trigger-release would not push a tag,
meaning that the Release workflow was not getting triggered. While we
could have changed the Release workflow to react on "Trigger Release"
workflow runs, the inter-dependency felt awkward and brittle:

    diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
    index b711c5cf..843fdb70 100644
    --- a/.github/workflows/release.yml
    +++ b/.github/workflows/release.yml
    @@ -7,10 +7,16 @@ on:
       push:
         tags:
         - v*
    +  workflow_run:
    +    workflows:
    +    - "Trigger Release"
    +    types:
    +    - completed

     jobs:
       goreleaser:
         runs-on: ubuntu-latest
    +    if: ${{ github.event.workflow_run.conclusion == 'success' }}
         defaults:
           run:
             shell: bash

Instead of doing the above, introducing duplication between "Trigger
Release" and "Release" seemed simpler from a cognitive perspective. Now,
releases are produced via the Release workflow when tags are pushed, and
also when releases are triggered via "Trigger Release", now renamed to
"Auto Release".

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-18 18:59:50 +00:00
Gerhard Lazu
fcdda0b072
Add GitHub Workflow that triggers a release at a regular time in the week
While talking with @aluzzardi, we thought that regular auto-releases
which happen with no intervention on our part would be a good idea. The
last Dagger release (0.1.0-alpha.31) was over 1 month ago, and there are
Europa-related changes which we want to make available in the Dagger
GitHub Action. We should never have more than 1 week of unreleased
changes. While more often is better, and we may need to tweak this later,
this is a decent starting point: release every Tuesday, 5pm UTC & 9am SFO.

We had to adjust the starting point slightly so that we do not start at
the top of the hour when there is high load on GitHub Actions (see the
inline comments for more details)

The workflow can also be triggered manually, and a custom tag can be
provided optionally. If no tag is provided, the last one will be
incremented as expected, e.g. v0.1.0-alpha.31 -> v0.1.0-alpha.32.
Before you get too carried away with custom tags, let's talk about the
unexpected side-effects which are not worth covering in this commit
message ("people over processes").

There is also a concurrency setting which will not prevent multiple
releases to be triggered, but at least these jobs will not run in
parallel. I looked into cancelling the current workflow if another one
of the same type is running, but I couldn't get it to work properly
within my 30 mins time-box so I stopped.

There is a lot to talk about our releases AFTER this gets merged, so
let's defer those conversations until we are happy with the first step
which I think is in the right direction.

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-14 20:28:45 +00:00
Andrea Luzzardi
200aca9f3d ci: handle non-release CUE versions
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-11 11:38:22 -08:00
Sam Alba
2601bc7de1 ci: moved website deploy in its own workflow to filter out paths
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-17 13:16:23 -08:00
Sam Alba
b082b1e5bc implemented integration tests for engine.#Pull + moved auth code separately for sharing code with other tasks later
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-17 10:07:33 -08:00
Sam Alba
40e1b28133 ci: reduced docs test suite timeout + disabled parallelization within a suite
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-16 17:10:49 -08:00
Sam Alba
e81feba213 ci: upgrade localstack to latest release
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-15 10:14:44 -08:00
Sam Alba
d03e345c52 ci: disabled several tests to debug CI issues in Docs tests
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-14 15:09:44 -08:00
jffarge
4d07e2af40 docs: 🐛 remove --frozen-lockfile option to update yarn.lock
Signed-off-by: jffarge <jf@dagger.io>
2021-11-18 15:39:53 +01:00
Jonathan Hult
d0e7d9b564
Link to new CUE GitHub repository
https://github.com/cue-lang/cue/issues/1078

Signed-off-by: Jonathan Hult <Jonathan@JonathanHult.com>
2021-10-20 14:49:26 -04:00
Andrea Luzzardi
285fe3fe39 ci: disable GHA caching
Getting a 400 Bad Request from GH API

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-05 12:39:50 -07:00
Andrea Luzzardi
f792199c98 ci: change gha scope
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-05 12:19:30 -07:00
Andrea Luzzardi
cb5634329c ci: gha export with mode=max
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:56:08 -07:00
Andrea Luzzardi
541d3775c3 ci: gha lint
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:29:23 -07:00
Andrea Luzzardi
4cce3de9df ci: re-enable GHA caching
Reverts #875 since moby/buildkit#2276 is now closed.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:28:44 -07:00
Tom Chauveau
0fe8b14e78 Improve ArgoCD and fix CI
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-09-10 09:10:21 -07:00