Commit Graph

93 Commits

Author SHA1 Message Date
Gerhard Lazu
8c009f9d59
Default semver bump type when scheduled
When this workflow is scheduled, instead of triggered manually, we need
a default bump type, otherwise it will fail:
https://github.com/dagger/dagger/runs/4942069177?check_suite_focus=true#step:5:11

If the SEMVER_BUMP environment variable is not set via an input, which
will be the case for all scheduled runs, default to the same 'prerel
alpha..' value.

Modified the auto-trigger to later on today so that we can check the
change in the final setting.

This is a Ship & Show follow-up to
https://github.com/dagger/dagger/pull/1476

What is Ship / Show / Ask?
https://martinfowler.com/articles/ship-show-ask.html

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 19:58:41 +00:00
Gerhard Lazu
fb09e2a1c6
Split CI workflow into individual workflows
I had to re-run all jobs in the CI workflow at least 10 times in the
past 2 days. The problem is that when one jobs fails, all jobs need to
re-run, which sometimes results in different jobs failing. It would be
great if we could only re-run the jobs that failed, rather than all the
jobs in the CI workflow. Going forward, we should focus on improving
flaky tests, and speed the jobs which take the longest, but for now this
is a good start.

Before this change, we were wasting a lot of dev time - 2h in total for
my last PR #1476 - but also wasting CI minutes. Some of us were even
tempted to ignore CI 😱. This is a very slipper slope, and while it may
feel liberating in the short-term, there are many "windmill monsters"
down this path - don't do it.

Have a look at the CI workflow before this change to see how many
failures we had:
https://github.com/dagger/dagger/actions/workflows/ci.yml

Without looking at the jobs that failed, can you guess which areas are
the flakiest and need our attention the most? Integration & Universe are
good guesses, and I wish we could see this without digging into the CI
workflow - this change does that.

There is a lot more that can be improved here, but I didn't want to get
too carried away. The biggest improvement that we can make is switch
this to Dagger, which has some challenges, but I definitely intend to
tackle them because it feels worth it. This is good enough for now.

This is a ship & show PR. If all tests pass, this is a straight merge. I
am keeping it atomic so that we can revert it if we don't like it.

cc @aluzzardi @talentedmrjones @jlongtine @samalba @shykes @grouville

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 18:44:42 +00:00
Gerhard Lazu
8682f63aaa
Unblock auto-releases
Last time this ran, GoReleaser built an artefact with the wrong version
- it didn't bump it correctly. It was meant to build 0.1.0-alpha.33, but
it built 0.1.0-alpha.32 instead:
https://github.com/dagger/dagger/runs/4860126130?check_suite_focus=true#step:7:94

This new approach is a simpler and more explicit tag bump by leveraging
the semver-tool directly. A link to this utility is included in the
comments. We version it in this repository so that it is all
self-contained.

We also use the gh CLI tool directly, instead of a GitHub Action that
hides the implementation detail behind Typescript. We now have two very
simple gh CLI invocations that do all that. While we still use the
https://github.com/lewagon/wait-on-check-action GitHub Action to wait
on running checks, and abort if any check failed, I didn't want to
bundle that improvement into this PR - it's already big enough.

As a meaningful improvement, we should have a Dagger package that bumps
versions.  It would have been so much easier to use that Dagger package.
That implies us switching our GitHub Actions to Dagger, which we should
totally do. Small steps ftw!

Next step: run 0.1.0 release manually
Step 2: run 0.2.0-alpha.1 release manually
Step 3: wait for 0.2.0-alpha.2 to be produced automatically, tomorrow.

Pair: @aluzzardi

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-25 11:42:46 +00:00
Andrea Luzzardi
5016cf5e30 netlify: Europa port
- Fix netlify.#Deploy (there's still FIXMEs)
- Externalize the `deploy.sh` script
- Add tests
- Misc engine fixes for more explicit error messages

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-20 16:10:14 -08:00
Andrea Luzzardi
568cd194a4 tests: Basic test infrastructure for Europa Universe
- Each package can include its own bats file (e.g. `universe.dagger.io/yarn/test/yarn.bats`)
- universe.dagger.io includes a common bash helper file
- bats is installed/launched through yarn with minimal setup
- shellcheck is done across the entire repo
- Integrated into our CI

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-19 12:39:39 -08:00
Gerhard Lazu
c5c4fdac29
Fix running-workflow-name in bump_version-tag-release job
This will prevent the job waiting on itself to complete (which will not
happen until GitHub Actions kills the run).

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-18 22:02:28 +00:00
Gerhard Lazu
59e9fae189
Also release in the Trigger Release workflow
The first implementation of the trigger-release would not push a tag,
meaning that the Release workflow was not getting triggered. While we
could have changed the Release workflow to react on "Trigger Release"
workflow runs, the inter-dependency felt awkward and brittle:

    diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
    index b711c5cf..843fdb70 100644
    --- a/.github/workflows/release.yml
    +++ b/.github/workflows/release.yml
    @@ -7,10 +7,16 @@ on:
       push:
         tags:
         - v*
    +  workflow_run:
    +    workflows:
    +    - "Trigger Release"
    +    types:
    +    - completed

     jobs:
       goreleaser:
         runs-on: ubuntu-latest
    +    if: ${{ github.event.workflow_run.conclusion == 'success' }}
         defaults:
           run:
             shell: bash

Instead of doing the above, introducing duplication between "Trigger
Release" and "Release" seemed simpler from a cognitive perspective. Now,
releases are produced via the Release workflow when tags are pushed, and
also when releases are triggered via "Trigger Release", now renamed to
"Auto Release".

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-18 18:59:50 +00:00
Gerhard Lazu
fcdda0b072
Add GitHub Workflow that triggers a release at a regular time in the week
While talking with @aluzzardi, we thought that regular auto-releases
which happen with no intervention on our part would be a good idea. The
last Dagger release (0.1.0-alpha.31) was over 1 month ago, and there are
Europa-related changes which we want to make available in the Dagger
GitHub Action. We should never have more than 1 week of unreleased
changes. While more often is better, and we may need to tweak this later,
this is a decent starting point: release every Tuesday, 5pm UTC & 9am SFO.

We had to adjust the starting point slightly so that we do not start at
the top of the hour when there is high load on GitHub Actions (see the
inline comments for more details)

The workflow can also be triggered manually, and a custom tag can be
provided optionally. If no tag is provided, the last one will be
incremented as expected, e.g. v0.1.0-alpha.31 -> v0.1.0-alpha.32.
Before you get too carried away with custom tags, let's talk about the
unexpected side-effects which are not worth covering in this commit
message ("people over processes").

There is also a concurrency setting which will not prevent multiple
releases to be triggered, but at least these jobs will not run in
parallel. I looked into cancelling the current workflow if another one
of the same type is running, but I couldn't get it to work properly
within my 30 mins time-box so I stopped.

There is a lot to talk about our releases AFTER this gets merged, so
let's defer those conversations until we are happy with the first step
which I think is in the right direction.

Signed-off-by: Gerhard Lazu <gerhard@lazu.co.uk>
2022-01-14 20:28:45 +00:00
Andrea Luzzardi
200aca9f3d ci: handle non-release CUE versions
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2022-01-11 11:38:22 -08:00
Sam Alba
2601bc7de1 ci: moved website deploy in its own workflow to filter out paths
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-17 13:16:23 -08:00
Sam Alba
b082b1e5bc implemented integration tests for engine.#Pull + moved auth code separately for sharing code with other tasks later
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-17 10:07:33 -08:00
Sam Alba
40e1b28133 ci: reduced docs test suite timeout + disabled parallelization within a suite
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-16 17:10:49 -08:00
Sam Alba
e81feba213 ci: upgrade localstack to latest release
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-15 10:14:44 -08:00
Sam Alba
d03e345c52 ci: disabled several tests to debug CI issues in Docs tests
Signed-off-by: Sam Alba <samalba@users.noreply.github.com>
2021-12-14 15:09:44 -08:00
jffarge
4d07e2af40 docs: 🐛 remove --frozen-lockfile option to update yarn.lock
Signed-off-by: jffarge <jf@dagger.io>
2021-11-18 15:39:53 +01:00
Jonathan Hult
d0e7d9b564
Link to new CUE GitHub repository
https://github.com/cue-lang/cue/issues/1078

Signed-off-by: Jonathan Hult <Jonathan@JonathanHult.com>
2021-10-20 14:49:26 -04:00
Andrea Luzzardi
285fe3fe39 ci: disable GHA caching
Getting a 400 Bad Request from GH API

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-05 12:39:50 -07:00
Andrea Luzzardi
f792199c98 ci: change gha scope
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-05 12:19:30 -07:00
Andrea Luzzardi
cb5634329c ci: gha export with mode=max
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:56:08 -07:00
Andrea Luzzardi
541d3775c3 ci: gha lint
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:29:23 -07:00
Andrea Luzzardi
4cce3de9df ci: re-enable GHA caching
Reverts #875 since moby/buildkit#2276 is now closed.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-10-04 17:28:44 -07:00
Tom Chauveau
0fe8b14e78 Improve ArgoCD and fix CI
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-09-10 09:10:21 -07:00
Tom Chauveau
2a91ae1d80 Update argoCD test
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-09-10 09:10:21 -07:00
Guillaume de Rouville
6ef0cfe321 1008: Implement on LocalStack
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-09-01 11:19:16 +02:00
Tom Chauveau
b145d03192 Update CI workflow to test documentation
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-09-01 11:14:09 +02:00
Guillaume de Rouville
8e9a56a024 Add doc-test to CI - Add Cue dependency to integration CI test
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-09-01 11:12:41 +02:00
Guillaume de Rouville
74b1ac3632 ci: re-enable localstack tests
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-31 14:39:45 +02:00
Guillaume de Rouville
58209b87d1 Doc update + linting + fix CI typo + fix port issue ECR
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-23 17:44:34 +02:00
Guillaume de Rouville
f42a243692 Add Localstack inside CI
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-08-23 17:44:34 +02:00
jffarge
7e13614cb4 docs: 🚚 rename cypress job
Signed-off-by: jffarge <jf@dagger.io>
2021-08-23 10:13:27 +02:00
jffarge
868339e174 docs: init cypress test e2e
Signed-off-by: jffarge <jf@dagger.io>
2021-08-10 15:39:13 +02:00
Andrea Luzzardi
5cc1a0f84a ci: disable GHA cache for buildkit
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-08-06 15:08:12 +02:00
Andrea Luzzardi
67d5609aee support buildkit cache export/import
- Add support for buildkit cache export/import
- Enable GHA cache for universe test (using buildkit v0.9 GHA support)

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-28 14:28:20 +02:00
Andrea Luzzardi
304959b3f5 automatically detect vendored buildkit version
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-19 16:26:18 +02:00
Andrea Luzzardi
cc004eb0ef ci: run universe tests as a separate CI job
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-07-01 18:55:03 +02:00
Tom Chauveau
794f27f116 Update setup kind cluster action
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-06-30 16:10:46 +02:00
Andrea Luzzardi
ec83ed5047 docs: add netlify build hook
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-21 15:41:12 +02:00
Andrea Luzzardi
479dc50402 docs: disable deploying using dagger
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-21 15:19:26 +02:00
Guillaume de Rouville
d8eaaf925a Change markdown linter github action to a more updated one
Signed-off-by: Guillaume de Rouville <guillaume.derouville@gmail.com>
2021-06-14 10:59:42 +02:00
Andrea Luzzardi
6d5142aa67 docs: use make docs to re-generate the docs
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-08 18:57:53 -07:00
Andrea Luzzardi
5537ab33de ci: add universe linting
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-08 18:29:02 -07:00
Andrea Luzzardi
e9b952a882 docs: rename tools/daggosaurus to website
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-04 13:49:20 -07:00
Tom Chauveau
ce65a65ede feat(CI): setup kind local kubernetes cluster in CI
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-06-03 15:19:54 +02:00
Andrea Luzzardi
ca03cd6e58
Merge pull request #526 from aluzzardi/docs-dogfood
docs: deploy using dagger
2021-06-02 16:49:09 -07:00
Andrea Luzzardi
335e2f7c24 docs: switch release flow to dagger
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-02 15:53:22 -07:00
Andrea Luzzardi
b76a4a2264 docs: enable dependabot
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-02 15:52:03 -07:00
Andrea Luzzardi
e86cb7be6d
Merge pull request #535 from aluzzardi/dagger-homedir
use ~/.config/dagger rather than ~/.dagger
2021-06-01 13:23:43 -07:00
Andrea Luzzardi
c5c586ff71 use ~/.config/dagger rather than ~/.dagger
since `.dagger` directories have a special meaning now because of gitflow,
it's better not to have a `~/.dagger` since it's not a workspace and
it confuses dagger (e.g. `dagger new` from $HOME).

We don't store state there anymore, just keys and the last version
check, so it's okay to be in ~/.config IMO

Looking at my system, in ~/.config there's `gcloud`, `gatsby`, `gh`,
`yarn`, and others so it seems like a pretty common location.

Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-06-01 12:38:14 -07:00
Tom Chauveau
242b61972c Update sops version to fix CI
Signed-off-by: Tom Chauveau <tom.chauveau@epitech.eu>
2021-05-29 19:40:01 +02:00
Andrea Luzzardi
868fd018f4 tests: fix remaining tests due to secrets
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
2021-05-28 14:00:27 -07:00