kjuulh/octopush
1
0
Fork 0
Code Issues 1 Pull Requests 2 Packages Projects Releases Wiki Activity

v0.2 #14

Merged
kjuulh merged 2 commits from v0.2 into main 2022-09-21 22:55:52 +02:00
Conversation 0 Commits 2 Files Changed 48 +938 -478
48 changed files with 938 additions and 478 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CONFIGURATION_SERVER.md Normal file
View File

@ -0,0 +1 @@
# Configuration server

249
README.md Normal file
View File

@ -0,0 +1,249 @@
<p align="center">
<image src="https://git.front.kjuulh.io/kjuulh/octopush/raw/branch/v0.2/assets/octopush.svg" width="300" height="300"/>
</p>
<h1 align="center">Octopush - Your cute action executor</h1>
## Purpose
The goal of this project is to easily do batch changes or queries on a host of
repositories. In large organisations using multi-repository strategies, it may
be painful to change even small things across many repositories, because there
are so many of them. Octopush aims to change that.
**DISCLAIMER:** It is still early days, and the api is subject to change.
## Features
- Uses an actions repository, where you store all your pending commands or
queries to be performed across your fleet of repositories. (See \_examples)
- Actions can both execute changes, open pull-requests or in some cases commit
directly to your preferred branch
- Actions natively use either shell, go or docker files to execute changes
(see \_examples/actions)
- Actions can also be analytical, so you can query your fleet for whatever you
would like
- Works both as a client, or as a server
- Supports SSH/https for fetching repos
- Supports GPG signing
- Supports dry-run mode for easy testing when developing your actions (enabled
by default on the cli)
## Roadmap
Refer to [roadmap.md](roadmap.md)
## Installation
Octopush comes in two modes. Client or Client -> Server. Octopush can stand alone as
a client, for smaller and less secure changes. However, for organisations, it
may be useful to use Octopush in server mode, which supports more features, and
has extra security built in.
### Client (CLI)
Download executable from [releases](https://github.com/kjuulh/octopush/releases)
#### Or Use docker image
```bash
docker run --rm kasperhermansen/octopushcli:latest version
```
#### Or Build from source
```bash
git clone https://github.com/kjuulh/octopush.git
cd octopush
go build cmd/octopush/octopush.go
./octopush version
```
#### Or Build with cuddle
```bash
git clone https://github.com/kjuulh/octopush.git
cd octopush
cuddle_cli x build_cli
```
### Server
We prefer to run the server directly as a docker image.
```bash
docker pull kasperhermansen/octopushserver:latest
docker run -p 9090:80 --rm kasperhermansen/octopushserver:latest
```
#### Or Build from source
```bash
git clone https://github.com/kjuulh/octopush.git
cd octopush
go build cmd/server/server.go
./server version
```
#### Or Build with cuddle
```bash
git clone https://github.com/kjuulh/octopush.git
cd octopush
cuddle_cli x build_server
```
## Usage
**DISCLAIMER:** It is still early days, and the api of the CLI is subject to
change, this provides the aim of the project, but as it is currently in flux,
there may not be as much handholding in the actual usage.
I will focus on the client here, as the server provides the same features,
though available through the cli, but instead as configuration options (see
[CONFIGURATION_SERVER.md](CONFIGURATION_SERVER.md))
Octopush ships with autocomplete built in (courtesy of spf13/cobra). To add:
- Bash: `echo 'source <(octopush completion bash)' >> ~/.bashrc`
- Zsh: `echo 'source <(octopush completion zsh)' >> ~/.zshrc`
### Creating a new action
Creating a new action
```bash
git init my-actions # should only be done once
cd my-actions
octopush tmpl init write-a-readme --command
cat write-a-readme/octopush.yml
# Output
# apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
# name: write-a-readme
# select:
# repositories: []
# actions:
# - type: shell
# entry: "main.sh"
```
Octopush also ships with yaml schema, which should help write the yaml
configuration.
#### Add upstream repositories (victims)
Now add a preferred repository
```
cat << EOF > write-a-readme/octopush.yml
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
providers: # new
- gitea: https://git.front.kjuulh.io # new
organisation: "kjuulh" # new
actions:
- type: shell
entry: "main.sh"
EOF
```
This will take all your repositories under an organisation and run the script
on.
Another could be to use
```bash
cat << EOF > write-a-readme/octopush.yml
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories: #new
- git@git.front.kjuulh.io:kjuulh/octopush.git #new
- git@git.front.kjuulh.io:kjuulh/octopush-test.git #new
actions:
- type: shell
entry: "main.sh"
EOF
```
This will just apply to those repositories instead. Both can also be combined
for a shared effect.
### Execute action
To run the script use
```bash
octopush process --path "write-a-readme"
```
This will cause the octopush process to automatically apply the action on the repo
and open a pr.
### Query repositories
Octopush can also be used to query.
```bash
cat << EOF > write-a-readme/octopush.yml
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories:
- git@git.front.kjuulh.io:kjuulh/octopush.git
- git@git.front.kjuulh.io:kjuulh/octopush-test.git
queries:
- type: grep
query: "# README"
EOF
```
Using the same command as above, will return the lines on each repo with those
criteria. Everything is run in docker, even locally, so no need to install fancy
tools.
Do note: All actions will be run as dry-run unless `--apply` is added. This is
to help test locally, as well as not cause serious issues. The server
configuration is pretty much the same, except the command would look like so:
`octopush server process --path "write-a-readme" --apply`. Octopush will try to
infer as much as possible, but it may be needed to apply some extra flags to
specify upstream repositories and such. Octopush will also help you setup keys and
such on the first run, using `octopush setup` or `octopush server setup`.
## Contributing
It is still early days, and as such things are moving fast, I may not be able to
implement features, because I am focusing my energy on the API. That said PRs
are welcome, though they are at your own risk.
### Bugs & features requests
Please use [issues](https://github.com/kjuulh/octopush/issues)
### Development
We use [cuddle](https://git.front.kjuulh.io/kjuulh/cuddle) to improve ease of
use, it is however, not a requirement, and probably won't need to be used
outside core maintainers.
Simply:
```bash
go run cmd/octopush/octopush.go # CLI
go run cmd/server/server.go # Server
```
We follow the `gofmt` formatting, along with optionally but recommend `golines`
If using cuddle
```
cuddle_cli x run # Run both server and client, will do a quick test sweep on the cli
cuddle_cli x watch_run # Automatically refresh both
cuddle_cli x fmt # will format the current code
```

11
_examples/actions/add_releaserc/go.mod Normal file
View File

@ -0,0 +1,11 @@
module write_a_readme
go 1.19
require github.com/bitfield/script v0.20.2
require (
bitbucket.org/creachadair/shell v0.0.7 // indirect
github.com/itchyny/gojq v0.12.7 // indirect
github.com/itchyny/timefmt-go v0.1.3 // indirect
)

20
_examples/actions/add_releaserc/go.sum Normal file
View File

@ -0,0 +1,20 @@
bitbucket.org/creachadair/shell v0.0.7 h1:Z96pB6DkSb7F3Y3BBnJeOZH2gazyMTWlvecSD4vDqfk=
bitbucket.org/creachadair/shell v0.0.7/go.mod h1:oqtXSSvSYr4624lnnabXHaBsYW6RD80caLi2b3hJk0U=
github.com/bitfield/script v0.20.2 h1:4DexsRtBILVMEn3EZwHbtJdDqdk43sXI8gM3F04JXgs=
github.com/bitfield/script v0.20.2/go.mod h1:l3AZPVAtKQrL03bwh7nlNTUtgrgSWurpJSbtqspYrOA=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
github.com/itchyny/gojq v0.12.7 h1:hYPTpeWfrJ1OT+2j6cvBScbhl0TkdwGM4bc66onUSOQ=
github.com/itchyny/gojq v0.12.7/go.mod h1:ZdvNHVlzPgUf8pgjnuDTmGfHA/21KoutQUJ3An/xNuw=
github.com/itchyny/timefmt-go v0.1.3 h1:7M3LGVDsqcd0VZH2U+x393obrzZisp7C0uEe921iRkU=
github.com/itchyny/timefmt-go v0.1.3/go.mod h1:0osSSCQSASBJMsIZnhAaF1C2fCBTJZXrnj37mG8/c+A=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

25
_examples/actions/add_releaserc/main.go Normal file
View File

@ -0,0 +1,25 @@
package main
import "github.com/bitfield/script"
func main() {
releaseRc := `
branches:
- "main"
- "v0.x"
plugins:
- "@semantic-release/commit-analyzer"
- "@semantic-release/release-notes-generator"
- "@semantic-release/changelog"
- "@semantic-release/git"
`
_, err := script.
Echo(releaseRc).
WriteFile(".releaserc.yml")
if err != nil {
panic(err)
}
}

12
_examples/actions/add_releaserc/octopush.yml Normal file
View File

@ -0,0 +1,12 @@
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories:
- git@git.front.kjuulh.io:kjuulh/octopush-test.git
#- git@git.front.kjuulh.io:kjuulh/octopush.git
# providers:
# - gitea: https://git.front.kjuulh.io
# organisation: "cibus"
actions:
- type: go
entry: "main.go"

2
_examples/actions/docker_action/Dockerfile
View File

@ -1,6 +1,6 @@
FROM debian:bullseye-slim
# Kraken relies on this path being the specified path
# Octopush relies on this path being the specified path
WORKDIR /src/work/
COPY entry.sh /src/script.sh

4
_examples/actions/docker_action/kraken.yml → _examples/actions/docker_action/octopush.yml
View File

@ -1,8 +1,8 @@
apiVersion: git.front.kjuulh.io/kjuulh/kraken/blob/main/schema/v1
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories:
- git@git.front.kjuulh.io:kjuulh/kraken-test.git
- git@git.front.kjuulh.io:kjuulh/octopush-test.git
# providers:
# - gitea: https://git.front.kjuulh.io
# organisation: "cibus"

4
_examples/actions/write_a_readme/kraken.yml → _examples/actions/write_a_readme/octopush.yml
View File

@ -1,8 +1,8 @@
apiVersion: git.front.kjuulh.io/kjuulh/kraken/blob/main/schema/v1
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories:
- git@git.front.kjuulh.io:kjuulh/kraken-test.git
- git@git.front.kjuulh.io:kjuulh/octopush-test.git
# providers:
# - gitea: https://git.front.kjuulh.io
# organisation: "cibus"

9
_examples/queries/find_semantic/octopush.yml Normal file
View File

@ -0,0 +1,9 @@
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
providers:
- gitea: https://git.front.kjuulh.io
organisation: "cibus"
queries:
- type: grep
query: "releaser"

4
_examples/queries/scrape_readme/kraken.yml → _examples/queries/scrape_readme/octopush.yml
View File

@ -1,8 +1,8 @@
apiVersion: git.front.kjuulh.io/kjuulh/kraken/blob/main/schema/v1
apiVersion: git.front.kjuulh.io/kjuulh/octopush/blob/main/schema/v1
name: write-a-readme
select:
repositories:
- git@git.front.kjuulh.io:kjuulh/kraken-test.git
- git@git.front.kjuulh.io:kjuulh/octopush-test.git
# providers:
# - gitea: https://git.front.kjuulh.io
# organisation: "cibus"

BIN
assets/octopush.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

10
assets/octopush.svg Normal file
View File

@ -0,0 +1,10 @@
<svg width="1000" height="1000" viewBox="0 0 1000 1000" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M975 568C975 830.335 762.335 497 500 497C237.665 497 25 830.335 25 568C25 305.665 237.665 93 500 93C762.335 93 975 305.665 975 568Z" fill="#52DB78"/>
<path d="M237.436 856.185C165.693 827.571 320.875 755.464 377.754 612.854C434.634 470.243 371.672 311.132 443.414 339.746C515.157 368.36 527.205 507.165 470.326 649.776C413.446 792.386 309.178 884.799 237.436 856.185Z" fill="#52DB78"/>
<path d="M428.353 849.847C353.918 829.229 500.308 740.627 541.293 592.663C582.278 444.699 502.337 293.405 576.773 314.023C651.208 334.641 678.325 471.303 637.34 619.267C596.355 767.231 502.788 870.465 428.353 849.847Z" fill="#52DB78"/>
<path d="M598.707 855.902C523.231 839.498 664.415 742.813 697.024 592.781C729.634 442.748 641.321 296.183 716.797 312.588C792.272 328.992 827.022 463.916 794.413 613.948C761.803 763.98 674.183 872.307 598.707 855.902Z" fill="#52DB78"/>
<path d="M47.896 838.855C-19.3259 800.817 144.17 750.323 219.784 616.698C295.397 483.073 254.495 316.918 321.717 354.956C388.938 392.994 382.136 532.155 306.522 665.78C230.909 799.405 115.118 876.893 47.896 838.855Z" fill="#52DB78"/>
<ellipse cx="605.35" cy="265.255" rx="57.5" ry="36.5" transform="rotate(-40.9544 605.35 265.255)" fill="white"/>
<ellipse cx="359.35" cy="265.255" rx="57.5" ry="36.5" transform="rotate(-40.9544 359.35 265.255)" fill="white"/>
<path d="M385.635 374C385.635 374 391.199 435.298 463.528 438.425C535.857 441.553 538.492 380.61 538.492 380.61" stroke="white" stroke-width="8"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

14
cmd/kraken/commands/root.go
View File

@ -1,14 +0,0 @@
package commands
import "github.com/spf13/cobra"
func CreateKrakenCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "kraken",
// Run: func(cmd *cobra.Command, args []string) { },
}
cmd.AddCommand(CreateKrakenProcessCmd())
return cmd
}

18
cmd/kraken/kraken.go
View File

@ -1,18 +0,0 @@
package main
import (
"os"
"git.front.kjuulh.io/kjuulh/kraken/cmd/kraken/commands"
)
func main() {
Execute()
}
func Execute() {
err := commands.CreateKrakenCmd().Execute()
if err != nil {
os.Exit(1)
}
}

61
cmd/octopush/commands/process.go Normal file
View File

@ -0,0 +1,61 @@
package commands
import (
"context"
"time"
"git.front.kjuulh.io/kjuulh/octopush/internal/cli"
"git.front.kjuulh.io/kjuulh/octopush/internal/commands"
"github.com/spf13/cobra"
"go.uber.org/zap"
)
func CreateOctopushProcessCmd(logger *zap.Logger) *cobra.Command {
var (
actionsRepo string
branch string
path string
)
cmd := &cobra.Command{
Use: "process",
RunE: func(cmd *cobra.Command, args []string) error {
if err := cmd.ParseFlags(args); err != nil {
return err
}
ctx := cmd.Context()
deps, cleanupFunc, err := cli.Start(ctx, logger)
if err != nil {
return err
}
defer func() {
ctx, _ = context.WithTimeout(ctx, time.Second*5)
if err := cleanupFunc(ctx); err != nil {
panic(err)
}
}()
err = commands.
NewProcessRepos(logger, deps).
Process(ctx, actionsRepo, branch, path)
if err != nil {
return err
}
return nil
},
}
pf := cmd.PersistentFlags()
pf.StringVar(&actionsRepo, "actions-repo", "", "actions repo is the location of your actions, not where to apply the actions themselves, that should be self contained")
cmd.MarkPersistentFlagRequired("actions-repo")
pf.StringVar(&branch, "branch", "main", "which branch to look for actions in, will default to main")
pf.StringVar(&path, "path", "", "the location of the path inside the repository")
cmd.MarkPersistentFlagRequired("path")
return cmd
}

18
cmd/octopush/commands/root.go Normal file
View File

@ -0,0 +1,18 @@
package commands
import (
"git.front.kjuulh.io/kjuulh/octopush/cmd/octopush/commands/server"
"github.com/spf13/cobra"
"go.uber.org/zap"
)
func CreateOctopushCmd(logger *zap.Logger) *cobra.Command {
cmd := &cobra.Command{
Use: "octopush",
}
cmd.AddCommand(CreateOctopushProcessCmd(logger))
cmd.AddCommand(server.CreateOctopushServerCmd(logger))
return cmd
}

4
cmd/kraken/commands/process.go → cmd/octopush/commands/server/process.go
View File

@ -1,4 +1,4 @@
package commands
package server
import (
"bytes"
@ -8,7 +8,7 @@ import (
"github.com/spf13/cobra"
)
func CreateKrakenProcessCmd() *cobra.Command {
func CreateOctopushProcessCmd() *cobra.Command {
var (
actionsRepo string

16
cmd/octopush/commands/server/server.go Normal file
View File

@ -0,0 +1,16 @@
package server
import (
"github.com/spf13/cobra"
"go.uber.org/zap"
)
func CreateOctopushServerCmd(logger *zap.Logger) *cobra.Command {
cmd := &cobra.Command{
Use: "server",
}
cmd.AddCommand(CreateOctopushProcessCmd())
return cmd
}

28
cmd/octopush/octopush.go Normal file
View File

@ -0,0 +1,28 @@
package main
import (
"os"
"git.front.kjuulh.io/kjuulh/octopush/cmd/octopush/commands"
"git.front.kjuulh.io/kjuulh/octopush/internal/logger"
"go.uber.org/zap"
)
func main() {
logger, err := logger.New()
if err != nil {
panic(err)
}
_ = logger.Sync()
zap.ReplaceGlobals(logger)
Execute(logger)
}
func Execute(logger *zap.Logger) {
err := commands.CreateOctopushCmd(logger).Execute()
if err != nil {
os.Exit(1)
}
}

2
cmd/server/commands/root.go
View File

@ -7,7 +7,7 @@ import (
func CreateServerCmd(logger *zap.Logger) *cobra.Command {
cmd := &cobra.Command{
Use: "krakenserver",
Use: "octopushserver",
}
cmd.AddCommand(NewStartServerCommand(logger))

4
cmd/server/commands/start.go
View File

@ -1,7 +1,7 @@
package commands
import (
"git.front.kjuulh.io/kjuulh/kraken/internal/server"
"git.front.kjuulh.io/kjuulh/octopush/internal/server"
"github.com/spf13/cobra"
"go.uber.org/zap"
)
@ -9,7 +9,7 @@ import (
func NewStartServerCommand(logger *zap.Logger) *cobra.Command {
cmd := &cobra.Command{
Use: "start",
Short: "Start the kraken server",
Short: "Start the octopush server",
RunE: func(cmd *cobra.Command, args []string) error {
return server.Start(logger)
},

4
cmd/server/server.go
View File

@ -3,8 +3,8 @@ package main
import (
"os"
"git.front.kjuulh.io/kjuulh/kraken/cmd/server/commands"
"git.front.kjuulh.io/kjuulh/kraken/internal/logger"
"git.front.kjuulh.io/kjuulh/octopush/cmd/server/commands"
"git.front.kjuulh.io/kjuulh/octopush/internal/logger"
"go.uber.org/zap"
)

2
cuddle.yaml
View File

@ -3,7 +3,7 @@
base: "git@git.front.kjuulh.io:kjuulh/cuddle-go-plan.git"
vars:
service: "kraken"
service: "octopush"
deployments: "git@git.front.kjuulh.io:kjuulh/deployments.git"
scripts:

14
go.mod
View File

@ -1,25 +1,25 @@
module git.front.kjuulh.io/kjuulh/kraken
module git.front.kjuulh.io/kjuulh/octopush
go 1.19
require (
git.front.kjuulh.io/kjuulh/curre v1.2.2
code.gitea.io/sdk/gitea v0.15.1
git.front.kjuulh.io/kjuulh/curre v1.3.5
github.com/ProtonMail/go-crypto v0.0.0-20220822140716-1678d6eb0cbe
github.com/ProtonMail/gopenpgp/v2 v2.4.10
github.com/gin-contrib/zap v0.0.2
github.com/gin-gonic/gin v1.8.1
github.com/go-git/go-git/v5 v5.4.2
github.com/google/uuid v1.3.0
github.com/spf13/cobra v1.5.0
github.com/stretchr/testify v1.8.0
github.com/whilp/git-urls v1.0.0
go.uber.org/zap v1.23.0
golang.org/x/net v0.0.0-20220909164309-bea034e7d591
gopkg.in/yaml.v3 v3.0.1
)
require (
code.gitea.io/sdk/gitea v0.15.1 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/ProtonMail/go-mime v0.0.0-20220302105931-303f85f7fe0f // indirect
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/cloudflare/circl v1.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
@ -43,13 +43,10 @@ require (
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421 // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/sergi/go-diff v1.2.0 // indirect
github.com/sirupsen/logrus v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/ugorji/go/codec v1.2.7 // indirect
github.com/whilp/git-urls v1.0.0 // indirect
github.com/xanzy/ssh-agent v0.3.2 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/multierr v1.8.0 // indirect
@ -59,5 +56,4 @@ require (
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

23
go.sum
View File

@ -1,9 +1,8 @@
code.gitea.io/gitea-vet v0.2.1/go.mod h1:zcNbT/aJEmivCAhfmkHOlT645KNOf9W2KnkLgFjGGfE=
code.gitea.io/sdk/gitea v0.15.1 h1:WJreC7YYuxbn0UDaPuWIe/mtiNKTvLN8MLkaw71yx/M=
code.gitea.io/sdk/gitea v0.15.1/go.mod h1:klY2LVI3s3NChzIk/MzMn7G1FHrfU7qd63iSMVoHRBA=
git.front.kjuulh.io/kjuulh/curre v1.2.2 h1:0OwWIfekrMykdQg9bdmG80I+Mjc2k4i+sy903phuDWs=
git.front.kjuulh.io/kjuulh/curre v1.2.2/go.mod h1:m7WpSehONLqPh/XF3F0BI0UOpLOfGuDmDEFI1XsM6fE=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
git.front.kjuulh.io/kjuulh/curre v1.3.5 h1:oKYh5Z0vInjViLnS4ppzK0G2Mnj7vXq8mA5i/rsWId4=
git.front.kjuulh.io/kjuulh/curre v1.3.5/go.mod h1:m7WpSehONLqPh/XF3F0BI0UOpLOfGuDmDEFI1XsM6fE=
github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA=
github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0=
github.com/Microsoft/go-winio v0.5.2 h1:a9IhgEQBCUEk6QCdml9CiJGhAws+YwffDHEMp1VMrpA=
@ -11,10 +10,6 @@ github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v
github.com/ProtonMail/go-crypto v0.0.0-20210428141323-04723f9f07d7/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo=
github.com/ProtonMail/go-crypto v0.0.0-20220822140716-1678d6eb0cbe h1:R2HeCk7SG/XpoYZlEeI1v7sId7w2AMWwzOaVqXn45FE=
github.com/ProtonMail/go-crypto v0.0.0-20220822140716-1678d6eb0cbe/go.mod h1:UBYPn8k0D56RtnR8RFQMjmh4KrZzWJ5o7Z9SYjossQ8=
github.com/ProtonMail/go-mime v0.0.0-20220302105931-303f85f7fe0f h1:CGq7OieOz3wyQJ1fO8S0eO9TCW1JyvLrf8fhzz1i8ko=
github.com/ProtonMail/go-mime v0.0.0-20220302105931-303f85f7fe0f/go.mod h1:NYt+V3/4rEeDuaev/zw1zCq8uqVEuPHzDPo3OZrlGJ4=
github.com/ProtonMail/gopenpgp/v2 v2.4.10 h1:EYgkxzwmQvsa6kxxkgP1AwzkFqKHscF2UINxaSn6rdI=
github.com/ProtonMail/gopenpgp/v2 v2.4.10/go.mod h1:CTRA7/toc/4DxDy5Du4hPDnIZnJvXSeQ8LsRTOUJoyc=
github.com/acomagu/bufpipe v1.0.3 h1:fxAGrHZTgQ9w5QqVItgzwj235/uYZYgbXitB+dLupOk=
github.com/acomagu/bufpipe v1.0.3/go.mod h1:mxdxdup/WdsKVreO5GpW4+M/1CE2sMG4jeGJ2sYmHc4=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 h1:kFOfPq6dUM1hTo4JG6LR5AXSUEsOjtdm0kw0FtQtMJA=
@ -130,8 +125,6 @@ github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNX
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.7.0 h1:ShrD1U9pZB12TX0cVy0DtePoCH97K8EtX+mg7ZARUtM=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/spf13/cobra v1.5.0 h1:X+jTBEBqF0bHN+9cSMgmfuvv2VHJ9ezmFNf9Y/XstYU=
github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM=
@ -173,7 +166,6 @@ go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY=
go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY=
golang.org/x/crypto v0.0.0-20190219172222-a4c6cb3142f2/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
@ -183,15 +175,7 @@ golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5y
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90 h1:Y/gsMcFOcR+6S6f3YeMKl5g+dZMEWqcz5Czj/GWYbkM=
golang.org/x/crypto v0.0.0-20220829220503-c86fa9a7ed90/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/exp v0.0.0-20190731235908-ec7cb31e5a56/go.mod h1:JhuoJpWY28nO4Vef9tZUw9qufEGTyX1+7lmHxV5q5G4=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20200801112145-973feb4309de/go.mod h1:skQtrUTUwhdJvXM/2KKJzY8pDgNr9I/FOMqDVRPBUS4=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191209134235-331c550502dd/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
@ -210,7 +194,6 @@ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@ -241,9 +224,7 @@ golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200117012304-6edc0a871e69/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200325010219-a49f79bcc224/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

2
integration_test/main_test.go
View File

@ -7,7 +7,7 @@ import (
"os"
"testing"
"git.front.kjuulh.io/kjuulh/kraken/internal/server"
"git.front.kjuulh.io/kjuulh/octopush/internal/server"
"go.uber.org/zap"
)

2
integration_test/storage_test.go
View File

@ -9,7 +9,7 @@ import (
"path"
"testing"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"github.com/stretchr/testify/require"
)

10
internal/actions/action.go
View File

@ -4,15 +4,15 @@ import (
"context"
"errors"
"git.front.kjuulh.io/kjuulh/kraken/internal/actions/builders"
"git.front.kjuulh.io/kjuulh/kraken/internal/actions/querier"
"git.front.kjuulh.io/kjuulh/kraken/internal/schema"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"git.front.kjuulh.io/kjuulh/octopush/internal/actions/builders"
"git.front.kjuulh.io/kjuulh/octopush/internal/actions/querier"
"git.front.kjuulh.io/kjuulh/octopush/internal/schema"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"go.uber.org/zap"
)
type Action struct {
Schema *schema.KrakenSchema
Schema *schema.OctopushSchema
SchemaPath string
}

16
internal/actions/action_creator.go
View File

@ -7,9 +7,9 @@ import (
"path"
"time"
"git.front.kjuulh.io/kjuulh/kraken/internal/schema"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/providers"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"git.front.kjuulh.io/kjuulh/octopush/internal/schema"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/providers"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"go.uber.org/zap"
)
@ -23,12 +23,12 @@ type (
ActionCreator struct {
logger *zap.Logger
storage *storage.Service
git *providers.Git
git *providers.GoGit
}
ActionCreatorDeps interface {
GetStorageService() *storage.Service
GetGitProvider() *providers.Git
GetGitProvider() *providers.GoGit
}
)
@ -59,19 +59,19 @@ func (ac *ActionCreator) Prepare(ctx context.Context, ops *ActionCreatorOps) (*A
return nil, fmt.Errorf("path is invalid: %s", ops.Path)
}
contents, err := os.ReadFile(path.Join(executorUrl, "kraken.yml"))
contents, err := os.ReadFile(path.Join(executorUrl, "octopush.yml"))
if err != nil {
return nil, err
}
krakenSchema, err := schema.Unmarshal(string(contents))
octopushSchema, err := schema.Unmarshal(string(contents))
if err != nil {
return nil, err
}
ac.logger.Debug("Action creator done")
return &Action{
Schema: krakenSchema,
Schema: octopushSchema,
SchemaPath: executorUrl,
}, nil
}

4
internal/actions/builders/docker.go
View File

@ -36,7 +36,7 @@ func (g *DockerBuild) Build(ctx context.Context, modulePath, entryPath string) (
return nil, err
}
tag := hex.EncodeToString(b)
buildDockerCmd := fmt.Sprintf("(cd %s; docker build -f %s --tag kraken/%s .)", modulePath, entryPath, tag)
buildDockerCmd := fmt.Sprintf("(cd %s; docker build -f %s --tag octopush/%s .)", modulePath, entryPath, tag)
g.logger.Debug("Running command", zap.String("command", buildDockerCmd))
cmd := exec.CommandContext(
@ -73,7 +73,7 @@ func (g *DockerBuild) Build(ctx context.Context, modulePath, entryPath string) (
ctx,
"/bin/bash",
"-c",
fmt.Sprintf("docker run --rm -v %s/:/src/work/ kraken/%s", victimPath, tag),
fmt.Sprintf("docker run --rm -v %s/:/src/work/ octopush/%s", victimPath, tag),
)
runDockerWriter := &zapio.Writer{

6
internal/api/process_command.go
View File

@ -4,9 +4,9 @@ import (
"context"
"net/http"
"git.front.kjuulh.io/kjuulh/kraken/internal/commands"
"git.front.kjuulh.io/kjuulh/kraken/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/jobs"
"git.front.kjuulh.io/kjuulh/octopush/internal/commands"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/jobs"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
"go.uber.org/zap"

2
internal/api/root.go
View File

@ -1,7 +1,7 @@
package api
import (
"git.front.kjuulh.io/kjuulh/kraken/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)

30
internal/cli/cli.go Normal file
View File

@ -0,0 +1,30 @@
package cli
import (
"context"
"git.front.kjuulh.io/kjuulh/curre"
"git.front.kjuulh.io/kjuulh/octopush/internal/server"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/signer"
"go.uber.org/zap"
)
func Start(ctx context.Context, logger *zap.Logger) (*serverdeps.ServerDeps, curre.CleanupFunc, error) {
deps := serverdeps.NewServerDeps(logger)
readyChan := make(chan curre.ComponentsAreReady, 1)
cleanupFunc, err := curre.NewManager().
Register(
server.NewStorageServer(logger.With(zap.Namespace("storage")), deps),
).
Register(
signer.NewOpenPGPApp(deps.GetOpenPGP()),
).
RunNonBlocking(ctx, readyChan)
<-readyChan
return deps, cleanupFunc, err
}

25
internal/commands/process_repos.go
View File

@ -7,11 +7,11 @@ import (
"sync"
"time"
"git.front.kjuulh.io/kjuulh/kraken/internal/actions"
"git.front.kjuulh.io/kjuulh/kraken/internal/gitproviders"
"git.front.kjuulh.io/kjuulh/kraken/internal/schema"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/providers"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"git.front.kjuulh.io/kjuulh/octopush/internal/actions"
"git.front.kjuulh.io/kjuulh/octopush/internal/gitproviders"
"git.front.kjuulh.io/kjuulh/octopush/internal/schema"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/providers"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
giturls "github.com/whilp/git-urls"
"go.uber.org/zap"
)
@ -20,14 +20,14 @@ type (
ProcessRepos struct {
logger *zap.Logger
storage *storage.Service
git *providers.Git
git *providers.GoGit
actionCreator *actions.ActionCreator
gitea *gitproviders.Gitea
}
ProcessReposDeps interface {
GetStorageService() *storage.Service
GetGitProvider() *providers.Git
GetGitProvider() *providers.GoGit
GetActionCreator() *actions.ActionCreator
GetGitea() *gitproviders.Gitea
}
@ -79,7 +79,7 @@ func (pr *ProcessRepos) Process(ctx context.Context, repository string, branch s
return nil
}
func (pr *ProcessRepos) getRepoUrls(ctx context.Context, schema *schema.KrakenSchema) ([]string, error) {
func (pr *ProcessRepos) getRepoUrls(ctx context.Context, schema *schema.OctopushSchema) ([]string, error) {
repoUrls := make([]string, 0)
repoUrls = append(repoUrls, schema.Select.Repositories...)
@ -161,7 +161,7 @@ func (pr *ProcessRepos) prepareAction(
return cleanupfunc, area, nil
}
func (pr *ProcessRepos) clone(ctx context.Context, area *storage.Area, repoUrl string) (*providers.GitRepo, error) {
func (pr *ProcessRepos) clone(ctx context.Context, area *storage.Area, repoUrl string) (*providers.GoGitRepo, error) {
pr.logger.Debug("Cloning repo", zap.String("path", area.Path), zap.String("repoUrl", repoUrl))
cloneCtx, _ := context.WithTimeout(ctx, time.Second*5)
repo, err := pr.git.Clone(cloneCtx, area, repoUrl)
@ -177,7 +177,7 @@ func (pr *ProcessRepos) clone(ctx context.Context, area *storage.Area, repoUrl s
return repo, nil
}
func (pr *ProcessRepos) commit(ctx context.Context, area *storage.Area, repo *providers.GitRepo, repoUrl string) error {
func (pr *ProcessRepos) commit(ctx context.Context, area *storage.Area, repo *providers.GoGitRepo, repoUrl string) error {
wt, err := pr.git.Add(ctx, area, repo)
if err != nil {
return fmt.Errorf("could not add file: %w", err)
@ -189,8 +189,9 @@ func (pr *ProcessRepos) commit(ctx context.Context, area *storage.Area, repo *pr
}
if status.IsClean() {
// TODO: check for pr
pr.logger.Info("Returning early, as no modifications are detected")
return nil
//return nil
}
err = pr.git.Commit(ctx, repo)
@ -231,7 +232,7 @@ func (pr *ProcessRepos) commit(ctx context.Context, area *storage.Area, repo *pr
return err
}
err = pr.gitea.CreatePr(ctx, fmt.Sprintf("%s://%s", "https", url.Host), org, semanticName, head, originHead, "kraken-apply")
err = pr.gitea.CreatePr(ctx, fmt.Sprintf("%s://%s", "https", url.Host), org, semanticName, head, originHead, "octopush-apply")
if err != nil {
return err
}

3
internal/logger/zap.go
View File

@ -12,7 +12,7 @@ func New() (*zap.Logger, error) {
return lvl >= zapcore.ErrorLevel
})
lowPriority := zap.LevelEnablerFunc(func(lvl zapcore.Level) bool {
return lvl < zapcore.ErrorLevel
return lvl < zapcore.ErrorLevel // && lvl > zapcore.DebugLevel
})
config := zap.NewDevelopmentEncoderConfig()
@ -28,5 +28,6 @@ func New() (*zap.Logger, error) {
)
logger := zap.New(core)
return logger, nil
}

6
internal/schema/kraken.go
View File

@ -2,7 +2,7 @@ package schema
import "gopkg.in/yaml.v3"
type KrakenSchema struct {
type OctopushSchema struct {
ApiVersion string `yaml:"apiVersion"`
Name string `yaml:"name"`
Select struct {
@ -22,8 +22,8 @@ type KrakenSchema struct {
} `yaml:"queries"`
}
func Unmarshal(raw string) (*KrakenSchema, error) {
k := &KrakenSchema{}
func Unmarshal(raw string) (*OctopushSchema, error) {
k := &OctopushSchema{}
err := yaml.Unmarshal([]byte(raw), k)
if err != nil {
return nil, err

4
internal/server/http_server.go
View File

@ -7,8 +7,8 @@ import (
"time"
"git.front.kjuulh.io/kjuulh/curre"
"git.front.kjuulh.io/kjuulh/kraken/internal/api"
"git.front.kjuulh.io/kjuulh/kraken/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/api"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
ginzap "github.com/gin-contrib/zap"
"github.com/gin-gonic/gin"
"go.uber.org/zap"

4
internal/server/server.go
View File

@ -4,8 +4,8 @@ import (
"context"
"git.front.kjuulh.io/kjuulh/curre"
"git.front.kjuulh.io/kjuulh/kraken/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/signer"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/signer"
"go.uber.org/zap"
)

2
internal/server/storage_server.go
View File

@ -5,7 +5,7 @@ import (
"time"
"git.front.kjuulh.io/kjuulh/curre"
"git.front.kjuulh.io/kjuulh/kraken/internal/serverdeps"
"git.front.kjuulh.io/kjuulh/octopush/internal/serverdeps"
"go.uber.org/zap"
)

14
internal/serverdeps/server_deps.go
View File

@ -1,12 +1,12 @@
package serverdeps
import (
actionc "git.front.kjuulh.io/kjuulh/kraken/internal/actions"
"git.front.kjuulh.io/kjuulh/kraken/internal/gitproviders"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/actions"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/providers"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/signer"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
actionc "git.front.kjuulh.io/kjuulh/octopush/internal/actions"
"git.front.kjuulh.io/kjuulh/octopush/internal/gitproviders"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/actions"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/providers"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/signer"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"go.uber.org/zap"
)
@ -53,7 +53,7 @@ func (deps *ServerDeps) GetStorageService() *storage.Service {
return storage.NewService(deps.logger.With(zap.Namespace("storage")), deps.storageConfig)
}
func (deps *ServerDeps) GetGitProvider() *providers.Git {
func (deps *ServerDeps) GetGitProvider() *providers.GoGit {
return providers.NewGit(deps.logger.With(zap.Namespace("gitProvider")), deps.gitCfg, deps.openPGP)
}

2
internal/services/actions/action.go
View File

@ -3,7 +3,7 @@ package actions
import (
"context"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"go.uber.org/zap"
)

338
internal/services/providers/git.go
View File

@ -1,339 +1 @@
package providers
import (
"context"
"errors"
"fmt"
"time"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/signer"
"git.front.kjuulh.io/kjuulh/kraken/internal/services/storage"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/config"
"github.com/go-git/go-git/v5/plumbing"
"github.com/go-git/go-git/v5/plumbing/object"
"github.com/go-git/go-git/v5/plumbing/transport"
"github.com/go-git/go-git/v5/plumbing/transport/http"
"github.com/go-git/go-git/v5/plumbing/transport/ssh"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
)
// Git is a native git provider, it can clone, pull
// , push and as in abstraction on native git operations
type Git struct {
logger *zap.Logger
gitConfig *GitConfig
openPGP *signer.OpenPGP
}
type GitRepo struct {
repo *git.Repository
}
func (gr *GitRepo) GetHEAD() (string, error) {
head, err := gr.repo.Head()
if err != nil {
return "", err
}
return head.Name().Short(), nil
}
type GitAuth string
const (
GIT_AUTH_SSH GitAuth = "ssh"
GIT_AUTH_USERNAME_PASSWORD GitAuth = "username_password"
GIT_AUTH_ACCESS_TOKEN GitAuth = "access_token"
GIT_AUTH_ANONYMOUS GitAuth = "anonymous"
GIT_AUTH_SSH_AGENT GitAuth = "ssh_agent"
)
type GitConfig struct {
AuthOption GitAuth
User string
Password string
AccessToken string
SshPublicKeyFilePath string
SshPrivateKeyPassword string
}
func NewGit(logger *zap.Logger, gitConfig *GitConfig, openPGP *signer.OpenPGP) *Git {
return &Git{logger: logger, gitConfig: gitConfig, openPGP: openPGP}
}
func (g *Git) GetOriginHEADForRepo(ctx context.Context, gitRepo *GitRepo) (string, error) {
auth, err := g.GetAuth()
if err != nil {
return "", err
}
remote, err := gitRepo.repo.Remote("origin")
if err != nil {
return "", err
}
refs, err := remote.ListContext(ctx, &git.ListOptions{
Auth: auth,
})
if err != nil {
return "", err
}
headRef := ""
for _, ref := range refs {
//g.logger.Debug(ref.String())
if !ref.Name().IsBranch() {
headRef = ref.Target().Short()
}
}
if headRef == "" {
return "", errors.New("no upstream HEAD branch could be found")
}
return headRef, nil
}
func (g *Git) CloneBranch(ctx context.Context, storageArea *storage.Area, repoUrl string, branch string) (*GitRepo, error) {
g.logger.Debug(
"cloning repository",
zap.String("repoUrl", repoUrl),
zap.String("path", storageArea.Path),
)
auth, err := g.GetAuth()
if err != nil {
return nil, err
}
cloneOptions := git.CloneOptions{
URL: repoUrl,
Auth: auth,
RemoteName: "origin",
ReferenceName: plumbing.NewBranchReferenceName(branch),
SingleBranch: false,
NoCheckout: false,
Depth: 1,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Tags: 0,
InsecureSkipTLS: false,
CABundle: []byte{},
}
repo, err := git.PlainCloneContext(ctx, storageArea.Path, false, &cloneOptions)
if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) {
return nil, err
}
g.logger.Debug("done cloning repo")
return &GitRepo{repo: repo}, nil
}
func (g *Git) Clone(ctx context.Context, storageArea *storage.Area, repoUrl string) (*GitRepo, error) {
g.logger.Debug(
"cloning repository",
zap.String("repoUrl", repoUrl),
zap.String("path", storageArea.Path),
)
auth, err := g.GetAuth()
if err != nil {
return nil, err
}
cloneOptions := git.CloneOptions{
URL: repoUrl,
Auth: auth,
RemoteName: "origin",
ReferenceName: "refs/heads/main",
SingleBranch: false,
NoCheckout: false,
Depth: 1,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Tags: 0,
InsecureSkipTLS: false,
CABundle: []byte{},
}
repo, err := git.PlainCloneContext(ctx, storageArea.Path, false, &cloneOptions)
if err != nil {
return nil, err
}
g.logger.Debug("done cloning repo")
return &GitRepo{repo: repo}, nil
}
func (g *Git) getProgressWriter() *zapio.Writer {
return &zapio.Writer{
Log: g.logger.With(zap.String("process", "go-git")),
Level: zap.DebugLevel,
}
}
func (g *Git) Add(ctx context.Context, storageArea *storage.Area, gitRepo *GitRepo) (*git.Worktree, error) {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return nil, err
}
err = worktree.AddWithOptions(&git.AddOptions{
All: true,
})
if err != nil {
return nil, err
}
status, err := worktree.Status()
if err != nil {
return nil, err
}
g.logger.Debug("git status", zap.String("status", status.String()))
return worktree, nil
}
func (g *Git) CreateBranch(ctx context.Context, gitRepo *GitRepo) error {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return err
}
refSpec := plumbing.NewBranchReferenceName("kraken-apply")
err = gitRepo.repo.CreateBranch(&config.Branch{
Name: "kraken-apply",
Remote: "origin",
Merge: refSpec,
Rebase: "",
})
if err != nil {
return fmt.Errorf("could not create branch: %w", err)
}
err = worktree.Checkout(&git.CheckoutOptions{
Branch: plumbing.ReferenceName(refSpec.String()),
Create: true,
Force: false,
Keep: false,
})
if err != nil {
return fmt.Errorf("could not checkout branch: %w", err)
}
remoteRef := plumbing.NewRemoteReferenceName("origin", "kraken-apply")
ref := plumbing.NewSymbolicReference(refSpec, remoteRef)
err = gitRepo.repo.Storer.SetReference(ref)
if err != nil {
return fmt.Errorf("could not set reference: %w", err)
}
auth, err := g.GetAuth()
if err != nil {
return err
}
err = worktree.PullContext(ctx, &git.PullOptions{
RemoteName: "origin",
ReferenceName: "refs/heads/main",
SingleBranch: false,
Depth: 1,
Auth: auth,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Force: true,
InsecureSkipTLS: false,
CABundle: []byte{},
})
if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) {
return fmt.Errorf("could not pull from origin: %w", err)
}
g.logger.Debug("done creating branches")
return nil
}
func (g *Git) Commit(ctx context.Context, gitRepo *GitRepo) error {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return err
}
_, err = worktree.Commit("some-commit", &git.CommitOptions{
All: true,
Author: &object.Signature{Name: "kraken", Email: "kraken@kasperhermansen.com", When: time.Now()},
Committer: &object.Signature{Name: "kraken", Email: "kraken@kasperhermansen.com", When: time.Now()},
SignKey: g.openPGP.SigningKey,
})
if err != nil {
return err
}
g.logger.Debug("done commiting objects")
return nil
}
func (g *Git) Push(ctx context.Context, gitRepo *GitRepo) error {
auth, err := g.GetAuth()
if err != nil {
return err
}
err = gitRepo.repo.PushContext(ctx, &git.PushOptions{
RemoteName: "origin",
RefSpecs: []config.RefSpec{},
Auth: auth,
Progress: g.getProgressWriter(),
Prune: false,
Force: true,
InsecureSkipTLS: false,
CABundle: []byte{},
RequireRemoteRefs: []config.RefSpec{},
})
if err != nil {
return err
}
g.logger.Debug("done pushing branch")
return nil
}
func (g *Git) GetAuth() (transport.AuthMethod, error) {
switch g.gitConfig.AuthOption {
case GIT_AUTH_SSH:
sshKey, err := ssh.NewPublicKeysFromFile(
g.gitConfig.User,
g.gitConfig.SshPublicKeyFilePath,
g.gitConfig.SshPrivateKeyPassword,
)
if err != nil {
return nil, err
}
return sshKey, nil
case GIT_AUTH_USERNAME_PASSWORD:
return &http.BasicAuth{
Username: g.gitConfig.User,
Password: g.gitConfig.Password,
}, nil
case GIT_AUTH_ACCESS_TOKEN:
return &http.BasicAuth{
Username: "required-username",
Password: g.gitConfig.AccessToken,
}, nil
case GIT_AUTH_ANONYMOUS:
return nil, nil
case GIT_AUTH_SSH_AGENT:
return ssh.NewSSHAgentAuth(g.gitConfig.User)
default:
return nil, nil
}
}

339
internal/services/providers/gogit.go Normal file
View File

@ -0,0 +1,339 @@
package providers
import (
"context"
"errors"
"fmt"
"time"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/signer"
"git.front.kjuulh.io/kjuulh/octopush/internal/services/storage"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/config"
"github.com/go-git/go-git/v5/plumbing"
"github.com/go-git/go-git/v5/plumbing/object"
"github.com/go-git/go-git/v5/plumbing/transport"
"github.com/go-git/go-git/v5/plumbing/transport/http"
"github.com/go-git/go-git/v5/plumbing/transport/ssh"
"go.uber.org/zap"
"go.uber.org/zap/zapio"
)
// GoGit is a native git provider, it can clone, pull
// , push and as in abstraction on native git operations
type GoGit struct {
logger *zap.Logger
gitConfig *GitConfig
openPGP *signer.OpenPGP
}
type GoGitRepo struct {
repo *git.Repository
}
func (gr *GoGitRepo) GetHEAD() (string, error) {
head, err := gr.repo.Head()
if err != nil {
return "", err
}
return head.Name().Short(), nil
}
type GitAuth string
const (
GIT_AUTH_SSH GitAuth = "ssh"
GIT_AUTH_USERNAME_PASSWORD GitAuth = "username_password"
GIT_AUTH_ACCESS_TOKEN GitAuth = "access_token"
GIT_AUTH_ANONYMOUS GitAuth = "anonymous"
GIT_AUTH_SSH_AGENT GitAuth = "ssh_agent"
)
type GitConfig struct {
AuthOption GitAuth
User string
Password string
AccessToken string
SshPublicKeyFilePath string
SshPrivateKeyPassword string
}
func NewGit(logger *zap.Logger, gitConfig *GitConfig, openPGP *signer.OpenPGP) *GoGit {
return &GoGit{logger: logger, gitConfig: gitConfig, openPGP: openPGP}
}
func (g *GoGit) GetOriginHEADForRepo(ctx context.Context, gitRepo *GoGitRepo) (string, error) {
auth, err := g.GetAuth()
if err != nil {
return "", err
}
remote, err := gitRepo.repo.Remote("origin")
if err != nil {
return "", err
}
refs, err := remote.ListContext(ctx, &git.ListOptions{
Auth: auth,
})
if err != nil {
return "", err
}
headRef := ""
for _, ref := range refs {
//g.logger.Debug(ref.String())
if ref.Target().IsBranch() {
headRef = ref.Target().Short()
}
}
if headRef == "" {
return "", errors.New("no upstream HEAD branch could be found")
}
return headRef, nil
}
func (g *GoGit) CloneBranch(ctx context.Context, storageArea *storage.Area, repoUrl string, branch string) (*GoGitRepo, error) {
g.logger.Debug(
"cloning repository",
zap.String("repoUrl", repoUrl),
zap.String("path", storageArea.Path),
)
auth, err := g.GetAuth()
if err != nil {
return nil, err
}
cloneOptions := git.CloneOptions{
URL: repoUrl,
Auth: auth,
RemoteName: "origin",
ReferenceName: plumbing.NewBranchReferenceName(branch),
SingleBranch: false,
NoCheckout: false,
Depth: 1,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Tags: 0,
InsecureSkipTLS: false,
CABundle: []byte{},
}
repo, err := git.PlainCloneContext(ctx, storageArea.Path, false, &cloneOptions)
if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) {
return nil, err
}
g.logger.Debug("done cloning repo")
return &GoGitRepo{repo: repo}, nil
}
func (g *GoGit) Clone(ctx context.Context, storageArea *storage.Area, repoUrl string) (*GoGitRepo, error) {
g.logger.Debug(
"cloning repository",
zap.String("repoUrl", repoUrl),
zap.String("path", storageArea.Path),
)
auth, err := g.GetAuth()
if err != nil {
return nil, err
}
cloneOptions := git.CloneOptions{
URL: repoUrl,
Auth: auth,
RemoteName: "origin",
ReferenceName: "",
SingleBranch: false,
NoCheckout: false,
Depth: 1,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Tags: 0,
InsecureSkipTLS: false,
CABundle: []byte{},
}
repo, err := git.PlainCloneContext(ctx, storageArea.Path, false, &cloneOptions)
if err != nil {
return nil, err
}
g.logger.Debug("done cloning repo")
return &GoGitRepo{repo: repo}, nil
}
func (g *GoGit) getProgressWriter() *zapio.Writer {
return &zapio.Writer{
Log: g.logger.With(zap.String("process", "go-git")),
Level: zap.DebugLevel,
}
}
func (g *GoGit) Add(ctx context.Context, storageArea *storage.Area, gitRepo *GoGitRepo) (*git.Worktree, error) {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return nil, err
}
err = worktree.AddWithOptions(&git.AddOptions{
All: true,
})
if err != nil {
return nil, err
}
status, err := worktree.Status()
if err != nil {
return nil, err
}
g.logger.Debug("git status", zap.String("status", status.String()))
return worktree, nil
}
func (g *GoGit) CreateBranch(ctx context.Context, gitRepo *GoGitRepo) error {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return err
}
refSpec := plumbing.NewBranchReferenceName("octopush-apply")
err = gitRepo.repo.CreateBranch(&config.Branch{
Name: "octopush-apply",
Remote: "origin",
Merge: refSpec,
Rebase: "",
})
if err != nil {
return fmt.Errorf("could not create branch: %w", err)
}
err = worktree.Checkout(&git.CheckoutOptions{
Branch: plumbing.ReferenceName(refSpec.String()),
Create: true,
Force: false,
Keep: false,
})
if err != nil {
return fmt.Errorf("could not checkout branch: %w", err)
}
//remoteRef := plumbing.NewRemoteReferenceName("origin", "octopush-apply")
//ref := plumbing.NewSymbolicReference(refSpec, remoteRef)
//err = gitRepo.repo.Storer.SetReference(ref)
//if err != nil {
// return fmt.Errorf("could not set reference: %w", err)
//}
auth, err := g.GetAuth()
if err != nil {
return err
}
err = worktree.PullContext(ctx, &git.PullOptions{
RemoteName: "origin",
ReferenceName: "",
SingleBranch: false,
Depth: 1,
Auth: auth,
RecurseSubmodules: 1,
Progress: g.getProgressWriter(),
Force: true,
InsecureSkipTLS: false,
CABundle: []byte{},
})
if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) {
return fmt.Errorf("could not pull from origin: %w", err)
}
g.logger.Debug("done creating branches")
return nil
}
func (g *GoGit) Commit(ctx context.Context, gitRepo *GoGitRepo) error {
worktree, err := gitRepo.repo.Worktree()
if err != nil {
return err
}
_, err = worktree.Commit("some-commit", &git.CommitOptions{
All: true,
Author: &object.Signature{Name: "octopush", Email: "octopush@kasperhermansen.com", When: time.Now()},
Committer: &object.Signature{Name: "octopush", Email: "octopush@kasperhermansen.com", When: time.Now()},
SignKey: g.openPGP.SigningKey,
})
if err != nil {
return err
}
g.logger.Debug("done commiting objects")
return nil
}
func (g *GoGit) Push(ctx context.Context, gitRepo *GoGitRepo) error {
auth, err := g.GetAuth()
if err != nil {
return err
}
err = gitRepo.repo.PushContext(ctx, &git.PushOptions{
RemoteName: "origin",
RefSpecs: []config.RefSpec{},
Auth: auth,
Progress: g.getProgressWriter(),
Prune: false,
Force: true,
InsecureSkipTLS: false,
CABundle: []byte{},
RequireRemoteRefs: []config.RefSpec{},
})
if err != nil {
return err
}
g.logger.Debug("done pushing branch")
return nil
}
func (g *GoGit) GetAuth() (transport.AuthMethod, error) {
switch g.gitConfig.AuthOption {
case GIT_AUTH_SSH:
sshKey, err := ssh.NewPublicKeysFromFile(
g.gitConfig.User,
g.gitConfig.SshPublicKeyFilePath,
g.gitConfig.SshPrivateKeyPassword,
)
if err != nil {
return nil, err
}
return sshKey, nil
case GIT_AUTH_USERNAME_PASSWORD:
return &http.BasicAuth{
Username: g.gitConfig.User,
Password: g.gitConfig.Password,
}, nil
case GIT_AUTH_ACCESS_TOKEN:
return &http.BasicAuth{
Username: "required-username",
Password: g.gitConfig.AccessToken,
}, nil
case GIT_AUTH_ANONYMOUS:
return nil, nil
case GIT_AUTH_SSH_AGENT:
return ssh.NewSSHAgentAuth(g.gitConfig.User)
default:
return nil, nil
}
}

2
internal/services/storage/storage.go
View File

@ -22,7 +22,7 @@ func NewDefaultStorageConfig() (*StorageConfig, error) {
return nil, err
}
return &StorageConfig{
Path: path.Join(tempDir, "kraken"),
Path: path.Join(tempDir, "octopush"),
}, nil
}

35
roadmap.md
View File

@ -19,26 +19,43 @@
## Version 0.1
- [x] Setup a way to choose actions and predicates
- [x] Allow instantiation of actions, kraken template repo etc.
- [x] Allow instantiation of actions, octopush template repo etc.
- [x] Implement docker action
- [x] Create pr for gitea provider
- [x] Providing query results
- [ ] Create CLI to trigger action
- [x] Create CLI to trigger action
### Not in scope
## Version 0.2
## Version 1.0
- [x] Write README
- [x] Fix git issues
- [x] Allow octopush to run directly on the cli
- [ ] Write README
## Version 0.3
- [ ] Make select depend on query
- [ ] Make configurable ssh user
- [ ] Make configurable gpg keyset
- [ ] Make configurable git provider
- [ ] Create templating function
- [ ] Add github
- [ ] Create templating function for easily creating new actions
- [ ] Add way to see progress of runners
- [ ] Implement global .kraken store for easy access
- [ ] Move builders to start instead of every time
- [ ] Implement global .octopush store for easy access to settings
- [ ] Move builders to start instead of every building on every action
- [ ] Setup releases on github
- [ ] Setup CI
- [ ] Setup static analysis
- [ ] Setup releases on gitea using drone
- [ ] Figure out a license (probably MIT)
## Version 1.x
## Version 0.4
- [ ] Create setup version for local actions
- [ ] Create setup version for server actions
- [ ] Create json schema
- [ ] Move roadmap to release / changelog
## Version 0.x
- Think about some sort of isolation
- Run authenticated on servers

2
scripts/push_github.sh
View File

@ -2,6 +2,6 @@
set -e
git remote add github git@github.com:kjuulh/kraken.git || true
git remote add github git@github.com:kjuulh/octopush.git || true
git push -f github main

9
scripts/run_client.sh
View File

@ -4,5 +4,10 @@ set -e
current_branch=$(git branch --show-current)
go run cmd/kraken/kraken.go process --actions-repo "git@git.front.kjuulh.io:kjuulh/kraken.git" --branch "$current_branch" --path "_examples/actions/write_a_readme"
go run cmd/kraken/kraken.go process --actions-repo "git@git.front.kjuulh.io:kjuulh/kraken.git" --branch "$current_branch" --path "_examples/queries/scrape_readme"
export $(cat .env | xargs)
#go run cmd/octopush/octopush.go process --actions-repo "git@git.front.kjuulh.io:kjuulh/octopush.git" --branch "$current_branch" --path "_examples/actions/write_a_readme"
go run cmd/octopush/octopush.go process \
--actions-repo "git@git.front.kjuulh.io:kjuulh/octopush.git"\
--branch "$current_branch" \
--path "_examples/actions/add_releaserc"