# Pipelines design docs

The goal of this experiment is to play with various code sharing features. The
pipelines in this experiment is supposed to model normal code sharing behavior
of libraries, with overriding capabilities of the downstream repository

The goal is to split the body of the work in three parts.

- libraries,
- compendiums
- articles

The terminology is as such:

## Libraries

Libraries provide raw functions, and is a general abstraction on an underlying
process, such as running a container, executing a shell script etc. Libraries
might define an API descripting how to interact with it. It is up to the
compendium, and article to use these in a sane manner.

Such that:

```rust
pub fn execute_shell(&self, input: &ShellOpts) -> Result<ShellOutput> {
   ...
}
```

These work similar to raw primitive functions, but should serve as an
opinionated flyweight. These may be extremely specific, such as building a go
binary, creating a github release etc. The details are left to the caller.

A version scheme of the library should follow semver, as that is the best model
at the moment for versioning. Libraries should be pulled using the native
package manager, or include as a submodule.

## Compendiums

A compendium is an opinionated collection of libraries, which consists of files,
configurations etc. A compendium is to be used by either other compendiums, or
articles. They are not to be used by libraries, this is to provide a natural
hierachy. The end result should be a directed acyclic graph.

A compendium should provide an API for either other compendiums, or articles.
These apis, need to remain flexible, and be open to mutations. As such all
primitive features, files, configurations need to be exposed as raw data or data
structures if suitable.

This is done using pipelines, or middleware for the specific parts. A data
object will pass from above, containing the data to implement the required
interfaces of the Compendium, these must be fulfilled for the construction of
the Compendium, else the compilation should fail.

The caller will have the ability to replace the specifics of the Compendium, by
replacing certain pipelines, or mutating the data. In case of mutations, the
data is only modified for that pipeline and below, if a fork occurs above, then
the divergent paths won't be affected.

Compendiums are driven by pipelines applied to them either from other
compendiums or articles. An article or compendium will only have access to their
direct dependent compendiums pipelines. An article will naturally expose
pipelines to be called by the user.

```rust
pub struct GoApplication;

impl Compendium for GoApplication {
  type Input = GoApplicationOpts;

  pub fn get_pipelines(&mut self) -> Result<Pipelines> {
    let pipelines = self.pipelines
      .clone()
      .add(self.get_application_pipelines())?
      .add(self.get_go_releaser_pipelines())?;

    Ok(pipelines)
  }
}
```

## Articles

An article is a specific implementation of a compendium, it is the end of the
chain, and is meant to be directly executed by the user, using a client
application.

It by default is supposed to be a golden path, I.e. it passes the defaults of
the Compendium, but on a case-by-case basis has the ability to modify its
pipelines to its needs. This may be changing certain default configurations,
mutate a dockerfile, add additional steps to a pipeline etc, and remove others.

We reason that once you stray from the golden path, you should be in control,
this may be done by forking the compendium's features.

It provides pipelines as actions, and implements a strict protocol for
communication.

```rust
pub struct MyGoService;

impl Article for MyGoService {

  fn get_pipelines(&mut self) -> Result<Pipelines> {
    let mut go_pipelines = GoApplication::new().get_pipelines()?;
    let api_pipeline = self.get_api_pipeline()?;
    let build_pipeline = self.get_docker_pipeline(go_pipelines.extract::<BuildPipeline>()?)?;;

    let pipelines = PipelineBuilder::new()
      .append(go_pipelines)
      .append(api_pipeline)
      .append(build_pipeline)
      .build()?

    Ok(pipelines)
  }
}
```

## Usage

A host app can now call these:

```bash
char ls
```

ls simply displays the information on what pipeline are available

```bash
char run build
```

run build will execute the pipeline build. It will validate input available from
char.toml, push these keys/values through to the pipeline, which will go through
all the steps.

- build
- MyGoService
- DockerBuildPipeline
  - (BuildPipeline)
- DockerLibrary
  - `fn docker_build(dockerfile_contents: string)`
- Native dependencies
  - "write dockerfile /tmp/abc/dockerfile"
  - "shell -> docker build -f /tmp/abc/dockerfile some-library-path"

## Common scenarios

### Replacing parts of a build script (dockerfile)

A compendium may embed/provide a dockerfile or any other resource, these are
provided through consistent interfaces.

```rust
pub struct GoDockerBuildPipeline;

impl GoDockerBuildPipeline {
  fn get_resources(&self) -> Result<(
    libraries::docker::DockerContents,
    libraries::docker::DockerBuildTags)> {
    return (self.contents, self.build_tags)
  }

  fn mutate_resources(
    &mut self,
    contents: libraries::docker::DockerContents,
    build_tags: libraries::docker::DockerBuildTags,
  ) -> Result<()> {
    self.contents = contents;
    self.build_tags = build_tags
  }
}

impl BuildPipeline for GoDockerBuildPipeline {
  fn execute(&mut self, config: Configuration) -> Result<()> {
    let (docker_contents, base_tags) = self.get_resources()
    libraries::docker::build(docker_contents, base_tags, config)
  }
}
```

In the article you can now replace the resources to fit your needs, that or
building your own pipeline.

```rust
pub struct MyGoService;

impl Article for MyGoService {

  fn get_pipelines(&mut self) -> Result<Pipelines> {
  let go_app = GoApplication::new();
  let go_pipelines = go_app.get_pipelines();

  let mut go_build_pipeline = go_pipelines.get_pipeline::<GoDockerBuildPipeline>()?;
  let go_resources = go_build_pipeline.get_resources()?;
  let go_resources = self.mutate_go_resources(&go_resources)?;
  go_build_pipeline.mutate_resources(go_resources)?;
  go_pipelines.replace::<GoDockerBuildPipeline>(go_build_pipeline)?;

  let pipelines = PipelineBuilder::new()
      .append(.get_pipelines())
      .build()?

    Ok(pipelines)
  }
}
```