examples/pipelines/readme.md

@@ -0,0 +1,208 @@
+# Pipelines design docs
+
+The goal of this experiment is to play with various code sharing features. The
+pipelines in this experiment is supposed to model normal code sharing behavior
+of libraries, with overriding capabilities of the downstream repository
+
+The goal is to split the body of the work in three parts.
+
+- libraries,
+- compendiums
+- articles
+
+The terminology is as such:
+
+## Libraries
+
+Libraries provide raw functions, and is a general abstraction on an underlying
+process, such as running a container, executing a shell script etc. Libraries
+might define an API descripting how to interact with it. It is up to the
+compendium, and article to use these in a sane manner.
+
+Such that:
+
+```rust
+pub fn execute_shell(&self, input: &ShellOpts) -> Result<ShellOutput> {
+   ...
+}
+```
+
+These work similar to raw primitive functions, but should serve as an
+opinionated flyweight. These may be extremely specific, such as building a go
+binary, creating a github release etc. The details are left to the caller.
+
+A version scheme of the library should follow semver, as that is the best model
+at the moment for versioning. Libraries should be pulled using the native
+package manager, or include as a submodule.
+
+## Compendiums
+
+A compendium is an opinionated collection of libraries, which consists of files,
+configurations etc. A compendium is to be used by either other compendiums, or
+articles. They are not to be used by libraries, this is to provide a natural
+hierachy. The end result should be a directed acyclic graph.
+
+A compendium should provide an API for either other compendiums, or articles.
+These apis, need to remain flexible, and be open to mutations. As such all
+primitive features, files, configurations need to be exposed as raw data or data
+structures if suitable.
+
+This is done using pipelines, or middleware for the specific parts. A data
+object will pass from above, containing the data to implement the required
+interfaces of the Compendium, these must be fulfilled for the construction of
+the Compendium, else the compilation should fail.
+
+The caller will have the ability to replace the specifics of the Compendium, by
+replacing certain pipelines, or mutating the data. In case of mutations, the
+data is only modified for that pipeline and below, if a fork occurs above, then
+the divergent paths won't be affected.
+
+Compendiums are driven by pipelines applied to them either from other
+compendiums or articles. An article or compendium will only have access to their
+direct dependent compendiums pipelines. An article will naturally expose
+pipelines to be called by the user.
+
+```rust
+pub struct GoApplication;
+
+impl Compendium for GoApplication {
+  type Input = GoApplicationOpts;
+
+  pub fn get_pipelines(&mut self) -> Result<Pipelines> {
+    let pipelines = self.pipelines
+      .clone()
+      .add(self.get_application_pipelines())?
+      .add(self.get_go_releaser_pipelines())?;
+
+    Ok(pipelines)
+  }
+}
+```
+
+## Articles
+
+An article is a specific implementation of a compendium, it is the end of the
+chain, and is meant to be directly executed by the user, using a client
+application.
+
+It by default is supposed to be a golden path, I.e. it passes the defaults of
+the Compendium, but on a case-by-case basis has the ability to modify its
+pipelines to its needs. This may be changing certain default configurations,
+mutate a dockerfile, add additional steps to a pipeline etc, and remove others.
+
+We reason that once you stray from the golden path, you should be in control,
+this may be done by forking the compendium's features.
+
+It provides pipelines as actions, and implements a strict protocol for
+communication.
+
+```rust
+pub struct MyGoService;
+
+impl Article for MyGoService {
+
+  fn get_pipelines(&mut self) -> Result<Pipelines> {
+    let mut go_pipelines = GoApplication::new().get_pipelines()?;
+    let api_pipeline = self.get_api_pipeline()?;
+    let build_pipeline = self.get_docker_pipeline(go_pipelines.extract::<BuildPipeline>()?)?;;
+
+    let pipelines = PipelineBuilder::new()
+      .append(go_pipelines)
+      .append(api_pipeline)
+      .append(build_pipeline)
+      .build()?
+
+    Ok(pipelines)
+  }
+}
+```
+
+## Usage
+
+A host app can now call these:
+
+```bash
+char ls
+```
+
+ls simply displays the information on what pipeline are available
+
+```bash
+char run build
+```
+
+run build will execute the pipeline build. It will validate input available from
+char.toml, push these keys/values through to the pipeline, which will go through
+all the steps.
+
+- build
+- MyGoService
+- DockerBuildPipeline
+  - (BuildPipeline)
+- DockerLibrary
+  - `fn docker_build(dockerfile_contents: string)`
+- Native dependencies
+  - "write dockerfile /tmp/abc/dockerfile"
+  - "shell -> docker build -f /tmp/abc/dockerfile some-library-path"
+
+## Common scenarios
+
+### Replacing parts of a build script (dockerfile)
+
+A compendium may embed/provide a dockerfile or any other resource, these are
+provided through consistent interfaces.
+
+```rust
+pub struct GoDockerBuildPipeline;
+
+impl GoDockerBuildPipeline {
+  fn get_resources(&self) -> Result<(
+    libraries::docker::DockerContents,
+    libraries::docker::DockerBuildTags)> {
+    return (self.contents, self.build_tags)
+  }
+
+  fn mutate_resources(
+    &mut self,
+    contents: libraries::docker::DockerContents,
+    build_tags: libraries::docker::DockerBuildTags,
+  ) -> Result<()> {
+    self.contents = contents;
+    self.build_tags = build_tags
+  }
+}
+
+impl BuildPipeline for GoDockerBuildPipeline {
+  fn execute(&mut self, config: Configuration) -> Result<()> {
+    let (docker_contents, base_tags) = self.get_resources()
+    libraries::docker::build(docker_contents, base_tags, config)
+  }
+}
+```
+
+In the article you can now replace the resources to fit your needs, that or
+building your own pipeline.
+
+```rust
+pub struct MyGoService;
+
+impl Article for MyGoService {
+
+  fn get_pipelines(&mut self) -> Result<Pipelines> {
+  let go_app = GoApplication::new();
+  let go_pipelines = go_app.get_pipelines();
+
+  let mut go_build_pipeline = go_pipelines.get_pipeline::<GoDockerBuildPipeline>()?;
+  let go_resources = go_build_pipeline.get_resources()?;
+  let go_resources = self.mutate_go_resources(&go_resources)?;
+  go_build_pipeline.mutate_resources(go_resources)?;
+  go_pipelines.replace::<GoDockerBuildPipeline>(go_build_pipeline)?;
+
+  let pipelines = PipelineBuilder::new()
+      .append(.get_pipelines())
+      .build()?
+
+    Ok(pipelines)
+  }
+}
+```