The first control plane node is now identical to any other server
node. The cluster initialization happens once in two steps: first,
make sure that the k3s cluster is initialized and then apply our
configurations while the other nodes join. This change makes the
initialization more resilient and even faster than before.
* To do so, we need to ensure that the generated kubeconfig is part of
terraforms dependency graph. This has the additional benefit of not
depending on local files anymore which should enable multi-user
setups.
* This also means that we can't deploy CCM, CSI & Traefik from our local
host, because we don't have kubeconfig.yaml locally while provisioning
the control plane, only afterwards.
* So we just run kubectl apply on the control plane itself, after k3s is
ready.
* To do so, we need to deploy all manifests. I've merged the patches
into a single kustomization.yaml file, because that makes the
deployment of those files to the control-plane server easier.
* we could also put the traefik config into the same kustomization file,
which would save us one of the file provisioner blocks. I didn't want
this PR to get any bigger, and will consider merging this config later
on. kustomization.yaml is small enough that we could yamlencode() for
it and store the patches in separate files again, not as
inline-strings which is kind of ugly.
Newly added ssh commands were missing the flag -i to pass an
identity file. This means that those commands use different
settings then the provisioners and their connection blocks
around them.
While adding this parameter, I decided it would be cleanest
to add local.ssh_args.
