A fully automated, optimized and auto-upgradable, HA-able, k3s cluster on <ahref="https://hetzner.com"target="_blank">Hetzner Cloud</a> 🤑
</p>
<hr/>
<br/>
</p>
## About The Project
![Product Name Screen Shot][product-screenshot]
[Hetzner Cloud](https://hetzner.com) is a good cloud provider that offers very affordable prices for cloud instances. The goal of this project was to create an optimal Kubernetes installation with it. We wanted functionality that was as close as possible to GKE's auto-pilot.
Here's what is working at the moment:
- Lightweight and resource-efficient Kubernetes with [k3s](https://github.com/k3s-io/k3s), and Fedora nodes to take advantage of the latest Linux kernels.
- Optimal [Cilium](https://github.com/cilium/cilium) CNI with full BPF support and Kube-proxy replacement. It uses the Hetzner private subnet underneath to communicate between the nodes, so no encryption is not needed.
- Automatic OS upgrades, supported by [kured](https://github.com/weaveworks/kured) that initiate a reboot of the node only when necessary and after having drained it properly.
- Automatic HA by setting the required number of servers and agents nodes.
- Automatic k3s upgrade by using Rancher's [system-upgrade-controller](https://github.com/rancher/system-upgrade-controller) and tracking the latest 1.x stable branch.
- Optional [Nginx ingress controller](https://kubernetes.github.io/ingress-nginx/) that will automatically use Hetzner's private network to allocate a Hetzner load balancer.
It uses Terraform to deploy as it's easy to use, and Hetzner provides a great [Hetzner Terraform Provider](https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs).
<!-- GETTING STARTED -->
## Getting started
Follow those simple steps and your world cheapest Kube cluster will be up and running in no time.
### Prerequisites
First and foremost, you need to have a Hetzner Cloud account. You can sign up for free [here](https://hetzner.com/cloud/).
Then you'll need you have both the [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli) and [helm](https://helm.sh/docs/intro/install/), and [kubectl](https://kubernetes.io/docs/tasks/tools/) cli installed. The easiest way is to use the [gofish](https://gofi.sh/#install) package manager to install them.
```sh
gofish install terraform
gofish install helm
gofish install kubectl
```
### Creating terraform.tfvars
1. Create a project in your Hetzner Cloud Console, and go to **Security > API Tokens** of that project to grab the API key.
2. Generate an ssh key pair for your cluster, unless you already have one that you'd like to use.
3. Rename terraform.tfvars.example to terraform.tfvars and replace the values from steps 1 and 2.
### (Optional) Customize other variables
The number of control plane nodes and worker nodes, and the Hetzner datacenter location, can be customized by adding the variables to your newly created [terraform.tfvars](terraform.tfvars) file.
See the default values in the [variables.tf](variables.tf) file, they correspond to (you can copy-paste and customize):
```tfvars
servers_num = 2
agents_num = 2
server_location = "fsn1"
```
### Installation
```sh
terraform init
terraform apply -auto-approve
```
It will take a few minutes to complete, and then you should see a green output with the IP addresses of the nodes. Then you can immediately kubectl into it without any further action.
<!-- USAGE EXAMPLES -->
## Usage
When the cluster is on, you can do whatever you wish with it! 🎉
### Useful commands
- List your nodes IPs, with either of those:
```sh
terraform outputs
hcloud server list
```
- See the Hetzner network config:
```sh
hcloud network describe k3s-net
```
- Log into one of your nodes (replace the location of your private key if needed):
- Sometimes the Hetzner network does not get deleted on its own (probably due to a temporary bug), we have to deleted manually after running the first command, and re-run it.
