/## Install K3s
Install the server version of Ubuntu on each Raspberry and assign a different hostname to each. In case you local router (wifi router etc.) does not resolve those hostnames add all to the /etc/hosts file on each Raspberry.
Update system on all Raspberry Pis
sudo apt-get update && sudo apt-get upgrade
Install will fail with error "Failed to find memory cgroup, you may need to add "cgroup_memory=1 cgroup_enable=memory" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)" if you do not add a line to each Raspberry's cmdline.txt file and then reboot. So, run:
sudo sed -i '$ s/$/ cgroup_memory=1 cgroup_enable=memory/' /boot/firmware/cmdline.txt
Then reboot.
sudo reboot
Install K3s on master node with (see also https://rancher.com/docs/k3s/latest/en/quick-start/)
curl -sfL https://get.k3s.io | sh -s - server --tls-san <IP of master> --write-kubeconfig-mode 644
And replace
If the master node does not start double check that you have run the side command above and reboot.
On the master node retrieve the node-token. You will need the token to add your worker nodes. Get it with:
sudo cat /var/lib/rancher/k3s/server/node-token
You will also need the hostname or IP that you have assigned to your master node for installing the workers.
Now install K3s on each worker with
curl -sfL https://get.k3s.io | K3S_URL=https://<hostname>:6443 K3S_TOKEN=<node-token> sh -
Replace
Kubernetes should be up and running now. On the Raspi running the master node run:
kubectl get nodes
This should list all you nodes (the master and the worker nodes).
Install Longhorn
See also https://www.jericdy.com/blog/installing-k3s-with-longhorn-and-usb-storage-on-raspberry-pi
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.2.2/deploy/longhorn.yaml
Setup Ingress for Longhorn
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: longhorn-system
name: longhorn-ingress
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- host: longhorn.k3s.lan
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: longhorn-frontend
port:
number: 80
Add physical storage
Mount the disks
Create a mount point with sudo mkdir /media/hdd1
, sudo mkdir /media/ssd1
etc. Find the disk's UUID with sudo blkid
. Add entries to the Raspi's /etc/fstab file for each disk:
UUID=<partition uuid> /media/hdd1 ext4 defaults,noatime,nodiratime 0 2
UUID=<partition uuid> /media/ssd1 ext4 defaults,noatime,nodiratime 0 2
Now add the disk to Longhorn by using the Web GUI. Click on "Node"; Then on "Edit Node and Disks" (menu item is hidden under the "Operation" dropdown/button) and finally click on "Add Disk". Give the disk a name and then fill in the mount point previously created (e.g. "/media/hdd1"). Click on the "enable" radio button under scheduling and then on "Save".
To use the Longhorn volumes specify the storage class name "longhorn" in your persistent volume claim (PVC). For example:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: funky-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 500Gi
You also may want to change the default storage class to make Longhorn the new default and not the local-path that comes with k3s. The following command lists the storage classes:
kubectl get storageclass
The output should be something like:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-path (default) rancher.io/local-path Delete WaitForFirstConsumer false 6h34m
longhorn driver.longhorn.io Delete Immediate true 3h32m
Make the k3s local-path not the default:
kubectl patch storageClass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
And make cool longhorn storage class the new default:
kubectl patch storageclass longhorn -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
Run the kubectl get storageclass
command again to double check all is ok.
Security related configurations
Update the host os automatically
By default the unattended updates feature will automatically install security updates only. We will modify the configuration slightly to also install regular updates automatically.
The unattended update should already be installed on the Raspberry Ubuntu image. If not you can install it with:
sudo apt-get install unattended-upgrades
Activate it with:
sudo dpkg-reconfigure --priority=low unattended-upgrades
And confirm activation it with "
Then edit /etc/apt/apt.conf.d/50unattended-upgrades
(with sudo vi
, sudo nano
or your editor of choice) by uncommenting (removing the "//") of the line
"// "${distro_id}:${distro_codename}-updates";
". It should then look like:
...
"${distro_id}:${distro_codename}-updates";
...
Leave the rest as is.
Repeat on all Raspberry Pis.
How about rolling out a Samba file server to your new cluster? Check out this post.