#+NAME: Raspberry pi k3s cluster guide #+AUTHOR: James Blair #+EMAIL: mail@jamesblair.net #+DATE: 24th December 2019 This file serves as a complete step by step guide for creating a bare metal raspberry pi kubernetes cluster using [[https://k3s.io/][k3s]] from [[https://rancher.com/][Rancher]]. My goal for this build is to replace a server I currently run at home that hosts several workloads via Docker with a scalable k8s cluster. Additionally in future I would like the cluster to be portable and operate via 3G-5G Cellular network and an array of batteries. I chose k3s as it incredibly lightweight but still CNCF certified and production grade software that is optimised for resource constraints of raspberry pis. * Pre-requisites ** Cluster machines For this guide I am using three [[https://www.pishop.us/product/raspberry-pi-4-model-b-4gb/][Raspberry Pi 4 4GB]] machines. The cluster will have one leader node and two worker nodes. For resiliency puposes in future I will update the cluster to run with two leader nodes. *** TODO Migration to high availability control plane For resiliency purposes in future I will update the cluster and this documentation to support a control plane of more than one machine. ** Boot media This guide requires each Raspberry Pi to have a removable SD card or other removable boot media. I am use three 32GB SD Cards though any USB or SD card at least 8GB in size should work fine. *** TODO Migration to network booting In future it would be preferable for the raspberry pi's to be able to network boot and setup automatically without an SD card. This is a nice to have that I will pursue at a later date once I have a deployed cluster that allows me to migrate off the current server setup I have deployed. * Step 1 - Prepare boot media for master ** Download the latest release Our first step is to create the bootable SD Card with a minimal install of [[https://www.raspbian.org/][Raspbian]], which is a free operating system based on [[https://www.debian.org/][Debian]] and is optimised for Raspberry Pi hardware. Rather than doing an installation and configuration of an operating system image from scratch I found [[https://github.com/FooDeas/raspberrypi-ua-netinst][this project]] on Github which automates the install and configuration process nicely. #+NAME: Download the latest release zip #+begin_src shell :results output verbatim replace :wrap example echo Downloading latest release zip from github curl -s https://api.github.com/repos/foodeas/raspberrypi-ua-netinst/releases/latest \ | grep "browser_download_url.*zip" \ | cut -d : -f 2,3 \ | tr -d \" \ | wget -i - echo Checking file is now present ls -l | grep *.zip echo Extracting the zip file unzip -q -d installer *.zip ls -l | grep installer #+end_src #+RESULTS: Download the latest release zip #+begin_example Downloading latest release zip from github Checking file is now present -rw-rw-rw- 1 james james 60299545 Aug 12 08:35 raspberrypi-ua-netinst-v2.4.0.zip Extracting the zip file drwxrwxrwx 1 james james 4096 Dec 24 19:35 installer -rw-rw-rw- 1 james james 1351 Dec 24 18:02 installer-config.txt #+end_example ** Apply custom configuration Our next step after downloading the latest release is to apply our own installation configuration using a simple txt file. There is great documentation online howing what configuration options are available [[https://github.com/malignus/raspberrypi-ua-netinst/blob/master/doc/INSTALL_CUSTOM.md][here]]. For our purposes we just over-write the file downloaded and extracted in the previous step with one we have prepared earlier :) #+NAME: Overwrite installer configuration file #+begin_src shell :results output code verbatim replace :wrap example echo Display wordcount of original file for comparison wc installer/raspberrypi-ua-netinst/config/installer-config.txt echo Overwriting /installer/raspberrypi-ua-netinst/config/installer-config.txt cp installer-config.txt /installer/raspberrypi-ua-netinst/config/ echo Display wordcount of file after copy to validate update wc installer/raspberrypi-ua-netinst/config/installer-config.txt #+end_src #+RESULTS: Overwrite installer configuration file #+begin_example Display wordcount of original file for comparison 33 64 1351 installer/raspberrypi-ua-netinst/config/installer-config.txt Overwriting /installer/raspberrypi-ua-netinst/config/installer-config.txt Display wordcount of file after copy to validate update 33 64 1351 installer/raspberrypi-ua-netinst/config/installer-config.txt #+end_example * Step 2 - Copy the install media to sd card Our next step is to copy the contents of the ~installer/~ folder to a *FAT32* formatted SD Card. Unfortunately this is currently a windows step as my dev environment is a Windows 10 laptop with Debian via Windows Subsystem for Linux which does not support ~lsblk~ or other disk management commands. Our first step is to insert the SD Card and ensure it is formatted correctly as ~FAT32~. To do that we need to know the number of the disk we want to format, we can find that via powershell. #+NAME: Get disks via windows powershell #+begin_src shell :results output code verbatim replace :wrap example echo Retrieving disk list via powershell powershell.exe -nologo get-disk | select Number, FriendlyName, Size exit #+end_src #+RESULTS: Get disks via windows powershell #+begin_example Retrieving disk list via powershell PS get-disk | select Number, FriendlyName, Size Number FriendlyName Size ------ ------------ ---- 1 Realtek PCIE Card Reader 31104958464 0 SAMSUNG MZVLB256HAHQ-000H1 256060514304 #+end_example Once we know the number of the disk we want to format we can proceed. In the example above I have a 32GB SD Card which shows as number ~1~. Our next step is to again user powershell, this time to format the sd. #+NAME: Format the sd card #+begin_src shell :results output code verbatim replace #+end_src