raspi-k3s/readme.org

#+NAME: Raspberry pi k3s cluster guide
#+AUTHOR: James Blair
#+EMAIL: mail@jamesblair.net
#+DATE: 24th December 2019


This file serves as a complete step by step guide for creating a bare
metal raspberry pi kubernetes cluster using [[https://k3s.io/][k3s]] from [[https://rancher.com/][Rancher]].

My goal for this build is to replace a server I currently run at home
that hosts several workloads via Docker with a scalable k8s cluster.

Additionally in future I would like the cluster to be portable and 
operate via 3G-5G Cellular network and an array of batteries.

I chose k3s as it incredibly lightweight but still CNCF certified and
production grade software that is optimised for resource constraints of
raspberry pis.


* Pre-requisites

** Cluster machines

   For this guide I am using three [[https://www.pishop.us/product/raspberry-pi-4-model-b-4gb/][Raspberry Pi 4 4GB]] machines.

   The cluster will have one leader node and two worker nodes. 
   For resiliency puposes in future I will update the cluster to run
   with two leader nodes.

*** TODO Migration to high availability control plane

   For resiliency purposes in future I will update the cluster and this
   documentation to support a control plane of more than one machine.


** Boot media

   This guide requires each Raspberry Pi to have a removable SD card or
   other removable boot media.  I am use three 32GB SD Cards though any
   USB or SD card at least 8GB in size should work fine.

*** TODO Migration to network booting

   In future it would be preferable for the raspberry pi's to be able
   to network boot and setup automatically without an SD card. 

   This is a nice to have that I will pursue at a later date once I
   have a deployed cluster that allows me to migrate off the current
   server setup I have deployed.




* Step 1 - Prepare boot media for master 

** Download the latest release

  Our first step is to create the bootable SD Card with a minimal install
  of [[https://www.raspbian.org/][Raspbian]], which is a free operating system based on [[https://www.debian.org/][Debian]] and is
  optimised for Raspberry Pi hardware.

  Rather than doing an installation and configuration of an operating system
  image from scratch I found [[https://github.com/FooDeas/raspberrypi-ua-netinst][this project]] on Github which automates the
  install and configuration process nicely.

  #+NAME: Download the latest release zip
  #+begin_src shell :results output verbatim replace :wrap example
  echo Downloading latest release zip from github
  curl -s https://api.github.com/repos/foodeas/raspberrypi-ua-netinst/releases/latest \
  | grep "browser_download_url.*zip" \
  | cut -d : -f 2,3 \
  | tr -d \" \
  | wget -i -
  
  echo Checking file is now present
  ls -l | grep *.zip
  
  echo Extracting the zip file
  unzip -q -d installer *.zip
  ls -l | grep installer
  #+end_src

  #+RESULTS: Download the latest release zip
  #+begin_example
  Downloading latest release zip from github
  Checking file is now present
  -rw-rw-rw- 1 james james 60299545 Aug 12 08:35 raspberrypi-ua-netinst-v2.4.0.zip
  Extracting the zip file
  #+end_example


** Apply custom configuration

   Our next step after downloading the 
  

* Step 2 - Write the image file to removable media

  Our next step is to write the downloaded image file to our removable
  media.  This step needs to be repeated for each raspberry pi in the
  cluster.

  *Note:* As my development environment is based on [[https://docs.microsoft.com/en-us/windows/wsl/about][wsl]] I need to use
  a third party tool to write the image.  If you are on a standard linux
  distribution you can use the ~dd~ utility to write the image.

  The image writing utility I use is [[https://www.balena.io/etcher/][balena etcher]].  After downloading
  the latest version:
   - Insert your removable media.
   - Select the image file you downloaded earlier.
   - Select your removable media and start writing the image.

 
* Step 3 - Enable ssh at startup

  As our cluster will be headless, i.e. have no screen keyboard or mouse
  plugged in we need to ensure ssh is configured from boot so that we can
  remotely connect.

  To do this we just need to add an empty file named ~ssh~ to our newly
  created sd card 

  #+NAME: Mount newly formatted sd card
  #+BEGIN_SRC shell
  sudo mkdir /media/sdcard
  sudo mount /dev/[SDCARD] /media/sdcard -o umask=000
  #+END_SRC


  #+NAME: Create the blank ssh file in the boot directory
  #+BEGIN_SRC shell
  sudo touch /media/sdcard/ssh
  #+END_SRC


* Step 4 - Enable wifi at startup

  For this guide we are running our cluser wirelessly.  To ensure we can
  access our pi's once they boot we need to ensure they boot with a wifi
  configuration that will connect to our desired network.

  To achieve this we need to set a configuration for the [[https://en.wikipedia.org/wiki/Wpa_supplicant][wpa_supplicant]]
  application that our raspberry pi's use for managing wireless.

  For security reasons I don't store wireless access point details here.
  Instead they are retrieved at runtime of the code block using the
  [[https://bitwarden.com/][bitwarden]] command line utility.

  #+NAME: Write the wireless configuration file
  #+BEGIN_SRC shell
  export WIRELESS_SSID=`bw get username wifi`
  export WIRELESS_PASS=`bw get password wifi`
  cat > /media/sdcard/wpa_supplicant.conf << EOF
  country=nz
  update_config=1
  ctrl_interface=/var/run/wpa_supplicant

  network={
    scan_ssid=1
    ssid=$WIRELESS_SSID
    psk=$WIRELESS_PASS
  }
  EOF 
  #+END_SRC

  After writing the file we will use ~cat~ to verify the details.
  If all details are correct you can unmount and remove the media.