tooling/wsl-setup.org

Windows Subsystem for Linux Setup

• Step 1 - Setup home folder structure
• Step 2 - Update and install packages
• Step 3 - Setup environment dotfiles
  • Obtain ssh keys from bitwarden
  • Clone and restore dotfiles
• Step 4 - Install kubemacs editor
  • Compile and install vanilla emacs
  • Overlay kubemacs
• Step 5 - Setup mutt email client

This guide will walk through setting up Windows Subsystem for Linux on Windows 10. This particular setup contains my opinionated view of a good foundation and layers on some pairing and development orientated tooling over top.

Caveats: Please note this guide is written for the Debian WSL distribution.

Acknowledgements: Large elements of this wsl setup came about through collaboration with the great people at ii.coop. I encourage you to explore and contribute to their work on gitlab as many elements form a core part of this setup and workflow.

Step 1 - Setup home folder structure

After installing the Debian WSL distribution no folders are present in your home folder.

In this section we create some quick standard folders to keep our home folder somewhat organised.

# Ensure we are in our home folder
cd ~/

# Create a documents folder for our git repositories
mkdir Documents

# Create a downloads folder for temporary objects
mkdir Downloads

Step 2 - Update and install packages

To get started we ensure the package manager is up to date.

sudo apt-get update && sudo apt-get upgrade

Next we install a series of standard packages that form part of our workflow or are dependencies for other tools in our environment.

# Install basic utilities
 sudo apt-get install -y git locales curl wget xclip xsel tmux tmate net-tools less wget htop screenfetch zip openssh-client dictd knockd

# Install pre-requisites for compiling emacs
sudo apt-get install -y make gcc libgnutls28-dev libtinfo-dev

# Install dpkg and apt management tools
sudo apt-get install -y software-properties-common apt-transport-https ca-certificates dirmngr

# Install terminal customisation packages
sudo apt install -y xterm xtermcontrol

We use pandoc for documentation export from spacemacs and other markup conversion tasks.

# Work from our downloads folder
cd ~/Downloads

# Download the latest release from github
curl -s https://api.github.com/repos/jgm/pandoc/releases/latest \
| grep "browser_download_url.*deb" \
| cut -d : -f 2,3 \
| tr -d \" \
| wget -i -

# Install the package with dpkg
sudo dpkg -i pandoc*

# Remove the package file after install
rm pandoc*

For additional package management we use node package manager. The code below installs node 12.

# Curl down the shell script for adding version 12 of nodejs to apt
sudo curl  -sL https://deb.nodesource.com/setup_12.x | sudo bash -

# Install the nodejs package via apt
sudo apt-get install -y nodejs

For managing secrets we use bitwarden which provides a great cli utility.

This section should be expanded in future to cover setting alias for common bitwarden tasks.

# Install the bitwarden cli via node package manager
sudo npm install -g @bitwarden/cli 

# Test login to bitwarden
bw login mail@jamesblair.net

For working with google cloud platform we use the GCP SDK, which provides our cli tools.

# Add the Cloud SDK distribution URI as a package source: 
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud public key: 
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

# Update then install the Google Cloud SDK & kubectl: 
sudo apt-get update && sudo apt-get install -y google-cloud-sdk kubectl

For working with Amazon Web Services we need the AWS CLI.

# Download the binary
cd ~/Downloads/
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

# Install
unzip awscliv2.zip 
sudo ./aws/install 

# Clean up
rm -rf ~/Downloads/aws*

For cloud infrastructure deployments we use terraforms.

# Download the binary
wget 'https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip'

# Unzip it
unzip *.zip

# Move the binary to path
sudo mv terraform /usr/local/bin/

# Clean up
rm *amd64.zip

For ad-hoc system administration we use ansible.

# Add a source entry to apt sources
echo 'deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main' | sudo tee -a /etc/apt/sources.list

# Add the required key
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367

# Install ansible via apt package manager
sudo apt-get update && sudo apt-get install -y ansible

Step 3 - Setup environment dotfiles

Within wsl we can use .dotfiles to further customise our environment. The script below restores my versions of key dotfiles automatically.

Note: The git clone below relies on having permission to clone the repository referenced. For me this means having an ssh key present which has been added to gitlab.

Obtain ssh keys from bitwarden

In order to be able to clone the repository in the next step we need to obtain our ssh keys from bitwarden. Given we have installed the bitwarden cli we can mostly automte this process minus the initial login to bitwarden.

# Generate a new blank key to overwrite
ssh-keygen -t rsa -f ~/.ssh/james -q -P ""

# Ensure we have an active bitwarden session
export BW_SESSION=$(bw unlock --raw > ~/.bw_session && cat ~/.bw_session)

# Export both keys
export key=$(bw get item desktop --pretty | grep notes)

# Extract private key
export private=${key:12}
export private=${private/END RSA*/END RSA PRIVATE KEY-----} 
echo $private | awk '{gsub(/\\n/,"\n")}1' > ~/.ssh/james

# Extract public key
export public=${key/*ssh-rsa/ssh-rsa} && echo ${public::-2} | awk '{gsub(/\\n/,"\n")}1' > ~/.ssh/james.pub

Clone and restore dotfiles

Once our keys are available to us we can clone down our dotfiles and get back to our comfortable normal terminal environment.

# Remove the interactive host prompt
ssh-keyscan -p 2224 gitlab.jamma.life >> ~/.ssh/known_hosts

# Clone down this repository
git clone ssh://git@gitlab.jamma.life:2224/jmhbnz/tooling.git ~/Documents/tooling/

# Restore all dotfiles
cp ~/Documents/tooling/.* ~/

# Reload bashrc with updated version
source ~/.bashrc

Step 4 - Install kubemacs editor

Compile and install vanilla emacs

A key component in our environment is the ii extension of spacemacs. The section below will setup emacs version 26.3 and then layer the ii version of spacemacs called kubemacs on top.

Our first step is to download the base emacs 26.3 source code.

# Work from our downloads directory
cd ~/Downloads/

# Download the tarball for emacs 26.3 source code
wget https://mirror.ossplanet.net/gnu/emacs/emacs-26.3.tar.xz

# Untar the source code archive
tar xf emacs-26.3.tar.xz

# Change to the extracted directory
cd emacs-26.3

After downloading and untarring the source code we are ready to attempt resolving dependencies and compiling.

We configure without-x as this environment is solely focussed on running within terminal i.e. emacs -nw.

# Run configure to resolve any dependencies minus x window support
./configure --without-x --with-gnutls=no

# Compile the application with make, using all available cpu cores
sudo make -j `nproc`

# Run make install to move/install compiled binaries
sudo make install

After compiling and installing emacs we should verify that version 26.3 is installed.

emacs --version

Overlay kubemacs

Once the right version of emacs is running we can then layer in kubemacs on top. Documentation for this is here: https://github.com/kubemacs/kubemacs

# Remove the default site-lisp file
sudo rm /usr/local/share/emacs/site-lisp/subdirs.el

# Clone kubemacs from github
sudo git clone --recursive https://github.com/kubemacs/kubemacs /usr/local/share/emacs/site-lisp/

# Ensure permissions are set for the cloned folder
sudo chown -R $USER:$USER /usr/local/share/emacs/site-lisp

After cloning down kubemacs we now need to launch emacs and install packages, this can take a while and several iterations may be neccessary before spacemacs will launch fully.

Note: As of <2020-04-11 Sat> there is one package org-plus-contrib that is refusing to install as normal, a manual install process is included below to work around this.

# Ensure the elpa folder is created
mkdir -p /usr/local/share/emacs/site-lisp/spacemacs/elpa/26.3/develop
cd /usr/local/share/emacs/site-lisp/spacemacs/elpa/26.3/develop

# Manually install org-plus-contrib
wget 'https://orgmode.org/elpa/org-plus-contrib-20200406.tar'
tar xf org-plus-contrib-20200406.tar 
rm org-plus-contrib-20200406.tar

# Start emacs and download packages
emacs --insecure

One final configuration step specific to wsl is to switch from osc52 to xsel for clipboard.

This is required because copy and paste osc52 sequences are not currently supported in wsl. You can add your voice to this issue to change that here.

# Replace the osc52 command with xsel
sed -i -e 's/osc52.sh/xsel -i -b/g' /usr/local/share/emacs/site-lisp/bin/osc52-tmate.sh

# Provide custom xclipboard functions
# https://github.com/syl20bnr/spacemacs/issues/2222

Step 5 - Setup mutt email client

For reading email we ideally use a cli based client for fast searching and lightweight mail reading.

The mutt mail client fills these roles well for imap mailboxes.

The first step to setup mutt is to ensure it is installed.

sudo apt-get install -y mutt urlscan

After installing mutt we then need to create configuration directories and files.

mkdir -p ~/.mutt/cache/headers
mkdir ~/.mutt/cache/bodies
touch ~/.mutt/certificates

One configuration folders and files exist we just need to populate our user mutt configuration file with a configuration for our particular mail provider.

The example provided in this repository utilises the bitwarden cli utility for secrets to ensure these are securely gathered at runtime and not stored in the file.