gitlab-gcp-deployment/README.org

Gitlab GCP Deployment

• Step 1 - Ensure GCP SDK is installed
• Step 2 - Configure cloud resources
• Step 3 - Create vertual machine
• Step 4 - Install docker on virtual machine
• Step 5 - Install gitlab via docker
• Step 6 - Teardown cloud resources

This org file is intended to capture and automate the end to end workflow to deploy an instance of Gitlab on Google Cloud Platform.

We'll use shell blocks inside this file which can be executed with Babel. Additionally we want to explore tangling these source code blocks to shell script files within this document so that the scripts can then be executed by a continous delivery pipeline.

Notes:

1. To interact with this org file we're using the Humacs distribution of Emacs.
2. This workflow has only been tested on the Ubuntu 20.04 linux distribution, via WSL 2.

Step 1 - Ensure GCP SDK is installed

To automate our interactions with Google Cloud Platform we'll use the GCP SDK which provides us with a number of command line tools to interact with the platform, such as gcloud, gsutil and kubectl.

Tangle the shell block below to a shell script by pressing , b t in emacs command mode:

# Add the Cloud SDK distribution URI as a package source
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee /etc/apt/sources.list.d/google-cloud-sdk.list

# Make sure apt-transport-https is installed
sudo apt-get install -y apt-transport-https ca-certificates gnupg

# Import the Google Cloud public key
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

# Update and install the SDK
sudo apt-get update && sudo apt-get install -y google-cloud-sdk

Step 2 - Configure cloud resources

With GCP SDK now installed we need to authenticate, create a project and then create a virtual machine instance that we will install Gitlab into later in the workflow.

First up is authentication so our GCP SDK installation can carry out actions in a given account and project. This part of the process is currently a manual step as the authentication process includes some interactive steps.

In future we could automate this process as part of a continous delivery pipeline using a GCP service account with permission to create virtual machine instances.

gcloud auth login

Once we have authenticated we can create a project and then create a new virtual machine instance within that project.

Firstly let's create a new project, a project is the logical boundary all our cloud resources for this deployment will live within. To be able to deploy resources we also need to enable billing.

Tangle the shell block below to a shell script by pressing , b t in emacs command mode:

# Create a project id based on date
export gcp_project_id="gitlab-gcp-"$(date +"%s")

# Create new project using a random project id
gcloud projects create $gcp_project_id

# Ensure billing is enabled for the project
export gcp_billing_account=$(gcloud alpha billing accounts list --limit=1 --format='value(name.basename())')
gcloud alpha billing projects link $gcp_project_id --billing-account $gcp_billing_account

# Make sure the project is set active
gcloud config set project $gcp_project_id

billingAccountName: billingAccounts/0175A9-AC77BE-CE5586
billingEnabled: true
name: projects/gitlab-gcp-1611720361/billingInfo
projectId: gitlab-gcp-1611720361

Step 3 - Create vertual machine

Once we have a project we can create a new virtual machine. To create a virtual machine we need to ensure compute engine apis are enabled.

Tangle the shell block below to a shell script by pressing , b t in emacs command mode:

# Ensure compute engine apis are enabled in the project
gcloud services enable compute.googleapis.com

# Create name for virtual machine based on date
export gcp_machine_name="gitlab-gcp-"$(date +"%s")

# Create the new machine
gcloud compute instances create $gcp_machine_name --zone australia-southeast1-a

Step 4 - Install docker on virtual machine

Next up we need to install Docker on the newly created virtual machine so that we can then deploy Gitlab as a container.

By default the virtual machine operating system for the vm we created on GCP is Debian. There are instructions for installing Docker on a debian machine here.

# Retrieve the vm name
export gcp_machine_name=$(gcloud compute instances list --limit=1 --format='value(name.basename())')

# Connect to the machine using ssh
gcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- "curl -fsSL https://get.docker.com -o get-docker.sh && sudo sh get-docker.sh"

# Install docker compose
gcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- "sudo curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose"

gcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- "sudo chmod +x /usr/local/bin/docker-compose"

Step 5 - Install gitlab via docker

Next step to create docker compose file for gitlab

   web:
     image: 'gitlab/gitlab-ce:latest'
     restart: always
     hostname: 'gitlab.example.com'
     environment:
       GITLAB_OMNIBUS_CONFIG: |
         external_url 'https://gitlab.example.com:3200'
     # Add any other gitlab.rb configuration here, each on its own line
     ports:
       - '3200:3200'
       - '443:443'
       - '2222:22'
     volumes:
       - '$GITLAB_HOME/config:/etc/gitlab'
       - '$GITLAB_HOME/logs:/var/log/gitlab'
       - '$GITLAB_HOME/data:/var/opt/gitlab'

Next up we need to tranfer file to gcloud vm and install the gitlab via docker composer

# Retrieve the vm name
export gcp_machine_name=$(gcloud compute instances list --limit=1 --format='value(name.basename())')

# Copy file to vm
gcloud compute scp docker-compose.yaml $gcp_machine_name:/home/$USER --ssh-key-file ~/.ssh/$USER --strict-host-key-checking=no

# Install gotlan with docker compose
sleep 2
gcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- 'sudo docker-compose up -d'

Step 6 - Teardown cloud resources

The Google Cloud Platform resources created by this process come at a cost, so it's important we have an easy way to teardown those resources as soon as we're finished with them!

The script below will delete any projects containing gitlab in the name along with any compute instances running in those projects.

Tangle the shell block below to a shell script by pressing , b t in emacs command mode:

# Iterate over any matching projects
for project in $(gcloud projects list | awk '{ print $1 }' | grep gitlab); do

  # Iterate over any instances in the project
  for instance in $(gcloud compute instances list --project $project --format="value(name)"); do

    # Delete the instance
    gcloud compute instances delete --quiet $instance --zone australia-southeast1-a --project $project

  done

  # Delete the project as well
  gcloud projects delete $project --quiet

done