Gitlab CI/CD
Setting up Gitlab CI/CD.
I setup Continuous Integration (CI) and Continuous Delivery (CD) through Gitlab on a couple projects of mine and wanted to briefly share my experiences.
Tools and Methods
Setup CI - Stacky
A recent project of mine, stacky, was a good candidate for running integration tests since it interacts heavily with other services through Docker and also the local filesystem.
At the time of this writing I ended up with the following .gitlab-ci.yml:
image: ubuntu:18.04
variables:
LC_ALL: C.UTF-8
LANG: C.UTF-8
test:
script:
- apt-get update -qy
- apt-get install -y python3 python3-dev python3-pip
- pip3 install pipenv
- make install
- make test
it:
script:
- apt-get update -qy
- apt-get install -y python3 python3-dev python3-pip
- apt-get install -y docker.io
- apt-get install -y curl jq
- service docker start
- pip3 install pipenv
- make install
- cd it
- make install
- make test
In this I’ve defined two pipelines:
- test - For running unit tests against mocked data or functions.
- it - For running integration tests which are a series of example stacky file project scenarios.
Trying to get Docker running inside of Docker proved to be little troubling and ultimately led to me using the base image ubuntu:18.04 as it’s a distro I’m familiar with and I knew would have most questions answered on a variety of Google results.
Setup CD - mega-mac.com
While I’m not extremely active on mega-mac.com I knew I wanted to streamline things by having updates to master be automatically published.
At the time of this writing I ended up with the following .gitlab-ci.yml:
image: ubuntu:18.04
variables:
LC_ALL: C.UTF-8
LANG: C.UTF-8
dist:
script:
- apt-get update -qy
- apt-get install -y build-essential hugo
- make dist
deploy:
script:
- apt-get update -qy
- apt-get install -y build-essential hugo
- apt-get install -y python python python-pip jq
- pip install awscli
- make publish
only:
- master
In this I’ve defined two pipelines:
- dist - Tests running Hugo to create the rendered HTML.
- deploy - On changes to master makes calls through the AWS CLI to publish the site.
Most of the AWS CLI was already figured out in the Hugo and AWS post where you can see the make publish command. Which has more recently changed to the following:
invalidate-cdn:
.core-devops/scripts/aws/cloudfront-manage-cdn.sh invalidate $(BUCKET)
publish: dist deploy invalidate-cdn
Due to the addition of using Amazon’s Cloudfront as a CDN.
In addition to the .gitlab-ci.yml file a few variables needed to be added to Gitlab’s CI/CD settings so that it could authenticate with AWS:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_DEFAULT_REGION
I should note that I had little success with using these when they were “Protected” or “Masked” in the Gitlab variable settings.
Results
Stacky is now running both unit and integration tests on PRs after being submitted and again after being merged into master.
mega-mac.com is now being published when there are changes to master — like when a PR is merged.
Discussion
A lot of the heavy lifting for this was on Gitlab. I just had to monkey around with YAML and fumbled with local Docker containers until the errors stopped flying around. There are plenty of other tools I could have used like CircleCI, Travis CI or Jenkins. Some of these tools I have used in the past or am currently using but I chose to give a Gitlab a shot for these projects.
While it’s not an issue right now I feel like the build times could be decreased if I chose a more slim base-package besides ubuntu:18.04. But for now it’s convenient.
Additionally, mega-mac.com doesn’t really need a Staging/QA environment to test changes against currently but in the future Gitlab Environments could be worth looking into.