GitLab CI | Stack | Fluid Attacks Help

GitLab CI

Rationale

GitLab CI is the system that orchestrates all the CI/CD workflows within our company. Such workflows are the backbone of our entire development cycle. By using it, we become capable of:

  1. Running automated processes on every commit
  2. Automatizing application testing
  3. Automatizing application deployment
  4. Automatizing every QA test we can think of

By having highly automated workflows, we become capable of deploying applications many times a day without sacrificing quality or security.

The main reasons why we chose GitLab CI over other alternatives are:

  1. It is Open source.
  2. Built-in support for GitLab: As GitLab is the platform we use for our product repository, it represents an advantage for us to be able to easily integrate our CI solution with it. All GUI related capabilities like pipelinesjobsCI/CD variablesenvironmentsschedules, and container registries are a consequence of such integration.
  3. It supports pipelines as code: It allows us to write all our pipelines as code.
  4. It supports horizontal autoscaling: In order to be able to run hundreds of jobs for many developers, all in real time, our system must support horizontal autoscaling.
  5. It supports directed acyclic graphs (DAG): Such capability allows us to make our integrations as fast as possible, as jobs exclusively depend on what they really should. It is a must when implementing a monorepo strategy like ours.
  6. Highly versatile configurations: As every piece of software usually has its own needs when it comes to building, testing and deploying, GitLab CI offers a vast set of configurations that range from parallelismstatic pages, and services to includesworkflows and artifacts.
  7. Highly versatile infrastructure: The AWS autoscaler allows configurations for s3 cachemachine typemax number of machinesspot instancesc5d instances ssd disk usageebs disksoff peak periodstaggingmax builds before destruction, among many others. The importance of having a highly versatile CI comes from the fact that our development cycle completely depends on it, making us to expect clockwork-like responsiveness and as-fast-as-possible computing speed.

Alternatives

The following alternatives were considered but not chosen for the following reasons:

  1. Jenkins: It did not support pipelines as code at the time it was reviewed.
  2. TravisCI: It required licensing for private repositories at the time it was reviewed.
  3. CircleCI: It did not support GitLab, it was very expensive, it was not as parameterizable.
  4. Buildkite: It is still pending for review.

Usage

We use GitLab CI for:

  1. Running all our CI/CD jobs.
  2. Managing all our CI pipelines as code.
  3. Configuring our AWS autoscaler as code.
  4. Implementing a Continuous Delivery approach for our development cycle. This means that although the whole process is automated, including deployments for both development and production, a manual merge request approval from a developer is still required in order to be able to deploy changes to production.

We do not use GitLab CI for:

  1. Highly time-consuming schedules that take longer than six hours, like Analytics ETL'sMachine learning training, among others. We use AWS Batch instead. The reason for this is that the GitLab CI is not meant to run jobs that take that many hours, often resulting in jobs being terminated before they can finish, mainly due to disconnections between the worker running the job and its GitLab CI Bastion.