Rationale

AWS EBS (Elastic Block Store) is the service we use for block-level storage. It allows us to have hard drives in the cloud. The main reasons why we chose it over other alternatives are the following:

• It seamlessly integrates with AWS EC2, allowing us to connect external hard drives to instances.
• It complies with several certifications from ISO and CSA. Many of these certifications are focused on granting that the entity follows best practices regarding secure cloud-based environments and information security.
• It provides a wide range of disk types that goes from SSDs with a size of 64 TiB and a throughput of 4,000 MiB/s to HHDs with a size of 16 TiB and a throughput of 500 MiB/s.
• It provides disks with different specializations. There are General Purpose and Provisioned IOPS SSDs and Throughput Optimized and Cold HHDs. By having all these different types of disks, we can easily select which one to work with, depending on the nature of the problem we are trying to solve.
• It supports point-in-time snapshots designed to back up all data that exists within a disk.
• Disks can be easily attached and detached from AWS EC2 machines, allowing us to easily change general machine configurations without losing any data.
• Disks can be encrypted using AWS KMS keys, which allows encryption of data moving between the disk and the instance using it, data at rest inside the volume, disk snapshots, and all volumes created from these snapshots.
• It supports data lifecycle policies, allowing us to create, retain and delete disks based on created policies.
• It supports monitoring and metrics using AWS CloudWatch.

Alternatives

Google Compute Engine (GCE) is one alternative that did not exist at the time we migrated to the cloud. GCP does not offer an equivalent to EBS. Instead, their entire disk service exists within GCE. It does not support disk encryption.

Note: Azure Disk Storage is another alternative that did not exist at the time we migrated to the cloud. A review is pending.

Usage

We use AWS EBS for

• GitLab CI bastion (we use a 16 GiB GP2 disk, as it only needs to have basic software installed such as GitLab Runner and Docker Machine; high disk throughput is not required);
• GitLab CI workers (we use 10 GiB GP3 disks just for hosting our workers' operating system. Additionally, workers come with high throughput 50 GiB internal NVMe disks, which are very useful for achieving as-fast-as-possible job performance within our CI);
• Batch processing workers (we use 8 GiB GP2 disks just for hosting the operating system. These workers also come with 50 GiB internal NVMe disks);
• Kubernetes cluster workers (we use 50 GiB GP2 disks for hosting the base operating system and stored containers for applications like our Platform. High disk throughput is not required as our ARM does not store any data within local disks);
• Okta RADIUS Agent (we use a 50 GiB GP2 disk. It is probably oversized since only the base operating system and RADIUS agent are required. High disk throughput is not required), and
• ERP (we use two disks: a 50 GiB GP2 disk for hosting the base operating system and a 200 GiB GP2 disk for hosting the ERP data).