MySQL DBaaS – A Performance/Cost Evaluation

Table of Contents

• Chapter 1: Key Insights
• Chapter 2: MySQL DBaaS Products Overview
  • AWS RDS for MySQL
  • MS Azure Database MySQL
  • GCP CloudSQL MySQL
• Chapter 3: Performance Results
  • Throughput
  • Latency
• Chapter 4: Performance/Cost Analysis
  • Costs
  • Performance / Cost Results
• Chapter 5: Conclusion
• Appendix
  • Benchmarking Specifications
  • Workload Specifications

Key Insights – Performance/Cost Results MySQL DBaaS

• AWS RDS for MySQL, Azure Database for MySQL and GCP CloudSQL are showing massive performance differences for the two workloads (eCommerce and web).
• Azure Database for MySQL is massively outperformed for the eCommerce workload, and nearly 20% worse than AWS RDS for MySQL for the web workload.
• GCP CloudSQL for MySQL shows great performance for the eCommerce workload, but not for the social workload.
• GCP CloudSQL MySQL has the most attractive pricing with nearly 15% lower monthly costs than AWS RDS for MySQL and Azure Database for MySQL.
• Azure Database for MySQL has the worst price/performance in these scenarios, while AWS RDS and GCP CloudSQL are sharing the first places among themselves.
• Price/performance is a useful indicator for pricy products such as Database-as-a-Service products
• Looking at performance/costs will help you to save money or to get more performance for your money.
• Similar to the recently analyzed PostgreSQL DBaaS price/performance, we can see the different strengths (and weaknesses) of the hyperscaler DBaaS products for different workloads. Workload matters a lot!

MySQL DBaaS Products Overview

We found over 35 commercial Database-as-a-Service products for MySQL. At first glance, many MySQL DBaaS products appear similar – technically and usually also in terms of cost structure and pricing.

Yet, performance measurements show greater differences than one would expect. This means that a pure cost analysis for decision-making lacks significance, and instead a price-performance evaluation is required.

Before we go into the results, let us take a quick look at the DBaaS MySQL products from AWS, Azure and GCP. An in-detail comparison can be found in our DBaaS Navigator.

AWS RDS for MySQL

AWS Relational Database System (RDS) was the first Database-as-a-Service product released for MySQL in 2013 for MySQL 5.6. AWS RDS for MySQL offers a fully managed databases service with all necessary deployment and management features, such as backup, alerting etc. AWS RDS for MySQL can be deployed on selected virtual machines from AWS as single-node, read replica or multi-availability-zone setup with one or two standby replica and load balancing.

The pricing model is similar to the regular virtual machine pricing. Users select dedicated resources with respect to machine size, storage type and backup sizing. They can either go with the “OnDemand” plan, or “Reserved Instances” with a 1- or 3-years contract period and respective reduction in price. Additionally, network costs and snapshot costs can occur.

An alternative DBaaS option for MySQL on AWS is the distributed MySQL solution from AWS Aurora. AWS Aurora is a cloud-native, automatic-scaling database, which is 100% compatible with MySQL and PostgreSQL.

Azure Database for MySQL

Azure Database for MySQL is the Microsoft Azure counterpart for AWS RDS for MySQL. The single server edition was released in 2017, but will be retired in 2024. The successor is called “Flexible Server” and allows more flexible server configurations. Features and pricing model are similar to the AWS product.

A major difference is that Azure Database for MySQL only offers a high-availability setup with 2 nodes and no read load balancing. Further, it is not possible to choose a storage type.

GCP CloudSQL for MySQL

Google Cloud’s DBaaS for MySQL is called CloudSQL for MySQL that was released in 2017. Compared to AWS RDS for MySQL and Azure Database for MySQL, one main difference is the flexible resource selection of the DBaaS flavor that allows the fine-grained selection of CPU and RAM per instance.

CloudSQL also offers single-node and HA setups with one replica, like Azure.

Performance Results – MySQL DBaaS

The first step of comparing Database-as-a-Service options for the Price/Performance KPI, are performance measurements.

We run the measurements with our approved benchANT framework for the following workloads:

• BenchBase TPC-C: read-heavy, eCommerce application workload (OLTP)
• BenchBase “Twitter”: read-heavy, web workload (simple transactions)

The full workload specifications can be found in the corresponding appendix section.

The performance measurements were executed on comparable resources on the respective DBaaS infrastructure, namely

• AWS RDS for MySQL: db.m5.4xlarge (16 vCPU/ 6 4 GB RAM), 500 GB GP3 storage
• Azure Database for MySQL: Gen5 General Purpose 16vCore (16 vCPU/ 64 GB RAM), 500 GB SSD storage
• CloudSQL for MySQL: flex 16 vCPU/ 64 GB RAM, 500 GB SSD storage

We used the default configuration of MySQL of each DBaaS provider.

All measurement scenarios were executed 3 times to capture variances coming from cloud resources.

The full benchmarking specifications can be found in the corresponding appendix section.

Throughput Results – MySQL DBaaS

• For the TPC-C workload, representing an eCommerce application, we see strong numbers for GCP CloudSQL MySQL, 15% better than AWS RDS for MySQL.
• Azure Database for MySQL show significant performant issues, delivering only 35% of GCP CoudSQL performance for this specific workload.
• For the web workload, “Twitter”, we see a completely different pattern. AWS RDS for MySQL outperforms Azure Database for MySQL by 21% and GCP CloudSQL for MySQL by 36%.
• Similar to the PostgreSQL DBaaS results we see massive performance differences for the same database for different workloads.

Latency 95th Results – MySQL DBaaS

• The latency results are congruent to the throughput results; the DBaaS products with high throughput show a better (= lower) latency.
• For the TPC-C workload we also measured significantly high latency values for Azure Database for MySQL, while AWS RDS MySQL (- 53%) and CloudSQL for MySQL (- 73%) have better and appropriate latency results.
• The latency results for the web workload show similar workloads for AWS RDS and Azure Database for MySQL and 23% higher latency values for GCP CloudSQL for MySQL.

Note: The latency results are not solely for read operations, but for whole transactions.

Performance / Cost Analysis

Cost Comparison

Estimating the monthly costs of a DBaaS product is more complex than for simple infrastructure resources. In the DBaaS Navigator we calculate the monthly costs adding up multiple components for a fair calculation due to different pricing models

• DBaaS flavor resource costs
• Storage costs
• Backup costs
• Network costs
• Support costs

Our research shows that the pricing models of the 3 hyperscalers, AWS, MS Azure and GCP are nearly identical. In consequence, we can simplify the calculation and just look at the two main cost positions – DBaaS flavor resource costs and storage costs.

Note: Please compare the detailed resource specs.

• The DBaaS flavor resource costs for AWS and MS Azure are nearly identical. The same goes for the storage pricing.
• GCP offers the lowest costs for the DBMS flavor, but has the most pricey storage. For our resource selection, GCP CloudSQL has the lowest monthly costs, which is 15% lower than MS Azure.

Note: Prices for cloud resources on hyperscalers are very dynamic. The costs were calculated in July 2023 and can be different now.

Performance / Costs

Considering only the costs, GCP CloudSQL would be the DBaaS product to select. But if we are looking of the performance per costs, which is a more reasonable KPI for decision-making, the picture changes.

Calculating price-performance ratios can be done in different ways. In this article we decided to calculate the costs per transaction. The mathematically unit would be ‘costs per month / transaction per second’.

• For the TPC-C workload, we see the best price/performance ratio for GCP CloudSQL for MySQL due to lowest price and best performance. GCP CloudSQL for MySQL offers a 36% better price/performance ratio than AWS RDS for MySQL, and 230% ratio than Azure Database for MySQL.
• For the web workload, “Twitter” we see a slightly different result. AWS RDS for MySQL delivers best performance per price, 16% better than GCP CloudSQL and 22% better than Azure Database for MySQL.

Conclusion

Similar to the PostgreSQL DBaaS price/performance evaluation we see valuable results coming from the performance benchmarks and the price/value calculation:

• DBaaS products for the same database technology show different performance results
• the performance results depend massively on the workload
• A solely cost analysis is worthless without the performance dimension.
• A performance/cost evaluation is useful before migrating your database into a DBaaS product. But specific measurements for your workload and your sizing need to be done.
• The identification of how and why these performance differences are huge for some workloads need to be identified in a larger, more complex study.

Appendix – Benchmarking Specification

Appendix – Workload & Database Specification