Open Telekom Cloud MySQL RDS: A Leading DBaaS

Key Findings

Feature Set: The features of DBaaS range from automatic deployment, backup and alerting to additional support. A high-level comparison of the three DBaaS services shows:

• Features: The DBaaS products compared here are native MySQL DBaaS solutions with a similar technical structure and comparable basic features in deployment, backup, alerting, and management.
• Pricing: Identical DBaaS pricing structure for all products and estimated monthly costs at nearly identical levels.
• Support: Only OTC offers free email and telephone support at the business level, instead of up to 10% additional DBaaS costs charged by the US hyperscalers AWS and MS Azure.
• GDPR compliance: Only OTC's DBaaS solution is not subject to the US Cloud Act.

Throughput & Latency: The performance benchmarks performed deliver excellent measurement results for OTC RDS for MySQL.

• The tested workload is based on the TPC-C benchmark which models an eCommerce system.
• OTC RDS and AWS RDS show throughput and latency results on a comparable level and OTC RDS surpasses the throughput of Azure Database for MySQL by over 160%.
• With the OTC setting "Private IP”, OTC RDS MySQL increased its throughput by 86%.

Price/Performance: When the DBaaS instance and storage costs are considered, the MySQL DBaaS solution from the Open Telekom Cloud is more than competitive overall.

• The OTC RDS MySQL product with Private IP shows the lowest costs per operation.
• OTC is thus establishing itself as a strong European alternative in the MySQL DBaaS sector.

1. Price/Performance:
The Decisive Indicator for DBaaS

Database-as-a-Service (DBaaS) solutions offer numerous advantages over self-managed database management systems. These include time savings during deployment and application development thanks to simple APIs, guaranteed service level agreements (SLAs) and an overall lower Total Cost of Ownership (TCO).

For these reasons, many companies rely on DBaaS solutions from Amazon Web Service (AWS), Microsoft Azure or the European solution from Open Telekom Cloud (OTC).

A comparison of these DBaaS solutions must be made holistically on various levels:

• Features: What technical and organizational features for deployment, management, support, and compliance does the respective DBaaS product offer?
• Price: What are the monthly costs for resources, storage, backup, network, and support for the respective DBaaS?
• Performance: What technical performance does the target DBaaS offers provide?

Currently, over 30 MySQL DBaaS products exist in the market. A lot of them offer a native MySQL managed database, but there are also advanced MySQL-compatible DBaaS products like AWS Aurora out there.

In this article, we only compare the following native MySQL DBaaS products with a focus on price and performance:

• AWS RDS for MySQL
• MS Azure Database for MySQL - Flexible Server
• OTC RDS MySQL

While AWS and MS Azure offer their product in multiple regions, OTC is providing a region for Germany, the Netherlands and Switzerland. Apart from the fact that AWS and MS Azure offer a larger number of instances, there is no significant technical difference in the basic functions for deployment, backup, alerting and management of the three DBaaS products.

However, one key point of difference is data protection for European companies. While Open Telekom Cloud, as a German company, is 100% EU GDPR compliant, its US competitors are subject to the US Cloud Act, which undermines GDPR compliance.

The price model of the three DBaaS solutions hardly differs and enables a fair price comparison. There are costs for:

• The DBaaS instance defined by number of nodes and number of vCPU and RAM per node.
• Storage costs for data storage per GB.
• Network costs for data ingress and egress.
• Costs for backup storage per GB
• Support costs.

While all providers offer a "free support tier", which is access to the documentation, only Open Telekom Cloud offers free business support by email and telephone.

A cost overview of all three DBaaS we used for the performance measurements in this technical analysis can be found in the appendix.

2. Performance
Same Sizing, Same Database, Same Performance?

The performance differences between DBaaS products are often underestimated and classified as almost identical for the same database technology such as MySQL. However, analyses show that the database configuration, the underlying hardware used internally and the implementation of the DBaaS layer have a major influence on the performance of the DBaaS and can result in significant differences.

For the evaluation, we consider a transactional eCommerce use-case. The workload is based on the TPC-C, with relational read and write queries simulating an eCommerce application.

For the performance measurements, we use comparable DBaaS resources.

For all DBaaS providers, the public endpoints of the DBaaS instances were used to run the benchmarks. In addition, for OTC we also measured the DBaaS performance by using the private endpoint which is equivalent to the virtual private network options of the US hyperscalers. It is important to emphasize that by using the virtual private network endpoints of the hyperscaler DBaaS, a performance improvement might be achieved as well.

The default MySQL configuration of each DBaaS offer was used and not adjusted. The full workload and benchmarking details can be found in the appendix.

2.1 Throughput Results

Throughput is one of the most important database performance indicators. It shows how many operations can be handled by the database infrastructure per time unit.

• The default OTC configuration with Public IP outperforms AWS by 11% and MS Azure by 168%.
• The Public IP option nearly doubles the performance of the OTC DBaaS solution.

2.2 Costs per Operation

Based on the throughput results and the transparent online available pricing details, we can calculate an even more important KPI. The price-performance-ratio.

The KPI is calculated by dividing the monthly costs by the operations per second to demonstrate how costly one operation per second is. A detailed cost calculation can be found in the appendix.

• AWS RDS and OTC RDS are on the same price-performance level with significantly lower costs per operation compared to MS Azure.
• OTC RDS for MySQL with the Private IP option has the best price-performance-ratio and shows nearly half the costs of AWS RDS.

2.3 Latency

To complete the technical performance analysis, we also look at the database latencies of the MySQL DBaaS.

The database latency indicates the delay of the operation. Database latency is an important parameter for many applications with users, as it influences the quality of interaction with the application from the user's perspective.

The specified latency is the 95th percentile of all database operations, which means that 95% of all operations were faster or equal to the measured values. In this case, this is the combined latency of read and write operations.

• For this transactional eCommerce use case, we see OTC RDS (Public IP) and AWS RDS on an identical level.
• Azure Database for MySQL shows with 300 milliseconds significantly higher latency values.
• Again, the Private IP option improves the performance of OTC RDS and reduces the latency from 114 milliseconds to 52 milliseconds.

3. Conclusion

Database management systems are the backbone of every modern application and Database-as-a-Service (DBaaS) may reduce the total cost of ownership of database operations.

The technical evaluation that we have undertaken in this article shows a holistic comparison of the most important MySQL DBaaS offerings from AWS, MS Azure and the Open Telekom Cloud.

Overall, the results state that the Open Telekom Cloud with its MySQL RDS service is more than at par in performance and price-performance with the DBaaS solutions of the US hyperscalers in the selected benchmarking scenarios. In most measurements, OTC RDS outperforms AWS and MS Azure.

Open Telekom Cloud is the only provider to offer inclusive e-mail and call support in the pricing. Such typical service is chargeable on other providers.

Together with the GDPR advantage, OTC offers a well-rounded and high-performance MySQL DBaaS product for European businesses.

Appendix

About the Authors

Jan Ocker is one of the founders and CGO of benchANT, a company that specializes in benchmarking and performance testing of databases. Jan started his career as an IT project manager in eCommerce. Additionally, he focused on data analytics and the optimization of SQL queries, before he founded benchANT.

Daniel Seybold is one of the founders and CTO of benchANT. Daniel started his career as a researcher focusing on distributed systems and databases. During his academic career he published over 20 papers on cloud and database performance-related topics at renowned scientific conferences and completed his PhD with the thesis “An automation-based approach for reproducible evaluations of distributed DBMS on elastic infrastructures”.

These research results are the technical foundation of benchANT which pursues the goal of supporting organizations in selecting the right database for their use case.

From his point of view, there is no "best" database, but only a better-suited and more efficient database solution for each use case.

Disclaimer

Open Telekom Cloud commissioned the work presented in this document. Open Telekom Cloud chose the competitors, the test, and the database sizes. benchANT chose the most compatible configurations for the other tested DBaaS, executed the benchmarks, analyzed the results, and wrote the text.

Cost Calculation

As already mentioned in the introduction, the price models of the three DBaaS offerings are comparable. The following price structure for DBaaS and 500GB storage resulted in the tested DBaaS flavors as of 01.02.2024.

The monthly costs were calculated for 730 hours and with an exchange rate of 1.08 USD/EUR in the region Frankfurt or comparable.

We have not included the costs for network, backup and support in the cost calculation, as these are very application-specific. Moreover, in our experience, these are in the region of 10% and are also almost identical, except for free support from OTC. Open Telekom Cloud is the only provider offering inclusive call and e-mail support.

Since all performance measurements were carried out on on-demand resources, we also used the on-demand prices for the cost calculation and price/performance calculation.

For completeness, we mention that all DBaaS providers also offer discount scales for 1-year and multi-year reserved DBaaS instances. With OTC, cost savings of over 45% are possible. More detailed information on the pricing of reserved instances can be found in the price calculators of the respective providers.

Benchmarking Methodology

All benchmarks are executed with benchANT’s benchmarking platform that fully automates the entire benchmarking process to ensure deterministic and reproducible benchmark results (for more details, see the associated publications on Mowgli and benchANT). A single benchmark execution comprises the following steps carried out by the benchANT platform:

• deploy a new (MySQL) DBaaS instance.
• deploy a benchmark VM (16 vCores / 64 GB RAM) to run the YCSB on the same cloud provider in the same region.
• trigger the LOAD phase and fill the database with the initial data set.
• wait for 5 minutes as a stabilization period before triggering the RUN phase.
• trigger the RUN phase of 30 minutes for the actual workload.
• collect and process the results and metadata.
• tear down the DBaaS instance and the benchmark VM.

Each benchmark scenario has been measured three times, i.e. the described process has been carried out three times per DBaaS provider and workload, resulting in 24 data sets. For each scenario, the three results sets are analyzed towards outliers, but no significant outliers were detected. The presented results are averages over the three runs.

The applied eCommerce workload is part of the BenchBase open-source benchmark suite maintained by Carnegie Mellon University.

The following table describes the most relevant benchmark settings per workload and the full benchmarking details and results can be found on GitHub.