benchANT Homepage
benchANT Homepage

The Cloud Database Benchmarking Blog

Are you interested in cloud database benchmarking and IT performance engineering?

The benchANT blog contains articles on the most important databases, cloud providers and,
of course, cloud database benchmarking results.

You would like to be informed about new articles? Follow us! Subscribe to our newsletter list.

Exploiting the YCSB for Custom Workloads

Updated: 2024-03-12

The Yahoo Cloud Serving Benchmark (YCSB) is a well-known tool to measure and evaluate the performance of database management systems (DBMS). One of the major advantages of YCSB is the very simple workload it uses as this allows support for many different DBMS. On the down-side, YCSB standard workloads do not make use of any even marginally sophisticated DBMS features.

In this article, we sketch how to exploit YCSB in order to build different workloads and also make use of other types of queries and updates to the dataset. While such an approach is still work, it relieves the programmer from building any feature beyond workload handling that are still important for benchmarking: metric collection, thread handling, and start-up and tear-down handling.

Thumbnail for Exploiting the YCSB for Custom Workloads
Read now

Competitive Yet? A Glance at OVHcloud's new MongoDB DBaaS

Updated: 2024-02-12

Database-as-a-Service (DBaaS) has the potential to reduce required manpower, lower time-to-delivery, and has the potential to reduce the total cost of ownership (TCO) of database services. As with most other cloud services, the DBaaS market worldwide is dominated by the American companies, in particular the hyperscalers AWS, Microsoft Azure, and Google Cloud Platform (GCP). Unfortunately, relying solely on U.S. companies is not without risk for European customers, for instance because of the interplay of Cloud Act and GDPR.

Therefore, it is beneficial to keep an eye on the European market and to evaluate European alternatives such as OVHcloud's recently released MongoDB DBaaS. In this article, we evaluate this new European service against AWS DocumentDB and Azure CosmosDB MongoDB vCore.

Thumbnail for Competitive Yet? A Glance at OVHcloud's new MongoDB DBaaS
Read now

A Database Benchmarking Recap on 2023

Updated: 2024-01-24

2024 is here and it's time to look back at the past year 2023.

We naturally want to focus on databases, Database-as-a-service, (cloud) infrastructures and their performance.

Let's review the year 2023 together.

Thumbnail for A Database Benchmarking Recap on 2023
Read now

New Azure VMs: Database Benchmark Performance Analysis (2024)

Updated: 2024-01-26

In the fall of 2023, Microsoft Azure launched a new generation of virtual machines with the latest hardware.

Our colleagues at Cloud-Mercato have already carried out CPU performance measurements and identified significant performance benefits. Do databases also benefit from these new VMs? And is it worth switching?

We have measured it!

Thumbnail for New Azure VMs: Database Benchmark Performance Analysis (2024)
Read now

What is Database Benchmarking?

Updated: 2023-10-05

Database benchmarking is a proven and clearly defined method for analysing and comparing the performance characteristics of databases or database management systems (DBMS).

Performance and scalability is a top current issue in IT. Data-driven applications and cloud infrastructures are current challenges for databases. Database benchmarking can provide reliable, quantitative answers to many questions and enable the selection and optimisation of performance, but also e.g. cloud costs.

Learn everything about (cloud) database benchmarking from classic SQL databases to NoSQL and NewSQL databases in distributed cloud systems.

Thumbnail for What is Database Benchmarking?
Read now

Navigating the Database Performance Landscape: A Comprehensive Guide to Database Benchmarking Suites (2024)

Updated: 2024-03-05

In the dynamic database management systems market, performance is a key differentiator.

Database benchmarking suites provide the tool to assess performance, guide us through the vast landscape of options, and help developers, engineers, and decision-makers make informed choices.

This article dives into the various benchmarking suites for different workload domains and provides an overview of their origin, implementations, and unique features.

Join us on a journey through YCSB, TSBS, TPC-C, and others.

Thumbnail for Navigating the Database Performance Landscape: A Comprehensive Guide to Database Benchmarking Suites (2024)
Read now

How-to: A Starting Point for PostgreSQL Configuration Tuning

Updated: 2023-12-21

There is always a lack of information and expertise regarding database configuration parameters. Tuning databases for specific use cases and workloads usually requires experience.

In this article, we show you how to start with database tuning and some alternative methods instead of hiring high-paid database consultants but still get better performance from your data infrastructure.

Thumbnail for How-to: A Starting Point for PostgreSQL Configuration Tuning
Read now

reCap of the AWS reInvent from the Perspective of a Database Enthusiast

Updated: 2024-01-09

End of November, Daniel, CTO & Founder of benchANT) had the chance to attend the AWS re--Invent in Las Vegas and it was (again) a blast.

Apart from getting first-hand insights into AWS's brand new cloud features, meeting with old acquaintances and making new ones, and enjoying the crazy vibe of Las Vegas, of course, there are also many database vendors presenting their latest features in the exhibition.

Thumbnail for reCap of the AWS reInvent from the Perspective of a Database Enthusiast
Read now

AlloyDB Omni: The better PostgreSQL?

Updated: 2023-11-27

AlloyDB from Google Cloud is the new modern DBaaS solution with a 100% PostgreSQL-API and competing with AWS Aurora and CosmosDB for PostgreSQL from MS Azure.

It claims to be more than 2x faster than standard PostgreSQL for transactional workloads, and a 100 times faster for analytical queries.

Since a few week AlloyDB Omni, a downloadable, self-managed database version of AlloyDB is available.

Let's get some independent numbers!

Thumbnail for AlloyDB Omni: The better PostgreSQL?
Read now

PostgreSQL DBaaS – Performance Tuning Differences of AWS, Azure and GCP

Updated: 2023-11-22

In the last weeks, we identified huge performance differences for the PostgreSQL DBaaS solutions from AWS, GCP and MS Azure.

We already did an in-depth query performance analysis for different queries and are now going to take a deeper look into performance-critical settings of PostgreSQL for these 3 DBaaS products.

We are getting closer!

Thumbnail for PostgreSQL DBaaS – Performance Tuning Differences of AWS, Azure and GCP
Read now

Database-as-a-Service: What, when, how?

Updated: 2024-04-01

Im November 2023 veröffentlichten die Gründer von benchANT einen Artikel in iX, einer renommierten deutschen IT-Zeitschrift. Hier fassen wir den ursprünglich deutschen Artikel in einer englischen Version zusammen. Der Artikel enthält Auszüge aus der DBaaS-Studie von benchANT und anderen Artikeln, die wir kürzlich über DBaaS veröffentlicht haben.

Der Artikel geht noch einmal auf den DBaaS-Markt und die Arten von Anbietern ein, erörtert, wann sich der Einsatz von DBaaS im Vergleich zum Betrieb eines selbstverwalteten DBMS lohnen kann, und zeigt schließlich anhand eines Beispiels, wie Benchmarking bei der Auswahl der richtigen DBaaS-Lösung helfen kann.

Thumbnail for Database-as-a-Service: What, when, how?
Read now

How-to: Data-Driven Capacity Planning with MongoDB on OTC Infrastructure

Updated: 2023-11-08

Ideally, an IT application grows its user base over time and in consequence the underlying data infrastructure needs to scale to handle the increasing workload demands.

We show a data-driven approach to efficient capacity planning for a self-hosted MongoDB-based IT application that takes both costs and performance into account.

This approach leads to an efficient data infrastructure - both from a technical and a business perspective.

Thumbnail for How-to: Data-Driven Capacity Planning with MongoDB on OTC Infrastructure
Read now

PostgreSQL DBaaS – In-depth Query Performance Analysis

Updated: 2023-11-08

A few weeks ago we published an article about cost/performance results for PostgreSQL DBaaS products of the three hyperscalers AWS, MS Azure and Google Cloud Platform.

We were surprised by the massive performance differences and decided to take a closer look at the query performance to identify possible reasons.

This is going to be an in-depth dive into query performance.

Thumbnail for PostgreSQL DBaaS – In-depth Query Performance Analysis
Read now

Database Performance for the Masses: A Glance on Auto-Tuning Tools for Database Management Systems

Updated: 2023-11-08

Database Management Systems (DBMS) are a central part of every modern application, and their proper configuration is crucial to guarantee quality of service on application level, but also cost-efficient operation of IT services.

Yet, finding a configuration of a DBMS that suits the workload and specific use case is difficult, time-consuming, and error-prone when done manually. The rise of AI-based tool support offers an opportunity to let algorithms do the heavy lifting.

In recent years some companies have been founded with the goal to help their users with database performance tweaking. In this article, we introduce three of them who have focused on traditional open source, relational DBMS such as PostgreSQL and MySQL. We glance over their history, solution, and marketing claims.

Thumbnail for Database Performance for the Masses: A Glance on Auto-Tuning Tools for Database Management Systems
Read now

How-to: Performance Benchmarking of MongoDB

Updated: 2023-10-17

"One accurate measurement is worth a thousand expert opinions." - Grace Hopper

This statement from one of the IT pioneers is still relevant today with the vastly growing amount of technologies to build data intensive and performance critical applications.

This article will explain what we learned and show you a how-to about performance benchmarks of one of the most popular NoSQL databases - MongoDB.

Let's dive in!

Thumbnail for How-to: Performance Benchmarking of MongoDB
Read now

MySQL DBaaS - Performance/Cost Evaluation

Updated: 2023-10-17

Last week we analyzed the PostgreSQL DBaaS products of the three hyperscalers AWS, MS Azure and Google Cloud Platform to spot deviations and understand their differences.

As the insights are more than interesting and useful, we proceed with this analysis for the MySQL DBaaS products of the same providers.

Check out the results of this week's DBaaS performance/cost analysis of the MySQL DBaaS products.

Thumbnail for MySQL DBaaS - Performance/Cost Evaluation
Read now

PostgreSQL DBaaS - Performance/Cost Evaluation

Updated: 2023-10-03

Database-as-a-Service (DBaaS) products are considered the future of database management, known to accelerate IT development, to compensate for skill shortages and to save costs.

However, our research shows that there are significant differences in terms of performance and cost structures across the market.

Recently, we analyzed the PostgreSQL DBaaS offering of the three hyperscalers AWS, MS Azure and Google Cloud Platform to spot deviations and understand their differences

Thumbnail for PostgreSQL DBaaS - Performance/Cost Evaluation
Read now

PostgreSQL 16 Performance

Updated: 2023-10-04

Last week, the newest major version of one of the most popular database management systems got released: PostgreSQL v16.

The new release comes with a lot of new features and improvements that also include performance optimizations for specific query types. But which performance improvements can you expect for standard workloads, or are there even performance regressions?

We ran some measurements. And here are the results!

Thumbnail for PostgreSQL 16 Performance
Read now

NoSQL Benchmark: MongoDB vs ScyllaDB

Updated: 2023-10-09

We compare the performance and scalability capabilities of MongoDB Atlas and ScyllaDB Cloud, two important NoSQL Database-as-a-Service solutions.

In more than 132 measurement results, we show the strengths of these databases for 5 important workloads - from caching to IoT sensors.

See the results of this extensive in-depth benchmarking study!

Thumbnail for NoSQL Benchmark: MongoDB vs ScyllaDB
Read now

NoSQL Comparison: MongoDB vs ScyllaDB

Updated: 2023-09-27

MongoDB is one of the most popular database management system and widely adapted. We compare MongoDB in an in-depth technical analysis with ScyllaDB, a rising star in the NoSQL database world.
Thumbnail for NoSQL Comparison: MongoDB vs ScyllaDB
Read now

DBaaS Deployment Times

Updated: 2023-09-19

In the course of our DBaaS benchmark measurements, another interesting metric drops - deployment time.

Here you can find the measurement results and some background on deployment time.

Thumbnail for DBaaS Deployment Times
Read now

Apache IoTDB Performance Benchmarks

Updated: 2023-09-19

Apache IoTDB is an emerging open source time series database. It is an Apache top-level project and from now on also represented in our database ranking.

And not just somehow!

Apache IoTDB convinces with outstanding performance results right from the start.

See for yourself.

Thumbnail for Apache IoTDB Performance Benchmarks
Read now

MongoDB Atlas vs. CosmosDB: An Holistic Databases-as-a-Service Benchmark Report (2023)

Updated: 2023-04-03

MongoDB Atlas and CosmosDB are two of the most popular Database-as-a-Service (DBaaS) products on the market. But which is the better one for your requirements?

In this article, we will perform a deep and comprehensive qualitative and quantitative benchmark of these two DBaaS products. We will discuss the differences, benefits and technical aspects to give you the information you need to make an informed decision.

See what our holistic benchmarking analysis revealed.

Thumbnail for MongoDB Atlas vs. CosmosDB: An Holistic Databases-as-a-Service Benchmark Report (2023)
Read now

DBaaS Navigator - Introduction & Disclaimer

Updated: 2024-03-12

DBaaS Navigator - holistic comparison of the most popular Database-as-a-Service products. More than 60 attributes on technical features, management tools, support contracts, performance measurements and price-performance assessment.

  • Step-by-Step Guide
  • Roadmap
  • Disclaimer
Thumbnail for DBaaS Navigator - Introduction & Disclaimer
Read now

The Top 208 Database-as-a-Service (DBaaS) Providers (2023)

Updated: 2023-10-10

Managed cloud databases, also called Database-as-a-Service (DBaaS), have experienced enormous growth in recent years.

We have analyzed more than 208 offers from more than 50 different providers and compiled them in a structured list with an exemplary cost calculation.

Thumbnail for The Top 208 Database-as-a-Service (DBaaS) Providers (2023)
Read now

The Data-Based MongoDB vs. Cassandra Study (2022)

Updated: 2023-10-05

MongoDB and Apache Cassandra are two of the most important NoSQL database management systems. For which use case is which DB suitable?

If you search for the differences, you will find many articles with theoretical statements about performance and scalability.

For us, this information was not enough.

We have measured!

Thumbnail for The Data-Based MongoDB vs. Cassandra Study (2022)
Read now

Bonus Material|List of Database Sources

Updated: 2023-10-05

Discover a variety of informative links to internal and external guides and lists.

These can help you choose the right database technologies and make the best decisions.

The cheat sheets for professionals.

Thumbnail for Bonus Material|List of Database Sources
Read now

ARM Instances for DBMS: Price-Efficient or Just Cheap?

Updated: 2023-10-05

Data centers and public cloud providers advertise low-cost ARM virtual machines.

But are they suitable for running database management systems?

We looked at quite small ARM instances for small-scale applications and benchmarked them against comparably large Intel-based x86 VMs.

The test results are available here!

Thumbnail for ARM Instances for DBMS: Price-Efficient or Just Cheap?
Read now

ARM Instances for DBMS: Part 2 – Are Larger ARM Instances Better for Databases?

Updated: 2023-10-05

Data centers and public cloud providers advertise low-cost ARM virtual machines.

But are they suitable for running database management systems?

We looked at quite small ARM instances for small-scale applications and benchmarked them against comparably large Intel-based x86 VMs.

The test results are available here!

Thumbnail for ARM Instances for DBMS: Part 2 – Are Larger ARM Instances Better for Databases?
Read now

Database Workload - Read-Write Ratio

Updated: 2023-10-05

Which impact does the distribution of read and write operations have on a database management system?

Does such an influence exist? Is it always similar? How does a specific database behave when the workload changes?

We did what we do best (and fastest), we benchmarked.

9 workload variations - 2 distributed cloud databases - 3 key insights.

Thumbnail for Database Workload - Read-Write Ratio
Read now

benchANT Guide|The benchANT-Score

Updated: 2023-08-28

Learn step-by-step how to efficiently measure cloud and database performance with the innovative benchmarking platform.

Learn the tips & tricks for the correct operation of benchANT.

Become a benchANT expert!

Thumbnail for benchANT Guide|The benchANT-Score
Read now

YCSB Benchmark Guide

Updated: 2023-12-08

The Yahoo! Cloud Serving Benchmark (YCSB) is the most popular NoSQL benchmark suite.

Learn all about the YCSB in this guide, including 4 exclusive articles from YCSB developers.

We show you what it can do and how you can customize it for your situation.

Thumbnail for YCSB Benchmark Guide
Read now

Performance Cassandra 4.0 Release

Updated: 2023-10-05

Apache Cassandra has released a new major release of their popular open-source, column-oriented No-SQL database management system (DBMS) after more than 6 years.

In doing so, they announce major improvements in terms of quality and performance!

How fast is the new Cassandra 4.0.0? And how does it perform against the previous versions?

We have made benchmark measurements on throughput and latency.

Thumbnail for Performance Cassandra 4.0 Release
Read now

Basisc|What is Cloud Database Benchmarking?

Updated: 2024-03-05

Learn the basics of cloud database benchmarking from scratch.

Learn everything worth knowing from classic database benchmarking to cloud database benchmarking.

Start here!

Thumbnail for Basisc|What is Cloud Database Benchmarking?
Read now

Basics|What is Database Benchmarking?

Updated: 2024-03-05

Learn the basics of cloud database benchmarking from scratch.

Learn everything worth knowing from classic database benchmarking to cloud database benchmarking.

Start here!

Thumbnail for Basics|What is Database Benchmarking?
Read now