benchANT Homepage
benchANT Homepage

The Cloud Database Benchmarking Experts

Classic database benchmarking is difficult, but cloud database benchmarking is even more complicated.

Two previously independent IT domains - databases and cloud - need to be understood and measured together.

The two co-founders Dr. Daniel Seybold and Dr. Jörg Domaschka combine their domain expertise and experience. The result is benchANT's unique Benchmarking-as-a-Service platform.

Find out what distinguishes this expertise and learn more about the long, insightful journey the two experts have taken together.

DBMS & Cloud -
independent IT domains.
benchmarking experts.
More than
20 years
performance engineering experience.

The Two Experts and Co-founders of benchANT

Dr. Daniel Seybold

Dr. Jörg Domaschka

The recognised leading benchmarking experts from benchANT
Database benchmarking experts and founders of benchANT: Dr. Jörg Domaschka (left) and Dr. Daniel Seybold (right)

Research & Publications on Clouds and Databases

Since 2011, benchANT's two technical founders have contributed to more than 120 scientific publications in the area of cloud computing, distributed systems, databases, and their performance, scalability, availability, and elasticity.

We highlight some core publications in more detail here, as they deepen the understanding and benefits of cloud & database benchmarking and clearly illustrate benchANT's approach.

The complete publication list of the two founders can be found on Google Scholar:

Publications Dr. Jörg Domaschka
Publications Dr. Daniel Seybold

2021 - Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads: Vision Paper

by Jörg Domaschka, Mark Leznik, Daniel Seybold, Simon Eismann, Johannes Grohmann, Samuel Kounev | Link

Designing synthetic benchmarks based on real workload traces for "what-if" benchmarking scenarios. Simulation of hypothetical scaling workloads as well as anomalies in benchmarking measurements.

2020 - Hathi: An MCDM-based Approach to Capacity Planning for Cloud-hosted DBMSs

by Jörg Domaschka, Simon Volpert, Daniel Seybold | Link

An evaluation-based Multi Criteria Decision-Making (MCDM) framework for planning cloud-hosted distributed DBMSs with respect to throughput, latency, cost, consistency, availability, and stability.

2020 - Baloo: Measuring and modeling the performance configurations of distributed DBMSs

by Johannes Grohmann, Daniel Seybold, Simon Eismann, Mark Leznik, Jörg Domaschka | Link

Framework for systematic measurement and modeling of various performance-relevant configurations of distributed DBMSs in cloud environments. Dynamic estimation of the required number of measurement configurations as well as the number of required measurement repetitions per configuration based on a desired target accuracy.

2020 - Towards Understanding the Performance of Distributed Database Management Systems in Volatile Environments

by Jörg Domaschka, Daniel Seybold | Link

Experiences with performance evaluation of DBMSs hosted in the cloud to find well-suited configurations for specific use cases. Workload dependencies, cloud environment and DBMS with different parameters.

2020 - King Louie: reproducible availability benchmarking of cloud-hosted DBMSs

by Daniel Seybold, Stefan Wesner, Jörg Domaschka | Link

Evaluation of availability and performance guarantees of distributed DBMSs with high availability mechanisms in case of cloud resource failures. Benchmarking process with fault injection and 16 availability evaluations.

2020 - Performance Results of a Containerized MongoDB DBMS

by Daniel Seybold, Christopher B Hauser, Georg Eisenhart, Simon Volpert, Jörg Domaschka | Link

Dataset with performance KPIs on different containerized MongoDB setups, focusing on the impact of storage hardware.

2019 - Kaa: Evaluating elasticity of cloud-hosted dbms

by Daniel Seybold, Simon Volpert, Stefan Wesner, André Bauer, Nikolas Herbst, Jörg Domaschka | Link

Automated evaluation process for the elasticity of distributed DBMSs. Case study with significant elasticity scenarios.

2019 - The cloud application modeling and execution language

by Achilleas P Achilleos, Kyriakos Kritikos, Alessandro Rossini, Georgia M Kapitsaki, Joerg Domaschka, Michal Orzechowski, Daniel Seybold, Frank Griesinger, Nikolay Nikolov, Daniel Romero, George A Papadopoulos | Link

Defining the Cloud Application Modeling and Execution Language (CAMEL) as a multi-cloud modeling language.

2019 - A survey on data storage and placement methodologies for cloud-big data ecosystem

by Somnath Mazumdar, Daniel Seybold, Kyriakos Kritikos, Yiannis Verginadis | Link

Management of Big Data and data storage in the cloud with a focus on non-functional aspects such as performance. Technology comparison and gap analysis.

2019 - SORRIR: A Resilient Self-organizing Middleware for IoT Applications

by Jörg Domaschka, Christian Berger, Hans P Reiser, Franz J Hauck, Gerhard Habiger, Frank Griesinger, Matthias Tichy, Jakob Pietron, Philipp Eichhammer | Link

Position paper of a robust and self-organizing execution platform for IoT applications.

2019 - Mowgli: Finding your way in the dbms jungle

by Daniel Seybold, Moritz Keppler, Daniel Gründler, Jörg Domaschka | Link

Database evaluation framework for data-intensive technologies like Big Data and IoT with cloud resources.

2018 - The impact of the storage tier: A baseline performance analysis of containerized dbms.

by Daniel Seybold, Christopher B Hauser, Georg Eisenhart, Simon Volpert, Jörg Domaschka | Link

Evaluation method for the performance overhead of containerized DBMSs by combining three operating models and two storage backends

2018 - A Provider-Agnostic Approach to Multi-cloud Orchestration Using a Constraint Language.

by Daniel Baur, Daniel Seybold, Frank Griesinger, Hynek Masata, Jörg Domaschka | Link

Multi-cloud constraint language for application and resource description for requirements analysis and bid matching with the goal of independent vendor selection.

2017 - Gibbon: An availability evaluation framework for distributed databases.

By Daniel Seybold, Christopher B Hauser, Simon Volpert, Jörg Domaschka | Link

Framework for analyzing the high availability of distributed database systems

2017 - Is distributed database evaluation cloud-ready?

by Daniel Seybold, Jörg Domaschka | Link

Cloud-centric analysis of distributed database evaluation frameworks based on performance, scalability, elasticity and consistency.

2016 - Is elasticity of scalable databases a myth?

by Daniel Seybold, Nicolas Wagner, Benjamin Erb, Jörg Domaschka | Link

Scalability and elasticity studies of Couchbase, Cassandra and MongoDB.

2015 - Cloudiator: a cross-cloud, multi-tenant deployment and runtime engine

by Jörg Domaschka, Daniel Baur, Daniel Seybold, Frank Griesinger | Link

Presentation of Cloudiator, a Corss cloud toolset for automated management of applications across multiple cloud providers.

2014 - The CACTOS Vision of Context-Aware Cloud Topology Optimization and Simulation

by Per-Olov Östberg, Henning Groenda, Stefan Wesner, James Byrne, Dimitrios S Nikolopoulos, Craig Sheridan, Jakub Krzywda, Ahmed Ali-Eldin, Johan Tordsson, Erik Elmroth, Christian Stier, Klaus Krogmann, Jörg Domaschka, Christopher B Hauser, Peter J Byrne, Sergej Svorobej, Barry Mccollum, Zafeirios Papazachos, Darren Whigham, Stephan Rüth, Dragana Paurevic | Link

Automation and optimization of cloud infrastructures with the 3 goals of 1) modeling applications and data center resources, 2) simulating applications and resources for planning and operations, and 3) autonomous optimization of application deployment and resource utilization.

2014 - Towards a generic language for scalability rules

by Jörg Domaschka, Kyriakos Kritikos, Alessandro Rossini | Link

Scalability Rules Language (SRL) for the specification of scalability rules in multi-cloud applications.

2014 - Reliability and availability properties of distributed database systems

by Jörg Domaschka, Christopher B Hauser, Benjamin Erb | Link

Classification of non-functional database properties such as replication, consistency, conflict management and partitioning.

2013 - Beyond IaaS and PaaS: An extended cloud taxonomy for computation, storage and networking

by Steffen Kächele, Christian Spann, Franz J Hauck, Jörg Domaschka | Link

Tiered taxonomy for computation, storage and networking services of cloud computing based on performance aspects

2011 - COSCA: an easy-to-use component-based PaaS cloud system for common applications

by Steffen Kächele, Jörg Domaschka, Franz J Hauck | Link

Comparison of cloud computing in theory with currently existing PaaS solutions for typical business applications. Identification of 11 requirements and comparison with the state of the art. Development of an ideal-typical PaaS system.

  • over 120 scientific research papers

  • more than 20 years of cloud and database research

Benchmarking findings from research and development

The connections are complex and measurements are very sensitive.
Simple measurement methods quickly lead to incorrect results.
Experience and expertise are required for reliable results.
Benchmarking enable huge potentials for more efficient IT infrastructures.