Running the TPC-C performance benchmark

To run the TPC-C benchmark with an efficiency of around 99.7%, choose a cluster specification (size and instance type) depending on your estimated IOPS, estimated data size, and expected latency.

You can use the following table for reference.

Instance types

The following table lists the recommended instance types on different cloud providers based on vCPU count.

After you have decided the instance type of the cluster that you need, use the following instructions to run the TPC-C workload.

Get TPC-C binaries

First, you need the benchmark binaries. To download the TPC-C binaries, run the following commands.

$ wget https://github.com/yugabyte/tpcc/releases/latest/download/tpcc.tar.gz
$ tar -zxvf tpcc.tar.gz
$ cd tpcc

Client machine

The client machine is where the benchmark is run from. An 8vCPU machine with at least 16GB memory is recommended. The following instance types are recommended for the client machine.

Cluster setup

{{<nav/tabs list="local,cloud" active="local"/>}}

{{<nav/panels>}} {{<nav/panel name="local" active="true">}}

{{<setup/local>}}

Store the IP addresses of the nodes in a shell variable for use in further commands.

IPS=127.0.0.1,127.0.0.2,127.0.0.3

{{</nav/panel>}}

{{<nav/panel name="cloud">}} {{<setup/cloud>}}

{{<warning title="Set up connection options correctly">}}

• When running the client machine outside the VPC, enable Public Access for the cluster. Also, make sure your client's IP is added to the IP allow list.
• Add the username (admin) and password from the credential file that you downloaded when you set up your cluster to the workload_all.xml file.
• Make sure you add the full path of the location of the SSL certificate of your cluster in the sslCert tag in the workload_all.xml file.
• If you are planning to do a horizontal scale test, set the fault tolerance level to None so that you can add a single node to the cluster. {{</warning>}}

Store the IP addresses/public address of the cluster in a shell variable for use in further commands.

IPS=<cluster-name/IP>

{{</nav/panel>}} {{</nav/panels>}}

Configure connection parameters

If needed, options like username, password, port, and so on, can be set up using the configuration file config/workload_all.xml.

<port>5433</port>
<username>yugabyte</username>
<password>***</password>
<sslCert>/Users/johnwick/.credentials/root.crt</sslCert>

Initialize the data

Initialize the database needed for the benchmark by following the instructions specific to your cluster.

{{<tabpane text=true >}}

{{% tab header="3 x 2vCPUs" lang="2vcpu" %}}

Set up the TPC-C database schema with the following command:

$ ./tpccbenchmark --create=true --nodes=${IPS}

Populate the database with data needed for the benchmark with the following command:

$ ./tpccbenchmark --load=true --nodes=${IPS} --warehouses=500 --loaderthreads 48

{{% /tab %}}

{{% tab header="3 x 4vCPUs" lang="4vcpu" %}}

Set up the TPC-C database schema with the following command:

$ ./tpccbenchmark --create=true --nodes=${IPS}

Populate the database with data needed for the benchmark with the following command:

$ ./tpccbenchmark --load=true --nodes=${IPS} --warehouses=1000 --loaderthreads 48

{{% /tab %}}

{{% tab header="3 x 8vCPUs" lang="8vcpu" %}}

Set up the TPC-C database schema with the following command:

$ ./tpccbenchmark --create=true --nodes=${IPS}

Populate the database with data needed for the benchmark with the following command:

$ ./tpccbenchmark --load=true --nodes=${IPS} --warehouses=2000 --loaderthreads 48

{{% /tab %}}

{{% tab header="3 x 16vCPUs" lang="16vcpu" %}}

Set up the TPC-C database schema with the following command:

$ ./tpccbenchmark --create=true --nodes=${IPS}

To populate the database with data needed for the benchmark, use two client machines. Run the following load commands on each of the machines respectively.

{{<note>}} initial-delay-secs has been introduced to avoid a sudden overload on the master to fetch catalogs. {{</note>}}

{{% /tab %}}

{{</tabpane>}}

Run the benchmark

Run the benchmark by following instructions specific to your cluster.

{{<tabpane text=true >}}

{{% tab header="3 x 2vCPUs" lang="2vcpu" %}}

$ ./tpccbenchmark --execute=true --warmup-time-secs=60 --nodes=${IPS} --warehouses=500 --num-connections=50

{{% /tab %}}

{{% tab header="3 x 4vCPUs" lang="4vcpu" %}}

$ ./tpccbenchmark --execute=true --warmup-time-secs=60 --nodes=${IPS} --warehouses=1000 --num-connections=100

{{% /tab %}}

{{% tab header="3 x 8vCPUs" lang="8vcpu" %}}

$ ./tpccbenchmark --execute=true --warmup-time-secs=60 --nodes=${IPS} --warehouses=2000 --num-connections=200

{{% /tab %}}

{{% tab header="3 x 16vCPUs" lang="16vcpu" %}}

Run the following commands on each of the client machines respectively.

{{<note>}} As initial-delay-secs is specified, to ensure that the tests start together, the time was adjusted in the warmup phase. {{</note>}}

{{% /tab %}}

{{</tabpane>}}

Analyze the results

{{<tabpane text=true >}}

{{% tab header="3 x 2vCPUs" lang="2vcpu" %}}

On a 2vCPU cluster, 6,415 tpmC was achieved with 99.78% efficiency keeping the new order latency around 64ms.

{{% /tab %}}

{{% tab header="3 x 4vCPUs" lang="4vcpu" %}}

On a 4vCPU cluster, 12,829 tpmC was achieved with 99.77% efficiency keeping the new order latency around 73ms.

{{% /tab %}}

{{% tab header="3 x 8vCPUs" lang="8vcpu" %}}

On an 8vCPU cluster, 25,646 tpmC was achieved with 99.71% efficiency keeping the new order latency around 54ms.

{{% /tab %}}

{{% tab header="3 x 16vCPUs" lang="16vcpu" %}}

On a 16vCPU cluster, 51,343 tpmC was achieved with 99.81% efficiency keeping the new order latency around 39ms.

{{% /tab %}}

{{</tabpane>}}

Conclusion

The TPC-C benchmark provides a rigorous and comprehensive method for evaluating the performance of OLTP systems, focusing on throughput, response time, and data consistency. The results will help you decide if YugabyteDB is the right database for your high-throughput, transaction-heavy applications.