Benchmark YCQL performance in YugabyteDB with YCSB

This document describes how to use a YCQL-specific binding to test the YCQL API using the YCSB benchmark.

For additional information about YCSB, refer to the following:

• YCSB Wiki
• Workload info

Running the benchmark

To run the benchmark, ensure that you meet the prerequisites and complete steps such as starting YugabyteDB and configuring its properties.

Prerequisites

The binaries are compiled with Java 13 and it is recommended to run these binaries with that version.

Run the following commands to download the YCSB binaries:

$ cd $HOME
$ wget https://github.com/yugabyte/YCSB/releases/download/1.0/ycsb.tar.gz
$ tar -zxvf ycsb.tar.gz
$ cd YCSB

Ensure that you have the YCQL shell ycqlsh and that its location is included in the PATH variable, as follows:

$ export PATH=$PATH:/path/to/ycqlsh

Start YugabyteDB

Start your YugabyteDB cluster by following the procedure described in Manual deployment. Note the IP addresses of the nodes in the cluster, as these addresses are required when configuring the properties file.

Configure the properties file

Update the file db.properties in the YCSB directory with the following contents, replacing values for the IP addresses in the hosts field with the correct values for all the nodes that are part of the cluster:

hosts=<ip>
port=9042
cassandra.username=yugabyte

The other configuration parameters are described in Core Properties.

Run the benchmark

Use the following script run_ycql.sh to load and run all the workloads:

$ ./run_ycql.sh --ip <ip>

The preceding command runs the workload on a table with a million rows. To run the benchmark on a table with a different row count, use the following command:

$ ./run_ycql.sh --ip <ip> --recordcount <number of rows>

To get the maximum performance out of the system, you would have to tune the threadcount parameter in the script. As a reference, for a c5.4xlarge instance with 16 cores and 32 GB RAM, we used a threadcount of 32 for the loading phase and 256 for the execution phase.

Verify results

The run_ycql.sh script creates two result files per workload: one for the loading, and one for the execution phase with the details of throughput and latency.

For example, for a workload it creates, inspect the workloada-ycql-load.dat and workloada-ycql-transaction.dat files.

Run individual workloads (optional)

Optionally, you can run workloads individually using the following steps:

1. Start the YCQL shell using the following command:

   shCopy

   $ ./bin/ycqlsh <ip>
   

2. Create the ycsb keyspace as follows:

   cqlCopy

   ycqlsh> CREATE KEYSPACE ycsb;
   

3. Connect to the keyspace as follows:

   cqlCopy

   ycqlsh> USE ycsb;
   

4. Create the table as follows:

   cqlCopy

   ycqlsh:ycsb> create table usertable (
                   y_id varchar primary key,
                   field0 varchar, field1 varchar, field2 varchar, field3 varchar,
                   field4 varchar, field5 varchar, field6 varchar, field7 varchar,
                   field8 varchar, field9 varchar);
   

5. Load the data before you start the yugabyteCQL workload:

   shCopy

   $ ./bin/ycsb load yugabyteCQL -s \
         -P db.properties           \
         -P workloads/workloada     \
         -p recordcount=1000000     \
         -p operationcount=10000000 \
         -p threadcount=32
   

6. Run the workload as follows:

   {{< note title="Note" >}} The recordcount parameter in the following ycsb commands should match the number of rows in the table. {{< /note >}}

   shCopy

   $ ./bin/ycsb run yugabyteCQL -s  \
         -P db.properties           \
         -P workloads/workloada     \
         -p recordcount=1000000     \
         -p operationcount=10000000 \
         -p threadcount=256
   

7. Run other workloads (for example, workloadb) by changing the corresponding argument in the preceding command, as follows:

   shCopy

   $ ./bin/ycsb run yugabyteCQL -s  \
         -P db.properties           \
         -P workloads/workloadb     \
         -p recordcount=1000000     \
         -p operationcount=10000000 \
         -p threadcount=256
   

Expected results

When run on a 3-node cluster with each a c5.4xlarge AWS instance (16 cores, 32 GB of RAM, and 2 EBS volumes) all belonging to the same availability zone with the client VM running in the same availability zone, you get the following results for 1 million rows: