Back to Rocksdb

Adding New Options to RocksDB Public API

claude_md/add_option.md

11.1.115.7 KB
Original Source

Adding New Options to RocksDB Public API

This document provides guidance on how to add new options to RocksDB's public API. There are two main categories of options:

  1. Standard Column Family Options (Options/DBOptions/AdvancedColumnFamilyOptions)
  2. BlockBasedTableOptions (options specific to block-based table format)

Overview of Files to Modify

For Standard Column Family Options

FilePurpose
include/rocksdb/advanced_options.hDefine the option with documentation
include/rocksdb/options.hAdd reference in related option groups if needed
options/cf_options.hAdd to MutableCFOptions or ImmutableCFOptions struct
options/cf_options.ccRegister option for serialization/deserialization and logging
options/options_helper.ccAdd to UpdateColumnFamilyOptions() for mutable options
options/options_settable_test.ccAdd to test string for option parsing
db_stress_tool/db_stress_common.hDeclare gflag
db_stress_tool/db_stress_gflags.ccDefine gflag with default value
db_stress_tool/db_stress_test_base.ccApply flag to options
tools/db_bench_tool.ccAdd flag definition and apply to options
tools/db_crashtest.pyAdd randomized values for stress testing
unreleased_history/new_features/Add release note markdown file

For BlockBasedTableOptions

FilePurpose
include/rocksdb/table.hDefine the option in BlockBasedTableOptions struct
table/block_based/block_based_table_factory.ccRegister for serialization, validation, and printing
options/options_settable_test.ccAdd to BlockBasedTableOptionsAllFieldsSettable test
options/options_test.ccAdd to MutableCFOptions test if applicable
db_stress_tool/db_stress_common.hDeclare gflag
db_stress_tool/db_stress_gflags.ccDefine gflag
db_stress_tool/db_stress_test_base.ccApply flag to block_based_options
tools/db_bench_tool.ccAdd flag definition and apply to block_based_options
tools/db_crashtest.pyAdd randomized values
java/src/main/java/org/rocksdb/BlockBasedTableConfig.javaJava API
java/rocksjni/portal.hJNI portal for Java bindings
java/rocksjni/table.ccJNI implementation
java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.javaJava unit test

Pattern 1: Adding a Standard Column Family Option

Example reference: commit 94e65a2e0b4f817aa4bfa4c96cdf867e7980d7bc (memtable_veirfy_per_key_checksum_on_seek)

Step 1: Define the Option in Public Header

File: include/rocksdb/advanced_options.h

Add the option with documentation in AdvancedColumnFamilyOptions struct:

cpp
// Enables additional integrity checks during seek.
// Specifically, for skiplist-based memtables, key checksum validation could
// be enabled during seek optionally. This is helpful to detect corrupted
// memtable keys during reads. Enabling this feature incurs a performance
// overhead due to additional key checksum validation during memtable seek
// operation.
// This option depends on memtable_protection_bytes_per_key to be non zero.
// If memtable_protection_bytes_per_key is zero, no validation is performed.
bool memtable_veirfy_per_key_checksum_on_seek = false;

Step 2: Add to Internal Options Structs

File: options/cf_options.h

Add to MutableCFOptions struct (or ImmutableCFOptions for immutable options):

cpp
// In MutableCFOptions constructor from Options:
memtable_veirfy_per_key_checksum_on_seek(
    options.memtable_veirfy_per_key_checksum_on_seek),

// In MutableCFOptions default constructor:
memtable_veirfy_per_key_checksum_on_seek(false),

// In MutableCFOptions struct member declarations:
bool memtable_veirfy_per_key_checksum_on_seek;

Step 3: Register for Serialization/Deserialization

File: options/cf_options.cc

Add to the options type info map for serialization:

cpp
{"memtable_veirfy_per_key_checksum_on_seek",
 {offsetof(struct MutableCFOptions,
           memtable_veirfy_per_key_checksum_on_seek),
  OptionType::kBoolean, OptionVerificationType::kNormal,
  OptionTypeFlags::kMutable}},

Add logging in MutableCFOptions::Dump():

cpp
ROCKS_LOG_INFO(log, "memtable_veirfy_per_key_checksum_on_seek: %d",
               memtable_veirfy_per_key_checksum_on_seek);

Step 4: Update Options Helper

File: options/options_helper.cc

Add to UpdateColumnFamilyOptions():

cpp
cf_opts->memtable_veirfy_per_key_checksum_on_seek =
    moptions.memtable_veirfy_per_key_checksum_on_seek;

Step 5: Add to Options Settable Test

File: options/options_settable_test.cc

Add to the test string in ColumnFamilyOptionsAllFieldsSettable:

cpp
"memtable_veirfy_per_key_checksum_on_seek=1;"

Step 6: Add db_stress Support

File: db_stress_tool/db_stress_common.h

cpp
DECLARE_bool(memtable_veirfy_per_key_checksum_on_seek);

File: db_stress_tool/db_stress_gflags.cc

cpp
DEFINE_bool(
    memtable_veirfy_per_key_checksum_on_seek,
    ROCKSDB_NAMESPACE::Options().memtable_veirfy_per_key_checksum_on_seek,
    "Sets CF option memtable_veirfy_per_key_checksum_on_seek.");

File: db_stress_tool/db_stress_test_base.cc

cpp
options.memtable_veirfy_per_key_checksum_on_seek =
    FLAGS_memtable_veirfy_per_key_checksum_on_seek;

Step 7: Add db_bench Support

File: tools/db_bench_tool.cc

cpp
// Flag definition (near related flags):
DEFINE_bool(memtable_veirfy_per_key_checksum_on_seek, false,
            "Sets CF option memtable_veirfy_per_key_checksum_on_seek");

// Apply flag to options (in InitializeOptionsFromFlags or similar):
options.memtable_veirfy_per_key_checksum_on_seek =
    FLAGS_memtable_veirfy_per_key_checksum_on_seek;

Step 8: Add Crash Test Support

File: tools/db_crashtest.py

python
"memtable_veirfy_per_key_checksum_on_seek": lambda: random.choice([0] * 7 + [1]),

Also add constraint handling in finalize_and_sanitize() if needed:

python
# only skip list memtable representation supports paranoid memory checks
if dest_params.get("memtablerep") != "skip_list":
    dest_params["memtable_veirfy_per_key_checksum_on_seek"] = 0

Step 9: Add Release Note

File: unreleased_history/new_features/<descriptive_name>.md

markdown
A new flag memtable_veirfy_per_key_checksum_on_seek is added to AdvancedColumnFamilyOptions. When it is enabled, it will validate key checksum along the binary search path on skiplist based memtable during seek operation.

Pattern 2: Adding a BlockBasedTableOptions Option

Example reference: commit 742741b175c5f238374c1714f9db3340d49de569 (super_block_alignment_size)

Step 1: Define the Option in Public Header

File: include/rocksdb/table.h

Add to BlockBasedTableOptions struct with documentation:

cpp
// Align data blocks on super block alignment. Avoid a data block split across
// super block boundaries. Works with/without compression.
//
// Here a "super block" refers to an aligned unit of underlying Filesystem
// storage for which there is an extra cost when a random read involves two
// such super blocks instead of just one. Configuring that size here suggests
// inserting padding in the SST file to avoid a single SST block splitting
// across two super blocks. Only power-of-two sizes are supported. See also
// super_block_alignment_space_overhead_ratio. Default to 0, which means super
// block alignment is disabled.
size_t super_block_alignment_size = 0;

// This option controls the storage space overhead of super block alignment.
// It is used to calculate the max padding size allowed for super block
// alignment. It is calculated in this way. If super_block_alignment_size is
// 2MB, and super_block_alignment_overhead_ratio is 128, then the max padding
// size allowed for super block alignment is 2MB / 128 = 16KB.
// Note that, when it is set to 0, super block alignment is disabled.
size_t super_block_alignment_space_overhead_ratio = 128;

Step 2: Register for Serialization in Table Factory

File: table/block_based/block_based_table_factory.cc

Add to the type info map:

cpp
{"super_block_alignment_size",
 {offsetof(struct BlockBasedTableOptions, super_block_alignment_size),
  OptionType::kSizeT, OptionVerificationType::kNormal}},
{"super_block_alignment_space_overhead_ratio",
 {offsetof(struct BlockBasedTableOptions,
           super_block_alignment_space_overhead_ratio),
  OptionType::kSizeT, OptionVerificationType::kNormal}},

Add validation in ValidateOptions():

cpp
if ((table_options_.super_block_alignment_size &
     (table_options_.super_block_alignment_size - 1))) {
  return Status::InvalidArgument(
      "Super Block alignment requested but super block alignment size is not "
      "a power of 2");
}
if (table_options_.super_block_alignment_size >
    std::numeric_limits<uint32_t>::max()) {
  return Status::InvalidArgument(
      "Super block alignment size exceeds maximum number (4GiB) allowed");
}

Add printing in GetPrintableOptions():

cpp
snprintf(buffer, kBufferSize,
         "  super_block_alignment_size: %" ROCKSDB_PRIszt "\n",
         table_options_.super_block_alignment_size);
ret.append(buffer);

Step 3: Add to Options Settable Test

File: options/options_settable_test.cc

Add to BlockBasedTableOptionsAllFieldsSettable test:

cpp
"super_block_alignment_size=65536;"
"super_block_alignment_space_overhead_ratio=4096;"

Step 4: Add to Options Test

File: options/options_test.cc

cpp
ASSERT_OK(GetColumnFamilyOptionsFromString(
    config_options, cf_opts,
    "block_based_table_factory.super_block_alignment_size=65536; "
    "block_based_table_factory.super_block_alignment_space_overhead_ratio=4096;",
    &cf_opts));
ASSERT_EQ(bbto->super_block_alignment_size, 65536);
ASSERT_EQ(bbto->super_block_alignment_space_overhead_ratio, 4096);

Step 5: Add db_stress Support

File: db_stress_tool/db_stress_common.h

cpp
DECLARE_uint64(super_block_alignment_size);
DECLARE_uint64(super_block_alignment_space_overhead_ratio);

File: db_stress_tool/db_stress_gflags.cc

cpp
DEFINE_uint64(
    super_block_alignment_size,
    ROCKSDB_NAMESPACE::BlockBasedTableOptions().super_block_alignment_size,
    "BlockBasedTableOptions.super_block_alignment_size");

DEFINE_uint64(
    super_block_alignment_space_overhead_ratio,
    ROCKSDB_NAMESPACE::BlockBasedTableOptions()
        .super_block_alignment_space_overhead_ratio,
    "BlockBasedTableOptions.super_block_alignment_space_overhead_ratio");

File: db_stress_tool/db_stress_test_base.cc

cpp
block_based_options.super_block_alignment_size =
    fLU64::FLAGS_super_block_alignment_size;
block_based_options.super_block_alignment_space_overhead_ratio =
    fLU64::FLAGS_super_block_alignment_space_overhead_ratio;

Step 6: Add db_bench Support

File: tools/db_bench_tool.cc

cpp
// Flag definitions:
DEFINE_uint64(
    super_block_alignment_size,
    ROCKSDB_NAMESPACE::BlockBasedTableOptions().super_block_alignment_size,
    "Configure super block size");

DEFINE_uint64(super_block_alignment_space_overhead_ratio,
              ROCKSDB_NAMESPACE::BlockBasedTableOptions()
                  .super_block_alignment_space_overhead_ratio,
              "Configure space overhead for super block alignment");

// Apply to block_based_options (in the block where other options are set):
block_based_options.super_block_alignment_size = FLAGS_super_block_alignment_size;
block_based_options.super_block_alignment_space_overhead_ratio =
    FLAGS_super_block_alignment_space_overhead_ratio;

Step 7: Add Crash Test Support

File: tools/db_crashtest.py

python
"super_block_alignment_size": lambda: random.choice(
    [0, 128 * 1024, 512 * 1024, 2 * 1024 * 1024]
),
"super_block_alignment_space_overhead_ratio": lambda: random.choice([0, 32, 4096]),

Step 8: Add Java API Support

File: java/src/main/java/org/rocksdb/BlockBasedTableConfig.java

Add getter and setter methods:

java
/**
 * Get the super block alignment size.
 *
 * @return the super block alignment size.
 */
public long superBlockAlignmentSize() {
  return superBlockAlignmentSize;
}

/**
 * Set the super block alignment size.
 * When set to 0, super block alignment is disabled.
 *
 * @param superBlockAlignmentSize the super block alignment size.
 *
 * @return the reference to the current option.
 */
public BlockBasedTableConfig setSuperBlockAlignmentSize(final long superBlockAlignmentSize) {
  this.superBlockAlignmentSize = superBlockAlignmentSize;
  return this;
}

Add member variable:

java
private long superBlockAlignmentSize;

Update constructor and native method signature.

File: java/rocksjni/portal.h

Update GetMethodID signature and add fields to Java object construction.

File: java/rocksjni/table.cc

Add parameters to JNI function and apply to options.

File: java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.java

Add unit tests:

java
@Test
public void superBlockAlignmentSize() {
  final BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig();
  blockBasedTableConfig.setSuperBlockAlignmentSize(1024 * 1024);
  assertThat(blockBasedTableConfig.superBlockAlignmentSize()).isEqualTo(1024 * 1024);
}

Pattern 3: Adding C API for Existing Option

Example reference: commit 429b36c22d76403d275dd0e6877b08d4cea2bc90 (block_align C API)

If an option already exists but needs C API support:

File: db/c.cc

cpp
void rocksdb_block_based_options_set_block_align(
    rocksdb_block_based_table_options_t* options, unsigned char v) {
  options->rep.block_align = v;
}

File: include/rocksdb/c.h

cpp
extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_block_align(
    rocksdb_block_based_table_options_t*, unsigned char);

Unit Testing Guidelines

For Standard Options

Add tests in appropriate test files (e.g., db/db_memtable_test.cc, db/db_options_test.cc):

cpp
TEST_F(DBMemTableTest, YourOptionTest) {
  Options options;
  options.your_new_option = true;
  Reopen(options);
  // Test the behavior
}

For BlockBasedTableOptions

Add tests in db/db_flush_test.cc, table/block_based/block_based_table_reader_test.cc, or table/table_test.cc:

cpp
TEST_P(DBFlushYourFeatureTest, YourFeature) {
  Options options;
  BlockBasedTableOptions block_options;
  block_options.your_new_option = some_value;
  options.table_factory.reset(NewBlockBasedTableFactory(block_options));

  ASSERT_OK(options.table_factory->ValidateOptions(
      DBOptions(options), ColumnFamilyOptions(options)));

  Reopen(options);
  // Test the behavior
}

Option Type Reference

Common option types used in serialization:

OptionTypeC++ TypeExample
kBooleanboolparanoid_memory_checks
kIntintmax_write_buffer_number
kInt32Tint32_tlevel0_file_num_compaction_trigger
kUInt32Tuint32_tmemtable_protection_bytes_per_key
kUInt64Tuint64_ttarget_file_size_base
kSizeTsize_tblock_size
kDoubledoublecompression_ratio
kStringstd::stringdb_log_dir

Checklist Summary

  • Public header file with option definition and documentation
  • Internal options struct (MutableCFOptions or ImmutableCFOptions)
  • Options serialization/deserialization registration
  • Options logging in Dump() method
  • UpdateColumnFamilyOptions() for mutable options
  • options_settable_test.cc
  • db_stress_common.h (DECLARE)
  • db_stress_gflags.cc (DEFINE)
  • db_stress_test_base.cc (apply flag)
  • db_bench_tool.cc (DEFINE and apply)
  • db_crashtest.py (randomized values)
  • Unit tests
  • unreleased_history markdown file
  • Java API (for BlockBasedTableOptions)
  • C API (if needed)