.. _cephfs-tool:

=========== cephfs-tool

cephfs-tool is a standalone C++ utility designed to interact directly with libcephfs. The initial implementation focuses on a bench command to measure library performance. This allows developers and administrators to benchmark the userspace library isolated from FUSE or kernel client overhead.

Key features include:

Multi-threaded Read/Write throughput benchmarking.
Configurable block sizes, file counts, and fsync intervals.
Detailed statistical reporting (Mean, Std Dev, Min/Max) for throughput and IOPS.
Support for specific CephFS user/group impersonation (UID/GID) via ceph_mount_perms_set.

Building

The tool can be built outside of the Ceph source tree:

.. code-block:: bash

g++ --std=c++20 -D_FILE_OFFSET_BITS=64 -O3 -o cephfs-tool cephfs-tool.cc -lcephfs -lpthread -lboost_program_options

Usage

.. code-block:: bash

cephfs-tool [general-options] <command> [command-options]

Commands

bench Run IO benchmark

Options

General Options

.. option:: -h, --help

Produce help message

.. option:: -c, --conf <path>

Ceph config file path

.. option:: -i, --id <id>

Client ID (default: admin)

.. option:: -k, --keyring <path>

Path to keyring file

.. option:: --filesystem <name>

CephFS filesystem name to mount

.. option:: --uid <uid>

User ID to mount as (default: -1)

.. option:: --gid <gid>

Group ID to mount as (default: -1)

Benchmark Options

These options are used with the bench command.

.. option:: --threads <n>

Number of threads (default: 1)

.. option:: --iterations <n>

Number of iterations (default: 1)

.. option:: --files <n>

Total number of files (default: 100)

.. option:: --size <size>

File size (e.g. 4MB, 0 for creates only) (default: 4MB)

.. option:: --block-size <size>

IO block size (e.g. 1MB) (default: 4MB)

.. option:: --fsync-every <size>

Call fsync every N bytes (default: 0)

.. option:: --prefix <str>

Filename prefix (default: benchmark_)

.. option:: --dir-prefix <str>

Directory prefix (default: bench_run_)

.. option:: --root-path <path>

Root path in CephFS (default: /)

.. option:: --per-thread-mount

Use separate mount per thread

.. option:: --no-cleanup

Disable cleanup of files

Examples

Benchmark throughput with 8 threads:

.. code-block:: bash

env CEPH_ARGS="--log-to-stderr=false --log-to-file=false --log-file=/tmp/bench.log" \
./cephfs-tool -c ~/ceph.conf -k ~/keyring -i scratch --filesystem scratch \
  bench --root-path=/pdonnell --files 256 --size=$(( 128 * 2 ** 20 )) \
  --threads=8 --iterations 3

Output:

.. code-block:: text

Benchmark Configuration:
  Threads: 8 | Iterations: 3
  Files: 256 | Size: 134217728
  Filesystem: scratch
  Root: /pdonnell
  Subdirectory: bench_run_d942
  UID: -1
  GID: -1

--- Iteration 1 of 3 ---
Starting Write Phase...
  Write: 2761.97 MB/s, 21.5779 files/s (11.864s)
Starting Read Phase...
  Read:  2684.36 MB/s, 20.9716 files/s (12.207s)
...
*** Final Report ***

Write Throughput Statistics (3 runs):
  Mean:    2727.06 MB/s
  Std Dev: 26.2954 MB/s
  Min:     2698.51 MB/s
  Max:     2761.97 MB/s

Read Throughput Statistics (3 runs):
  Mean:    2687.24 MB/s
  Std Dev: 5.68904 MB/s
  Min:     2682.16 MB/s
  Max:     2695.18 MB/s

File Creates Statistics (3 runs):
  Mean:    21.3051 files/s
  Std Dev: 0.205433 files/s
  Min:     21.0821 files/s
  Max:     21.5779 files/s

File Reads (Opens) Statistics (3 runs):
  Mean:    20.994 files/s
  Std Dev: 0.0444456 files/s
  Min:     20.9544 files/s
  Max:     21.0561 files/s

Cleaning up...

Benchmark file creation performance (size 0):

.. code-block:: bash

env CEPH_ARGS="--log-to-stderr=false --log-to-file=false --log-file=/tmp/bench.log" \
./cephfs-tool -c ~/ceph.conf -k ~/keyring -i scratch --filesystem scratch \
  bench --root-path=/pdonnell --files=$(( 2 ** 16 )) --size=0 \
  --threads=8 --iterations 3

Output:

.. code-block:: text

Benchmark Configuration:
  Threads: 8 | Iterations: 3
  Files: 65536 | Size: 0
...
*** Final Report ***

File Creates Statistics (3 runs):
  Mean:    4001.86 files/s
  Std Dev: 125.337 files/s
  Min:     3863.7 files/s
  Max:     4167.1 files/s

File Reads (Opens) Statistics (3 runs):
  Mean:    14382.3 files/s
  Std Dev: 556.594 files/s
  Min:     13636.3 files/s
  Max:     14972.8 files/s

Cleaning up...