Back to Lance

Zone Map Index

docs/src/format/table/index/scalar/zonemap.md

4.0.12.4 KB
Original Source

Zone Map Index

Zone maps are a columnar database technique for predicate pushdown and scan pruning. They break data into fixed-size chunks called "zones" and maintain summary statistics (min, max, null count) for each zone, enabling efficient filtering by eliminating zones that cannot contain matching values.

Zone maps are "inexact" filters - they can definitively exclude zones but may include false positives that require rechecking.

Index Details

protobuf
%%% proto.message.ZoneMapIndexDetails %%%

Storage Layout

The zone map index stores zone statistics in a single file:

  1. zonemap.lance - Zone statistics for query pruning

Zone Statistics File Schema

ColumnTypeNullableDescription
min{DataType}trueMinimum value in the zone
max{DataType}trueMaximum value in the zone
null_countUInt32falseNumber of null values in the zone
nan_countUInt32falseNumber of NaN values (for float types)
fragment_idUInt64falseFragment containing this zone
zone_startUInt64falseStarting row offset within the fragment
zone_lengthUInt32falseNumber of rows in this zone

Schema Metadata

KeyTypeDescription
rows_per_zoneStringNumber of rows per zone (default: "8192")

Accelerated Queries

The zone map index provides inexact results for the following query types:

Query TypeDescriptionOperationResult Type
Equalscolumn = valueIncludes zones where min ≤ value ≤ maxAtMost
Rangecolumn BETWEEN a AND bIncludes zones where ranges overlapAtMost
IsIncolumn IN (v1, v2, ...)Includes zones that could contain any valueAtMost
IsNullcolumn IS NULLIncludes zones where null_count > 0AtMost