Back to Redisearch

Search on Disk MVP: Feature Blocking Status

docs/design/search_on_disk_mvp_feature_blocking.md

99.99.9910.5 KB
Original Source

Search on Disk MVP: Feature Blocking Status

This document maps the features that should be blocked/disallowed for Disk mode according to the Search on Disk MVP Feature Map and compares them against the actual implementation status in the codebase.


Quick Reference Summary

Commands Allowed in SearchDisk (Flex) Mode - MVP

CommandStatusNotes
FT.CREATE✅ AllowedRequires SKIPINITIALSCAN; only HASH type; only TEXT/TAG/VECTOR fields
FT.DROPINDEX✅ AllowedDD option blocked
FT.SEARCH✅ AllowedRequires NOCONTENT or RETURN 0; no SLOP/INORDER/HIGHLIGHT/SUMMARIZE/SORTBY/LOAD
FT.PROFILE SEARCH✅ AllowedOnly with SEARCH subcommand
FT.INFO✅ Allowed
FT._LIST✅ Allowed
FT.EXPLAIN✅ Allowed
FT.EXPLAINCLI✅ Allowed
FT.ALIASADD✅ Allowed
FT.ALIASUPDATE✅ Allowed
FT.ALIASDEL✅ Allowed
FT.CONFIG✅ Allowed
FT.DEBUG✅ Allowed

Commands Blocked in SearchDisk (Flex) Mode - MVP

CommandStatusReason
FT.AGGREGATE❌ BlockedNot supported
FT.HYBRID❌ BlockedNot supported
FT.CURSOR READ/DEL/PROFILE/GC❌ BlockedNo cursor support
FT.ALTER❌ BlockedSchema modification not supported
FT.DROP❌ BlockedUse FT.DROPINDEX instead
FT.DROPINDEX DD❌ BlockedDocument deletion not supported
FT.DICTADD❌ BlockedDictionary not supported
FT.DICTDEL❌ BlockedDictionary not supported
FT.DICTDUMP❌ BlockedDictionary not supported
FT.SUGADD❌ BlockedSuggestions not supported
FT.SUGGET❌ BlockedSuggestions not supported
FT.SUGDEL❌ BlockedSuggestions not supported
FT.SUGLEN❌ BlockedSuggestions not supported
FT.SYNUPDATE❌ BlockedSynonyms not supported
FT.SYNDUMP❌ BlockedSynonyms not supported
FT.SYNADD❌ BlockedSynonyms not supported (deprecated)
FT.SPELLCHECK❌ BlockedNot supported

Deprecated Commands (Blocked)

CommandStatusNotes
FT.DROP❌ BlockedUse FT.DROPINDEX instead
FT.MGET❌ BlockedDeprecated, not supported
FT.ADD❌ BlockedDeprecated, use HSET instead
FT.SAFEADD❌ BlockedDeprecated, use HSET instead
FT.DEL❌ BlockedDeprecated, use DEL instead
FT.GET❌ BlockedDeprecated, use HGETALL instead
FT.TAGVALS❌ BlockedDeprecated, not supported
FT.SYNADD❌ Always ErrorDeprecated, returns error regardless

Future Enhancements (Post-MVP)

Once the MVP is complete, the following features should be unblocked:

FeatureCurrent StatusTarget Status
NUMERIC field type❌ Blocked✅ Allow
GEO field type❌ Blocked✅ Allow
GEOSHAPE field type❌ Blocked✅ Allow
FT.AGGREGATE❌ Blocked✅ Allow
FT.HYBRID❌ Blocked✅ Allow
FT.CURSOR commands❌ Blocked✅ Allow
FT.ALTER❌ Blocked✅ Allow
SORTBY argument❌ Blocked✅ Allow
ON JSON❌ Blocked✅ Allow
Vector Range queries⚠️ Allowed✅ Keep allowed
FLAT vector algorithm❌ Blocked⚠️ TBD
SVS vector algorithm❌ Blocked⚠️ TBD
Infix/Suffix/Wildcard/Fuzzy⚠️ Returns empty⚠️ TBD

Legend

  • BLOCKED - Feature is properly blocked in code
  • NOT BLOCKED - Feature should be blocked but isn't
  • ⚠️ PARTIAL - Partially blocked or unclear
  • N/A - Not applicable or implicitly blocked

1. Index Field Types

FeatureShould Block?Actually Blocked?Code Location
NUMERIC field✅ Yes✅ BLOCKEDspec.c:1367 - SearchDisk_MarkUnsupportedFieldIfDiskEnabled(SPEC_NUMERIC_STR, ...)
GEO field✅ Yes✅ BLOCKEDspec.c:1373 - SearchDisk_MarkUnsupportedFieldIfDiskEnabled(SPEC_GEO_STR, ...)
GEOSHAPE field✅ Yes✅ BLOCKEDspec.c:1360 - SearchDisk_MarkUnsupportedFieldIfDiskEnabled(SPEC_GEOMETRY_STR, ...)
TEXT fieldNo➖ Allowed
TAG fieldNo➖ Allowed
VECTOR fieldNo➖ Allowed

2. FT.CREATE Arguments

ArgumentShould Block?Actually Blocked?Code Location
ON JSON✅ Yes✅ BLOCKEDspec.c:1761-1770 - invalid_flex_on_type check
NOOFFSETS✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts, triggers error
NOHL✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts
NOFIELDS✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts
NOFREQS✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts
MAXTEXTFIELDS✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts
ASYNC✅ Yes✅ BLOCKEDspec.c:1741-1759 - Not in flex_argopts
Missing SKIPINITIALSCAN✅ Yes✅ BLOCKEDspec.c - Requires SKIPINITIALSCAN for Flex
WITHSUFFIXTRIE✅ Yes✅ BLOCKEDspec.c:1141-1148,1181-1188 - Blocked in parseTextField/parseTagField

3. FT.SEARCH Arguments

ArgumentShould Block?Actually Blocked?Code Location
Without NOCONTENT or RETURN 0✅ Yes✅ BLOCKEDaggregate_request.c:764-767
LOAD✅ Yes✅ BLOCKEDaggregate_request.c:1004-1008
SLOP✅ Yes✅ BLOCKEDaggregate_request.c:698-704
INORDER✅ Yes✅ BLOCKEDaggregate_request.c:705-708
HIGHLIGHT✅ Yes✅ BLOCKEDaggregate_request.c:639-642
SUMMARIZE✅ Yes✅ BLOCKEDaggregate_request.c:627-630
GEOFILTER✅ Implicit✅ BLOCKED (implicit)GEO field type is blocked, so no GEO index
FILTER (numeric)✅ Implicit✅ BLOCKED (implicit)NUMERIC field type is blocked
SORTBY✅ Yes✅ BLOCKEDaggregate_request.c:310-312

Scorer Reference

The following table lists all available scorers and their Disk mode compatibility:

ScorerShould Block?Actually Blocked?ReasonCode Location
TFIDF✅ Yes✅ BLOCKEDUses SLOP in calculation (tfidf /= slop)aggregate_request.c:1433-1437, default.c:116-117
TFIDF.DOCNORM✅ Yes✅ BLOCKEDUses SLOP (same as TFIDF, different normalization)aggregate_request.c:1438-1442, default.c:116-117
BM25✅ Yes✅ BLOCKEDUses SLOP in calculation (score /= slop) - deprecatedaggregate_request.c:1443-1447, default.c:211-212
BM25STDNo➖ AllowedDefault scorer - does not use SLOP
BM25STD.TANHNo➖ AllowedNormalized BM25STD with tanh - does not use SLOP
BM25STD.NORMNo➖ AllowedNormalized BM25STD - does not use SLOP
DISMAXNo➖ AllowedSum of term frequencies - does not use SLOP
DOCSCORENo➖ AllowedReturns raw document score only - does not use SLOP
HAMMINGNo➖ AllowedHamming distance scorer - does not use SLOP

Note: Scorers that use SLOP require term offset information to calculate proximity between matched terms. Since SLOP is blocked for Disk mode (see above), any scorer that relies on SLOP must also be blocked.


4. Commands - Completely Blocked

CommandShould Block?Actually Blocked?Code Location
FT.AGGREGATE✅ Yes✅ BLOCKEDmodule.c:1780,4662 - DiskDisabledCmd(RSAggregateCommand)
FT.HYBRID✅ Yes✅ BLOCKEDmodule.c:1779,4665 - DiskDisabledCmd(RSShardedHybridCommand)
FT.CURSOR (all)✅ Yes✅ BLOCKEDmodule.c:3804-3832 - All cursor commands
FT.ALTER✅ Yes✅ BLOCKEDmodule.c:1753,4677 - DiskDisabledCmd(AlterIndexCommand)
FT.DICTADD✅ Yes✅ BLOCKEDmodule.c:1755,4682 - DiskDisabledCmd(DictAddCommand)
FT.DICTDEL✅ Yes✅ BLOCKEDmodule.c:1756,4683 - DiskDisabledCmd(DictDelCommand)
FT.DICTDUMP✅ Yes✅ BLOCKEDmodule.c:1771 - DiskDisabledCmd(DictDumpCommand)
FT.SUGADD✅ Yes✅ BLOCKEDmodule.c:1764 - DiskDisabledCmd(RSSuggestAddCommand)
FT.SUGGET✅ Yes✅ BLOCKEDmodule.c:1765 - DiskDisabledCmd(RSSuggestGetCommand)
FT.SUGDEL✅ Yes✅ BLOCKEDmodule.c:1766 - DiskDisabledCmd(RSSuggestDelCommand)
FT.SUGLEN✅ Yes✅ BLOCKEDmodule.c:1767 - DiskDisabledCmd(RSSuggestLenCommand)

5. Vector Index Features

FeatureShould Block?Actually Blocked?Code Location
FLAT algorithm✅ Yes✅ BLOCKEDspec.c:1228-1236 - Error for FLAT on disk
SVS algorithm✅ Yes✅ BLOCKEDspec.c:1271-1278 - Error for SVS on disk
Range query⚠️ Per doc❌ NOT BLOCKEDvector_index.c:155-173 - Range query still allowed
Multi-value vectors✅ Implicit✅ BLOCKED (implicit)JSON is blocked

6. Query Syntax Features

FeatureShould Block?Actually Blocked?Code Location
Infix search (*foo*)⚠️ Per doc❌ NOT BLOCKEDWill return empty results (no suffix trie)
Suffix search (*foo)⚠️ Per doc❌ NOT BLOCKEDWill return empty results (no suffix trie)
Wildcard (w'pattern')⚠️ Per doc❌ NOT BLOCKEDWill return empty results
Fuzzy search (%term%)⚠️ Per doc❌ NOT BLOCKEDWill return empty results
Numeric range queries✅ Implicit✅ BLOCKED (implicit)NUMERIC field blocked
Geo queries✅ Implicit✅ BLOCKED (implicit)GEO field blocked

7. Configuration / Limits

FeatureShould Block?Actually Blocked?Code Location
Max 10 indexes✅ Yes✅ BLOCKEDsearch_disk_utils.c:13-18 - FLEX_MAX_INDEX_COUNT check
WORKERS = 0✅ Yes✅ BLOCKEDCorrected to 1 automatically