docs/design-docs/design_docs/20260624-function-chain-api.md
Function Chain introduces a typed, ordered, stage-aware pipeline for scoring and reranking search results. A chain is sent as structured protobuf, not as an opaque JSON string, so Milvus can validate dependencies, fetch required fields internally, execute built-in scoring functions, and project final search results back through the existing result schema.
The first release focuses on ordinary SearchRequest L2 rerank:
function_chains.map, sort, and limit operators on a DataFrame converted from reduced search results.$score is serialized through the existing search score/distance field.Milvus already has legacy rerank entry points such as function_score and ranker parameters. They are useful for predefined scoring formulas, but they do not provide a general ordered plan for composing multiple rerank steps.
Users need to express pipelines such as:
Representing this as typed operations gives Milvus:
$score semantics;SearchRequest.function_chains.output_fields.decaynum_combineround_decimalrerank_modelThe first release does not include:
function_chains support for hybrid search execution;function_score or legacy rank parameters;The public stage enum reserves room for future stages, but ordinary Search initially accepts only L2_RERANK.
A user builds a chain with FunctionChain, FunctionChainStage, col, and helper functions under fn:
from pymilvus import FunctionChain, FunctionChainStage
from pymilvus.function_chain import col, fn
chain = (
FunctionChain(FunctionChainStage.L2_RERANK, name="fresh_popular_rerank")
.map(
"freshness",
fn.decay(
col("published_at"),
function="exp",
origin=current_time,
scale=86400,
offset=0,
decay=0.5,
),
)
.map(
"$score",
fn.num_combine(
col("$score"),
col("freshness"),
col("popularity"),
mode="weighted",
weights=[0.7, 0.2, 0.1],
),
)
.map("$score", fn.round_decimal(col("$score"), decimal=4))
.sort(col("$score"), desc=True, tie_break_col=col("$id"))
.limit(10)
)
client.search(
collection_name="articles",
data=[query_vector],
anns_field="embedding",
search_params={"metric_type": "IP"},
limit=100,
output_fields=["title"],
function_chains=chain,
)
For model rerank:
chain = (
FunctionChain(FunctionChainStage.L2_RERANK, name="model_rerank")
.map(
"$score",
fn.rerank_model(
col("doc"),
queries=["renewable energy developments"],
provider="voyageai",
model_name="rerank-2.5",
truncation=True,
max_client_batch_size=128,
),
)
.sort(col("$score"), desc=True, tie_break_col=col("$id"))
)
External model credentials are resolved by Milvus server-side provider configuration. SDK requests should not carry API keys.
Ordinary Search accepts function_chains:
client.search(..., function_chains=chain)
client.search(..., function_chains=[chain])
SDK and server validation reject ambiguous combinations:
function_chains with SDK ranker / proto function_score;function_chains for hybrid search in the first release.The public protobuf models a chain as an ordered logical plan:
enum FunctionChainStage {
FunctionChainStageUnspecified = 0;
FunctionChainStageIngestion = 1;
FunctionChainStagePreProcess = 2;
FunctionChainStageL0Rerank = 3;
FunctionChainStageL1Rerank = 4;
FunctionChainStageL2Rerank = 5;
FunctionChainStagePostProcess = 6;
}
message FunctionChain {
string name = 1;
FunctionChainStage stage = 2;
repeated FunctionChainOp ops = 3;
}
message FunctionChainOp {
string op = 1;
FunctionChainExpr expr = 2;
repeated string inputs = 3;
repeated string outputs = 4;
map<string, FunctionParamValue> params = 5;
}
message FunctionChainExpr {
string name = 1;
repeated FunctionChainExprArg args = 2;
map<string, FunctionParamValue> params = 3;
}
message FunctionChainExprArg {
oneof arg {
FunctionChainColumnArg column = 1;
FunctionParamValue literal = 2;
}
}
message FunctionChainColumnArg {
string name = 1;
}
message FunctionParamValue {
oneof value {
bool bool_value = 1;
int64 int64_value = 2;
double double_value = 3;
string string_value = 4;
FunctionParamArray array_value = 5;
FunctionParamObject object_value = 6;
bytes bytes_value = 7;
}
}
message FunctionParamArray {
repeated FunctionParamValue values = 1;
}
message FunctionParamObject {
map<string, FunctionParamValue> fields = 1;
}
SearchRequest carries chains through:
repeated schema.FunctionChain function_chains = 24;
Hybrid request proto may reserve a field for future support, but first-version execution rejects it.
$score$score is a system virtual column, not a collection field.
Runtime behavior:
$score is initialized from the current search result score/distance.$score through col("$score").map("$score", expr) overwrites the current score register.sort(col("$score"), desc=True) sorts candidates by the current rewritten score.$score is serialized through existing result score/distance fields.Representation:
| Layer | Representation |
|---|---|
| Python DSL | "$score" |
| Proto | FunctionChainColumnArg.name = "$score" |
| Runtime | score register / DataFrame column |
| Search result | existing distance/score field |
$id is also available as a read-only system value for tie-breaking. Other $xxx system names are rejected in first-version L2 rerank.
mapmap(output, expr) evaluates an expression and writes the result to output.
output may be a temporary variable such as freshness.output may be writable system value $score.$id or unknown $xxx values.sortsort(by, desc=True, tie_break_col=None) sorts the current candidate chunk.
by is encoded as an op input and parameter.tie_break_col is optional and is also encoded as an input.limitlimit(limit, offset=0) trims each query chunk after previous operators.
This is part of the user-provided plan. Search does not append implicit limit or offset operators to public function chains.
decayComputes a numeric decay score from one numeric input column.
Parameters:
function: gauss, exp, or linearoriginscaleoffsetdecaynum_combineCombines two or more numeric inputs.
Modes:
multiplysummaxminavgweightedweighted mode requires one numeric weight per input.
round_decimalRounds one Float32 score column to a fixed number of decimal places in [0, 6].
rerank_modelCalls an external rerank model provider for a text column. It is only runnable at L2 rerank stage in the first release.
Required parameters:
queries: one query per search query chunk.provider, model_name, max_client_batch_size, and provider-specific options.Provider credentials and endpoint defaults are resolved on the Milvus server using existing function provider configuration.
Function Chain separates chain execution names from final result projection.
expr-based op read names = column references in FunctionChainExpr.args
non-expr op read names = FunctionChainOp.inputs
op write names = FunctionChainOp.outputs
final result projection = Search-owned output projection
Example:
FunctionChain(FunctionChainStage.L2_RERANK) \
.map("freshness", fn.decay(col("published_at"), ...)) \
.map("$score", fn.num_combine(col("$score"), col("freshness"), mode="sum")) \
.sort(col("$score"), desc=True)
Dependency analysis sees:
published_at, $score;freshness, $score;freshness is not fetched from collection schema because a previous op writes it;published_at is fetched internally for rerank even if it is not in user output_fields;$id, final $score, and user-requested output fields.Intermediate variables such as freshness are not returned to the user.
SearchRequest.function_chains
-> SDK serialization
-> Proxy request validation
-> chain.ProtoChainToRepr
-> Proxy L2 input planning
-> normal search/reduce/requery pipeline
-> rerankOperator builds DataFrame
-> chain.FuncChainFromRepr
-> FuncChain.Execute
-> Search-owned final projection
Function Chain L2 rerank is treated as a rerank source. It reuses the existing rerankOperator rather than adding a separate functionChainOperator.
The chain package converts public proto to a caller-independent representation:
type ChainRepr struct {
Name string
Stage string
Operators []OperatorRepr
Info ChainReprInfo
}
type ChainReprInfo struct {
RequiredInputs []string
WrittenNames []string
Ops []OperatorReprInfo
}
type OperatorReprInfo struct {
Type string
ReadNames []string
WriteNames []string
}
ChainRepr.Info.RequiredInputs only means "the chain reads these names before any previous op writes them." It does not decide whether a name is a schema field, runtime system value, or invalid. That classification is caller-owned.
For ordinary Search L2 rerank, Proxy classifies each required input:
$score and $id are runtime system inputs.$xxx system inputs are rejected.First-version supported schema input field types:
Unsupported input field types include vector fields, JSON, Array, Geometry, and dynamic field subkeys.
function_score conflictfunction_score and function_chains are mutually exclusive:
function_score and function_chains cannot be used together
Both APIs define rerank score behavior. Combining them would make ordering ambiguous.
SDK ranker maps to legacy rerank/function-score behavior, so SDK rejects ranker plus function_chains before RPC.
The same FunctionChainStage may appear at most once in one request.
The first release supports only one ordinary Search stage:
FunctionChainStageL2Rerank
Users who need multiple rerank steps should put multiple ops in one L2 chain instead of sending multiple L2 chains.
First-version hybrid search rejects function_chains:
function_chains is not supported for hybrid search yet
Hybrid support needs a separate design for whether public chains apply to sub-searches, merged candidates, or both.
No new fetch mechanism is required. Function-chain input fields flow through existing rerank metadata:
rerankMeta.GetInputFieldNames()
rerankMeta.GetInputFieldIDs()
When requery is needed, the requery operator includes rerank-required field names so the DataFrame can be built before rerank execution. Final projection still uses user output fields and does not expose internally fetched rerank inputs.
rerankOperator follows the existing function-score flow:
SearchResultData
-> chain.FromSearchResultData(..., neededFields)
-> build FuncChain from rerank metadata
-> ExecuteWithContext
-> chain.ToSearchResultDataWithOptions(...)
The chain builder dispatches by rerank metadata type:
FuncChainFromRepr / FuncChainFromReprWithContext.A public FunctionChain is executed as sent.
Milvus does not implicitly append tail operators such as:
Users or SDK helpers must add explicit ops when they want those effects.
First-version validation includes:
$id and $score.$score.Additional ordering constraints such as "at most one sort" or "sort must be last" can be considered as future stricter validation. The first release executes the ordered plan as sent unless a runtime operator rejects it.
function_chains are unchanged.function_score and legacy rank behavior remain supported.No deprecation is introduced in this MEP.
Users can migrate from function_score or ranker APIs to function_chains when they need explicit ordered composition. There is no automatic conversion in the first release.
External model rerank can call third-party services from Milvus server processes.
Security requirements:
function_chains requests should not include raw API keys.Initial observability reuses existing search/function error paths and provider HTTP errors.
Useful follow-up metrics:
map, sort, limit.col(...) validation.decay, num_combine, round_decimal, and rerank_model.function_chains plus ranker.function_chains for hybrid search.function_score plus function_chains is rejected.function_chains is rejected.$score-only chain succeeds with no schema input fields.field + $score chain fetches only required schema fields.$xxx system input is rejected.buildChainFromMeta builds a FuncChain from functionChainRerankMeta.rerankOperator executes a proto-derived chain through DataFrame.$score changes result order and scores.limit updates per-query TopKs.function_score and legacy rank behavior remain unchanged.rerank_model, such as VoyageAI, gated on server-side credentials.Run targeted Go tests with Milvus test flags:
go test -tags dynamic,test -gcflags="all=-N -l" -count=1 ./internal/util/function/chain/...
go test -tags dynamic,test -gcflags="all=-N -l" -count=1 ./internal/proxy/... -run 'FunctionChain|Rerank'
Run SDK tests from the PyMilvus repository or local checkout as appropriate.
KeyValuePairRejected because function-chain parameters may contain nested objects, arrays, booleans, integers, floating-point values, and bytes. JSON-in-string encoding would cause:
FunctionParamValue keeps the public plan typed.
function_score for all chain behaviorRejected because function_score is not an ordered operator pipeline and cannot naturally express multiple map/sort/limit/model steps with explicit dependencies.
functionChainOperator to the search pipelineRejected for the first release because function chains are another rerank implementation. Reusing the existing rerankOperator keeps fetch/requery/final-projection behavior consistent with legacy rerank.
Rejected because only the caller knows whether a name is a schema field, request payload field, runtime system value, or invalid. The chain package only reports structural dependencies.
sort and only as the last ordering op?