Back to Calico

Felix — Architecture & Design Index

felix/DESIGN.md

3.34.0-0.dev17.0 KB
Original Source
<!-- Copyright (c) 2026 Tigera, Inc. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 -->

Felix — Architecture & Design Index

Felix is Calico's per-node agent. It watches the datastore for configuration, computes per-endpoint state through its calculation graph, and programs one or more dataplanes (BPF, iptables, nftables, Windows) to enforce policy and route traffic.

This document has two parts:

  1. Architecture overview — the shape of Felix as a whole. Read this first; it sets the context every sub-design depends on.
  2. Sub-design index — pointers to per-topic design docs under felix/design/ with a path-to-doc mapping. Invariants and review criteria live in the sub-designs, not here.

Operational guidance (how to build, test, debug, use tooling) is separate and lives in felix/CLAUDE.md.

1. Architecture overview

Data flow

Datastore syncer
   → AsyncCalcGraph
   → CalcGraph (dispatcher + calculation nodes)
   → EventSequencer
   → InternalDataplane
   → dataplane-specific managers → kernel objects
  1. Datastore syncer receives updates from the Calico datastore (Kubernetes CRDs or etcd).
  2. CalcGraph processes them through a graph of calculation nodes — policy resolution, route resolution, IP set indexing, service-endpoint synthesis, VTEP calculation, encapsulation mode.
  3. EventSequencer buffers and coalesces the outputs and flushes them in dependency-safe order (e.g. IP sets before the policies that reference them).
  4. The dataplane driver (dataplane/driver.go) selects a dataplane implementation based on configuration.
  5. InternalDataplane (Linux; dataplane/linux/int_dataplane.go) or the Windows equivalent (dataplane/windows/win_dataplane.go) fans updates out to managers, each owning a slice of dataplane state.

Key orientation files: daemon/daemon.go, calc/calc_graph.go, dataplane/linux/int_dataplane.go, dataplane/driver.go.

Calculation graph (the "brain")

The calc graph in felix/calc/ is an event-processing pipeline that transforms raw datastore updates into dataplane-ready instructions. Key calculation nodes:

NodeRole
AllUpdDispatcherFans out datastore updates by resource type to downstream nodes
ActiveRulesCalculatorTracks which policies/profiles are active based on endpoint labels
RuleScannerScans rules for selector references; feeds the IP-set index
PolicyResolverResolves per-endpoint policy ordering (tiers, priorities)
L3RouteResolverComputes routes from IP pools, workload endpoints, and host IPs
VXLANResolverComputes VTEP (VXLAN tunnel endpoint) entries
EncapsulationResolverDetermines encapsulation mode from IP-pool config
IstioCalculatorMarks WEPs in the Istio ambient mesh (see bpf-observability "Istio ambient mode integration")

PipelineCallbacks (calc/calc_graph.go) is the composite interface the graph emits through. EventSequencer (calc/event_sequencer.go) is the primary implementation — it buffers updates in pending* maps/sets and flushes via Flush() in dependency-safe order, coalescing rapid updates so the dataplane only sees the final state.

Full invariants and per-node review notes — the node contract, the upstream syncer contract, inter-node ordering, the label indexes, the EventSequencer flush order, and the calc-graph FV testing framework — are in calc-graph.md.

Dataplane: managers and drivers

The Linux dataplane is a single codebase (InternalDataplane, dataplane/linux/) switched between iptables, nftables and eBPF modes; all three share the layering, event loop and restart-and-resync doctrine sketched here. Windows (dataplane/windows/) is a separate dataplane of the same overall shape.

The dataplane is split into two layers:

  • Managers (dataplane/linux/*_mgr.go) take the calc graph's Calico-internal desired state (local WEPs, abstract policy rules, resolved IP sets) and convert it into this dataplane's terms — iptables/nftables rules, IP-set contents and routes; BPF map entries; or Windows HNS policy. This conversion is the manager layer's defining job.
  • Drivers (iptables/, nftables/, ipsets/, routetable/, routerule/, vxlanfdb/) bring actual kernel state into sync with the desired state — read back what's there, compute a minimal delta, apply it. Some managers reconcile directly; others delegate to a driver and stay declarative.

Beyond convert and reconcile, the dataplane also reacts to expected kernel changes (interfaces coming and going, via ifacemonitor/), detects unexpected drift (periodic full resyncs), and reports programming status back to the datastore and to the CNI plugin.

Each manager implements a two-method Manager interface — OnUpdate to receive desired state cheaply, CompleteDeferredWork to program the kernel during a throttled apply() cycle. Key managers:

ManagerHandles
endpointManager / bpfEndpointManagerWorkload/host endpoint programming
policyManager / rawEgressPolicyManagerPolicy chain/rule generation
ipsetsManagerIP-set synchronisation
noEncapManager / vxlanManagerRoute management for encap modes
ipipManagerIPIP tunnel interfaces
wireguardManagerWireGuard tunnel setup
masqManagerIP masquerade rules
hostIPManagerHost-IP tracking
floatingIPManagerFloating-IP NAT
dscpManagerDSCP marking
serviceLoopManagerService-loop prevention
failsafeMgrBPF failsafe port programming

IPv4 and IPv6 each get their own manager instances; dataplane/driver.go is the factory that constructs and wires the dataplane.

The Manager contract and extended interfaces, the apply() ordering, the restart/resync (mark-and-sweep) doctrine, the *tables Table abstraction, IP sets, and the calc-graph→dataplane proto contract are all detailed in dataplane.md. eBPF mode reuses this architecture; its mode-specific managers, maps and packet path are in the bpf-* family.

Dataplane backends

Felix runs against one dataplane at a time (selected by BPFEnabled and NFTablesMode config):

  • BPF — eBPF programs on TC and cgroup hooks, BPF maps for NAT / conntrack / policy. See the bpf-* sub-designs under design/, starting with bpf-overview.md.
  • iptables — legacy netfilter via iptables-restore. Code in felix/iptables/.
  • nftables — modern netfilter via the nftables API. Code in felix/nftables/.
  • Windows (HNS/HCN) — separate dataplane in dataplane/windows/.

NFTablesMode=Auto resolves by following the detected kube-proxy mode: an nftables-mode kube-proxy selects the nftables dataplane. That signal is about coexistence — Felix must use the same netfilter generation as kube-proxy — not host capability, so it must not be replaced by a capability probe on cluster hosts. See useNftables() in dataplane/linux/int_dataplane.go. The per-host escape hatch is NFTablesMode=Disabled/Enabled set locally (env var or config file), which overrides any datastore-inherited value.

The iptables and nftables backends share a common rule- generation layer in felix/rules/ and a common table-abstraction interface in felix/generictables/. Backend-neutral rule generation: dispatch.go (per-endpoint dispatch chains), policy.go (policy chain generation), endpoints.go (endpoint chain setup), static.go (boilerplate filter/NAT/mangle chains), nat.go. A PR adding policy semantics usually touches felix/rules/ and needs matching changes on both backends.

Shared networking subsystems

Used by more than one dataplane:

PackagePurpose
routetable/Linux route-table management via netlink
routerule/Policy-based routing rules
vxlanfdb/VXLAN forwarding-database management
wireguard/WireGuard tunnel setup and key management
ifacemonitor/Interface state monitoring (link up/down, address changes)
nfnetlink/Conntrack and nflog via netfilter netlink
netlinkshim/Netlink abstraction layer for testing and portability

The route-sync drivers (routetable/, routerule/, vxlanfdb/) fit the dataplane manager/driver architecture and resync doctrine covered in dataplane.md; their deeper netlink-level design (resync grace periods, conntrack cleanup on IP moves) is reserved for a future route-sync.md sub-design. flow-logs-collector.md is likewise still to be written.

Cross-component designs Felix takes part in

Some subsystems are split between Felix and another component, so their design lives at the repo level rather than under felix/design/:

DesignWhat it covers in Felix
design/cluster-route-programming/DESIGN.mdWhether Felix or confd/BIRD programs the routes to workloads on other nodes, per encapsulation type. Covers ipipManager, noEncapManager, EncapsulationResolver.NoEncapNeeded, and the ProgramClusterRoutes config parameter.
design/ipam/DESIGN.mdFelix is a read-only consumer of IPAM state (IPAM blocks feed the L3RouteResolver).

2. Sub-design index

Per-topic design docs under felix/design/. Each is the authoritative source for its area's architecture, invariants, and review notes.

A PR that touches files across multiple "applies to" scopes must load every matching sub-design before acting. The applies to column is the authoritative mapping from source path to design doc.

The bpf-* rows form a single sub-design family for the BPF dataplane, deliberately split so a PR touching one area pulls only the relevant knowledge. The bpf-overview umbrella row is the always-pulled foundation (packet-path mental model, fast-path cost rule, cross-cutting review notes); the others have tight applyTo globs scoped to their topic. Load each of them either when you touch a matched file or when you're working on the related topic — the globs cover the common cases, but a change in a central file (e.g. tc.c, bpf.h) may legitimately need a sub-design even if the immediate edit site doesn't match its glob narrowly, and conversely a PR description that says "this fixes the conntrack scanner" should pull bpf-conntrack-flowstate.md even if the edit happens to land in code paths the glob doesn't list. Other sub-designs should split the same way once they grow large enough to bloat AI-tool context.

TopicApplies toStatus
bpf-overviewfelix/bpf/**, felix/bpf-gpl/**, felix/dataplane/linux/bpf_*.go, felix/dataplane/linux/vxlan_mgr.go (umbrella — pulled by every BPF change)✅ exists
bpf-tc-programsfelix/bpf-gpl/tc.c, tc_preamble.c, xdp_preamble.c, jump.h, bpf.h, globals.h, types.h, felix/bpf/hook/**, felix/bpf/tc/**, felix/bpf/jump/**, felix/bpf/ifstate/**✅ exists
bpf-xdpfelix/bpf-gpl/xdp.c, xdp_preamble.c, metadata.h, felix/bpf/xdp/**✅ exists
bpf-servicesfelix/bpf/proxy/**, felix/bpf/nat/**, felix/bpf/consistenthash/**, felix/bpf-gpl/connect*.{c,h}, nat*.h, nat_lookup.h, maglev.h, ctlb*.h, sendrecv.h, felix/dataplane/linux/bpf_ep_mgr.go✅ exists
bpf-host-networkingfelix/dataplane/linux/bpf_ep_mgr.go, dataplanedefs/dataplane_defs.go, felix/bpf-gpl/fib_co_re.h✅ exists
bpf-conntrack-flowstatefelix/bpf/conntrack/**, felix/bpf-gpl/conntrack*.{c,h}, rpf.h, felix/bpf/allowsources/**, felix/rules/static.go✅ exists
bpf-encap-fragments-icmpfelix/bpf/ipfrags/**, felix/bpf-gpl/ip_v4_fragment.h, tc_ip_frag.c, icmp*.h, fib*.h, felix/bpf/routes/**, felix/dataplane/linux/vxlan_mgr.go✅ exists
bpf-observabilityfelix/bpf/filter/**, events/**, ringbuf/**, qos/**, felix/bpf-gpl/log.h, events*.h, qos.h, ringbuf.h✅ exists
bpf-testsfelix/bpf/ut/**, felix/fv/bpf_*_test.go✅ exists
dataplanefelix/dataplane/linux/** (the shared loop/manager/resync architecture, all modes — BPF-specific files here are also matched by the bpf-* rows, intentionally), felix/iptables/**, felix/nftables/**, felix/generictables/**, felix/ipsets/**, felix/markbits/**, felix/rules/**; also the manager/driver architecture & resync doctrine for felix/routetable/**, felix/routerule/**, felix/vxlanfdb/**✅ exists
calc-graphfelix/calc/**, felix/labelindex/**, felix/dispatcher/**✅ exists
route-sync (deep netlink design only)felix/routetable/**, felix/routerule/**, felix/vxlanfdb/**architecture covered by dataplane.md; this row reserved for the deeper netlink-level resync designnot yet written
flow-logs-collectorfelix/collector/**not yet written
config-enginefelix/config/**not yet written
windows-dataplanefelix/dataplane/windows/**not yet written

A missing sub-design means the area's invariants have not been written down yet — not that the area has no constraints. Treat absence as "read the code and ask"; do not assume anything goes.

3. For coding agents and reviewers

  • Follow links. Every sub-design may reference sibling sub-designs, .github/instructions/*.instructions.md files, code, or external references. Load them. A design is a graph, not a single node.
  • Load what applies — by path or by topic. The applies to globs above are the path-based trigger: if a PR touches both BPF and route-sync code, both sub-designs are needed. The topic of the change matters too — a PR described as "fixing the conntrack scanner" should pull bpf-conntrack-flowstate.md even if the edit happens to land only in a central file the glob covers under a broader umbrella. When in doubt, pull the topic-relevant sub-design.
  • Review notes are the checklist. Each sub-design embeds per-section review notes describing the invariants a PR must respect. At write-time, respect them; at review-time, apply them.
  • Update rule. A change to how Felix works in a given area must update the relevant file under felix/design/ in the same PR — typically the sub-design covering the area. This index (felix/DESIGN.md) is also updated when the sub-design table, a applies to scope, or §1's architecture overview changes. Exemptions: (a) a bug fix that restores behaviour the doc already describes, (b) a mechanical refactor with no observable change, (c) comment or log-message edits, (d) dependency bumps. If in doubt, update. The path-scoped .github/instructions/*.instructions.md files wire this rule into Copilot's automated review.

4. Adding a new sub-design

When a topic above graduates from not yet written to a real doc:

  1. Create felix/design/<topic>.md. Follow the shape of an existing sub-design (e.g. the BPF family): narrative prose, architecture, per-section review notes at the end of each section, and a "keep this in sync" tail.
  2. Update the sub-design index above: replace not yet written with a link to the new file and the ✅ exists marker.
  3. Move any orientation content that belongs to the new sub-design out of this file into the new doc (most of the §1 architecture overview is cross-cutting and stays here).
  4. Create a matching .github/instructions/<topic>.instructions.md with the applyTo globs from the table above plus a pointer to the new design doc. Keep it thin — see any of the bpf-*.instructions.md files as the template.

When a sub-design grows too big, split it

A single sub-design that grows large enough to bloat AI-tool context for narrow PRs (in practice: ~2000+ lines, ~25k+ tokens) should split into a family of focused files the way the BPF dataplane did:

  • A short always-pulled <topic>-overview.md containing the mental model, the cost / discipline rules, and cross-cutting review notes.
  • Per-area files (<topic>-<area>.md) covering specific feature groupings, each with a tight applyTo glob in its own .github/instructions/<topic>-<area>.instructions.md.

The multi-file family appears in this index as multiple rows, all sharing the topic prefix. A PR touching multiple areas matches multiple instruction files; only the union of the matched sub-designs loads, not the whole family.