felix/design/bpf-xdp.md
XDP's narrow role in Calico (untracked-policy fast-drop / early-accept), how packets that XDP accepted hand off to TC via xdp2tc metadata, and the BPFForceTrackPacketsFromIfaces per-interface opt-out for 3rd-party DNAT interoperability.
This is one of several sub-designs for the eBPF dataplane. See
bpf-overview.md for the packet-path mental
model, the fast-path cost rule, and the cross-cutting review notes
that apply to every BPF change. The full set of sub-designs is
listed in felix/DESIGN.md.
XDP programs run in the NIC driver's receive path, before the kernel
allocates an skb. In Calico they are attached to HEPs that support
XDP (hardware or generic-XDP fallback) alongside the TC programs.
Their purpose is narrow but important:
skb
path at all — the NIC DMA buffer is recycled. This is the fast
defence against volumetric attacks and obvious-hostile traffic.XDP is stateless: no conntrack, no NAT. It only carries rules that the admin has marked as not needing connection tracking. Anything subtler — service NAT, policy that depends on flow state, fragments — falls through to TC.
A packet that XDP has accepted still needs to go through the rest of the dataplane — most obviously it still needs conntrack, NAT and forwarding decisions. The handoff is via packet metadata:
xdp2tc_set_metadata(ctx, CALI_META_ACCEPTED_BY_XDP)
(felix/bpf-gpl/metadata.h, used from xdp.c). The metadata is
written into a region of the packet buffer that XDP reserves and
TC can read.xdp2tc_get_metadata(skb) and, on
CALI_META_ACCEPTED_BY_XDP, sets
skb->mark = CALI_SKB_MARK_BYPASS_XDP (bpf.h) and skips the
policy step.The same jump-map / preamble machinery from bpf-tc-programs.md → TC program layout applies. XDP has its
own preamble (xdp_preamble.c) and its own jump map
(xdp_cali_progs).
XDP untracked policy means a packet can bypass the regular
tracked-flow path. For some interfaces (e.g. ones that carry
*tables-managed DNAT, see bpf-conntrack-flowstate.md → 3rd-party DNAT on host traffic) this is wrong — the packet must
be tracked so that netfilter's conntrack can match on return.
Calico supports per-interface opt-out via
BPFForceTrackPacketsFromIfaces. The generated *tables rule
(ChainRawUntrackedFlows via BPFForceTrackPacketsFromIfaces in
felix/rules/static.go) forces traffic on those interfaces through
the regular tracked path regardless of what XDP would have done.
CALI_META_*), corresponding code in
xdp2tc_set_metadata / xdp2tc_get_metadata, and a TC-side
handler. Do not overload skb->mark directly from XDP — the
metadata path is the only place where XDP→TC signal travel is
preserved across the skb allocation boundary.CALI_SKB_MARK_BYPASS_XDP must keep conntrack and NAT working
— those happen after the XDP-accept check in the TC main
program and are not skipped.*tables
conntrack (existing or new 3rd-party DNAT) needs to end up in
BPFForceTrackPacketsFromIfaces; without it, XDP's fast-drop
can make the packet invisible to netfilter.A change to how the BPF dataplane works in the area this file covers must update the relevant section in the same PR — new mechanism, new flag, new map field, new config knob, or any change to the packet path. Exemptions: (a) bug fix restoring documented behaviour, (b) mechanical refactor with no observable change, (c) comment / log-message edits, (d) dependency bumps. If in doubt, update.
Cross-cutting rules that apply to every BPF change (map
versioning, mark discipline, sub-program registration, kernel-
version sensitivity) live in
bpf-overview.md → Cross-cutting review notes.