ContextQMD
Back to Cilium

README

contrib/containerlab/multi-homing/README.md

1.20.04.5 KB
Original Source

Multi-homing BGP

Multi-homing BGP means Cilium node is peering with two upstream routers over different links.

Common design requirements

  • Node is connected to two different top-of-rack switches for redundancy.
  • Separation of east-west and north-south traffic via different links on the node. This may require different advertisements to each peer.

BGP Resources

BGP instance configuration

In the below example, we define two eBGP peers - fd00:10:0:1::1 and fd00:11:0:1::1, both of which have identical peering configuration defined in CiliumBGPPeerConfig resource with name cilium-peer.

yaml
  bgpInstances:
  - name: "65001"
    localASN: 65001
    peers:
    - name: "65000"
      peerASN: 65000
      peerAddress: fd00:10::1
      peerConfigRef:
        name: "cilium-peer"
    - name: "65011"
      peerASN: 65011
      peerAddress: fd00:11::1
      peerConfigRef:
        name: "cilium-peer"

BGP peer configuration

Peer configuration contains various peering settings like transport, authentication and AFI/SAFI configurations.

yaml
  families:
    - afi: ipv4
      safi: unicast
      advertisements:
        matchLabels:
          advertise: "pod-cidr"
    - afi: ipv6
      safi: unicast
      advertisements:
        matchLabels:
          advertise: "pod-cidr"

In the above example, both IPv4 and IPv6 address families are enabled and for each address family we advertise CiliumBGPAdvertisement resource which matches the label "advertise=pod-cidr".

BGP advertisement

Snippet of 'PodCIDR' advertisement is defined below. BGP control plane will advertise pod cidr prefix with BGP community attribute of 'no-export'.

yaml
    - advertisementType: "PodCIDR"
      attributes:
        communities:
          wellKnown: [ "no-export" ]

BGP node configuration override

In below example, router ID is configured manually for each node. Name of the CiliumBGPNodeConfigOverride resource matches the node name on which this configuration will be applied.

yaml
---
apiVersion: cilium.io/v2alpha1
kind: CiliumBGPNodeConfigOverride
metadata:
  name: bgp-cplane-dev-multi-homing-control-plane
spec:
  bgpInstances:
    - name: "65001"
      routerID: "1.2.3.4"

---
apiVersion: cilium.io/v2alpha1
kind: CiliumBGPNodeConfigOverride
metadata:
  name: bgp-cplane-dev-multi-homing-worker
spec:
  bgpInstances:
    - name: "65001"
      routerID: "5.6.7.8"

Verification

BGP Peering

root@bgp-cplane-dev-multi-homing-worker:/home/cilium# cilium-dbg shell -- bgp/peers
Instance   Peer    Session State   Uptime   Family         Received   Accepted   Advertised
65001      65000   established     1m9s     ipv4-unicast   0          0          1
                                            ipv6-unicast   0          0          1
           65011   established     1m9s     ipv4-unicast   0          0          1
                                            ipv6-unicast   0          0          1

BGP Routes

PodCIDR is 10.1.1.0 on this node, which is advertised with communities attribute 'no-export'.

root@bgp-cplane-dev-multi-homing-worker:/home/cilium# cilium-dbg shell -- bgp/routes out ipv4 unicast -a
Instance   Peer    Prefix        NextHop          Age     Attrs
65001      65000   10.1.1.0/24   fd00:10:0:2::2   1m24s   [{Origin: i} {AsPath: 65001} {Communities: no-export}]
           65011   10.1.1.0/24   fd00:11:0:2::2   1m24s   [{Origin: i} {AsPath: 65001} {Communities: no-export}]

On peering routers we can see 10.1.1.0/24 prefix with appropriate route attributes and configured router ID.

FRR Router0

 docker exec -it clab-bgp-cplane-dev-multi-homing-router0 vtysh -c 'sh bgp ipv4 10.1.1.0'
BGP routing table entry for 10.1.1.0/24, version 2
Paths: (1 available, best #1, table default, not advertised to EBGP peer)
  Not advertised to any peer
  65001
    fd00:10:0:2::2 from fd00:10:0:2::2 (5.6.7.8)               <<<<<<<<< Router ID
      Origin IGP, valid, external, best (First path received)
      Community: no-export                                     <<<<<<<<< Community
      Last update: Fri Jun 28 15:30:33 2024

FRR Router1

docker exec -it clab-bgp-cplane-dev-multi-homing-router1 vtysh -c 'sh bgp ipv4 10.1.1.0'
BGP routing table entry for 10.1.1.0/24, version 1
Paths: (1 available, best #1, table default, not advertised to EBGP peer)
  Not advertised to any peer
  65001
    fd00:11:0:2::2 from fd00:11:0:2::2 (5.6.7.8)
      Origin IGP, valid, external, best (First path received)
      Community: no-export
      Last update: Fri Jun 28 15:30:31 2024