Home > Networking > Lab to configure a Cisco VXLAN EVPN-BGP fabric with a spine-leaf topology

Lab to configure a Cisco VXLAN EVPN-BGP fabric with a spine-leaf topology

By /

Let’s set up a lab to configure a Cisco VXLAN EVPN-BGP fabric with a spine-leaf topology, including normal leaf switches with vPC for server connectivity, border leaf switches, and border gateway switches for Layer 2 VM failover to another data center (DC). This lab will simulate a modern data center fabric with EVPN as the control plane and VXLAN as the data plane, focusing on a single site with connectivity to a remote DC. I’ll provide a step-by-step configuration for clarity, assuming Nexus 9000 series switches (e.g., NX-OS 9.3(x)) and a basic understanding of EVPN/VXLAN concepts.

Lab Topology Overview

  • Spine Switches: 2 spines (Spine-1, Spine-2) for underlay routing and BGP route reflection.
  • Leaf Switches:
    • 2 normal leaf switches (Leaf-1, Leaf-2) in a vPC pair for server connectivity.
    • 1 border leaf (Border-Leaf-1) for external Layer 3 connectivity within the site.
  • Border Gateway Switches: 2 BGWs (BGW-1, BGW-2) in vPC mode for inter-DC Layer 2 extension.
  • Server: Dual-homed to Leaf-1 and Leaf-2 via vPC.
  • Remote DC: Simulated as a single switch (Remote-BGW) for Layer 2 extension.
  • Underlay: OSPF for IP reachability.
  • Overlay: BGP EVPN for control plane, VXLAN for data plane.

Assumptions

  • IP addressing: Spine/Leaf underlay uses 10.1.x.x, loopbacks in 172.16.x.x range.
  • VLAN 10 (VNI 10010) for server traffic, extended to the remote DC.
  • VRF “Tenant1” for Layer 3 segmentation.
  • Multicast (PIM ASM) for intra-site BUM traffic; ingress replication for inter-site BUM.

Step 1: Base Configuration (All Switches)

Enable required features on all switches:

bash

feature ospf
feature bgp
feature pim
feature vn-segment-vlan-based
feature nv overlay
feature interface-vlan
feature vpc  # On Leaf-1, Leaf-2, BGW-1, BGW-2

Step 2: Underlay Configuration

Spine-1

bash

interface loopback0
  ip address 172.16.1.1/32
  ip router ospf 1 area 0
  ip pim sparse-mode

interface Ethernet1/1  # To Leaf-1
  ip address 10.1.1.1/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2  # To Leaf-2
  ip address 10.1.2.1/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3  # To Border-Leaf-1
  ip address 10.1.3.1/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4  # To BGW-1
  ip address 10.1.4.1/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/5  # To BGW-2
  ip address 10.1.5.1/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

router ospf 1
  router-id 172.16.1.1

ip pim rp-address 172.16.1.1 group-list 239.0.0.0/8

Spine-2 is similar (adjust IPs, e.g., loopback0 172.16.1.2, Eth1/1 10.1.1.5/30, etc.).

Leaf-1 (vPC Primary)

bash

vpc domain 1
  role priority 1000
  peer-keepalive destination 172.16.2.2 source 172.16.2.1

interface loopback0
  ip address 172.16.2.1/32
  ip router ospf 1 area 0
  ip pim sparse-mode

interface Ethernet1/1  # To Spine-1
  ip address 10.1.1.2/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2  # To Spine-2
  ip address 10.1.1.6/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface port-channel1  # vPC Peer-Link
  switchport mode trunk
  vpc peer-link

interface Ethernet1/3
  channel-group 1 mode active
  no shutdown

router ospf 1
  router-id 172.16.2.1

Leaf-2 (vPC Secondary)

Similar to Leaf-1, adjust IPs (e.g., loopback0 172.16.2.2, peer-keepalive destination 172.16.2.1).

Border-Leaf-1

bash

interface loopback0
  ip address 172.16.3.1/32
  ip router ospf 1 area 0
  ip pim sparse-mode

interface Ethernet1/1  # To Spine-1
  ip address 10.1.3.2/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2  # To Spine-2
  ip address 10.1.3.6/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

router ospf 1
  router-id 172.16.3.1

BGW-1 (vPC Primary)

bash

vpc domain 2
  role priority 1000
  peer-keepalive destination 172.16.4.2 source 172.16.4.1

interface loopback0
  ip address 172.16.4.1/32
  ip router ospf 1 area 0
  ip pim sparse-mode

interface loopback100  # Multi-Site VIP
  ip address 172.16.100.1/32
  ip router ospf 1 area 0
  ip pim sparse-mode

interface Ethernet1/1  # To Spine-1
  ip address 10.1.4.2/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2  # To Spine-2
  ip address 10.1.4.6/30
  ip router ospf 1 area 0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3  # To Remote-BGW (DCI Link)
  ip address 10.2.1.1/30
  ip router ospf 1 area 0
  no shutdown

interface port-channel2  # vPC Peer-Link
  switchport mode trunk
  vpc peer-link

interface Ethernet1/4
  channel-group 2 mode active
  no shutdown

router ospf 1
  router-id 172.16.4.1

BGW-2 is similar (loopback0 172.16.4.2, loopback100 172.16.100.1, adjust IPs).

Remote-BGW (Simulated Remote DC)

bash

interface loopback0
  ip address 172.16.5.1/32
  ip router ospf 1 area 0

interface Ethernet1/1  # To BGW-1
  ip address 10.2.1.2/30
  ip router ospf 1 area 0
  no shutdown

router ospf 1
  router-id 172.16.5.1

Step 3: Overlay Configuration (EVPN-BGP)

Spine-1 (Route Reflector)

bash

router bgp 65001
  router-id 172.16.1.1
  address-family l2vpn evpn
    neighbor 172.16.2.1 remote-as 65001
      update-source loopback0
      address-family l2vpn evpn
        send-community extended
        route-reflector-client
    neighbor 172.16.2.2 remote-as 65001
      update-source loopback0
      address-family l2vpn evpn
        send-community extended
        route-reflector-client
    neighbor 172.16.3.1 remote-as 65001
      update-source loopback0
      address-family l2vpn evpn
        send-community extended
        route-reflector-client
    neighbor 172.16.4.1 remote-as 65001
      update-source loopback0
      address-family l2vpn evpn
        send-community extended
        route-reflector-client
    neighbor 172.16.4.2 remote-as 65001
      update-source loopback0
      address-family l2vpn evpn
        send-community extended
        route-reflector-client

Spine-2 mirrors this (router-id 172.16.1.2).

Leaf-1 (vPC Pair with Leaf-2)

bash

fabric forwarding anycast-gateway-mac 0000.1111.2222

vlan 10
  vn-segment 10010
vlan 100
  vn-segment 50100  # L3 VNI

vrf context Tenant1
  vni 50100
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface vlan10
  no shutdown
  vrf member Tenant1
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface vlan100
  no shutdown
  vrf member Tenant1
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010 mcast-group 239.1.1.10
  member vni 50100 associate-vrf

interface port-channel10  # To Server
  switchport mode trunk
  switchport trunk allowed vlan 10
  vpc 10

interface Ethernet1/10
  channel-group 10 mode active
  no shutdown

router bgp 65001
  router-id 172.16.2.1
  neighbor 172.16.1.1 remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  neighbor 172.16.1.2 remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf Tenant1
    address-family ipv4 unicast
      advertise l2vpn evpn

evpn
  vni 10010 l2
    rd auto
    route-target import auto
    route-target export auto

Leaf-2 mirrors this (router-id 172.16.2.2, same vPC domain, anycast-gateway-mac).

Border-Leaf-1

bash

vlan 100
  vn-segment 50100

vrf context Tenant1
  vni 50100
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface vlan100
  no shutdown
  vrf member Tenant1
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 50100 associate-vrf

router bgp 65001
  router-id 172.16.3.1
  neighbor 172.16.1.1 remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  neighbor 172.16.1.2 remote-as 65001
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  vrf Tenant1
    address-family ipv4 unicast
      advertise l2vpn evpn

BGW-1 (vPC Pair with BGW-2)

bash

evpn multisite border-gateway 100
  delay-restore time 30

fabric forwarding anycast-gateway-mac 0000.1111.2222

vlan 10
  vn-segment 10010
vlan 100
  vn-segment 50100

vrf context Tenant1
  vni 50100
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn

interface vlan10
  no shutdown
  vrf member Tenant1
  ip address 192.168.10.1/24
  fabric forwarding mode anycast-gateway

interface vlan100
  no shutdown
  vrf member Tenant1
  ip forward

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  multisite border-gateway interface loopback100
  member vni 10010
    multisite ingress-replication  # For inter-site BUM
    mcast-group 239.1.1.10  # Intra-site BUM
  member vni 50100 associate-vrf

router bgp 65001
  router-id 172.16.4.1
  neighbor 172.16.1.1 remote-as 65001  # Spine RR
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  neighbor 172.16.1.2 remote-as 65001  # Spine RR
    update-source loopback0
    address-family l2vpn evpn
      send-community extended
  neighbor 172.16.5.1 remote-as 65002  # Remote-BGW (eBGP)
    update-source loopback0
    ebgp-multihop 5
    address-family l2vpn evpn
      send-community extended
      peer-type fabric-external
  vrf Tenant1
    address-family ipv4 unicast
      advertise l2vpn evpn

evpn
  vni 10010 l2
    rd auto
    route-target import auto
    route-target export auto

BGW-2 mirrors this (router-id 172.16.4.2, same loopback100 IP for vPC).

Remote-BGW

bash

vlan 10
  vn-segment 10010

interface vlan10
  no shutdown
  ip address 192.168.10.2/24  # Different IP for testing

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback0
  member vni 10010 ingress-replication

router bgp 65002
  router-id 172.16.5.1
  neighbor 172.16.4.1 remote-as 65001  # BGW-1
    update-source loopback0
    ebgp-multihop 5
    address-family l2vpn evpn
      send-community extended
  neighbor 172.16.4.2 remote-as 65001  # BGW-2
    update-source loopback0
    ebgp-multihop 5
    address-family l2vpn evpn
      send-community extended

evpn
  vni 10010 l2
    rd auto
    route-target import auto
    route-target export auto

Step 4: Server Connectivity

  • Configure the server with NIC teaming (LACP) to connect to Leaf-1 (Eth1/10) and Leaf-2 (Eth1/10) via vPC.
  • Assign VLAN 10 and an IP in 192.168.10.0/24 (e.g., 192.168.10.10).

Verification

  1. Underlay: show ip ospf neighbor, show ip route.
  2. Overlay:
    • show bgp l2vpn evpn summary (check peering).
    • show nve peers (check VXLAN tunnels).
    • show l2route evpn mac all (verify server MAC learned locally and remotely).
  3. vPC: show vpc on Leaf-1/Leaf-2 and BGW-1/BGW-2.
  4. Layer 2 Extension: Ping from server (192.168.10.10) to Remote-BGW’s VLAN 10 IP (192.168.10.2).

VM Failover

  • The Layer 2 extension (VLAN 10, VNI 10010) allows a VM to move from the server on Leaf-1/Leaf-2 to the remote DC (behind Remote-BGW).
  • EVPN updates MAC reachability dynamically, ensuring seamless failover without ARP issues due to the anycast gateway (192.168.10.1).

This lab demonstrates a scalable EVPN-BGP fabric with vPC for server redundancy and multi-site Layer 2 extension for VM mobility. Let me know if you’d like to adjust or expand any part!

Related posts:

  1. Difference between MTU, tcp-adjust-mss and PMTUD
  2. Define abnormal network traffic on cisco NDR – Secure Network Analytics (Previous name steathwatch)
  3. Cisco Router QoS with Policy Map: Optimizing WAN Links for Better Performance
  4. Should Your First Networking Certification Be CCNA? A Beginner’s Guide to Starting Strong

Powered by YARPP.

Leave a Comment