Routing Design
The Aruba ESP Campus Routing Design section describes the technologies and design principals used in the design of an Aruba ESP Layer 3 LAN topology and control plane.
Table of contents
OSPF Routing
Aruba ESP best practice uses OSPF for its simplicity and ease of configuration. OSPF is a dynamic, link-state, standards-based routing protocol commonly deployed in campus networks. OSPF provides fast convergence and excellent scalability, making it a good choice for large networks because it can grow with the network without requiring redesign.
OSPF defines areas to limit routing advertisements and to allow for route summarization. The Aruba ESP campus design uses a single area for the campus LAN. Multi-area, backbone designs are considered when connecting multiple campus or WAN topologies. OSPF is often used to exchange routes between the campus LAN and a WAN gateway or a DMZ firewall.
The ESP underlay best practice configuration uses OSPF point-to-point links between aggregation and core devices. Interfaces on aggregation switches providing layer 3 services to downstream hosts are configured as members of the OSPF domain. Configure the OSPF router for passive-interface default to prevent unintended adjacencies from forming with devices plugged into a layer 2 access port. When an OSPF neighbor is expected on a port, disable passive operation.
When configuring access switches, best practice is to configure an IP address in the management VLAN and to enable OSPF on that VLAN IP interface. Adding /32 loopback interfaces to OSPF also lays the foundation for a high-reliability management network.
The diagram below illustrates the OSPF fundamentals of a three-tier campus LAN.
OSPF in Three-Tier Wired
IP Multicast
Aruba Networks ESP uses Protocol Independent Multicast — Sparse Mode (PIM-SM) to route multicast traffic on the network. Additional mechanisms required to route multicast traffic are:
- Bootstrap router protocol (BSR)
- The rendezvous point (RP)
- Multicast Source Discovery Protocol (MSDP)
- Internet Group Management Protocol (IGMP)
Configure PIM-SM on all routed links to enable multicast traffic on the network. PIM-SM uses reverse path forwarding (RPF) based on the active routing table to find the best path toward an RP or multicast source.
The BSR protocol is enabled on all routers in a PIM multicast domain to share RP information dynamically. After a single router is elected as the BSR, it advertises RP information to all participating routers, freeing the administrator from having to configure an RP address on each network router. After the BSR is elected, all routers with RP candidate interfaces send the candidate RP IP address to the BSR. The active RP is selected from the list of candidates.
The RP contains information on active multicast sources in a PIM multicast domain and is the root of the Rendezvous Point Tree (RPT). Anycast networking enables multiple RPs to be active at the same time for redundancy and traffic flow optimization. Configure core switches to announce an anycast loopback IP address as a candidate-RP to be selected by the BSR.
MSDP facilitates active-active RP anycast redundancy by sharing multicast source information between RPs, ensuring that all multicast sources in a PIM multicast domain are known by the full set of anycast RPs. MSDP is enabled on campus core switches.
Enable IGMP on aggregation switch layer 3 interfaces with downstream clients. If a wireless gateway or another layer 2 device is connected directly to the core switches, enable IGMP on layer 3 interfaces facing those devices. Enable IGMP snooping on access layer switches.
Note: IGMP timers must match across all switches on the network, including switches from other vendors.
Overlay Networks
Overlay networks provide a mechanism to deploy flexible topologies that meet the demands of everchanging endpoints and applications. By fully decoupling user data traffic and associated policy from the physical topology of the network, overlays enable on-demand deployment of L2 or L3 services. Overlays also make it simpler to carry device or user role information across the network without requiring all devices in the path to understand or manage the roles. Overlay networks are virtual networks, built on top of an underlay network composed of physical infrastructure. The underlay should be designed for stability, scalability, and predictability. Technologies such as GRE and IPSEC have been used to create overlays in the campus and WAN space for many years. Virtual Extensible LAN (VXLAN) is now an option to create distributed overlay networks in the campus.
Aruba provides the flexibility to choose between centralized overlays or distributed overlays to address a range of traffic engineering and policy enforcement requirements. User Based Tunneling is a centralized overlay architecture that provides simplified operations and advanced security features on gateway devices. EVPN-VXLAN is a distributed overlay architecture that enables dynamic tunnels between switches anywhere on a campus, providing consistent and continuous policy enforcement across the network. Both overlay models support the “Colorless Ports” feature, which enables automated client onboarding and access control for ease of operations.
Centralized Overlay Summary
User-Based Tunneling (UBT) is a centralized overlay that enables administrators to tunnel specified user traffic to a gateway cluster to enforce policy using services such as firewalling, DPI, application visibility, and bandwidth control. UBT selectively tunnels traffic based on a user or device role. The policies associated with each client usually are assigned through a RADIUS server such as ClearPass Policy Manager.
Distributed Overlay Summary
Distributed overlays are an evolution of the traditional campus network design. Distributed overlays are built using EVPN-VXLAN on highly available underlays and are tied to a full policy-based micro-segmentation, based on global roles, across the entire network infrastructure. Role-based policies abstract policy from the underlaying network and enable flexible and simplified policy definition and enforcement. This is provisioned by fully automating the overlay to provide a single-pane-of-glass management overview.
A Distributed Fabric is formed by assigning personas to various devices in the network. The list below describes the purpose of each persona.
- Route Reflector (RR) - Core switches are configured as BGP route reflectors (the RR persona) to share EVPN reachability information. This reduces the number of peering sessions required across the fabric.
- Stub - Wireless aggregation switches are configured with the stub persona to extend policy enforcement to wireless gateways, which only support static VXLAN tunnels. The aggregation switches carry GPID values from the campus fabric VXLAN tunnels forward into static VXLAN tunnels configured between the aggregation switches and the gateways.
- Border - Internet edge switches use the border persona to provide connectivity between the fabric and services outside the fabric.
- Edge - The edge persona is applied to access switches that provide primary VXLAN tunnel ingress/egress and policy enforcement for endpoint traffic into or out of the fabric.
- Intermediate Devices - Wired aggregation switches are underlay devices with no fabric persona assigned. They do not run a VTEP and must support jumbo frames.
Distributed Overlay
Distributed overlay networks in the Aruba ESP Campus are created using EVPN-VXLAN. This suite of protocols creates a dynamically formed network fabric that extends layer 2 connectivity over an existing physical network and layer 3 underlay. It is an open standards suite to create more agile, secure, and scalable networks in campuses and data centers. EVPN-VXLAN consists of:
- Ethernet VPN (EVPN), a BGP-driven control plane for overlays that provides virtual connectivity between different layer 2/3 domains over an IP or MPLS network.
- Virtual extensible LANs (VXLAN), a common network virtualization tunneling protocol that expands the number of layer 2 broadcast domains to 16 million from the 4,000 available using traditional VLANs.
Aruba ESP implements EVPN-VXLAN overlays on an IP underlay using redundant, layer 3 links for high-speed resiliency and maximum bandwidth utilization.
EVPN-VXLAN Benefits
Uniform bridging and routing across a diverse campus topology:
- Efficient layer 2 extension across layer 3 boundaries.
- Anycast gateways to ensure consistent first-hop routing services across the campus.
- End-to-end segmentation using VXLAN-Group Based Policies (VXLAN-GBP) provides the ability to propagate policy anywhere in the campus.
- Transported across any IP network supporting jumbo frames, VXLAN must be deployed only on the edge devices of the fabric.
EVPN-VXLAN Control Plane
In Aruba ESP, the EVPN-VXLAN control plane is Multi-Protocol BGP (MP-BGP) which communicates MAC addresses, MAC/IP bindings, and IP Prefixes to ensure endpoint reachability across the fabric. This approach is far superior to both inefficient flood-and-learn communication on the data plane and centralized control plane approaches with inherent scaling limitations.
The use of MP-BGP with EVPN address families between virtual tunnel endpoints (VTEPs) provides a standards-based, highly scalable control plane for sharing endpoint reachability information with native support for multi-tenancy. For many years, service providers have used MP-BGP to offer secure layer 2 and layer 3 VPN services at a very large scale. An iBGP design with route reflectors simplifies design by eliminating the need for a full mesh of BGP peerings across the full set of switches containing VTEPs. BGP peering is required only between VTEP terminating switches (access, stub, and service aggregation) and the core.
BGP control plane constructs include:
- Address Family (AF) - MP-BGP enables the exchange of network reachability information for multiple address types by categorizing them into address families (IPv4, IPv6, L3VPN, etc.). The layer 2 VPN address family (AFI=25) and the EVPN subsequent address family (SAFI=70) advertise IP and MAC address information between MP-BGP speakers. The EVPN address family contains reachability information for establishing VXLAN tunnels between VTEPs.
- Route Distinguisher (RD) - A route distinguisher enables MP-BGP to carry overlapping layer 3 and layer 2 addresses within the same address family by prepending a unique value to the original address. The RD is only a number with no inherent meaningful properties. It does not associate an address with a route or bridge table. The RD value supports multi-tenancy by ensuring that a route announced for the same address range via two different VRFs can be advertised in the same MP-BGP address family.
- Route Target (RT) - Route targets are MP-BGP extended communities used to associate an address with a route or bridge table. In an EVPN-VXLAN network, importing and exporting a common VRF route target into the MP-BGP EVPN address family establishes layer 3 reachability for a set of VRFs defined across a number of VTEPs. Layer 2 reachability is shared across a distributed set of L2 VNIs by importing and exporting a common route target in the L2 VNI definition. Additionally, layer 3 routes can be leaked between VRFs using the IPv4 address family by importing route targets into one VRF that are exported by other VRFs.
- Route Reflector (RR) - To optimize the process of sharing reachability information between VTEPs, the use of route reflectors at the core enables simplified iBGP peering. This design allows all VTEPs to have the same iBGP peering configuration, eliminating the need for a full mesh of iBGP neighbors.
The Aruba ESP Campus design uses two layer 3 connected core switches as iBGP route reflectors. The number of destination prefixes and overlay networks consume physical resources in the form of forwarding tables and should be considered when designing the network. Refer to the “Reference Architecture” section for hardware guidelines when scaling the fabric underlay design.
VXLAN Network Model
VXLAN encapsulates layer 2 Ethernet frames in layer 3 UDP packets. These VXLAN tunnels provide both layer-2 and layer-3 virtualized network services to connected endpoints. A VTEP is the function within a switch that handles the origination or termination of VXLAN tunnels. Similar to a traditional VLAN ID, a VXLAN Network Identifier (VNI) identifies an isolated layer-2 segment in a VXLAN overlay topology. Symmetric Integrated Routing and Bridging (IRB) enables the overlay networks to support contiguous layer-2 forwarding and layer-3 routing across leaf nodes.
Note: Configure jumbo frames on all underlay links in the fabric to ensure the accurate transport of additional encapsulation.
VXLAN networks comprise two key virtual network constructs: Layer 2 VNI and Layer 3 VNI. The relationship between an L2VNI, L3VNI, and VRF is described below:
- L2VNIs are analogous to a VLAN and, for AOS-CX use, the configuration of a VLAN. An L2VNI bridges layer 2 traffic between endpoints attached to different VTEPs.
- L3VNIs are analogous to VRFs and route between the subnets of L2VNIs between VTEPs.
- Multiple L2VNIs can exist within a single VRF.
VXLAN Encapsulation
An overlay network is implemented using Virtual Extensible LAN (VXLAN) tunnels that provide both layer 2 and layer 3 virtualized network services to endpoints connected to the campus. The VXLAN Network Identifier (VNI) associates tunneled traffic with the correct corresponding layer 2 VLAN or layer 3 route table so the receiving VTEP can forward the encapsulated frame appropriately. The Symmetric Integrated Routing and Bridging (IRB) capability allows the overlay networks to support contiguous layer 2 forwarding and layer 3 routing across leaf nodes.
A VTEP encapsulates a frame in the following headers:
IP header: IP addresses in the header can be VTEPs or VXLAN multicast groups in the transport network. Intermediate devices between the source and destination forward VXLAN packets based on this outer IP header.
UDP header for VXLAN: The default VXLAN destination UDP port number is 4789.
VXLAN header: VXLAN information for the encapsulated frame.
- 8-bit VXLAN Flags: The first bit signals if a GBP ID has been set on the packet and the fifth bit signals if the VNI is valid. All other bits are reserved and set to “0”.
- 16-bit VXLAN Group Policy ID: The group ID identifies the policy enforced on tunneled traffic.
- 24-bit VXLAN Network Identifier: Specifies the virtual network identifier (VNI) of the encapsulated frame.
- 24-bit Reserved field
