Link Search Menu Expand Document
calendar_month 23-Mar-23

Layer 2 Two-Tier Core

Configure Layer 2 two-tier core switches as a VSX pair for Layer 2 aggregation of the data center access switches, IP data center services, and routing to the main campus.

Table of contents

Configure Core VSX ISL Interface

To prepare for establishing a VSX relationship between the core switches, a link aggregation (LAG) interface must be created for assignment as the VSX data plane’s interswitch link (ISL). The LAG can be defined at the Central UI group level when using the same ports for the VSX ISL on both core switches.

Step 1 On the left Aruba Central menu, click the current context, then click the data center core switch group name in the Groups column.

Note: The current context in the screenshot above is Global.

Step 2 On the left navigation menu, click Devices.

Step 3 On the upper right of the Switches pane, click Config.

Step 4 In the Interfaces tile, click Ports & Link Aggregations

Step 5 Scroll to the right of the Ports & Link Aggregations table, and click + (plus sign) at the upper right.

Step 6 On the Add LAG page, enter the following values:

  • Name: lag256
  • Description: VSX_ISL_LAG
  • Port Members: 1/1/31, 1/1/32
  • Speed Duplex: <no value> (default)
  • VLAN Mode: trunk
  • Native VLAN: 1 (default)
  • Allowed VLANs: <no value> (default)
  • Admin Up: checked
  • Aggregation Mode: LACP Active

Step 7 In the Ports & Link Aggregations table’s title row, click ← (left arrow) to return to the main configuration page.

Configure Core Switch VLANs

In this topology, one VLAN is required per switch for connecting to upstream external networks, and an additional VLAN is defined to establish a routed transit link between the core switches.

If one of the core switches loses its external network peering, the routed transit link shares external reachability information with the other core switch.

Step 1 In the Bridging tile, click VLANs.

Step 2 In the upper right of the VLANs table, click + (plus sign).

Step 3 On the Add VLAN page, enter the following field values, then click ADD.

  • ID: 4000

  • Name: CORE-ROUTED-TRANSIT

  • Description: <no value> (default)

  • Admin Up: checked (default)

  • Voice: unchecked (default)

Step 4 Repeat the procedure to create VLANs with the following non-default values.

IDName
4001CORE1-1-TO-EXT-FW
4002CORE1-2-TO-EXT-FW

Step 5 In the VLANs table’s title row, click ← (left arrow) to return to the main configuration page.

Spanning Tree

Multi-chassis link aggregation (MC-LAG) links provide the primary loop prevention mechanism in a Layer 2 Two-Tier architecture. MC-LAGs allow loop-free forwarding on all inter-switch links simultaneously in both directions, when configured on both core and access switches.

MC-LAGs also provide higher performance, hash-based load balancing compared to individual mapping of VLANs to Multiple Spanning-Tree (MST) instances.

Spanning-tree (STP) is configured only as a backup loop prevention mechanism in case of operator cabling errors when connecting hosts to top of rack switches.

Setting the spanning-tree priority to 0 ensures that the core VSX pair of switches is the STP root.

Step 1 In the Bridging tile, click Loop Prevention.

Step 2 In the Loop Prevention window, set the following Spanning Tree values, then click SAVE.

  • Priority: 0
  • Region: RSVDC

Enter MultiEdit Configuration

The Central UI interface for switches provides simplified access to the most commonly required switch features. MultiEdit is a tool in the Central UI for CX switches that provides the ability to edit a switch configuration directly using CLI syntax, with the benefit of syntax checking, colorization, and command completion. This enables configuration of any CX feature displayed in the running configuration of a switch.

For complete details on using MultiEdit, refer to the Editing Configuration on AOS-CX section of the Central online help.

The text configurations in the following steps are intended to be copied and pasted into MultiEdit. To prevent potential copy and paste errors, scroll to the bottom of the configuration, create a new line, then paste the new configuration lines. MultiEdit automatically positions the new lines correctly within the configuration.

Step 1 In the upper left of the Switches pane, click the MultiEdit enable slider.

Step 2 Click both core switches in the Devices lists, then click EDIT CONFIG.

Note: When using the Central MultiEditor, it is beneficial to save small sets of configuration at a time. This reduces the volume of configuration that must be inspected when errors occur and makes troubleshooting configuration elements faster.

Configure Core Switch VSX

The core switches are configured as a VSX pair to support Layer 2 multi-chassis link aggregation (MC-LAG) to the access layer switches. The previously defined LAG is assigned as the VSX data path inter-switch link (ISL). The out-of-band (OOB) mgmt interface is used for VSX keepalives to maximize the number of ports available to connect access switches.

Step 1 Enter the initial VSX configuration.

vsx
	system-mac 02:00:00:00:10:00
	inter-switch-link lag 256
	role primary
	keepalive peer 172.16.104.22 source 172.16.104.21 vrf mgmt

Note: When the mgmt vrf is specified, the keepalive peer addresses are the IPs assigned to the out-of-band management interfaces. When using DHCP IP address assignments on the OOB management network, it is required that DHCP reservations are created for VSX paired switches to avoid future keepalive failures.

Step 2 Mouse-over the role value of primary to display the values for each individual switch, then right-click.

Step 3 In the Modify Parameters window, click primary under RSVDC-CORE1-2, select secondary from the menu, then click SAVE CHANGES.

Note: Hovering the mouse over per-switch values displays the specific value assigned per switch.

Step 4 Modify the VSX keep alive peer and source parameters by right-clicking on the values.

Switchpeersource
RSVDC-CORE1-1172.16.104.22172.16.104.21
RSVDC-CORE1-2172.16.104.21172.16.104.22

Step 5 Assign the maximum MTU value to the VSX ISL physical interfaces.

interface 1/1/31
    mtu 9198
interface 1/1/32
    mtu 9198

Configure Core Switch MC-LAGs

Step 1 Create core-to-access MC-LAG interfaces.

interface lag 1 multi-chassis
    description RACK-1
    no shutdown
    no routing
    vlan trunk native 1
    vlan trunk allowed all
    lacp mode active
    spanning-tree root-guard
interface lag 2 multi-chassis
    description RACK-2
    no shutdown
    no routing
    vlan trunk native 1
    vlan trunk allowed all
    lacp mode active
    spanning-tree root-guard

Note: MC-LAG interfaces can scope trunked VLANs to only those required for a specific downstream rack. Tagging all VLANs on all core-to-access MC-LAGs supports ubiquitous host mobility across all racks within the two-tier structure and reduces the administrative overhead of maintaining VLAN assignments per rack.

Step 2 Assign physical interfaces to MC-LAGs supporting access switches.

interface 1/1/1
    description RSVDC-ACCESS1-1
    no shutdown
    mtu 9198
    lag 1
interface 1/1/2
    description RSVDC-ACCESS1-2
    no shutdown
    mtu 9198
    lag 1
interface 1/1/3
    description RSVDC-ACCESS2-1
    no shutdown
    mtu 9198
    lag 2
interface 1/1/4
    description RSVDC-ACCESS2-2
    no shutdown
    mtu 9198
    lag 2

Step 3 Remove the following configuration line from interfaces 1/1/1–1/1/4: vlan access 1

Note: Interface LAG assignments and VLAN access statements cannot be assigned to an interface simultaneously. An error occurs when saving the MultiEdit configuration, if the vlan access 1 statement is not removed.

Configure Data Center Host VLAN SVIs

Step 1 Configure VLAN switched virtual interfaces (SVIs) for data center host VLANs. Core switches provide the default gateway to downstream data center hosts. An active gateway IP and MAC address are configured for each VLAN to allow both core switches to represent the same IP gateway.

interface vlan101
    description PROD-WEB-SVI
    ip mtu 9198
    ip address 10.1.101.2/24
    active-gateway ip mac 02:00:0a:01:65:01
    active-gateway ip 10.1.101.1
interface vlan102
    description PROD-DB-SVI
    ip mtu 9198
    ip address 10.1.102.2/24
    active-gateway ip mac 02:00:0a:01:66:01
    active-gateway ip 10.1.102.1

Note: The example active gateway MAC address associated with the virtual IP sets the locally administered bit to “1” and embeds a hexadecimal representation of the active gateway IP in the last four octets.

Step 2 Mouse-over the VLAN 101 ip address value of 10.1.101.2, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.1.101.3, and click SAVE CHANGES.

Step 3 Mouse-over the VLAN 102 ip address value of 10.1.102.2, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.1.102.3, and click SAVE CHANGES.

Configure Routed Transit VLAN

The routed transit VLAN allows external network reachability over the VSX ISL if an external link fails on one of the core switches.

VSX combines two separate switches into a single, logical Layer 2 switch. Layer 3 functions remain independent. A routed transit VLAN is created to support IP routing between the core switches. An OSPF adjacency is defined between the switches over the VLAN, which shares loopback IP address reachability used later in BGP configuration. External reachability information is shared over an iBGP peering.

Step 1 Create the OSPF process.

router ospf 1
    router-id 10.18.0.1
    passive-interface default
    area 0.0.0.0

Step 2 Mouse-over the OSPF router ID value, right-click to set per switch values, set the router-id of RSVDC-CORE-1-2 to 10.18.0.2, and click SAVE CHANGES.

Step 3 Create core switch loopback interfaces. The loopback IP should be the same value assigned to the OSPF router-id.

interface loopback 0
	ip address 10.18.0.1/32
	ip ospf 1 area 0.0.0.0

Step 4 Mouse-over the loopback 0 ip address value, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.18.0.2/32, and click SAVE CHANGES.

Step 5 Configure the transit VLAN SVI.

interface vlan4000
    description CORE-ROUTED-TRANSIT-SVI
    ip mtu 9198
    ip address 10.18.0.254/31
    ip ospf 1 area 0.0.0.0
    ip ospf network point-to-point
    no ip ospf passive

Step 6 Mouse-over the transit VLAN ip address value 10.18.0.254, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.18.0.255/31, and click SAVE CHANGES.

Configure External Connectivity

In a Layer 2 two-tier architecture, the core switches typically connect directly to external networks. This example deployment connects to an upstream campus network. It is common to place firewalls between the data center and a campus network for granular policy enforcement. Access control lists (ACLs) also can be used to augment firewall policy or in place of a firewall, when policy complexity does not require a dedicated appliance.

This example deployment uses a typical active/passive redundant firewall pair. The IP assignment on the firewall side is used only by the currently active firewall.

The external BGP (eBGP) peering advertises data center networks and typically accepts a default (0/0) route. The internal BGP (iBGP) session is configured to share the incoming default route between core switches in case one core switch loses its external connectivity.

In the diagram below, an eBGP session is depicted between the core data center switches and RSVDC-FW-1, which is the active member of the firewall cluster pair. When RSVDC-FW1-2 becomes the active member of the cluster, the eBGP sessions are moved to RSVDC-FW1-2.

Define Shared Core Switch Config

Step 1 Configure the border ACL.

access-list ip RSVDC_Border_ACL
    10 permit tcp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0 eq https
    10 comment Allow_All_HTTPS_to_Prod_Web
    20 permit any 10.2.120.0/255.255.255.0 10.1.0.0/255.255.0.0
    20 comment Allow_Supporting_Services
    30 permit any 10.254.1.0/255.255.255.0 10.1.0.0/255.255.0.0
    30 comment Allow_Admins_to_DC_Hosts
    40 permit any 10.254.1.0/255.255.255.0 10.18.0.0/255.255.255.0
    40 comment Allow_Admins_to_DC_loopbacks
    50 permit icmp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0
    50 comment Allow_ICMP_to_Prod_Web
    60 permit udp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0 range 33434 33535
    60 comment Allow_Traceroute_to_Prod_Web
    70 permit tcp 10.0.0.0/255.255.255.0 10.0.0.0/255.255.255.0 eq bgp
    70 comment Allow_BGP_peering

Step 2 Configure the data center core interfaces connected to the campus firewall.

interface 1/1/5
    no shutdown
    mtu 9198
    description EXT-FW-1
    no routing
    vlan access 4001
interface 1/1/6
    no shutdown
    mtu 9198
    description EXT-FW-2
    no routing
    vlan access 4001

Step 3 Mouse-over the vlan access VLAN number, right-click to set per switch values, and set the value to 4002 on RSVDC-CORE1-2 for both firewall facing interfaces.

Step 4 Configure prefix lists to define allowed routes to and from external peers.

ip prefix-list PL_DC-Prefixes-In seq 10 permit 0.0.0.0/0 le 0
ip prefix-list PL_DC-Prefixes-Out seq 10 permit 10.1.0.0/16 le 24

Step 5 Configure route maps for use with BGP to specify accepted and advertised prefixes.

route-map RM_DC-Prefixes-In permit seq 10
    match ip address prefix-list PL_DC-Prefixes-In
route-map RM_DC-Prefixes-In deny seq 20

route-map RM_DC-Prefixes-Out permit seq 10
    match ip address prefix-list PL_DC-Prefixes-Out
route-map RM_DC-Prefixes-Out deny seq 20

Step 6 In the bottom right of the MultiEdit Configuration window, click SAVE.

Configure BGP on Core Switch 1

Step 1 Select RSVDC-CORE1-1, then click EDIT CONFIG.

Step 2 Configure the eBGP VLAN SVI interface on RSVDC-CORE1-1.

interface vlan 4001
    description CORE1-1-FW-SVI
    apply access-list ip RSVDC_Border_ACL routed-in
    ip address 10.0.0.21/31
    ip mtu 9198

Step 3 Configure BGP peering information on RSVDC-CORE1-1.

router bgp 65001
    bgp router-id 10.18.0.1
    bgp fast-external-fallover
    neighbor 10.0.0.20 remote-as 65520
    neighbor 10.18.0.2 remote-as 65001
    neighbor 10.18.0.2 update-source loopback 0
    address-family ipv4 unicast
        neighbor 10.0.0.20 activate
        neighbor 10.0.0.20 route-map RM_DC-Prefixes-In in
        neighbor 10.0.0.20 route-map RM_DC-Prefixes-Out out
        neighbor 10.18.0.2 activate
        neighbor 10.18.0.2 next-hop-self
        redistribute connected

Step 4 In the bottom right of the MultiEdit Configuration window, click SAVE.

Configure BGP on Core Switch 2

Step 1 Select RSVDC-CORE1-2, then click EDIT CONFIG.

Step 2 Configure the eBGP VLAN SVI interface on RSVDC-CORE1-2.

interface vlan4002
    description CORE1-2-FW-SVI
    ip mtu 9198
    ip address 10.0.0.23/31
    apply access-list ip RSVDC_Border_ACL routed-in

Step 3 Configure BGP peerings on RSVDC-CORE1-2.

router bgp 65001
    bgp router-id 10.18.0.2
    bgp fast-external-fallover
    neighbor 10.0.0.22 remote-as 65520
    neighbor 10.18.0.1 remote-as 65001
    neighbor 10.18.0.1 update-source loopback 0
    address-family ipv4 unicast
        neighbor 10.0.0.22 activate
        neighbor 10.0.0.22 route-map RM_DC-Prefixes-In in
        neighbor 10.0.0.22 route-map RM_DC-Prefixes-Out out
        neighbor 10.18.0.1 activate
        neighbor 10.18.0.1 next-hop-self
        redistribute connected

Step 4 In the bottom right of the MultiEdit Configuration window, click SAVE.

Verify Operational State

Step 1 On the left navigation menu, click Tools.

Step 2 On the Tools menu at the top, click the Commands tab.

Step 3 Click the Available Devices dropdown menu, select both data center core switches, then click elsewhere on the page.

Step 4 In the Categories list, click All Category. Enter vsx in the commands list filter, click show vsx status, then click Add >.

Step 5 Add the following additional commands to the Selected Commands list.

  • show bgp ipv4 unicast summary
  • show ip route all-vrfs

Step 6 In the lower left of the Commands pane, click RUN.

Step 7 Scroll down to review the CLI command output for each switch. Verify key results for each command.

  • show vsx status
    • ISL channel: In-Sync
    • ISL mgmt channel: operational
    • Config Sync Status: In-Sync
    • Device Role: set to primary and secondary on corresponding switches
    • Other VSX attributes display equal values for both VSX members

  • show bgp ipv4 unicast summary
    • External BGP peering is in “Established” state to upstream firewall.
    • Internal BGP peering to VSX peer is in “Established” state.

  • show ip route all-vrfs
    • Verify that a default route is learned from the BGP protocol and installed in the route table with the upstream firewall as the next hop.
    • Verify that the loopback address of the peer data center core switch is learned via OSPF and installed in the route table.

Back to top

© Copyright 2022 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Aruba Networks and the Aruba logo are registered trademarks of Aruba Networks, Inc. Third-party trademarks mentioned are the property of their respective owners. To view the end-user software agreement, go to Aruba EULA.