Configure Layer 2 two-tier core switches as a VSX pair for Layer 2 aggregation of the data center access switches, IP data center services, and routing to the main campus.
Table of contents
- Layer 2 Two-Tier Core
- Configure Core VSX ISL Interface
- Configure Core Switch VLANs
- Spanning Tree
- Enter MultiEdit Configuration
- Configure Core Switch VSX
- Configure Core Switch MC-LAGs
- Configure Data Center Host VLAN SVIs
- Configure Routed Transit VLAN
- Configure External Connectivity
- Verify Operational State
To prepare for establishing a VSX relationship between the core switches, a link aggregation (LAG) interface must be created for assignment as the VSX data plane’s interswitch link (ISL). The LAG can be defined at the Central UI group level when using the same ports for the VSX ISL on both core switches.
Step 1 On the left Aruba Central menu, click the current context, then click the data center core switch group name in the Groups column.
Note: The current context in the screenshot above is Global.
Step 2 On the left navigation menu, click Devices.
Step 3 On the upper right of the Switches pane, click Config.
Step 4 In the Interfaces tile, click Ports & Link Aggregations
Step 5 Scroll to the right of the Ports & Link Aggregations table, and click + (plus sign) at the upper right.
Step 6 On the Add LAG page, enter the following values:
- Name: lag256
- Description: VSX_ISL_LAG
- Port Members: 1/1/31, 1/1/32
- Speed Duplex: <no value> (default)
- VLAN Mode: trunk
- Native VLAN: 1 (default)
- Allowed VLANs: <no value> (default)
- Admin Up: checked
- Aggregation Mode: LACP Active
Step 7 In the Ports & Link Aggregations table’s title row, click ← (left arrow) to return to the main configuration page.
In this topology, one VLAN is required per switch for connecting to upstream external networks, and an additional VLAN is defined to establish a routed transit link between the core switches.
If one of the core switches loses its external network peering, the routed transit link shares external reachability information with the other core switch.
Step 1 In the Bridging tile, click VLANs.
Step 2 In the upper right of the VLANs table, click + (plus sign).
Step 3 On the Add VLAN page, enter the following field values, then click ADD.
Description: <no value> (default)
Admin Up: checked (default)
Voice: unchecked (default)
Step 4 Repeat the procedure to create VLANs with the following non-default values.
Step 5 In the VLANs table’s title row, click ← (left arrow) to return to the main configuration page.
Multi-chassis link aggregation (MC-LAG) links provide the primary loop prevention mechanism in a Layer 2 Two-Tier architecture. MC-LAGs allow loop-free forwarding on all inter-switch links simultaneously in both directions, when configured on both core and access switches.
MC-LAGs also provide higher performance, hash-based load balancing compared to individual mapping of VLANs to Multiple Spanning-Tree (MST) instances.
Spanning-tree (STP) is configured only as a backup loop prevention mechanism in case of operator cabling errors when connecting hosts to top of rack switches.
Setting the spanning-tree priority to 0 ensures that the core VSX pair of switches is the STP root.
Step 1 In the Bridging tile, click Loop Prevention.
Step 2 In the Loop Prevention window, set the following Spanning Tree values, then click SAVE.
- Priority: 0
- Region: RSVDC
The Central UI interface for switches provides simplified access to the most commonly required switch features. MultiEdit is a tool in the Central UI for CX switches that provides the ability to edit a switch configuration directly using CLI syntax, with the benefit of syntax checking, colorization, and command completion. This enables configuration of any CX feature displayed in the running configuration of a switch.
For complete details on using MultiEdit, refer to the Editing Configuration on AOS-CX section of the Central online help.
The text configurations in the following steps are intended to be copied and pasted into MultiEdit. To prevent potential copy and paste errors, scroll to the bottom of the configuration, create a new line, then paste the new configuration lines. MultiEdit automatically positions the new lines correctly within the configuration.
Step 1 In the upper left of the Switches pane, click the MultiEdit enable slider.
Step 2 Click both core switches in the Devices lists, then click EDIT CONFIG.
Note: When using the Central MultiEditor, it is beneficial to save small sets of configuration at a time. This reduces the volume of configuration that must be inspected when errors occur and makes troubleshooting configuration elements faster.
The core switches are configured as a VSX pair to support Layer 2 multi-chassis link aggregation (MC-LAG) to the access layer switches. The previously defined LAG is assigned as the VSX data path inter-switch link (ISL). The out-of-band (OOB) mgmt interface is used for VSX keepalives to maximize the number of ports available to connect access switches.
Step 1 Enter the initial VSX configuration.
vsx system-mac 02:00:00:00:10:00 inter-switch-link lag 256 role primary keepalive peer 172.16.104.22 source 172.16.104.21 vrf mgmt
Note: When the mgmt vrf is specified, the keepalive peer addresses are the IPs assigned to the out-of-band management interfaces. When using DHCP IP address assignments on the OOB management network, it is required that DHCP reservations are created for VSX paired switches to avoid future keepalive failures.
Step 2 Mouse-over the role value of primary to display the values for each individual switch, then right-click.
Step 3 In the Modify Parameters window, click primary under RSVDC-CORE1-2, select secondary from the menu, then click SAVE CHANGES.
Note: Hovering the mouse over per-switch values displays the specific value assigned per switch.
Step 4 Modify the VSX keep alive peer and source parameters by right-clicking on the values.
Step 5 Assign the maximum MTU value to the VSX ISL physical interfaces.
interface 1/1/31 mtu 9198 interface 1/1/32 mtu 9198
Step 1 Create core-to-access MC-LAG interfaces.
interface lag 1 multi-chassis description RACK-1 no shutdown no routing vlan trunk native 1 vlan trunk allowed all lacp mode active spanning-tree root-guard interface lag 2 multi-chassis description RACK-2 no shutdown no routing vlan trunk native 1 vlan trunk allowed all lacp mode active spanning-tree root-guard
Note: MC-LAG interfaces can scope trunked VLANs to only those required for a specific downstream rack. Tagging all VLANs on all core-to-access MC-LAGs supports ubiquitous host mobility across all racks within the two-tier structure and reduces the administrative overhead of maintaining VLAN assignments per rack.
Step 2 Assign physical interfaces to MC-LAGs supporting access switches.
interface 1/1/1 description RSVDC-ACCESS1-1 no shutdown mtu 9198 lag 1 interface 1/1/2 description RSVDC-ACCESS1-2 no shutdown mtu 9198 lag 1 interface 1/1/3 description RSVDC-ACCESS2-1 no shutdown mtu 9198 lag 2 interface 1/1/4 description RSVDC-ACCESS2-2 no shutdown mtu 9198 lag 2
Step 3 Remove the following configuration line from interfaces 1/1/1–1/1/4: vlan access 1
Note: Interface LAG assignments and VLAN access statements cannot be assigned to an interface simultaneously. An error occurs when saving the MultiEdit configuration, if the vlan access 1 statement is not removed.
Step 1 Configure VLAN switched virtual interfaces (SVIs) for data center host VLANs. Core switches provide the default gateway to downstream data center hosts. An active gateway IP and MAC address are configured for each VLAN to allow both core switches to represent the same IP gateway.
interface vlan101 description PROD-WEB-SVI ip mtu 9198 ip address 10.1.101.2/24 active-gateway ip mac 02:00:0a:01:65:01 active-gateway ip 10.1.101.1 interface vlan102 description PROD-DB-SVI ip mtu 9198 ip address 10.1.102.2/24 active-gateway ip mac 02:00:0a:01:66:01 active-gateway ip 10.1.102.1
Note: The example active gateway MAC address associated with the virtual IP sets the locally administered bit to “1” and embeds a hexadecimal representation of the active gateway IP in the last four octets.
Step 2 Mouse-over the VLAN 101 ip address value of 10.1.101.2, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.1.101.3, and click SAVE CHANGES.
Step 3 Mouse-over the VLAN 102 ip address value of 10.1.102.2, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.1.102.3, and click SAVE CHANGES.
The routed transit VLAN allows external network reachability over the VSX ISL if an external link fails on one of the core switches.
VSX combines two separate switches into a single, logical Layer 2 switch. Layer 3 functions remain independent. A routed transit VLAN is created to support IP routing between the core switches. An OSPF adjacency is defined between the switches over the VLAN, which shares loopback IP address reachability used later in BGP configuration. External reachability information is shared over an iBGP peering.
Step 1 Create the OSPF process.
router ospf 1 router-id 10.18.0.1 passive-interface default area 0.0.0.0
Step 2 Mouse-over the OSPF router ID value, right-click to set per switch values, set the router-id of RSVDC-CORE-1-2 to 10.18.0.2, and click SAVE CHANGES.
Step 3 Create core switch loopback interfaces. The loopback IP should be the same value assigned to the OSPF router-id.
interface loopback 0 ip address 10.18.0.1/32 ip ospf 1 area 0.0.0.0
Step 4 Mouse-over the loopback 0 ip address value, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.18.0.2/32, and click SAVE CHANGES.
Step 5 Configure the transit VLAN SVI.
interface vlan4000 description CORE-ROUTED-TRANSIT-SVI ip mtu 9198 ip address 10.18.0.254/31 ip ospf 1 area 0.0.0.0 ip ospf network point-to-point no ip ospf passive
Step 6 Mouse-over the transit VLAN ip address value 10.18.0.254, right-click to set per switch values, set the ip address of RSVDC-CORE1-2 to 10.18.0.255/31, and click SAVE CHANGES.
In a Layer 2 two-tier architecture, the core switches typically connect directly to external networks. This example deployment connects to an upstream campus network. It is common to place firewalls between the data center and a campus network for granular policy enforcement. Access control lists (ACLs) also can be used to augment firewall policy or in place of a firewall, when policy complexity does not require a dedicated appliance.
This example deployment uses a typical active/passive redundant firewall pair. The IP assignment on the firewall side is used only by the currently active firewall.
The external BGP (eBGP) peering advertises data center networks and typically accepts a default (0/0) route. The internal BGP (iBGP) session is configured to share the incoming default route between core switches in case one core switch loses its external connectivity.
In the diagram below, an eBGP session is depicted between the core data center switches and RSVDC-FW-1, which is the active member of the firewall cluster pair. When RSVDC-FW1-2 becomes the active member of the cluster, the eBGP sessions are moved to RSVDC-FW1-2.
Step 1 Configure the border ACL.
access-list ip RSVDC_Border_ACL 10 permit tcp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0 eq https 10 comment Allow_All_HTTPS_to_Prod_Web 20 permit any 10.2.120.0/255.255.255.0 10.1.0.0/255.255.0.0 20 comment Allow_Supporting_Services 30 permit any 10.254.1.0/255.255.255.0 10.1.0.0/255.255.0.0 30 comment Allow_Admins_to_DC_Hosts 40 permit any 10.254.1.0/255.255.255.0 10.18.0.0/255.255.255.0 40 comment Allow_Admins_to_DC_loopbacks 50 permit icmp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0 50 comment Allow_ICMP_to_Prod_Web 60 permit udp 0.0.0.0/0.0.0.0 10.1.101.0/255.255.255.0 range 33434 33535 60 comment Allow_Traceroute_to_Prod_Web 70 permit tcp 10.0.0.0/255.255.255.0 10.0.0.0/255.255.255.0 eq bgp 70 comment Allow_BGP_peering
Step 2 Configure the data center core interfaces connected to the campus firewall.
interface 1/1/5 no shutdown mtu 9198 description EXT-FW-1 no routing vlan access 4001 interface 1/1/6 no shutdown mtu 9198 description EXT-FW-2 no routing vlan access 4001
Step 3 Mouse-over the vlan access VLAN number, right-click to set per switch values, and set the value to 4002 on RSVDC-CORE1-2 for both firewall facing interfaces.
Step 4 Configure prefix lists to define allowed routes to and from external peers.
ip prefix-list PL_DC-Prefixes-In seq 10 permit 0.0.0.0/0 le 0 ip prefix-list PL_DC-Prefixes-Out seq 10 permit 10.1.0.0/16 le 24
Step 5 Configure route maps for use with BGP to specify accepted and advertised prefixes.
route-map RM_DC-Prefixes-In permit seq 10 match ip address prefix-list PL_DC-Prefixes-In route-map RM_DC-Prefixes-In deny seq 20 route-map RM_DC-Prefixes-Out permit seq 10 match ip address prefix-list PL_DC-Prefixes-Out route-map RM_DC-Prefixes-Out deny seq 20
Step 6 In the bottom right of the MultiEdit Configuration window, click SAVE.
Step 1 Select RSVDC-CORE1-1, then click EDIT CONFIG.
Step 2 Configure the eBGP VLAN SVI interface on RSVDC-CORE1-1.
interface vlan 4001 description CORE1-1-FW-SVI apply access-list ip RSVDC_Border_ACL routed-in ip address 10.0.0.21/31 ip mtu 9198
Step 3 Configure BGP peering information on RSVDC-CORE1-1.
router bgp 65001 bgp router-id 10.18.0.1 bgp fast-external-fallover neighbor 10.0.0.20 remote-as 65520 neighbor 10.18.0.2 remote-as 65001 neighbor 10.18.0.2 update-source loopback 0 address-family ipv4 unicast neighbor 10.0.0.20 activate neighbor 10.0.0.20 route-map RM_DC-Prefixes-In in neighbor 10.0.0.20 route-map RM_DC-Prefixes-Out out neighbor 10.18.0.2 activate neighbor 10.18.0.2 next-hop-self redistribute connected
Step 4 In the bottom right of the MultiEdit Configuration window, click SAVE.
Step 1 Select RSVDC-CORE1-2, then click EDIT CONFIG.
Step 2 Configure the eBGP VLAN SVI interface on RSVDC-CORE1-2.
interface vlan4002 description CORE1-2-FW-SVI ip mtu 9198 ip address 10.0.0.23/31 apply access-list ip RSVDC_Border_ACL routed-in
Step 3 Configure BGP peerings on RSVDC-CORE1-2.
router bgp 65001 bgp router-id 10.18.0.2 bgp fast-external-fallover neighbor 10.0.0.22 remote-as 65520 neighbor 10.18.0.1 remote-as 65001 neighbor 10.18.0.1 update-source loopback 0 address-family ipv4 unicast neighbor 10.0.0.22 activate neighbor 10.0.0.22 route-map RM_DC-Prefixes-In in neighbor 10.0.0.22 route-map RM_DC-Prefixes-Out out neighbor 10.18.0.1 activate neighbor 10.18.0.1 next-hop-self redistribute connected
Step 4 In the bottom right of the MultiEdit Configuration window, click SAVE.
Step 1 On the left navigation menu, click Tools.
Step 2 On the Tools menu at the top, click the Commands tab.
Step 3 Click the Available Devices dropdown menu, select both data center core switches, then click elsewhere on the page.
Step 4 In the Categories list, click All Category. Enter vsx in the commands list filter, click show vsx status, then click Add >.
Step 5 Add the following additional commands to the Selected Commands list.
- show bgp ipv4 unicast summary
- show ip route all-vrfs
Step 6 In the lower left of the Commands pane, click RUN.
Step 7 Scroll down to review the CLI command output for each switch. Verify key results for each command.
- show vsx status
- ISL channel: In-Sync
- ISL mgmt channel: operational
- Config Sync Status: In-Sync
- Device Role: set to primary and secondary on corresponding switches
- Other VSX attributes display equal values for both VSX members
- show bgp ipv4 unicast summary
- External BGP peering is in “Established” state to upstream firewall.
- Internal BGP peering to VSX peer is in “Established” state.
- show ip route all-vrfs
- Verify that a default route is learned from the BGP protocol and installed in the route table with the upstream firewall as the next hop.
- Verify that the loopback address of the peer data center core switch is learned via OSPF and installed in the route table.