Two-Tier Server Access

Configure Two-Tier access switches as VSX pairs for redundant multi-chassis link aggregation (MC-LAG) connections to the core and downstream data center hosts.

Configure Access VSX ISL Interface

To establish a VSX relationship between each pair of access switches, a link aggregation (LAG) interface must be created for assignment as the VSX data plane’s inter-switch link (ISL). Standardizing the ToR model enables configuring the same ports on all access switches for the VSX ISL link at the UI group level.

Step 1 On the left navigation menu, click DC-RSVCORE, then click the data center access switch group name in the Groups column.

Step 2 On the left navigation menu, click Devices.

Step 3 At the upper right of the Switches pane, click Config.

Step 4 In the Interfaces tile, click Ports & Link Aggregations

Step 5 Scroll to the right of the Ports & Link Aggregations table, and click the + (plus sign) in the upper right.

Step 6 On the Add LAG page, assign the following values:

• Name: lag256
• Description: VSX_ISL_LAG
• Port Members: 1/1/49, 1/1/50
• Speed Duplex: <no value> (default)
• VLAN Mode: trunk
• Native VLAN: 1 (default)
• Allowed VLANs: <no value> (default)
• Admin Up: checked
• Aggregation Mode: LACP Active

Step 7 In the Ports & Link Aggregations table’s title row, click ← (left arrow) to return to the main configuration page.

Spanning Tree

MC-LAGs provide loop prevention in a Two-Tier architecture. Spanning-tree (STP) is configured as a backup loop prevention mechanism in case of host cabling errors to ToR switches.

Step 1 In the Bridging tile, click Loop Prevention.

Step 2 In the Loop Prevention window, set the Spanning Tree Region to RSVDC, leave all other settings at their default, then click SAVE.

Enter MultiEdit Configuration

Step 1 At the upper left of the Switches pane, click the MultiEdit enable slider.

Step 2 Select all access switches in the Devices lists, then click EDIT CONFIG.

Configure Access Switch VSX Pairs

The access switches are configured as VSX pairs to support Layer 2 multi-chassis link aggregation to the core layer and downstream data center hosts. A two-port link aggregation is configured and assigned as the VSX data path inter-switch link (ISL). The out-of-band mgmt interface is used for VSX keepalives to maximize the number of ports available for connecting access switches.

Step 1 Enter the initial VSX configuration.

vsx
	system-mac 02:00:00:00:10:01
	inter-switch-link lag 256
	role primary
	keepalive peer 172.16.104.25 source 172.16.104.24 vrf mgmt

Step 2 Mouse-over the field values in the table column headings below, right-click, and set the appropriate values for each switch.

Step 3 Assign a description and maximum MTU value for the VSX ISL physical interfaces.

interface 1/1/49
    description VSX-ISL
    mtu 9198
interface 1/1/50
    description VSX-ISL
    mtu 9198

Configure Access to Core MC-LAGs

Step 1 Create the core-facing MC-LAG interface.

interface lag 255 multi-chassis
    no shutdown
    description DC-CORE
    no routing
    vlan trunk native 1
    vlan trunk allowed all
    lacp mode active

Note: Tag all VLANs on all inter-switch MC-LAGs to support ubiquitous host mobility across all racks in the Two-Tier structure.

Step 2 Assign physical interfaces to the core MC-LAG interface.

interface 1/1/53
    no shutdown
    mtu 9198
    description RSVDC-CORE1-1
    lag 255
interface 1/1/54
    no shutdown
    mtu 9198
    description RSVDC-CORE1-2
    lag 255

Note: The same physical interface on each access switch in the data center should connect to the same upstream core switch. For example, interface 1/1/53 on every ToR access switch can be configured to connect to the primary switch in the VSX core pair. This creates a consistent configuration that is easy to troubleshoot.

Step 3 Remove the following configuration line from interfaces 1/1/53 and 1/1/54: vlan access 1

Configure Access Switch to Host MC-LAGS

Step 1 Configure the host-facing MC-LAG interface.

interface lag 1 multi-chassis
    no shutdown
    description ESXi-01
    no routing
    vlan trunk native 1
    vlan trunk allowed 101-102
    lacp mode active
    spanning-tree root-guard

Note: MC-LAG interfaces to downstream hosts should scope allowed VLANs only to those required for host.

Step 2 Mouse-over the description field, right-click, then modify values appropriately for each switch.

Note: Additional field configuration values, such as allowed VLANs on the trunk, also can be modified, when appropriate.

Step 3 Associate physical interfaces with the MC-LAG. The following configuration assigns an interface MTU and associates the interfaces with the previously created MC-LAG interface.

interface 1/1/1
    no shutdown
    mtu 9198
    lag 1

Note: Standardize the association of LAG index values to physical interfaces across all access switches for efficient configuration of switch interfaces. The example above assigns LAG index 1 to interface 1/1/1 on all selected switches in MultiEdit.

Step 4 Remove the following configuration line from interfaces 1/1/1: vlan access 1

Step 5 Repeat this process for each host facing MC-LAG.

Configure Multicast

Server Access switches do not perform multicast routing or client services, but IGMP snooping observes IGMP requests from hosts to optimize multicast forwarding at Layer 2. IGMP populates Multicast MAC addresses corresponding to IP multicast groups in the MAC table. This conserves bandwidth by ensuring that multicast traffic is forwarded only to interested receivers.

Step 1 Enable IGMP snooping on all VLAN interfaces.

vlan 101
    ip igmp snooping enable
vlan 102
    ip igmp snooping enable

Step 2 At the bottom right of the MultiEdit Configuration window, click SAVE.

Configure Physical Port Speeds

The default port speed on a switch may be different than the supported speed of a connected device. When attached host speeds are not common across racks, MultiEdit can be used to select only the ToR VSX pair of switches to be modified.

Aruba CX 8325 and CX 10000 switches set physical interface speeds in groups. Every non-uplink interface is associated with an interface group. All members of an interface group use the same operational speed. The size of the group depends on the switch model. This sample topology uses a CX 8325, which groups sets of 12 non-uplink physical interfaces to four distinct interface groups.

Step 1 Select switches that require interface group speed settings, then click EDIT CONFIG.

Step 2 Set the interface group port speed to 10Gbps.

system interface-group 1 speed 10g

Note: The command above sets physical ports 1/1/1–1/1/4 on an 8360 and ports 1/1/1–1/1/12 on an 8325 to operate at 10 Gbps.

Step 3 At the lower right of the MultiEdit Configuration window, click SAVE.

Verify Configuration

Step 1 On the left navigation menu, click Tools.

Step 2 On the Tools menu at the top, click the Commands tab.

Step 3 Click the Available Devices dropdown, select all access switches, then click elsewhere on the page.

Step 4 In the Categories list, click All Category. Enter vsx in the commands list filter, click show vsx status, then click Add >.

Step 5 Add the following additional commands to the Selected Commands list.

• show lacp interfaces
• show spanning-tree mst detail
• show ntp status

Step 6 At the lower left of the Commands pane, click RUN.

Step 7 Scroll down to review the CLI command output for each switch. Verify key result data for each command.

• show vsx status
  • ISL channel: In-Sync
  • ISL mgmt channel: operational
  • Config Sync Status: In-Sync
  • Device Role: set to primary and secondary on corresponding switches
  • Other VSX attributes display equal values for both VSX members

• show lacp interfaces
  • Both Actor and Partner have a corresponding interface for each MC-LAG.
  • All Actor interfaces have a State of “ALFNCD”.
  • All Actor interfaces have a Forwarding State of “up” for all host facing MC-LAGs and the upstream core switch facing MC-LAGs.
  • All Partner interfaces have a state of “PLFNCD” or “ALFNCD”.

Note: “(mc)” in the Aggr Name column indicates an MC-LAG. The switch on which the show lacp interfaces command is run is considered the Actor. The other VSX member switch is considered the Partner.

• show spanning-tree mst detail
  • Verify that the Root Address value is the virtual VSX MAC address on the core switches.
  • Verify that the Role for LAG 255 connected to the core switches is “Root” with a State of “Forwarding”.
  • Verify that the Role for all other LAGs and ports with connections is “Designated” with a State of “Forwarding”.

• show ntp status
  • Verify that NTP Server is populated with a configured NTP server IP address
  • Verify that the Time Accuracy field is populated.