Link Search Menu Expand Document
Table of contents

Aruba Reference Architecture for Data Center

The Aruba ESP Data Center reference architecture is designed to support high availability compute racks using redundant top-of-rack switches connected in a Layer 3 spine and leaf topology. The spine and leaf topology optimizes performance and also provides a horizontally scalable design that can expand to accommodate a growing data center without disrupting the existing network components. A data center can be initially built with as few as two spine switches. When additional capacity is required up to four spine switches can be deployed in a single fabric. The following figure shows the reference architecture with three spine switches.

**Spine and Leaf: Dual Top of Rack**

Certain application environments do not require high availability at the individual compute host. In this case, a single ToR switch per rack will provide a more cost effective data center network. In this type of implementation, the number of compute hosts deployed per rack should be kept low as a ToR switch under maintenance will impact connectivity to all compute hosts in the rack. The following topology shows a single ToR design with two spine switches.

**Spine and Leaf: Single Top of Rack**

Reference Architecture Components Selection

The following section provides guidance for hardware selection based on your compute host, availability and bandwidth requirements.

Aruba CX 8300 Data Center Switches

The Aruba CX 8300 portfolio offers two models of fixed configuration data center switches. The CX 8325 model offers the highest port density while the new CX 8360 model offers a variety of port configurations for small and medium spine and leaf topologies. Both models offer the following data center switching capabilities:

  • High-speed, fully distributed architecture with line-rate forwarding
  • High availability and in-service ToR upgrades with VSX
  • Cloud-native and fully programmable modern operating system built on a microservices architecture
  • Error free network configuration with software defined orchestration tools
  • Distributed analytics and guided troubleshooting provide full visibility and rapid issue resolution
  • Hot swappable and redundant load-sharing fans and power supplies
  • Power-to-port and port-to-power cooling options for different data center designs
  • Jumbo frame support for 9,198 byte frames
  • Advanced L2 and L3 features to support VXLAN spine and leaf with MP-BGP / EVPN control plane
  • Distributed active gateways for supporting host mobility

Spine Switches

The Aruba ESP Data Center reference architecture is built around two choices of 1RU high density spine switch with QSFP ports capable of 40GbE/100GbE speeds. The 8325 model can support up to 32 compute racks in a single ToR switch topology or up to 16 compute racks in a dual ToR switch topology. The 8360 model can support up to 12 compute racks in a single ToR switch topology or up to 6 compute racks in a dual ToR switch topology.

The primary function of the spine switches is to make routing decisions for the overlay. The primary design considerations when choosing the best switch for your data center spine are:

  • Port density
  • Ports speeds
  • Routing table sizes
SKUDescriptionMaximum Rack Capacity
JL626A8325: 32-port 40GbE/100GbE QSFP+/QSFP28, port-to-power airflow32 Single ToR / 16 Dual ToR
JL627A8325: 32-port 40GbE/100GbE QSFP+/QSFP28, power-to-port airflow32 Single ToR / 16 Dual ToR
JL708A8360: 12-port 40GbE/100GbE QSFP+/QSFP28, port-to-power airflow12 Single ToR / 6 Dual ToR
JL709A8360: 12-port 40GbE/100GbE QSFP+/QSFP28, power-to-port airflow12 Single ToR / 6 Dual ToR

Leaf Switches

There are two leaf switch models to choose from in the Aruba ESP Data Center reference architecture. Both models are 1RU ToR switches that support high density racks using 10GbE copper or SFP+ ports. SFP ports on the 8360 model also support 10GBASE-T transceivers

For redundant ToR designs, the high and medium density SKUs provide the minimum of four uplink ports that are required for a two spine switch topology. For non-redundant ToR design, medium and low density SKUs provide the minimum of two uplink ports required for a two spine switch topology.

You can mix and match racks of different ToR configurations in a common spine and leaf topology in order to optimize your host aggregation needs. The following table summarize the leaf SKUs available with their corresponding supported designs.

SKUDescriptionRack DesignSpine Design
JL624A8325: 48-port 1/10/25GbE SFP/SFP+/SFP28, 8-port 40/100GbE QSFP+/QSFP28, port-to-power airflowHigh Density - Dual ToR2 to 4 switches
JL625A8325: 48-port 1/10/25GbE SFP/SFP+/SFP28, 8-port 40/100GbE QSFP+/QSFP28, power-to-port airflowHigh Density - Dual ToR2 to 4 switches
JL706A8360: 48-port 100M/1GbE/10GbE 10GBASE-T, 4-port 40/100GbE QSFP+/QSFP28, port-to-power airflowHigh Density - Dual ToR2 switches
JL707A8360: 48-port 100M/1GbE/10GbE 10GBASE-T, 4-port 40/100GbE QSFP+/QSFP28, power-to-port airflowHigh Density - Dual ToR2 switches
JL700A8360: 32-port 1/10/25GbE SFP/SFP+/SFP28, 4-port 40/100GbE QSFP+/QSFP28, port-to-power airflowMedium Density - Dual ToR2 switches
JL701A8360: 32-port 1/10/25GbE SFP/SFP+/SFP28, 4-port 40/100GbE QSFP+/QSFP28, power-to-port airflowMedium Density - Dual ToR2 switches
JL710A8360: 24-port 1/10GbE SFP/SFP+, 2-port 40/100GbE QSFP+/QSFP28, port-to-power airflowMedium Density / Single ToR2 switches
JL711A8360: 24-port 1/10GbE SFP/SFP+, 2-port 40/100GbE QSFP+/QSFP28, port-to-power airflowMedium Density / Single ToR2 switches
JL702A8360: 16-port 1/10/25GbE SFP/SFP+/SFP28, 2-port 40/100GbE QSFP+/QSFP28, port-to-power airflowLow Density / Single ToR2 switches
JL703A8360: 16-port 1/10/25GbE SFP/SFP+/SFP28, 2-port 40/100GbE QSFP+/QSFP28, power-to-port airflowMedium Density / Single ToR2 switches

Out-of-Band Management Switches

The Aruba ESP Data Center reference architecture uses a management LAN built on dedicated switching infrastructure to ensure reliable connectivity to data center infrastructure for automation, orchestration, and traditional management access. The following table lists the recommended switch models.

SKUDescriptionHost ports
JL667AAruba CX 6300F 48-port 1GbE and 4-port SFP56 Switch48
JL668AAruba CX 6300F 24-port 1GbE and 4-port SFP56 Switch24
JL663AAruba CX 6300M 48-port 1GbE and 4-port SFP56 Switch48
JL664AAruba CX 6300M 24-port 1GbE and 4-port SFP56 Switch24
JL724AAruba 6200F 24G 4SFP+ Switch24
JL726AAruba 6200F 48G 4SFP+ Switch48
JL678AAruba 6100 24G 4SFP+ Switch24
JL676AAruba 6100 48G 4SFP+ Switch48

Aruba Fabric Composer

Aruba Fabric Composer is offered as a self-contained ISO or virtual machine OVA and can be installed in both virtual and physical host environments as a single instance or as a high-availability, 3-node cluster. Aruba Fabric Composer is available as an annual, per switch software subscription.

Aruba Fabric Composer supports Aruba CX 8325 and 8360 switches.

Ordering information for Aruba Fabric Composer is at the end of the solutions overview.

NetEdit

NetEdit runs as an virtual machine OVA on a host. Aruba NetEdit is available from the Aruba Service Portal. Customers must visit the Aruba Airheads Community and create an Airheads account in order to download the NetEdit software.

Ordering information for Aruba NetEdit is at the end of the data sheet.

Reference Architecture Physical Layer Planning

The following section provides guidance for physical layer planning of your data center switches.

Cables and Transceivers

Please refer to the following documents to make sure that you select supported cables and transceivers as you plan for physical connectivity inside your data center:

HPE Server Networking Transceiver and Cable Compatibility Matrix

ArubaOS-Switch and ArubaOS-CX Transceiver Guide

Port Speed Groups

When planning for ToR configurations that require server connectivity at multiple speeds, it is important to note that setting the speed of a port might require adjacent ports to then operate at that same speed.

The CX 8325 series switch has a default speed of 25GbE. To change the speed to 10GbE will impact groups of 12 ports. The CX 8360 series switch allows individual ports to operate at different speeds without impacting adjacent ports unless MACSec is in use. Ports configured to use MACSec must all be configured to operate at the same speed.

Split Ports

Breakout cables can be used to split a 40 Gb/s or 100 Gb/s port, into four lower speed connections (4x10 Gb/s and 4x25 Gb/s). Please refer to the ArubaOS-Switch and ArubaOS-CX Transceiver Guide for selecting supported breakout cables and switch support for split ports.

Media Access Control Security

Media Access Control security (MACsec) is a standard defined in IEEE 802.1AE which extends standard Ethernet to provide frame level encryption on point-to-point links. This feature is typically used in environments where additional layers of data confidentiality are required or where it is impossible to physically secure the network links between systems. Please refer to the following table for details of MACsec support in the Aruba switching portfolio:

SKUDescriptionSupported Ports
JL700A8360: 32-port 1/10/25GbE SFP/SFP+/SFP28, 4-port 40/100GbE QSFP+/QSFP28, port-to-power airflow1-4 SFP+/SFP28
JL701A8360: 32-port 1/10/25GbE SFP/SFP+/SFP28, 4-port 40/100GbE QSFP+/QSFP28, power-to-port airflow1-4 SFP+/SFP28

Reference Architecture Capacity Planning

The following section provides capacity planning guidance for the Aruba ESP Data Center spine and leaf reference architecture.

Bandwidth Calculations

A spine and leaf network design provides maximum flexibility and throughput for your Aruba ESP Data Center implementation. To achieve the greatest level of performance, a spine and leaf topology can be designed for zero oversubscription of bandwidth. This results in a data center network that will never be congested because the bandwidth available to hosts is equal to the bandwidth between leaf and spine switches.

A significant advantage of a spine and leaf design is that additional capacity can be added as needed by adding additional spine switches and/or increasing the speed of the uplinks between leaf and spine switches. A rack with 40 dual homed servers with 10GbE NICs could theoretically generate a total load of 800G of traffic. For that server density configuration a 1:1 (non-oversubscribed) fabric could be built with four spine switches using 4x100GbE links on each. In practice most spine and leaf topologies are built between 2.4:1 and 6:1 server to fabric oversubscription ratio.

Network and Compute Scaling

The Aruba ESP Data Center reference architecture provides enough capacity for most deployments. With the use of distributed gateways and symmetric IRB forwarding the MAC and ARP tables are localized to directly attached compute nodes and are not impacted by the amount of racks deployed. The amount of IP prefixes will be a function of the total number of nodes, fabric links, and also the number of physical and/or virtualized servers. The border leaf is typically the node with the highest control plane load as it handles both internal and external connections. Route summarization is a good practice to reduce the redistribution of IP prefixes between domains.

The Aruba data center reference architecture was thoroughly tested in an end-to-end solution environment that incorporates best practices deployment recommendations, applications, and load profiles that represent production environments.

Please refer to the product data sheets on Aruba Campus Core and Aggregation Switches for detailed specifications not included in this guide. The following table provides validated multi-dimensional profiles that you can use for capacity planning of your spine and leaf design.

Feature8325 Leaf8360 Leaf8325 Spine8360 Spine
Host Scale - IPv4/ARP30,00050,000N/AN/A
Host Scale - IPv6/ND15,00050,000N/AN/A
Routing - IPv4 Routes10,00016,00072,000100,000
Routing - IPv6 Routes1,0008,00020,000100,000
Routing - OSPF Neighbors4412864
VXLAN - Overlay VRFs (L3 VNI)3232N/AN/A
VXLAN - Host VLANs (L2 VNI)1024512N/AN/A
Active Gateway SVIs1000512N/AN/A

Back to top

© Copyright 2021 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. Aruba Networks and the Aruba logo are registered trademarks of Aruba Networks, Inc. Third-party trademarks mentioned are the property of their respective owners. To view the end-user software agreement, go to Aruba EULA.