EdgeConnect SD-Branch Branch Design
This section covers the following aspects of branch design:
Table of contents
Zero Touch Provisioning
Zero Touch Provisioning (ZTP) is the preferred method to deploy a branch gateway. ZTP automatically communicates with Central and downloads the branch’s group configuration without requiring user interaction. Group assignments for gateways are made in Central before provisioning, so the group configuration can be applied without the need for admin personnel to move the gateway from the default group or perform configuration functions.
When deploying a new branch site, remember that the gateway(s) must be provisioned and online before APs and switches at the branch can complete the ZTP process and receive their group configuration. The APs and switches must receive IP addressing and domain name server (DNS) information from their respective gateways before they can communicate with Central. In order for a gateway to perform ZTP successfully, one or more ZTP ports must be connected to a WAN service that provides the following:
- DHCP addressing
- DHCP options 3 (Router) and 5 (Name Server)
- Internet access.
Physical ports should be consistent across BGWs to ensure consistency between branches and reduce the number of groups required. The objective is to select a common set of ports that work for as many branch configurations as possible.
WAN uplinks can be connected to any switchport on a BGW except for Gigabit Ethernet 0/0/1 which is reserved for One Touch Provisioning (OTP).
If the gateway is connected to multiple WAN services, it will attempt to perform ZTP over each service until it is successful. The first WAN service to respond to the DHCP’s discover message is selected first. The figure below illustrates the interfaces reserved for OTP for some of the different form factors.
The video below explains the topics covered in the Zero Touch provisioning section.
Sites, Clustering and System IPs
Branch gateways are generally deployed as a single device or as a redundant pair. Both deployment methods are placed automatically in a cluster based on the site. The cluster can contain one or two gateways. Automatic clustering works only for a maximum of two branch gateways at site. For both redundant and non-redundant sites, if gateways are not placed in a cluster, the access points are not able to tunnel to the gateway. It is always important to place both devices in a site, cluster automatically.
Caution: Do not use auto group clustering, this will require each site to have a unique group. This will make managing a standard configuration very cumbersome.
Non-redundant sites and redundant sites vary slightly in the manner the system IP address is used.
- In a non-redundant site, the gateway uses the system IP address as the source for functions such as Radius, SNMP, Tunneling, etc. See EdgeConnect SD-Branch Overview for more details.
- In a redundant deployment, a VRRP Virtual IP (VIP) is required and serves as the source IP address for Radius and TACACS functions. The gateway proxies all COA requests sourced from the VIP. The system IP address is still used for functions such as tunneling and cluster formation.
“Split brain” situations can occur if LAN connectivity is interrupted and the gateways are unable to communicate with one another. Both gateways assume primary functions and use VIP. After LAN connectivity is restored, the secondary cluster member no longer assumes the primary VIP role and cedes the primary role after 280s. Gateways should have layer 2 connectivity, either directly or through an intermediary switch. Gateways also should use LACP in both cases for fault tolerance.
The video below explains the topics covered in the Sites, Clustering and System IPs section.
WAN Redundancy
It is very common for branch sites to have limited WAN drops on-site in deployments with more than one branch gateway and limited WAN transports.
Administrators can take advantage of uplink sharing. This enables the branch gateways to sync DHCP state, and share uplinks between each device over the LAN. Each branch gateway can share selective uplinks, no uplinks, or all available uplinks. When sharing uplinks, it is critical to use a unique VLAN for each share uplink. For example, the INET uplink should be VLAN 4085 and the MPLS uplink should be VLAN 4086. The VLANs can be mutually shared, as illustrated below.
Administrators Identify the uplink interfaces to share based on the VLAN ID. If administrators want to take advantage of DHCP state sync between gateways without sharing uplinks, use the same VLAN ID for WAN uplinks on both gateways. In the illustration below, the VLAN 4085 is in use for both BGW1 and BGW2. That uplink is also connected. In this example, the WAN uplink is not shared.
Branch Security
Branch gateways can serve as a site’s first line of defense against external and internal threats when implemented with the following features:
- Aruba Role-Based Stateful Firewall - The firewall supports scalable configuration using firewall aliases, ALGs, and role-based policies.
- Deep Packet Inspection - Qosmos’s application engine and signatures provide the capability to identify nearly 3500 applications.
- Web Content, Reputation and Geo-location Filtering - WebRoot’s machine learning technology classifies content, reputation, and geolocation for billions of URLs.
- Aruba Threat Defense (IPS/IDS) - Powered by ProofPoint’s Threat Intelligence, Aruba 9000 Series gateways can perform IDS/IPS functions for all branch traffic. See the IPDS Guide for more detail.
Before employing the security features available for the Aruba branch gateway, it is especially important to identify critical business assets, such as groups of users, point-of-sale systems, and applications (cloud based or on-premise). Assets must be clearly identified before comprehensive protection requirements and security protocols can be defined.
On a branch gateway, assets are identified clearly by their destinations, services, and individual roles:
- Network Aliases are used to identify and differentiate a specific host, a specific network, or a combination.
- Service Aliases are used to identify and differentiate services, such as DHCP, FTP, SIP, etc.
- Applications are identified clearly so they can be targeted individually or as a group for implementation of specialized security operations. For example, Deep Packet Inspection can be performed on a list of more than 3000 common applications, such as YouTube, Facebook, and Twitter.
- Roles are used to identify and categorize end user devices. Access control lists and policies are applied using assigned roles.
After assets are clearly identified, the organization must map which assets communicate with one another. For example, employees likely need to communicate with Office 365 or Google Workspace, to complete daily tasks. Create flows to outline the connections between assets.
The next step is developing and enforcing policies based on the assets and their communication flows. The branch Gateway can use four methods to enforce policy: Geolocation Filtering, Web Reputation Rating, Intrusion Prevention Systems, and Access Control Lists (ACLs).
To enforce policy using Geolocation Filtering, Web Reputation, and Intrusion Prevention Systems, the gateway must have the Firewall Visibility, Deep Packet Inspection and Web Content Classification (Webcc) functions enabled. Also note that Geolocation Filtering has an implicit default “allow all” by default, so additional configuration is required to block unwanted countries.
Web Reputation settings should be established to block malicious URLs with both inbound and outbound policies. If a legitimate website is accidentally blocked by the settings, it can be added individually to the list of allowed URLs. Logs are available for administrators to review the blocked URLs based on Web Reputation and identify URLs to unblock when required.
Depending on the direction of the traffic different enforcement rules take precedence. ACLs can be applied specifically to interfaces and roles, each with their own enforcement precedence.
In the image below, Role A has a VLAN assignment ACL and routing policy applied. The route policy within the role takes precedence over the route policy applied to the VLAN. The next policy hit is the IPS/IDS which inspects traffic to verify it is not malicious. Depending on the organization’s policy requirements, the gateway’s IPS or IDS can be enabled. The IPS (prevention) blocks malicious traffic and the IDS (detection) logs instances of malicious traffic. If traffic is not determined malicious, it continues to the Geo Filtering/Web Reputation system. Finally, the packet is NATed or PATed through the firewall where it hits the WAN ACL.
When traffic is coming into the Gateway from its WAN interface, the first policy hit is the WAN ACL. From there, incoming traffic is routed through the Geo Filtering/Web Reputation system. Traffic then hits the NAT (1:1), then moves to IPS/IDS for inspection. The traffic then hits a global routing policy which is slightly different from the example above. Finally, the traffic hits the user role for its final ACL and policy checks.
When traffic is received from a tunneled endpoint this could be a VPNC, or another branch site. The enforcement points for traffic is slightly different. Flowing directly to the IPS/IDS, then to the global routing policy and finally to the user role.
Within a gateway, each user role is tracked as an instance in the stateful firewall. Any traffic going between roles within the same gateway is inspected on ingress even when on the same gateway. Additional security for untrusted ports is described in detail in the “Trusted Ports” section.
In the example below, the role initiating traffic (Role A) passes through the stateful firewall. ACLs are enforced for the role and IPS/IDS inspection is performed. The traffic is routed to the destination role (Role B), where it must pass through another stateful firewall, with ACL enforcement for Role B.
Trusted Ports
Each port on a gateway can be configured as Trusted or Untrusted. The trust parameter determines how the gateway processes incoming user traffic. When a port or VLAN is configured as untrusted, the gateway tracks user sessions for each IPv4 address. User devices are assigned one of the predefined roles that determines the session ACLs applied to incoming and outgoing user traffic. The trust configuration for each port and VLAN depends on the role of the gateway and what is connected to that port. Aruba recommends the following trust configuration for VPNC and BGW ports:
- VPNCs – Configure all ports and VLANs as trusted.
- BGWs – Configure all WAN ports and VLANs as trusted.
- BGWs – Configure all LAN ports as untrusted.
There is no reason to track user sessions on VPNCs, so all ports and VLANs are configured as trusted. This also applies to BGW and VPNC ports that are directly connected to WAN services. If a VPNC or BGW is directly connected to a public WAN service, Aruba recommends configuring and assigning a restrictive session ACL to the port and configuring all BGW LAN ports as untrusted. This results in BGW tracking of all user sessions for internal IPv4 addresses. Each LAN/VLAN requires its own AAA profile. Each AAA profile can trigger MAC, 802.1X, or captive portal authentication and determine an initial role assignment. Device and user authentication is completely optional.
Aruba SD-LAN
After the SD-WAN is deployed, LAN side, also called the SD-LAN, the branch gateway must be designed. Generally, the SD-LAN design aligns with the campus Two-Tier and Three-Tier designs, except that applications are not hosted on-site.
This section covers the same architectures with a focus LAN interaction with the SD-WAN side and how deployment is affected at a branch site.
L2 and L3 Boundaries
When deploying a branch site, the LAN can go two directions, depending on the size of the branch. Organizations can use layer 2 or layer 3 from the branch gateways down, or a combination of both. In either case Centralized overlays can be leveraged or traditional segmentation methodologies.
Two/Three Tier Layer 2
- Allows for simple deployment of sites
- Increases broadcast domains
- Requires more configuration on BGW (the BGW would be the default gateway for all VLANs)
- Involves spanning tree complexity
Three Tier Layer 3 (routed to the BGW Layer 2 to access)
- Increases scale
- Reduces broadcast domain
- Minimizes spanning tree complexity
- Requires more switch-level configuration (the switch would be the default gateway for all VLANs)
- Requires more complex deployment of sites (point-to-point routes)
Each deployment type has benefits and drawbacks. Generally, for a smaller branch site with Two-Tier architecture, layer 2 facilitates bringing up a site quickly with minimal scale issues.
Organizations should consider using layer 3 if they have a Three-Tier architecture. Layer 3 increases the deployment complexity; however, it reduces the spanning tree complexity that can arise in a larger L2 domain.
Layer 2 Wired Access
In this design, the BGW provides layer 2 services for the site. The switches use VLANs for segmentation, enabling identical configuration of access switches to reduce design complexity.
Design Considerations
Cluster gateways using (Site Cluster).
Enable Default-GW mode.
System IP is required for cluster formation, AP and switch tunneling.
Both gateways must be able to reach each other through all configured VLANs for VRRP communication .
Radius will be sourced from the VRRP, Virtual IP address.
Use Default gateway mode to prevent asymmetrical routing. Traffic will be pinned to VRRP preferred member, will failover to the backup member.
The BGW can be the DHCP server or point to a centralized DHCP server.
Easy ZTP for switching infrastructure
Connected VLANs should be redistributed into the SD-WAN overlay.
Layer 2 Centralized Overlay
The following design uses user-based tunneling (UBT) also known as a centralized fabric. Access switches, and AP’s tunnel clients to the BGW to apply user roles and for layer 3 termination. Administrators can take advantage of the user roles applied to clients to enforce role to role policy and designate WAN policies using Deep Packet Inspection (DPI). For example, guest traffic can be completely segmented and broken out directly to the Internet egress or to a cloud security provider.
Layer 2 Two-Tier Design Considerations
- Cluster gateways using (Site Cluster).
- Enable Default-GW mode.
- System IP is required for cluster formation, AP and switch tunneling.
- Radius is sourced from the VRRP, Virtual IP address.
- Use Default gateway mode to prevent asymmetrical routing. Tunnels go to the VRRP preferred member, tunnels will failover to the backup member.
- Policy is enforced at the BGW
- Gateways should run LACP trunks to the L2 aggregation block and trunk all tunneled VLANs. Gateways must have L2 adjacency for VRRP and Cluster formation.
- VLANs between the aggregation and access must have different VLAN IDs and subnets from the tunneled VLANs. (These VLANs can NOT be tunneled)
- All devices must be on the same management VLAN, do not tunnel this VLAN.
- Run LACP trunks between the Aggregation and BGWs.
- MTU/Jumbo frames MUST be enabled on all VLAN’s
Layer 2 Single-Tier Design Considerations
- Cluster gateways using (Site Cluster).
- Enable Default-GW mode.
- System IP is required for cluster formation, AP and switch tunneling.
- Radius is sourced from the VRRP, Virtual IP address.
- Use Default gateway mode to prevent asymmetrical routing. Tunnels go to the VRRP preferred member, tunnels will failover to the backup member.
- Policy is enforced at the BGW
- Gateways should run LACP trunks to the L2 Access block and trunk all tunneled VLANs.
- Gateways must have L2 adjacency for VRRP and Cluster formation. VLANs between the gateway and access switch must have the tunneled VLAN’s. Alternatively a dedicated link between the gateways can be used.
- All devices must be on the same management VLAN, do not tunnel this VLAN.
- MTU/Jumbo frames MUST be enabled on all VLAN’s
Layer 3 Wired Access
In this design, the aggregation switches provide layer 3 services for the site. The layer 2 access switches use multiple VLANs trunked to the aggregation switches to map the VLANs between them. The aggregation switches act as the IP default gateway for each IP subnet and provide DHCP services to the end devices. DHCP also can be centralized at the headend location or in the data center. The layer 2 access switch obtains its IP address using a DHCP client on the management VLAN. The aggregation switches are routed to the BGWs using layer 3 ports.
Design Considerations
Clustering is not needed in this design.
Gateways should act only as edge routers/firewalls.
OSPF should be used to redistribute routes into the SD-WAN overlay.
Centralized Overlay
The following design uses user-based tunneling (UBT) also known as a centralized fabric. Access switches, and AP’s tunnel clients to the BGW to apply user roles and for layer 3 termination. Administrators can take advantage of the user roles applied to clients to enforce role to role policy and designate WAN policies using Deep Packet Inspection (DPI). For example, guest traffic can be completely segmented and broken out directly to the Internet egress or to a cloud security provider.
Design Considerations
- Cluster gateways using (Site Cluster).
- Enable Default-GW mode.
- System IP is required for cluster formation, AP and switch tunneling.
- Radius is sourced from the VRRP, Virtual IP address.
- Use Default gateway mode to prevent asymmetrical routing. Tunnels go to the VRRP preferred member, tunnels will failover to the backup member.
- Policy is enforced at the BGW
- Gateways should run LACP trunks to the L3 aggregation block and trunk all tunneled VLANs. Gateways must have L2 adjacency for VRRP and Cluster formation.
- VLANs between the aggregation and access should have different IDs and subnets from the tunneled VLANs. (These VLANs can NOT be tunneled)
- OSPF should run between the LACP trunks to Aggregation and BGWs to learn about non-tunneled subnets.
- OSPF should NOT advertise the tunneled VLANs to the aggregation switch.
- MTU/Jumbo frames MUST be enabled on all VLAN’s and routed links
Note: User-based tunneling can be enabled in a layer 2 or layer 3 LAN deployment. This example uses only layer 3 architecture.
Centralized Multi-Site Fabric
The design below illustrates two branch deployments, both branches implmented user roles. Both branches have enabled role propagation to enforce policy across the WAN. Role propagation is done by mapping the user role to VXLAN Group-Based Policy (GBP) tag then encapsulating the tag in IPSEC. The fabric only uses the VXLAN GBP header and does not transport layer 2 payload.
For successful role propagation, two factors must be considered: the implementation of user roles and the gateway’s awareness of user roles. There are several methods to apply user roles, including static user role to VLAN mapping or 802.1x. Additionally, it is essential for the gateway to accurately recognize user roles. To achieve consistency across all locations utilize the Global Policy Manager.
The method of applying user roles can determine whether Microsegmentation or VLAN segmentation is enabled across various sites. For instance, a static mapping of VLANs to user roles would appear as follows: The user role “MGMT” is mapped to VLAN 100. Consequently, whenever the gateway receives traffic on VLAN 100, it associates it with the “MGMT” user role. By creating a WAN policy, communication between MGMT user roles across the WAN can be permitted, instead of specifying particular subnets. Similarly, using 802.1x requires the gateway to have knowledge of the VLANs associated with each user role.
Another example is the utilization of Microsegmentation with User-Based Tunneling (UBT). In this scenario, each user authenticates using 802.1x and receives a distinct user role. For instance, User-Role-A and User-Role-B. Instead of relying on VLAN mapping, the gateway obtains awareness of each user role through the tunnel. This concept extends to tunneled SSIDs as well. As each user role possesses a distinct name, each user role can now be assigned a different Group-Based Policy (GBP) tags while still being part of the same VLAN. This enables Microsegmentation at the local branch, with the user role enforcing policies. Once the traffic traverses the WAN, the GBP tag ensures policy enforcement at the destination site.
Design Considerations
- Gateway Must be aware of the user roles of devices.
- Intra-branch policy is enforced at the BGW
- Branch to Branch policy is enforced at the destination branch.
- Use Global policy manager and NetConductor to define roles, this is also where the GBP-ID is set.
- VXLAN-GBP packets are fragmented and reassembled at each gateway. (No MTU changes required)
- Policy is enforced within enabled groups and across enabled groups.
Note: This example shows role propagation with user based tunneling, however UBT is not required to allow for role propagation.