Orchestrated Deployment Workflows

This section describes the configurations needed to do the orchestrated integration.

Orchestration between Aruba Central and Zscaler Internet Access

For the orchestration to take place, the first step is to enable communication between Aruba Central and ZIA through APIs. That will enable Central to define locations in Zscaler, query for the best Service Edges, etc.

Establishing API communication between Aruba Central and Zscaler

Configuring Zscaler for API access

To add a partner key for Aruba SD-Branch, complete the following steps:

Step 1 Log in to the Zscaler admin portal.

Step 2 Click Administration > Partner Integrations > SD-WAN in the Partner Integrations page in the ZIA portal.

Click Add Partner Key. Create a Partner Key.

Step 3 Create a Partner Administrator Role to provide credentials for the API access. This is done from Administration > Role Management:

Step 4 Create a partner account for Aruba Central using the recently created role. This can be done from Administration > Administrator Management:

Once that’s done, it should look like this in the Zscaler portal:

Configure Aruba Central for Zscaler orchestration

The first step enable the orchestrated integration is to enter the partner credentials and API Key for the Cloud Connect Service to communicate to the ZIA service. This can be configured from the “Global” context, under Network Services > Cloud Connect > Settings > Accounts.

After that, the account will be marked as “ACCESS VERIFIED”. If needed, multiple Zscaler accounts can be added.

Site and Tunnel Orchestration from Aruba Central

At this point Aruba Central is able to communicate with ZIA through its API, and it’s therefore capable of orchestrating communications between Gateways or Microbranches and the ZIA service.

The first step would be to go to Network Services > Cloud Connect > Settings > Deployment and select Gateway or Microbranch groups that are to be connected to ZIA. For any device present in such groups, Aruba Central will create a “location” in ZIA with the desired attributes. Select the groups that are to be orchestrated by clicking on the “Connection” check box.

Defining location-specific configurations from Aruba Central

When doing so, you’ll notice a small “cog” icon appearing on the right. Clicking this icon allows editing locations from Aruba Central.

It also allows specifying the Service Edges a certain group of Gateways or Microbranches would connect to. If the “Override ZIA Service Edge” option is not selected, Aruba Central will simply build tunnels from every Gateway or Microbranch to the nearest two ZIA Service Edges.

Zscaler sub-locations

For sites that may have a wider variety of users, Aruba Central also allows creating sub-locations. By implementing sub-locations, customers can apply different policies to different VLANs behind a Branch Gateway. More information about sub-locations can be found in the Zscaler help. To create sublocations, simply click the “caret” on the left of the table and then the “+” on the top right, as shown in the image below.

Please note that a sublocation named “other” is automatically created when the first user-defined sub-location is created. This sub-location serves as a “catch all” policy for all traffic not belonging to any user-defined sub-location.

Sub-locations can then be defined and edited in the same way as site options.

Note: The ability to edit sub-locations from Aruba Central is only available for Branch Gateways.

Applying configurations

Depending on the number of sites that are present in each group, defining the sites in ZIA can take several minutes. For this reason, Aruba Central holds on pushing configurations until all changes are made. When ready, simply go to “Preview” and then click “Submit” if all changes make sense.

Monitoring the Orchestration Process

The time required to orchestrate connectivity to ZIA (or any other 3rd party service) will depend on number of gateways in the group as well as the load on the partner API (in this case Zscaler). Aruba Central will provide an indication of the remaining time. This can be verified from Network Services > Cloud Connect > List > Zscaler. Once the orchestration process is finished, the deployment status bar will indicate so.

Validating Site and Tunnel Orchestration

After a group is enabled for ZIA Service Edge Orchestration, any device belonging to it (Microbranch or Gateway). or any new device that gets added to the group in the future triggers the creation of a location in Zscaler (with all the necessary attributes). For example, adding a site in Redwood City, California with 3 sublocations, triggers the creation of the subsequent site in Zscaler:

It also triggers establishing IPsec tunnels from all public WAN interfaces (INET, LTE, Metro Ethernet) to appropriate ZIA Service Edges. As an example, a Microbranch site in Madrid, Spain, gets automatically connected to Public Service Edges in Madrid, Spain and Marseille, France.

Additionally, all tunnels orchestrated from Branch Gateways are automatically set to probe http://gateway.<zscaler cloud domain>/vpntest to assess the quality of communications through every tunnel. This will allow the gateway to make smarter forwarding decisions.

The state of the tunnels can easily be verified from the interface of Aruba Central. This can be checked from the Tunnels tab in the Gateway monitoring dashboard. Please note that tunnels will be marked as “ORCH-IKE” in the case of orchestrated tunnels and “IKE” in the case of manual tunnels.

Forwarding Traffic through ZIA

Once tunnels between Aruba Gateways and ZIA are established, the only remaining step would be to select which traffic is to be sent through ZIA. This is done using Policy Based Routing, but the details on how to implement the forwarding policies vary based on whether this is done using a Gateway or a Microbranch.

Traffic forwarding with a Gateway

Branch Gateways automatically establish tunnels to ZIA through all active (public) uplink interfaces and (when using ArubaOS version 10.3 or higher), send probes to ZIA to monitor quality over each tunnel. The next step would then be to organize tunnels in a “next-hop-list” so they can be used by WAN and routing policies.

To better understand how routing and WAN policies interact with each other. Routing/PBR determines next-hop(s); It provides up to 4 active (best cost/priority) paths to the WAN engine. WAN Policies (SaaS/DPS) then determine which path should be taken for each traffic flow. The diagram below explains how PBR and global routing coexist with WAN policies defined in DPS or SaaS Express:

To do so, first create a “next-hop list” with the tunnels, then include them in a PBR policy, and finally attach that policy to the right object (Role, VLAN or Overlay). This can all be handled in a single workflow by going to Policies > PBR in the Basic Configuration mode at group level (see screenshot below).

To get a more in-depth understanding of how this mechanism works, this document will go through all the necessary steps using Advaced Mode to configure the Gateways.

Create a Next-Hop List with the tunnels and (optionally) define an SLA

The key aspect of this configuration is to make sure tunnels going to the same Service Edge always have the same priority. This will allow Dynamic Path Steering (DPS) to determine which traffic flow should go on every one of the tunnels to the Service Edge. This can be configured from Routing > Next Hop Configuration.

Step 1 Create a Next Hop list.

Step 2 Add an IPsec map for Site-to-Site” tunnels under IPSec maps. When doing so, you can optionally set an SLA.

Step 3 Use the same priority for all tunnels going to the same Service Edge. In the case of orchestrated tunnels, those marked as primary should have the same priority between them, and those marked as secondary should also have the same priority between them.

Step 4 Use different priorities for different ZIA Service Edges.

Step 5 Ensure that Preemptive failover is enabled, as this will allow Gateways to steer traffic back after recovering from any service-impacting issue on the primary tunnels.

SLA profiles can optionally be associated to the next-hop list. By doing so, gateways can selectively remove non-compliant tunnels from the next hop list (the IPsec tunnels would continute to be up, but not be used to forward traffic). There are two modes of operation:

• Liveness: In this mode, tunnels through which the ZIA probe responder (http://gateway.<zscaler cloud domain>/vpntest) is not reachable are marked as passive and are not used to forward traffic.
• Performance: In this mode, a certain SLA goal (latency, packet loss) must be met in order to consider a tunnel for traffic forwarding. If none of the tunnels in a next-hop list are “compliant”, the gateway will simply pick whichever deviates the least from the defined SLA.

In both cases, every time a next-hop has been marked as “non-compliant”, it will remain in that state for a minimum of 3 minutes in order to avoid flapping between tunnels.

IP-SLA Profiles are managed from the WAN > Health Check >IP-SLA Profiles page in the Branch Gateway configuration page.

Add Next-Hop to a Routing Policy

After the next-hop list with tunnels to ZIA is created, add it to a routing policy in Routing > Policy-Based Routing.

In the example below, the policy is sending all traffic to certain applications (Zoom and Microsoft Teams) as well as private-networks (an alias representing RFC1918 addresses) through the regular path, and it’s sending the rest of the traffic through ZIA Public Service Edges using the next-hop list previously defined.

Apply Routing Policy

After the routing policy is created, the last step would be to apply it to relevant traffic.

In the case of Branch Gateways, these policies would be applied to roles or VLANs where there are devices that should be sent through ZIA:

• To apply a policy to a VLAN, go to Security > Apply Policies and select it from the dropdown next to each VLAN.
• To apply a policy to a role, go to Security > Roles” and edit the role you want to send through ZIA by adding a routing policy (routing policies always come at the end).

In case of VPNCs, routing policies would normally be applied to incoming SD-WAN traffic. This can be configured from VPN > SDWAN Overlay > Advanced.

Validating Routing Policy

Once PBR is configured and applied to the appropriate role, interface, etc, we can validate that traffic is effectively taking that path. To do so, we can start by checking which user-role is associated to the client device:

We should also check that the traffic is actually going to the Internet through the tunnels to ZIA:

As an additional verification, client devices can browse to https://ip.zscaler.com. This page shows if the client is accessing the Internet through ZIA, and which Service Edge is being used.

Traffic Forwarding with a Microbranch AP

Configuring a Microbranch AP to forward traffic through ZIA is also very straightforward, and it follows very similar steps to those used for Branch Gateways. As with gateways, a PBR policy has to be created and then applied to the corresponding user roles.

Define a PBR policy to send Internet traffic through ZIA

When using PBR to force traffic through ZIA, general best practice is to only do so for public addressing space, leaving security policies for internal traffic to on-premises appliances such as NGFWs or the Microbranch APs themselves. For that reason, it makes sense to start by creating a network alias representing corporate IP addresses. This can be done (at the group level) from Security > Policies & Access Control > Aliases.

The next step would be to create a “next-hop” representing the tunnels to the ZIA service. This next-hop will ensure traffic automatically fails-over in the rare event that one of the tunnels were to go down. This can be done from Tunnels & Routing > NextHop List, as displayed in the image below.

With that, a PBR policy sending all non-RFC1918 traffic through ZIA can easily be created from Tunnels & Routing > Policy-based Routing. A good example would be the policy below:

Apply PBR policies to relevant user-roles

The last step is to add the PBR policy to all the user-roles that would require it. This can be done from Security > Policies & Access Control > Roles, as shown in the image below:

Validating Routing Policy

Just as in the case of Gateways, to validate that clients are being forwarded as per PBR policy, the first step is to validate they have the right user-role. This can be checked from the client details and client list pages. If the role is the right one, make sure that role has a PBR policy associated to it.

Further debugging can be done via the AP CLI.

“show acces-list” can be used to validate the routing ACLs being used to redirect the traffic:

EC-Microbranch# show access-list    

Access list table
-----------------

Name                  AclId  Type   Use Count  Roles

----                  -----  ----   ---------  -----

"default policy"      471    route              
internet-through-zia  472    route  1          EC-Microbranch 
EC-Microbranch# 
EC-Microbranch# show access-list internet-through-zia

Route Access-List Rules
-----------------------

Src IP   Src Mask    Dest IP  Dest Mask   Eth Type  Dest Match  Protocol (id:sport:eport)  Application  Action                         Log  TOS  802.1P  Blacklist  App Throttle (Up:Down)  Mirror  DisScan  ClassifyMedia  TimeRange

------   --------    -------  ---------   --------  ----------  -------------------------  -----------  ------                         ---  ---  ------  ---------  ----------------------  ------  -------  -------------  ---------

netdest  rfc1918(2)  netdest  rfc1918(2)  IPv4/6    match       any                                     permit                                                                                               ClassifyMedia  
any      any         any      any         IPv4/6    match       any                                     route nexthop-list ZIA-Bundle                                                                        ClassifyMedia
EC-Microbranch#

“show ip nexthop-list” can be used to verify that the right tunnels are being attached to the next hop list that is being used by the PBR policy, and that one of these tunnels is active (marked with a *).

EC-Microbranch# 
EC-Microbranch# sh ip nexthop-list  

Nexthop-List Entries
--------------------

Name        Dest    Preemptive Failover  Nexthop                                       Nexthop Dest  Nexthop Priority

----        ----    -------------------  -------                                       ------------  ----------------

ZIA-Bundle  0x4401  Enabled              *zs-init-samuel-production-primary-comcast    0x443e        128
                                          zs-init-samuel-production-secondary-comcast  0x443f        64
EC-Microbranch# 

“show datapath session”, can be used to verify that traffic is being redirected into the previously defined next-hop-list:

EC-Microbranch# sh datapath session | i -
------------------------------

Flags: A - Application Firewall Inspect
       C - client, D - deny, E - Media Deep Inspect
       F - fast age, G - media signal, H - high prio
       I - Deep inspect, L - ALG session, M - mirror, N - dest NAT
       O - Session is programmed through SDN/Openflow controller
       P - set prio, R - redirect, S - src NAT, 
       T - set ToS, U - Locally destined, V - VOIP
       X - Http/https redirect for dpi denied session
       Y - no syn
       a - rtp analysis, h - Https redirect error page
       i - in offload flow, m - media mon
       p - Session is marked as permanent
       s - media signal
       d - DPI cache hit
RAP Flags: 0 - Q0, 1 - Q1, 2 - Q2, r - redirect to conductor, t - time based, i - in flow
Flow Offload Blacklist Flags: O - Openflow, E - Default, U - User os unknown, T - Tunnel

----------------  --------------  ---- ----- ----- ---- ---- --- --- ----------- ---- ------- ----- ------ -------------

151.101.194.137   10.127.17.126   6    443   55954 0    0    0   1   pbr-nhl 1   501  22      49b0  Fi                   
13.227.76.59      10.127.17.126   6    443   55942 0    0    0   1   pbr-nhl 1   508  5ab     18395b Fi                   
13.35.122.126     10.127.17.126   6    443   55944 0    0    0   1   pbr-nhl 1   505  16      1bee  Fi                   
13.35.122.26      10.127.17.126   6    443   55957 0    0    0   1   pbr-nhl 1   4fe  17      1e99  F                    
13.35.122.43      10.127.17.126   6    443   55958 0    0    0   1   pbr-nhl 1   4fd  22      2c46  F                    
10.127.17.126     151.101.194.137 6    55954 443   0    0    8   1   pbr-nhl 1   501  1f      a32   FCi                  
10.127.17.126     13.35.122.126   6    55944 443   0    0    8   1   pbr-nhl 1   505  16      755   FCi                  
10.127.17.126     13.35.122.43    6    55958 443   0    0    8   1   pbr-nhl 1   4fd  26      1852  FC                   
10.127.17.126     13.35.122.26    6    55957 443   0    0    8   1   pbr-nhl 1   4fe  1a      a23   FC                   
10.127.17.126     99.84.226.109   6    55953 443   0    0    8   1   pbr-nhl 1   501  22      b6b   FC                   
10.127.17.126     13.227.76.59    6    55942 443   0    0    8   1   pbr-nhl 1   508  13d     4569  FCi                  
10.127.17.126     99.84.226.140   6    55945 443   0    0    8   1   pbr-nhl 1   505  16      761   FC                   
10.127.17.126     99.84.226.173   6    55946 443   0    0    8   1   pbr-nhl 1   505  17      77d   FC                   
10.127.17.126     99.84.226.186   6    55949 443   0    0    8   1   pbr-nhl 1   505  3a      10ea  FC                   
10.127.17.126     99.84.226.201   6    55952 443   0    0    8   1   pbr-nhl 1   502  1f      b49   FC                   
99.84.226.201     10.127.17.126   6    443   55952 0    0    0   1   pbr-nhl 1   502  22      3a34  F                    
99.84.226.186     10.127.17.126   6    443   55949 0    0    0   1   pbr-nhl 1   505  5a      1572b F                    
99.84.226.173     10.127.17.126   6    443   55946 0    0    0   1   pbr-nhl 1   505  15      1b8f  F                    
99.84.226.140     10.127.17.126   6    443   55945 0    0    0   1   pbr-nhl 1   505  14      1edd  F                    
99.84.226.109     10.127.17.126   6    443   55953 0    0    0   1   pbr-nhl 1   501  29      6170  F                    
EC-Microbranch# 

Finally, additional validation can be obtained by browsing to https://ip.zscaler.com from the client device. This page shows if the client is accessing the Internet through ZIA as well as through which node.