Citrix NetScaler SD-WAN 101 – Build a Lab Infrastructure

Reading Time: 20 minutes


I think most of the people in the Citrix community have been reading a lot about the NetScaler SD-WAN the last months. I want to show you, how to configure the appliances to give you a straight start with this technology. First of all: Why should a company consider investing money in Software Defined WAN?

  • Improve application performance in the datacenter and branch offices
  • Reduce your operating costs dramatically. You don’t need to spend money for the expensive MPLS connection to South America/Asia when you can bundle n-WAN connections from different Internet Service Providers.
  • Maintain network connectivity (without interrupt) to the Datacenter/Branch even multiple network connections go down.
  • Reduce network hardware (stateful firewall function)
  • Centralized and easy management
  • Detailed reports and network analysis

If you check the SDWAN datasheet, you will notice there are three kind of appliances.

  • Standard (SE): Bundle different Internet Service Provider links to a Virtual WAN
  • WANOP (WO): Optimize the WAN Connection. Many people will know this as Branch Repeater or CloudBridge.
  • Enterprise (EE): You get the ability to use Standard and WANOP on a single appliance.

My opinion to WANOP: I made bad experience with Citrix CloudBridge some time ago. I think this should only be used when there really is no other ISP at the branch available and you need to squeeze the last drop out of the connection. Regarding HDX optimization –> With adaptive transport (EDT) the traffic will be UDP based and there will be nothing to optimize. Keep this in mind.

NetScaler SD-WAN is available as a physical appliance or virtualized on the hypervisor of your choice.

  • Hyper-V (2012 R2)
  • XenServer 6.5 SP1
  • VMware ESX 5.5 & ESX 6.0
  • KVM (Ubuntu 16.04)

The Enterprise edition is only available as a physical appliance.

Like in the NetScaler ADC family the purchased license will specify how much bandwidth you can use and how many virtual paths you will be able to create. What is a virtual path? Let’s say your datacenter/branch is getting WAN access from two different ISPs. If you now create a Virtual WAN to the branch office, this will result in a total of 4 virtual paths.

  1. ISP1_Datacenter –> ISP1_BranchOffice
  2. ISP1_BranchOffice –> ISP1_Datacenter
  3. ISP2_Datacenter –> ISP2_BranchOffice
  4. ISP2_BranchOffice –> ISP2_Datacenter

When running a VPX the license upgrade will also give you the ability stock up the available virtual paths. A physical appliance can use the maximum of the virtual paths starting from the smallest license. Depending on your company size you should size the appliance properly.


The quality of each path will be measured by the following information:

  • BOWT (Latency – Best one-way time)
  • Jitter (mS)
  • Packet-loss (%)
  • kbps (Bandwidth)
  • Congestion (will detect if there is a problem on the ISP backbone. Example: unexcepted packet flow)

SD-WAN will provide out-of-the-box configured QoS settings for the most common protocols used in an enterprise network:

Prioritization 1 – Real Time – Minimum Bandwidth 30% – VOIP (Skype for Business, ICA Audio)

Prioritization 2 – Interactive – Minimum Bandwidth 40 % –  XenDesktop, Exchange

Prioritization 3 – Bulk – Minimum Bandwidth 30 % –  Moving Data (FTP, Backup Replication)

If you are not comfortable with the default settings, you can create a custom “Classes” for prioritizing specific protocols.

Deployment Methods

You have a variety of possibilities to connect the SD-WAN to your corporate network.  Before starting to show you the different deployment modes it’s important to know that you can configure the ethernet interfaces as “Fail-To-Wire” or “Fail-To-Block”.

Fail-To-Wire: Only available on physical appliances. If the appliance goes offline, the specified ports will bridge the traffic from the SD-WAN to the LAN side. This will be used for a MPLS connection to prevent a complete network outage. The communication to the branch office is still guaranteed.

Fail-To-Block: When you have connected an unsecure Internet connection (without Firewall) the port will block the traffic. This is the default setting on a VPX.

For High Availability you have the possibility to create an active-passive setup.

Datacenter Gateway Deployment

The SD-WAN appliance will be put in the physical path between LAN and WAN (Two-Arm Deployment). SD-WAN will be the default gateway for the entire datacenter LAN network. If the appliance will go down, you will not have any internet or MPLS connection because SD-WAN is acting as Layer 3 device only (no Fail-To-Wire functionality).


Branch Inline Deployment

The SD-WAN appliance will be put in the physical path between LAN and WAN (Two-Arm Deployment). The branch LAN will be connected directly to the appliance.


Virtual Inline Mode (PBR)

The SD-WAN appliance will be connected with a single interface to the datacenter router (One-Arm Deployment). Policy Based Routing (PBR) needs to be configured on the core switch/router.


Edge Mode

In Edge Mode the SD-WAN is connected directly to the WAN links. You can seamlessly replace the legacy routers.


Building a NetScaler SD-WAN Lab

Now let’s come to the interesting part. How can I create a NetScaler SD-WAN Lab? You should bring some network knowledge with you. If you are new to this stuff I will try to explain as detailed as possible. My lab environment is hosted on an “Intel NUC- i7-6770HQ” with XenServer Release 7.2

Network Diagram


Overview Lab Infrastructure

VM vCPU RAM HDD Description
RTR_INET_DC 1 256 MB 5 GB Internet Router (DC)
RTR_INET_BR 1 256 MB 5 GB Internet Router (Branch)
WANEM-INET 1 256 MB 5 GB WAN Emulator Internet
RTR_MPLS_DC 1 256 MB 5 GB MPLS Router (DC)
RTR_MPLS_BR 1 256 MB 5 GB MPLS Router (Branch)
WANEM-MPLS 1 256 MB 5 GB WAN Emulator MPLS
SDWAN_BR 4 4 GB 40 GB SDWAN (Branch)
DC_Client 2 2 GB 25 GB Windows Client (DC)
BR_Client 2 2 GB 25 GB Windows Client (Branch)


Create the virtual networks

Network Description VLAN ID (optional)

Before we can start to configure the VyOS routers we need to create the virtual networks. If you build the lab on a single machine you don’t need to create external networks with a VLAN ID. A single server private network is enough. When using a single XenServer for the lab, it’s very important to disable TCP checksum offload on the virtual networks otherwise no request except ICMP will reach your clients/servers in a different network. Thanks to Benjamin Ruoff from Citrix who discovered this issue. Here is the bash script from Benjamin to disable the checksum offload on all virtual network interfaces. This was consuming most of my time during the lab build.

  1. Click on “Networking” in XenCenter and select “Add Network…”n1
  2. Choose “Single-Server Private Network”, when running on a single lab machine. Otherwise you need to take “External Network” and specify the VLAN ID.n2
  3. Provide the name and a description for the new network.n3
  4. Uncheck the box “Automatically add this network to new virtual machines”.n4
  5. Create the missing networks. After this we can start to install the VyOS routers.

Install VyOS Routers

VM Network 1 Network 2 Description


Internet Router (DC)


Internet Router (Branch)


MPLS Router (DC)


MPLS Router (Branch)
  1. Right click on the XenServer and select “New VM…”. Now we need to create the first router called “RTR_INET_DC”.vyos1
  2. Put in the name for the first router “RTR_INET_DC”.vyos2
  3. Choose the downloaded VyOS image as install media.vyos3
  4. Assign 1vCPU and 256MBvyos4
  5. Configure the used virtual disk. 5GB is enough.vyos5
  6. Attach the needed network interfaces (refer to the table).vyos6
  7. Click on “Create Now” and start the new VM automatically.vyos7
  8. Before we can start with the configuration we need to install the VyOS image to the harddisk and install the XenServer Guest Tools. The default login is vyos;vyos.vyos8
  9. Use the command “install Image”, confirm the user prompts and set a new password for the vyos account. After the successful installation restart the VM with the command “reboot”.vyos9vyos11
  10. Mount the “guest-tools.iso” to the virtual dvd drive and login with the vyos user.
  11. Run the following commands to install the XenServer Tools.
    sudo su
    mount /dev/cdrom /mnt
  12. After the successful reboot login again and execute the command “show interfaces” you should now be able to see the virtual network adapters inside VyOS.vyos12
  13. Repeat these steps to create the missing routers

Configure VyOS Routers

Execute the commands on the routers.

###  RTR_INET_DC ###

### RTR_INET_BR ###

### RTR_MPLS_DC ###



Install & Configure the WAN Emulators

VM Network 1 Network 2 IP Address
  1. Create the VM “WANEM-INET”, attach the needed networks and mount the WANEMv2.3.iso.
  2. Press “Enter” to boot from the CD.
  3. Do you want to configure all interfaces via DHCP: y
  4. Enter new UNIX password: ********
  5. WANemControl@Perc> exit2shell
  6. root!tty1:/# knx2hdwanem1
  7. Initialization of Knoppix-Installation: OK
  8. Initialization of Knoppix-Installation: OK
  9. Knoppix Partition-Menu: “ Partition”
  10. Partitioning harddisk for Knoppix: “had” and NEXT
  11. Partitioning harddisk for Knoppix: “ Template 1”
  12. Partitioning Harddisk: Yes
  13. Partitioning Harddisk: Yes
  14. Partitioning harddisk for Knoppix: OK
  15. Knoppix Partition-Menu: “Quit”
  16. Save configuration?: Yes
  17. root!tty1:/# knx2hd
  18. Initialization of Knoppix-Installation: OK
  19. Knoppix Main-Menu: “1. Configure Installation”
  20. Choose your system type: “debian”
  21. Creating Knoppix Configuration (Step1/7): “/dev/hda2”
  22. Choose filesystem-type: ext3
  23. Input your whole name (name surname): “dummy”
  24. Input your user name: “ddummy”
  25. Input user password: ***** (will not be used)
  26. Input your administration password: *****
  27. Input your preferred hostname: “WANEM-INET”
  28. Choose where the bootloader (grub) shall be installed: mbr
  29. Knoppix Main-Menu: “2. Start Installation”
  30. Starting Knoppix Installation: Next
  31. Creating floppy disk: No
  32. root!tty1:/# reboot and remove the mounted image
  33. The WAN Emulator will boot from harddisk and you can login with the admin password you choose in Step 26. Now we need to configure the ethernet interfaces to bridge mode. Execute “vi /etc/network/interfaces”wanem33
  34. Remove/Uncomment the DHCP interfaces (eth0, eth1) settings and add this to the configuration file.
    auto br0
    iface br0 inet static
         bridge_ports all
         bridge_fd 0
         bridge_stp off
  35. Save the changes with “:wq!”wanem35
  36. Reboot the VM.
  37. Now create the WAN MPLS Emulator. The only difference is the IP configuration of the bridged network interface.
    auto br0
    iface br0 inet static
         bridge_ports all
         bridge_fd 0
         bridge_stp off

Network Test

At this point you should have installed and configured the VyOS routers and the WAN emulators. Time for a short test if the components can communicate with each other. Run a “ping” command from the VyOS console. If there is a problem with the network, please check your VyOS configuration.


Destination: (WANEM INET)

Destination: (RTR_INET_BR – ISP WAN Side)

Destination: (RTR_INET_BR – INET Branch Side)

Destination: (WANEM MPLS)

Destination: (RTR_MPLS_BR –  MPLS WAN Side)

Destination: (RTR_MPLS_BR – MPLS Branch Side)

Create the Datacenter and Branch Client

Before we can start with the import and configuration of the SD-WAN appliance we need to create the Datacenter and Branch client that, we can access the management ip address of the VPX.

VM vCPU RAM HDD IP Address Network Description
DC_Client 2 2 GB 25 GB

LAN-DC Windows Client (DC)
BR_Client 2 2 GB 25 GB

LAN-BR Windows Client (Branch)
  1. Right click on the XenServer and select “New VM…”cl1
  2. Select “Legacy Windows” as Template and click Next.cl2
  3. Enter the Name of the datacenter client “DC_Client”.cl3
  4. Select your Windows 10/Server 2016 install media.cl4
  5. Place the VM on the server which is simulating the datacenter.cl5
  6. Assign 2vCPU and 2048 MBcl6
  7. Configure the virtual disk.cl7
  8. Attach the needed network interface “LAN-DC”.cl8
  9. Click on “Create Now” and start the new VM automatically.cl9
  10. Complete the Windows installation and install the XenServer Guest Tools.cl10
  11. Configure the TCP/IPv4 Properties.cl11
  12. Repeat the steps and create the branch client “BR_Client”.

Import the NetScaler SD-WAN VPX (Datacenter)

VM eth0 eth1 eth2 eth3
  1. In XenCenter click on File -> Importimp1
  2. Browse to the “ns-sdw-os-xxxx.xva”file and click Next.imp2
  3. Set a location for the VM.imp3
  4. Select the storage where the VM should be imported.imp4
  5. Assign the Datacenter LAN (LAN-DC) to interface 0. This is always the management port. Connect the missing interfaces.imp5
  6. Uncheck “Start VM(s) after import” and click “Finish”.imp6
  7. Right click on the imported VPX and select “Properties”imp7
  8. Change the Name to “SDWAN_DC”.imp8

Configure Management IP Address

Note: If there is a DHCP server available on the management interface, the SD-WAN appliance will request an IP address.

  1. Start the VM “SDWAN_DC”
  2. After the run once commands are complete, login to the console as “admin” and “password”.mgmtip2
  3. Enter “management_ip” and “set interface”mgmtip3
  4. Enter “apply” and “y” to save the new configured management ip address.mgmtip4


  1. Download your License.lic1
  2. Select “Citrix Store Cloud Bridge VPX Virtualization” and Continue.lic2
  3. Enter the MAC address from the VM “SDWAN_DC” interface 0 (MGMT) and download the license file.lic3
  4. Open the console of “DC_Client” and browse to the URL:
  5. Login with default user “admin” and “password”.lic5
  6. Click on “Configuration”lic6
  7. Click on “Appliance Settings” and Licensing and upload your trial license file. Its also possible to import the .lic file to your Citrix Licensing Server and activate the VPX via this method.lic7

General Settings (NTP, TimeOut)

  1. Click on Appliance Settings, Administrator Interface and choose “Miscellaneous”. Change the Time Out of the Management Web Console to 500 minutes. This will save you some login procedures while building the lab. Not recommended for production environment.settings1
  2. Under “System Maintenance” and “Date/Time Settings” specify your NTP server and change to the correct time zone.settings2

Create the Master Control Node (MCN)

The Master Control Node is the heart of the Virtual WAN. The MCN will create the config for the Datacenter and for the branch offices. Any changes you will make will happen on this machine.

  1. Go to “Appliance Settings”, “Administrator Interface” and switch to “Miscellaneous”. Hit the Button to “Switch Console” to make this appliance to the primary MCN.mcn1
  2. Your Session will be terminated and you need to login again with admin;password.mcn2

Create the Site “Datacenter”

  1. Go to “Configuration” and expand the “Virtual WAN” menu.mcn3
  2. Click on “Configuration Editor” and create a “New” configuration.sitedc1
  3. Select the arrow icon to expand the window size.sitedc3
  4. Change the mode to “Advanced”.sitedc4
  5. Click on the “+ Add” button under “Sites”.sitedc5
  6. Specify the Site Name to “DC” and change the Model to “CBVPX”. Click “Add”.sitedc6
  7. Click on the “+” button.sitedc7
  8. Mark the Ethernet Interface “1”, set the Bypass Mode to “Fail-To-Block and change the Security to “Untrusted”. Click on the “+” button on the left side. The Interface is untrusted because on Interface 1 will be connected the Public Internet connection without a firewall.sitedc8
  9. Click again on the “+” button.sitedc9
  10. Specify the name “DC-INET” for the virtual interface.sitedc10
  11. Repeat the procedure to create the virtual interface for the MPLS connection. (Ethernet 2, Trusted Security).sitedc11
  12. Repeat the procedure to create the virtual interface for the LAN connection. (Ethernet 3, Trusted Security).sitedc12
  13. Now its time to specify the virtual ip addresses.sitedc13
  14. Enter the ip addresses/subnet mask and the matching virtual interface for the VPX.sitedc14
  15. The next step is to create the WAN links.sitedc15sitedc16
  16. Click on the pencil to edit the WAN link configuration.sitedc16_1
  17. Specify the available bandwidth for the Public Internet connection (16000kbps/4000kbps). There is a text field with the description “Public IP Address”. We don’t need this setting in the lab environment. In a real-world environment, the MCN needs to be configured with at least one public ip address. Otherwise the branch appliance will not be able to connect to the MCN.sitedc17
  18. Create the missing access interface “DC-INET-AI-1” (,
  19. Repeat the steps for the MPLS WAN Link. a. Access Type: Private Intranet, b.Bandwidth: 6000kbps/6000kbps, c. IP Address:, Gateway:
  20. Save the configuration.sitedc20

Create the Site – Branch

  1. Create the Site “Branch” with the Configuration Editor.br1
  2. Specify the Name “Branch”, Model: “CBVPX” and the mode to “client”.br2
  3. Now we need to create the interface groups for the branch site. When finished it should look like this.br3
  4. Specify the virtual ip addresses for the branch.br4
  5. Create the WAN Links for the Branch.
    a.BR-INET – 16000kbps/4000kbps – Public Internet – IP:, Gateway
    b.BR-MPLS – 6000kbps/6000kbps – Private Internet – IP:, Gateway
  6. Save the configuration.

MPLS Path Association

You always need to specify that a MPLS Link will be used for a connection. For Public Internet connections this will be done automatically.

  1. Go to “Connections”, select “WAN Links” and edit the “DC-MPLS” link.mplsp1
  2. Mark the path with “Use” and apply the change.mplsp2
  3. Do the same with the MPLS link in the branch site.mplsp3
  4. Head to “Connections” –> “Branch” –> “Virtual Paths” –> “DC-Branch” –> “Paths” and create a new path.mplsp4
  5. From Site: Branch, From WAN LINK: BR-MPLS, To Site: DC, To WAN Link: DC-MPLS Make sure you check the “Reverse Also” option. (This will create the path in thedatacenter)                                                            mplsp5mplsp6
  6. Now your configuration editor shouldn’t throw any errors and you have completed the configuration of the two sites. Congratulations. Was quiet easy hum? Its no rocket science 😊 Please save the config one more time!

Import the NetScaler SD-WAN VPX (Branch)

VM eth0 eth1 eth2 eth3
        1. Import the .ova Template and create the VPX for the branch site. The network configuration should look like this.brvpx1
        2. Rename the VM to “SDWAN_BR” and set the management ip address.
          set interface
        3. Import a valid license file.
        4. Change NTP, time zone and timeout interval.

Provision the Configuration (Staging)

  1. Login to the SDWAN_DC again –
  2. Go to the “Configuration Editor” of the Virtual WAN and click on “Export”. If the configuration is not loaded just open it.staging1
  3. Export the Configuration the “Change Management Inbox”.staging3
  4. Switch to “Change Management”staging4
  5. If you start the staging process with “Stage Appliances” you will get an error message because the software package is missing. The software packing is containing the firmware file and needs to be imported to the MCN. This is needed because imagine your physical MCN was shipped with Firmware 9.3 but the branch appliance is still running on Firmware 9.2. With this procedure its secured that the firmware will be same on every appliance you build the Virtual WAN.staging5
  6. Download the Software Package for the matching SD-WAN Release.staging6
  7. Upload the package and try again to start the staging.staging7
  8. Accept the EULAstaging8
  9. Go to “Next”staging9
  10. Click on “Activate Staged”.staging10
  11. Click again on “Activate Staged” and wait until the appliance is ready again. Can take up to 180 seconds.staging11
  12. Enable the “Citrix Virtual WAN Service” under Virtual WAN à “Enable/Disable/Purge Flows”staging12staging13

Setup the Branch VPX

There is also a way provision the configuration file without the following manual steps. This is called Zero-Touch Deployment. The configuration file will be uploaded to the Citrix Cloud. The configuration file need to be linked to the serial number of the appliance. The co-worker in the branch office just need connect an ethernet link on the management interface with DHCP and active internet connection. The config will be downloaded, and the appliance should be ready to use.

  1. Go to Configuration, Virtual WAN, Change Management on the MCN. Download the active configuration file for the branch appliance.brsetup1
  2. Login to the Branch VPX –
  3. Go to Configuration, Local Change Management and upload the configuration file from step 1 –> Nextbrsetup3.png
  4. Click on “Activate Staged”brsetup4
  5. Select “OK” and wait until the webpage reloaded.brsetup5
  6. Enable the Citrix Virtual WAN Service.brsetup6
  7. Click on “Monitoring” and you should see that all the WAN Links are up and running. That’s it 🙂brsetup7.png

Working with WANEM

With WANEM we have the ability to simulate stuff like lower bandwith and packetloss. Its only a short introduction I think you can handle the rest without problems.

  1.  Connect the branch or datacenter Client to the network “WAN-INET-BR” or WAN-INET-DC”wanemsim1
  2. Set a static ip address for the ethernet interface (
  3. Browse to  (case sensitive!)
  4. Select “Advanced Mode” and click on Start.wanemsim4
  5. Create a packet-loss of 25% and apply the settings.wanemsim5
  6. Check the SD-WAN Dashboard (Monitoring).wanemsim6

SD-WAN as DHCP Server

You can use the SD-WAN appliance in the branch office as a DHCP server. Very usefull when there is no other server infrastructure available.

    1. Open the Configuration Editor and import your current configuration.dhcp1dhcp2
    2. Click on the “Branch Site” and create a new DHCP “Server Subnet”dhcp3
    3. Specify the DHCP settings.dhcp4
    4. Save and Export the configuration to the Change Management.
    5. Start the Staging procedure. dhcp5
    6. Activate the new configuration.dhcp6
    7. The configuration rollout was succesfull.dhcp7
    8. Switch to your Branch Client and change the Ethernet Interface to DHCP to verify if its working.                             dhcp8

Direct Internet Breakout at Branch

If the Branch Office should use the WAN Links directly to surf in the WWW  its possible to configure a direct Internet Breakout.

  1. Open the Configuration Editor and import your current configuration.
  2. Click on the “Branch Site” –> Connections –> Virtual Paths –> Addinetbreakout1
  3. Click on “WAN Links” and press the edit button.inetbreakout2
  4. Now you can specify which WAN Links should be used for the direct internet access. You have the ability to choose three different modes.
    1. Primary
    2. Second
    3. Balance
  5. After you have applied the setting the routing table will be updated.inetbreakout3
  6. Rollout the new configuration.



  1. Hi thanks for the documentation, I am trying to build this and step 14. to 15. of creating the DataCenter Site, I do not understand. The Boxes for the untrusted Internet (Private and Identity) are both checked even thou the Box for Private is greyed out in step 14 how is this possible?

  2. I am new to SD-WAN and IT for that matter. You are a big help for me understanding SD-WAN. My Links are now showing as UP and I can ping the client, but I can’t send Data across to the client at the other side or RDP on to the client. Do you have any clues or tips for me?

      1. The lab is on a single XenServer 7.4, in Monitoring, there are 0 flows 🙁 LAN to WAN and WAN to LAN. Under statistics, the bandwidth is considerably lower than the configured bandwidth.

  3. one question, in your lab, you set the gw of the client with the ip of the SD-WAN right? but i can’t see the gateway on the SD-WAN for the configuration of address
    thanks again.

      1. yes, i saw, but on the SD-WAN (DC) i can’t see the route and the GW definition for the LAB-DC interface, it use the same of the management interface?

  4. Great blog, thank you very much!!
    I have only one problem, the Virtual WAN service is not starting.

    This is the message:
    The Citrix Virtual WAN Service is currently disabled.
    The Citrix service has reset 4 times in less than 120 seconds. The service has been disabled to prevent unnecessary network disturbance.
    The Citrix Virtual WAN Service was disabled at: Wed Oct 28 18:49:47 2020

Leave a Reply

Your email address will not be published. Required fields are marked *