Redundant Internet with SD-WAN

Facebooktwittergoogle_plusredditpinterestlinkedinFacebooktwittergoogle_plusredditpinterestlinkedin

The following example demonstrates how to configure redundant Internet using the new SD-WAN feature in FortiOS 5.6. 

The goal of SD-WAN is to seamlessly manage traffic at the Layer 2 level of the OSI model without the need to manage hardware-based switches or WAN controllers.

Find this recipe for other FortiOS versions
5.2.0 | 5.2.1 +  | 5.4 | 5.6

The example includes volume-based weighted load balancing so that 75% of your Internet traffic is handled by the ISP connected to WAN1, with the remaining 25% handled by the ISP connected to WAN2.

This configuration also means that in the event of a failure connecting to one ISP, all traffic will divert to the other WAN interface (this is commonly referred to as ‘failover’).

1. Connecting your ISPs to the FortiGate

Connect your ISP devices to your FortiGate so that the ISP you wish to use for most traffic is connected to WAN1 and the other connects to WAN2.

2. Modifying existing policies

You will not be able to add any interface to the SD-WAN interface that is already used in the FortiGate’s configuration. So, in this scenario, you must delete any security policies that use either WAN1 or WAN2, such as the default Internet access policy. Traffic will not be able to reach WAN1 or WAN2 through the FortiGate after you delete the existing policies.

It is also advisable to check for any other references to WAN1 or WAN2 and make the necessary modifications.

If you have many policies that reference WAN1 and/or WAN2, a simple method is to redirect those policies to unused ports, rather than delete them, to avoid having to recreate each policy from scratch. Obviously, you should redirect those same policies back to the SD-WAN interface once it is created.

Go to Policy & Objects > IPv4 Policy and delete any policies that use WAN1 or WAN2.

3. Creating the SD-WAN interface

Go to Network > SD-WAN.

Set the Interface State to Enable.

Under SD-WAN, add the two WAN interfaces.

Under Load Balancing Algorithm, select Volume and prioritize the WAN1 interface to serve more traffic.

In the example, the ISP connected to WAN1 is a 40Mb link, and the ISP connected to WAN2 is a 10Mb link, so we balanced the weight 75% to 25% in favor of WAN1.

To help visualize the effectiveness of the algorithm selected, the WAN Links Usage graph shows you the Bandwidth and Volume usage.

4. Configuring SD-WAN Status Check

You can optionally configure SD-WAN Status Check to verify the health and status of the links that make up the virtual WAN link.

This configuration uses the Ping protocol to verify the status of the SD-WAN.

Go to Network > SD-WAN Status Check and (if you wish to use Google) enter the values shown here.

5. Allowing traffic from the internal network to the SD-WAN interface

Go to Policy & Objects > IPv4 and create a new policy.

Set Incoming Interface to your internal network’s interface and set Outgoing Interface to the SD-WAN interface.

Enable NAT and apply Security Profiles as required.

Enable Log Allowed Traffic for All Sessions to allow you to verify the results later.

At this point, you should recover any policies that may have been redirected or deleted in Step 2 and point them to the SD-WAN interface.

6. Results

Browse the Internet using a computer on the internal network and then go to Network > SD-WAN > SD-WAN Usage.

You can see the bandwidth and volume of traffic traversing the SD-WAN interfaces.

Verify that Status Check is working by viewing the table at Network > SD-WAN > SD-WAN Status Check.

Go to Monitor > SD-WAN Monitor to view the number of sessions for each interface, bit rate, and more.

7. Testing failover

To test failover of the redundant Internet configuration, you must simulate a failed Internet connection to one of the ports. Do so by physically disconnecting the Ethernet cable connected to WAN1.

Verify that users still have Internet access by navigating to Monitor > SD-WAN Monitor. Note the Upload/Download of each WAN interface.

Furthermore, go to Network > SD-WAN > SD-WAN Usage to see that bandwidth and volume have diverted entirely through WAN2.

Users on the internal network should have no knowledge of the WAN1 failure. Likewise, if you are using the WAN1 gateway IP to connect to the admin dashboard, nothing should change from your perspective. It will appear as though you are still connecting through WAN1.

Reconnect the WAN1 Ethernet cable when you have verified successful failover.

For further reading, check out Installing a FortiGate in NAT/Route mode in the FortiOS 5.6 Handbook.

Keith Leroux

Keith Leroux

Technical Writer at Fortinet
Keith Leroux is a writer on the FortiOS 'techdocs' team in Ottawa, Ontario. He obtained a Bachelor's degree from Queen's University in English Language and Literature, and a graduate certificate in Technical Writing from Algonquin College. He spent a year teaching ESL in South Korea. Annyeong!
Keith Leroux
  • Was this helpful?
  • Yes   No
You can use any stable server that responds to ICMP requests, such as the ISP’s gateway. We recommend something with the fewest hops.
  • Kamil Filipiński

    How to set specyfic device to use always only WAN1 with SD-WAN?
    It is device to test health of ISP1 from WAN1, so it cannot change to ISP2

    • Victoria Martin

      Hello Kamil,

      When you create an SD-WAN interface, the FortiGate treats WAN1 and WAN2 (or any other ports added) as a single interface. To be able to configure WAN1 and WAN2 separately, this
      method of configuring redundant Internet isn’t going to work.

      There is another way to configure redundant Internet; however, we don’t have a 5.6 version of that recipe written yet. There is an older 5.0 version that might be helpful: https://docs.fortinet.com/uploaded/files/1646/using-two-ISPs-for-redundant-Internet-connections.pdf

  • Larry

    Good cookbook… only items I think is missing is the need to have both routes added and make sure the Interfaces are correctly configured (for example one of my connections is statically assigned by ISP and other is DHCP from the cable box). I ended up adding a second 0/0 route with a larger admin distance and then when the connection went down it worked. Did I mi-configure something?

  • Harshil Gupta

    I am using banking sites a lot. The FGT100E is configured with 3 ISPs and SD-WAN is created. Due to SD-WAN the banking sites terminate connection due to frequent change in WAN-IP. Is there any way to resolve this issue. Any suggestion ?

  • hardyanto

    How if SDWAN use 100% ISP1 and 100% ISP2 (all ISP are fully utilize)? And at the same time, it can do failover in case one of the ISP down.

    • bdickie

      If a failover occurs, then all of the traffic will attempt to use the ISP that is still connected. The amount of traffic would be limited by the available bandwidth so some traffic may be delayed or slowed until both ISPs are connected again. You could consider using traffic shaping to make sure that high priority traffic gets more of the available bandwidth.

  • Erick Mori Yamamoto

    I
    would like to know what are the requirements to enable this feature if
    the communication will become dynamic (closing VPN automatically between
    models with the same OS version) and from which box models support OS
    5.6.

  • Is there a way to use the SDWAN failover with an HA cluster of FortiGates?

    • bdickie

      Yes HA with SD-WAN is supported. If your cluster is already setup you just have to work out the redundant connections and then set up SD-WAN. If SD-WAN is already set up you can add the backup FortiGate to your network and configure HA. The complex part is setting up all the redundant network connections. We have plans to create a recipe showing this configuration some time this fall.

  • Dirk Damaso III

    i am having issues with my SSL VPN fortios5.4 i can connect to internal resources but no internet everytime i am connected to vpn.. i am planning to upgrade the firmware to 5.6, is there an option to create policy for SDWAN-SSL.root? in WLLB i cannot create..

  • Ahmed Oteify

    Hello ALL
    i need to know how these steps work without we create static route ?
    how fortigate will know to route to external destination 0.0.0.0 through SD-Link Interface
    i created all these steps but no traffic forward until i created static route

    Thanks All

  • Tanguy Le Loch

    Hi !
    Thanks for your article…
    But how to just do a failover?

    Will it still be work if I uodate firmware? (without using SDWan)?

    Thanks

  • bdickie

    We have received many requests for a basic failover redundant internet connections example so its on our To Do list. Until it actually happens, here is some information about how to set it up.

    You now setup the basic failover configuration from the CLI only. It is also referred to as dead-gateway detection or link-monitoring.

    See this link for details about the command:

    http://help.fortinet.com/cli/fos50hlp/54/index.htm#FortiOS/fortiOS-cli-ref-54/config/system/link-monitor.htm

    Here is the configuration:

    config system link monitor

    edit wan1

    Echo1 (wan1) # get
    name : wan1
    srcintf :
    server :
    protocol : ping
    gateway-ip : 0.0.0.0
    source-ip : 0.0.0.0
    interval : 5
    timeout : 1
    failtime : 5
    recoverytime : 5
    ha-priority : 1
    update-cascade-interface: enable
    update-static-route : enable
    status : enable

    Typically you would configure the link monitor for both interfaces.
    Set the Distance to be equal on the routes for each interface.
    Set the priority to be higher, on the redundant interface.

    To test the feature, remove/disconnect the cable upstream.

    This configuration requires you to create redundant routes and redundant firewall policies. See this old FortiOS 5.0 recipe for the GUI steps http://docs.fortinet.com/uploaded/files/1646/using-two-ISPs-for-redundant-Internet-connections.pdf

    • Casey Phillips

      @bdickie:disqus So would this sorta setup also still allow you to allow SSL-VPN traffic to browse out on one of the lan connections. Secondly do you need to be bread the SD-WAN pairing before setting up the two in this configuration in order to get it to work? If I do break up the pair I think I will have no problem setting up the polices allows my VPN user to browse out on either WAN connection, but when they are paired I can’t add the SD-WAN (in my case still called the LLB) to any polices.

    • Casey Phillips

      @bdickie:disqus So would this sorta setup also still allow you to allow SSL-VPN traffic to browse out on one of the lan connections. Secondly do you need to be break the SD-WAN pairing before setting up the two WAN connections in this configuration in order to get it to work? If I do break up the pair I think I will have no problem setting up the polices allows my VPN user to browse out on either WAN connection, but when they are paired I can’t add the SD-WAN (in my case still called the LLB) to any polices.

      I’m still on 5.4 so perhaps the above issue is corrected in the new versions of 5.6 where I can add the linked pair to policies but wanted to get confirmation before I upgraded.

      Thanks for the help!

      • bdickie

        Yes this sort of setup allows you to configure separate policies for each of the LAN connections. Also, this setup is not compatible with SD-WAN. You can’t have both of these set up at the same time using the same interfaces. This is true for 5.4 and 5.6.

  • Markone

    Hi Keith,
    thank you very much for this helpful article.

    I was able to setup the SD-WAN. Failover is OK but load balancing seems not to work.
    If WAN1 fails all traffic goes correctly through WAN2 but normally all the traffic goes through WAN1, even if the weight is set. Of couse failover is the most important thing but if possible I would be able to use load balancing too.

    Thank you very much again.

    • Ahmed Oteify

      Hello
      Any updated about weight and load balancing issue, because i have the same problem
      Thanks

  • Ondrej Rolien Dvoulety

    set the distance on the inteface for the secondary connection to a higher value. this will enable the failover without load balancing

  • Enrique Montero

    I can´t delete the rules of SD-WAN, Who can help me?

  • Simon

    Hi there, what kind of routing protocol does it support ? It seems that not work with bgp

  • Anders Kortsen

    Hi Rajarshi and David
    I am trying to acomplish the “only failover without any load balancing” solution but it is not working. I tried the workaround David proposed before I found this SD guide.
    David: Did you manage to get a hold of the Fortinet support and discuss the case with them?

    • Jerry

      Is there a way to do only fail-over with no load balancing in 5.6.1 ?

  • malik nedjm

    Hi,

    I deleted all the policies but I still can’t use the My wan 1 and wan 2 interfaces. I even deleted the Routing POlicy but same thing. I have a lot VPN Connections. Do you think in need to delete all the vpns depending on WAN1 and WAN2?

    • Keith Leroux

      I believe that’s correct–or at least you’ll have to point those VPNs to other interfaces until the sd-wan setup is complete, then you can point them back.

      • malik nedjm

        Hi, thank you for your reply.
        Your article helped me a lot understanding how to implememnt the LB.

        But I still have the issue, I have too much VPN IP SEC connections, MPLS and Policies. I am not willing to go one by one and change the interfaces name, do you know any method to change the interface at once it would help a lot? Or any hack to do it?

        Last time i worked late night because i can’t interrupt internet Availability :/.

        Thank you keith for your help.

        Regards.

  • Mohammed Fathi

    Any news about Ip Sec VPN Recipe over SD-Wan ???

    • Keith Leroux

      Hi Mohammed,

      I’m working on it! Thanks for checking in~

      • Gregor

        I need this… When is the recipe ready?

        • Keith Leroux

          Hi Gregor, it will be posted very soon. It’s scheduled for testing and review today.

          • Mohammed Fathi

            Waiting for this recipe….

          • Gregor

            +1

          • Adriano

            Me too… any progress ?

          • Keith Leroux

            Sorry all, the VM lab I was testing on went down, so this recipe is delayed for the time being. Hope to have it back soon!

          • Adriano

            Keith, while you do not have it, can you give us a help? My
            problem is that I followed all the steps in the cookbook for VPN L2TP /
            IPSec for windows clients (dial up) but at the time of creating the
            firewall policy, when selecting the action type for “IPSEC” I can not
            select the configuration of IPSEC phase 1 created previosly. There are no entries (I’ve also tried via CLI without success). Any tips? Fortigate 51E Firmware 5.6.0. Thanks

          • Gregor

            it’s time to get a recipe Keith

          • Keith Leroux

            I agree! Unfortunately, due to the lab being down and other priorities (5.6.1 and CLI Reference updates), I had to shelve this. Please accept my sincere apology on this delay.

          • Dean Tree

            Now that 5.6.1 is out for 24 hours or so…….could we get a beta copy of the recipe?:)

          • Gregor

            5.6.1 is available since 2 weeks and still there is no info 🙁

      • Victoria Martin

        We are still working on the IPsec VPN with SD-WAN recipe; however, due to circumstances beyond our control, we do not know when this recipe will be available.

        • Casey Phillips

          Victoria, can you please at least let me know if this is possible in the current firmware (6.1)? I can probably figure it out but as of now on 5.4 I can’t select the LLB on policy routes screen so I’m a bit stuck. I can update the unit but don’t want to do it unless I’m sure its at least possible currently, regardless of ya’lls instructions being ready.

  • David

    I’m guessing you could just use the load balancing option and weight the volume 100% on the primary WAN connection and 0% on the secondary WAN. You then would use the SD-WAN Status Check for failover. I have this exact same need and will be testing this solution tomorrow.

    • Patrick

      Hi, did this solution work?

  • Adi

    I have lost GUI for the WAN Link Monitoring in 5.4.x . Is this where we find it now ?

  • Joe

    Is there not a need to specify a default route anymore, or is that just missing from this recipe?

    • Einer

      do you have tested it? when i try to created the default route, it says “Error: You cannot have duplicated routes on SD-WAN and non SD-WAN interfaces.”

      • Joe

        When I created mine, I had to specify a default route or I wasnt able to get out. I did not get the error that you received. Not sure why

  • V-P

    I tried to get SD-WAN working with my FGT30E 3g4g but it seems that traffic cannot be routed over wwan (DHCP). Isn’t that scenario supported? I wanted to try this as normal link monitoring and using static routes with different priorities (same distances) there is still SNAT problem as existing session will stay alive as long as there is active route in routing table. So if WAN1 (primary) is down, traffic will flow over WWAN (secondary) but when WAN1 comes back there is still active route over WWAN (secondary) and existing sessions will use that and not WAN1 (primary).
    Also is there any recipe how to get route based IPsec site to site tunnels working in dual WAN scenario?

  • Unai Falagan

    Hello. With SD-WAN is possible to use to ip nat pools? We need to do an ISP load balance of two isp link with one ip nat pool each one.

    Regards

  • Keith Leroux

    Hi Peter,

    This configuration includes failover. Health Link is now located under SD-WAN Status Check.

  • Victoria Martin

    Hello Peter,

    I have added this topic to our to-do list.

  • Victoria Martin

    Hi Alexander,

    The WWLB feature has been re-named to SD-WAN, so you are correct that it is essentially the same feature. However, new functionality has also been added to it in FortiOS 5.6, which will be featured in future Cookbook recipes.

  • nirmal roshan

    Hi Keith,

    In this kind of set up how will I be able to set up port forwarding to an internal web server, or how can i do a VPN configuration.

    • Keith Leroux

      Hello nirmal roshan,

      We are currently working on an IPsec VPN over SD-WAN recipe, so stay tuned. I’ll see if I can add port forwarding to the docket!

      • Yoann

        Hi,
        I just setup my fgt with sdwan, but we can’t use port forwarding to an internal server.
        Virtual IP allow us to select WAN1, WAN2 but no SDWAN
        Rules won’t allow to select WAN1 or WAN2 virtual IP
        We simply loose our messaging server and all connexion from outside.
        Will the next firmware allow us to put this in place ?

        • Rodrigo de los Santos

          you just need to create your VIPs in the correct WAN interface and when you create the policy you need to use SD-WAN as SRC Interface.

          • Yoann

            Thank you a lot for your reply. I try again and it’s ok now.

      • Jean-Francois Gauthier

        Hi Keith,

        Any news on that IPsec VPN over SD-WAN recipe ??
        Thanks
        JF

        • Joe

          Looking for this as well

      • Jasper

        Anny news on the combination with IPsec VPN?

  • Ian

    In Step 4 you cite using Google’s public DNS server, 8.8.8.8. However, they have recently been more stringent about allowing this and in some cases have started filtering ICMP traffic to those DNS servers making them unreliable for these kinds of tests. One possible source for this is: https://groups.google.com/forum/#!msg/public-dns-discuss/p1o62SJElck/w0flYsmqBQAJ

    • Keith Leroux

      Hi Ian,

      Thank you for the excellent information! As the asterisk in Step 4 indicates, you can use any stable server that responds to ICMP requests. We recommend one with the fewest hops.