Using IOS to create a “fatpipe” type service

Although less common today, it use to be quite common for people to want to take two disparate carrier type connections, and make use of them as some sort of “fatpipe” type service.  This can be done using CEF and some NAT wizardry.  It is preferable to use some other methods, such as OER/PFR if possible, however in some cases you are dealing with certain constraints where “quick and dirty” is what is needed.  It’s important to note that the service is not aggregating at the packet level, so you do not get the aggregate amount of bandwidth between the connections available to a single flow. Instead it is balancing the traffic at the flow level, and each flow can use no more bandwidth than it’s single egress connection has available.

This is accomplished primary using CEF, as CEF will do deterministic hashing for its next-hop.  Without CEF, you could have bad things happen such as a NAT session could timeout, and a new session created over a different egress link.  This would break things like online shopping carts, statefull connections, etc.

Conditional NAT is used on both egress connections.  If a connection is lost, CEF will rebalance the hash over the remaining connections.

This only works for balancing outbound traffic.  You could try to do some inbound balancing using DNS round robin with the appropriate NAT statements.

In order to manage the device itself remotely (telnet, ssh, etc) you must use local policy routing.

You can see in the above interface configuration, policy based routing is being used.  This is because the server referenced in the map has a static NAT IP address.  Any server such as this will not be able to take advantage of the multiple connections and will effectively be “pinned” to the single connection from which it’s IP comes from.

To remotely reach the router, we must make sure the IP we are managing it on always goes out the interface that it was assigned from (If its an IP from ISP #1 it must take that path, if its an IP from ISP #2 it must take that path. In this example I am managing it using the IP address which is from ISP #1 so I must make sure local policy uses that path.

For ISP #2 we use a pool of one address, for ISP #1 we are doing interface overload

If we have a packet trying to exit ISP #1 interface, based on CEF next hop, use an egress IP from ISP #1

If we have a packet trying to exit ISP #2 interface, based on CEF next hop, use an egress IP from ISP #2 is our server, it runs a webserver, inbound and only reachable from ISP #1

Dual defaults, let CEF build a hash and load share traffic over the two external links

Make sure you static route the DNS Servers if you use external DNS from one of your providers, even though may be reachable from any ISP, its only going to allow recursive lookups from IP’s they assigned, which is why we have to make sure we hit that DNS from one of ISP #1’s IP’s.  To eliminate this single point of failure, run your own internal DNS with a root cache file

In the ACL’s deny the server IP, as it has its own rules (uses ISP #1 always)

Any local management (telnet, etc) use ISP #1, as we are managing from that IP, if ISP #1 goes down we will not be able to reach the router for management, there are ways around this, easiest being try an ISP #2 IP as a second try

server policy route-map, we only have 1 server, this is so inbound connection to the server don’t get screwed up

Route-maps to ensure we use correct interfaces

As I said this definitely has its limitations, but is an interesting quick and dirty approach to getting the functionality of using two circuits simultaneously and even allows for a certain amount of resiliency/failover.

This entry was posted in CCIE Routing and Switching, Cisco and tagged , . Bookmark the permalink.

Leave a Reply