Using Secondary Addresses with VRRP

VRRP on the MDS in it’s basic form looks something like this:

Switch 1

In the above configuration, 192.168.10.2 is the actual physical interface address and 192.168.10.2 is also the VIP address.  This is common.  The other side may looking something like this:

Switch 2

On this side of the link 192.168.10.3 is the physical interface address and 192.168.10.2 is the VIP address.  These two MDS switches Switch 1 and Switch 2 are both members of VR10.  As per the VRRP specification, VRRP VIP addresses are for passing traffic onto real server IP addresses that they front end, they are not for the origination or destination of traffic.  If you try to send traffic to a VRRP VIP, these packets are by default dropped.  Some application’s on the  MDS may need to use the VRRP IP as an actual IP address to terminate traffic on.  For example iSNS (removed from current versions of SAN-OS / NX-OS) and IPSec.  With IPSec you may wish to use the VRRP as a destination of a IPSec tunnel for high availability.  To do this you have to add the “secondary” option like so:

Switch 2

Realize on Switch 1, the primary IP address of G2/1.10 is configured the same as the VIP.  So in this case when Switch 1 is master it will be able to accept traffic terminated to the VIP address.  Switch 2 will not be able to receive traffic destined to the VIP address even with the secondary option configured unless it is the master!  So the key is, the secondary option allows a switch to be able to terminate traffic destined to the VIP, when it’s the master.  If we were terminated an IPSec session on the VIP of Switch 1 and Switch 2, and wanted it to failover, we would need to configure the secondary address for Switch 2.

Also realize this has an effect on the switch VRRP priorities.  A switch that uses the same VRRP address as its interface address automatically has a VRRP priority of 255.  A switch using a different address for its VRRP than it has on its interface has a priority of 100.  This applies to the use of the secondary command as well.  With or without the secondary command the interface will have a default priority of 100 if its interface address does not match the VRRP address.

Also realize, you cannot configure a secondary address to be the same as the interface address, if you try to do so you will get an error:

You can test the functionality of the secondary address option by telnetting to the VIP.  Obviously on Switch 1 you can telnet to the VIP regardless since a) it is the master because of it’s higher configured priority and b) It’s interface IP address matches the VRRP VIP address.  But on Switch 2, configure the secondary address, bring down Switch 1’s VRRP interface peer, and you will see master switch to Switch 2.  Then you should be able to telnet to the VIP of Switch 2.  You will not be able to telnet to the VIP of switch 2 unless it’s the master and it has secondary address configured.
This entry was posted in CCIE Storage, VRRP and tagged , , . Bookmark the permalink.

13 Responses to Using Secondary Addresses with VRRP

  1. livelybrowsers says:

    Thanks for good stuff

  2. roclafamilia says:

    Helpful blog, bookmarked the website with hopes to read more!

  3. badmash says:

    I just signed up to your blogs rss feed. Will you post more on this subject?

  4. brian says:

    Are you talking specifically about VRRP on the MDS or on MDS in general? I can talk a bit about whatever subjects you find beneficial, just let me know.

  5. Michael says:

    Hi Brian, How do you telnet to the gigE interfaces? I am able to ping the gigE interfaces but can telnet only to the management interface.

    • brian says:

      You have to configure the “secondary” keyword. If you do so, the interface will accept connections to the gig interface when it is the master. This may be necessary for some services such as iSNS. So 1) it has to be the master, and 2) you have to configure the “secondary” keyword as I describe.

      Obviously from the host your are telnetting from you also need routing to be there, so you must have a route that will take you to the gig-E interface and routing that will allow the MDS to return the traffic to you.

  6. Michael says:

    Hi Brian,

    I tried using the secondary option. Then to rule out vrrp ,I tried using a normal gigE interface without any vrrp config.
    still I can’t telnet. Routing is in place because I am able to ping the gigE interface. I came across the below text in CLI config guide so I was wondering if you had to do some specific configuration to get telnet working on gigE interface or may be it is specific to some modules?

    “A new port mode, called IPS, is defined for Gigabit Ethernet ports on each IPS module or MPS-14/2
    module. IP storage ports are implicitly set to IPS mode, so it can only be used to perform iSCSI and FCIP
    storage functions. IP storage ports do not bridge Ethernet frames or route other IP packets.”

    Thanks for the quick response

    • brian says:

      Michael,

      You need to have VRRP enabled, you have to use the secondary keyword, and you must be telnetting to the VRRP master. Can you show me your VRRP configuration and the output of the VRRP status?

      Also are you using iSLB? Also are you trying to telnet to the VIP address or physical interface address?

      Brian

      • brian says:

        Also realize that iSLB does have some strange glitches/bugs. A common thing that needs to be done to “fix” iSLB when its acting weird is:

        islb commit
        no islb vrrp load-balance
        islb commit
        islb vrrp load-balance
        islb commit

        You would substitute the appropriate vrrp group where I have . So when things with iSLB are acting strange remember to do this.

        Brian

  7. Michael says:

    mds2(config-if-vrrp)# do sh run int gig1/1.101
    version 3.3(3)

    interface GigabitEthernet1/1.101
    ip address 10.0.1.3 255.255.255.0
    no shutdown
    vrrp 101
    address 10.0.1.1 secondary
    no shutdown

    mds2(config-if-vrrp)# do sh vrrp
    Interface VR IpVersion Pri Time Pre State VR IP addr
    —————————————————————————
    GigE1/1.10 10 IPv4 100 1 s backup 192.168.10.2
    GigE1/2.10 10 IPv4 100 1 s master 192.168.10.2
    GigE1/1.20 20 IPv4 100 1 s backup 192.168.20.2
    GigE1/2.20 20 IPv4 100 1 s master 192.168.20.2
    GigE1/1.101 101 IPv4 100 1 s master 10.0.1.1

    C:\Users\Administrator>ping 10.0.1.1

    Pinging 10.0.1.1 with 32 bytes of data:
    Reply from 10.0.1.1: bytes=32 time<1ms TTL=254
    Reply from 10.0.1.1: bytes=32 time<1ms TTL=254
    Reply from 10.0.1.1: bytes=32 time<1ms TTL=254
    Reply from 10.0.1.1: bytes=32 time

    C:\Users\Administrator>telnet 10.0.1.1
    Connecting To 10.0.1.1…Could not open connection to the host, on port 23: Connect failed

  8. Michael says:

    I am trying to telnet to VIP address.
    I am not using islb. however I think i had tried it with islb on too.

    • brian says:

      Do you have any ACL’s applied? Anything like that? It should work. Config looks good. SAN-OS can be weird so you may wish to restart the box to see if that “fixes” it but it should accept telnet.

Leave a Reply