NetApp Experience: Networking

For reasons unknown, a disproportionate amount of the trouble tech people run into is network.  I encountered some of these issues recently that forced me to take a close look at the specifics of ONTAP networking.  Here are some important details:

• Make sure the switch ports are hard-coded for LACP if you are creating LACP interface groups (vif's) in ONTAP.
• These are the options for status:
• Up: the link is sending and receiving data.
• Down: the link is down but believed to be operational.
• Broken: the link is inactive and believed to be non-operational.

Here's a problem I ran into, see if you can spot the issue.  Upon boot, ONTAP presents these messages:

vif: Cannot create a multi-level 802.3ad compliant vif: vif1
vif: Cannot create a multi-level 802.3ad compliant vif: vif2
vif: vif1 cannot create multi level 802.3ad vif
vif: Failure adding vif1. Continuing with other interfaces
vif: vif2 cannot create multi level 802.3ad vif
vif: Failure adding vif2. Continuing with other interfaces

Here's some more hints:

rdfile /etc/rc
#Auto-generated by setup Thu Jul 14 19:22:30 GMT 2011
hostname NAME
vif create lacp vif1 -b ip e0a e0b
vif create lacp vif2 -b ip e0c e0d
vif create lacp supervif1 -b ip vif1 vif2
ifconfig supervif1 `hostname`-supervif1 mediatype auto
netmask 255.255.255.0
route add default 10.18.33.1 1
routed on
options dns.domainname acme.corp.com
options dns.enable on
options nis.enable off
savecore

ifconfig -a
e0a: flags=0xa508866 mtu 1500

        ether 00:00:00:00:00:00 (auto-100tx-fd-cfg_down) flowcontrol full
        trunked vif1
e0b: flags=0xa508866 mtu 1500
        ether 00:00:00:00:00:00 (auto-unknown-cfg_down) flowcontrol full
        trunked vif1
e0c: flags=0xa508866 mtu 1500
        ether 00:00:00:00:00:00 (auto-unknown-cfg_down) flowcontrol full
        trunked vif2
e0d: flags=0xa508866 mtu 1500
        ether 00:00:00:00:00:00 (auto-unknown-cfg_down) flowcontrol full
        trunked vif2
lo: flags=0x1948049 mtu 9188
        inet 127.0.0.1 netmask-or-prefix 0xff000000 broadcast 127.0.0.1
vif1: flags=0x22408862 mtu 1500
        ether 00:00:00:00:00:00 (Disabled virtual interface)
vif2: flags=0x22408862 mtu 1500
        ether 00:00:00:00:00:00 (Disabled virtual interface)
supervif1: flags=0x2354b863 mtu 1500
        inet 10.18.33.33 netmask-or-prefix 0xffffff00 broadcast 10.18.33.255
        ether 02:a0:98:2c:01:9c (Disabled virtual interface)

Solution: You can't create multi-level multi-mode or LACP vifs.  Essentially, if you have two vifs that are load balanced, you can't create a load balanced vif out of those two vifs.  This is documented as an unsupported configuration by netapp.

https://kb.netapp.com/support/index?page=content&id=3011251 (requires NetApp login)

So what to do?  Simple!  Just create one big multi or LACP vif out of the ports you wanted to use in the first place.