Clustered ONTAP Networking

Basic

• ifgrps are made up of physical ports.  ifgrps are to be named "a," e.g. a0a, a0b.
• All members of a ifgrp must have the same role set (data, mgmt)
• Cluster ports cannot be in ifgrps
• A port with its own failover designation cannot be added to a ifgrp
• All ports in an ifgrpmust be on the same physical node
• A lif (logical interface group) is basically the IP addresses assigned to an ifgrp.  That IP address entity is given other properties like "home port," VLAN, failover group, etc.  Each IP needs to be able to have these properties independent of other IP's.
• Spanning tree should not be enabled on the switch for any ports connecting to the controllers except the management ports.
• It is a best practice to set flowcontrol to none for all ports except UTA ports.
• UTA ports set to full.
• You can of course add VLAN tagging and multiple IP's per LIF, named "-," e.g. a0b-441
• Each LIF gets a routing group that you may or may not want to alter.  
• You'll need to create a default routing group (default per vserver) for LIF's to default to.

Once you have those concepts figured out, you can move on to the advanced.

• For snapmirror/snapvault, each node needs its own "intercluster LIF" to identify which ports and IP to be used for replication.
• You'll need inter-cluster routing groups on each node to set up replication.
• You have to create a cluster peer to establish a relationship for replication.
• Network Interface Failover Groups are simply the cluster-wide list of ifgrps whose LIFs can migrate over to each other.  For example, you don't want your 10Gb NFS LIF failing over to another controller's 1Gb port ifgrp.
• Your node management LIF can't fail over to other nodes (obviously, it's the portal to your physical node).  So it should have its own failover group that contains only local ports.
• You want to be cognizant of not including e0M on any failover groups with data ports.

Some examples of real life setups:

• A vserver (virtual storage machine) that was strictly SAN would need only one IP address, assigned to a physical port for management.