New Years Technical Blast!

·         General NetApp:

o   2040’s can only run up to ONTAP 8.1.4

o   2240’s can’t run below 8.1

o   8.2 licenses are worthless in 8.1, but you can generate temp licenses easily here: http://support.netapp.com/NOW/download/special/evaluation.cgi?

o   If your telnet connection dies right as you log in, there’s a stale connection.  Kill it with “logout telnet”

·         Snapmirror:

o   When snapmirror initialize gives you a “network error,” it might mean the source volume doesn’t exist/is misspelled/is too large.

o   “sysstat 1” is a quick easy way to view the throughput of a system

o   Options replication.throttle.enable combined with options replication.throttle.incoming/outgoing is a quick, easy way to turn down snapmirror traffic.  The value is in kilobytes/s, so “50,000” = ~50MB

o   A single 1Gb link can transfer as much as 130MB/s if the disks can handle it

o   Reverting a snapmirror destination is a very, very tough process.  Avoid it.

·         ISCSI:    

o   There’s a tool that will generate a script to re-set LUN serial numbers.   Ntstp.netapp.com, at the top menu bar under tools.

o   LUN ID’s, LUN SN’s, and iqn nodenames matter.  Luckily all are easy to set.

o   If you’re updating ONTAP to 8.2, Oracle Linux-based systems using ASMlib can crash entirely.  There’s an easy workaround, see page 61 in the 8.2 release notes.

·         CIFS:

o   CIFSCONFIG_SETUP.CFG contains the list of shares and permissions on the system.  You can just copy that to a new system to emulate them.

·         Bonus: you can separate snapmirror streams by editing your snapmirror.conf file.  Format:

Replication_stream1 = multi(first ip address of source, first ip address of destination)

Replication_stream2 = multi(second ip address of source, second ip address of destination)

Then for each snapmirror relationship, replace the source system’s name with the replication stream’s name.  Example:

rep1 = multi(172.16.0.7,172.16.0.5)

rep2 = multi(172.16.0.8,172.16.0.6)

rep1:volname destnetapp:volname_data kbs=5000 0 1

rep2:volname destnetapp:volname kbs=5000 0 1

·         Double bonus: Here’s a list of other important config files in the system.  Check them out!

KRB (DIRECTORY)
CIFSSEC.CFG
CIFSCONFIG_SHARE.CFG
CIFS_HOMEDIR.CFG
CIFS_NBALIAS.CFG
CIFSCONFIG_SETUP.CFG
EXPORTS
FILERSID.CFG
GROUP
HOSTS
HOSTS.EQUIV
KRB5.KEYTAB
KRB5AUTO.CONF
LCLGROUPS.CFG
NSSWITCH.CONF
PASSWD
QUOTAS
RESOLV.CONF
USERMAP.CFG

iSCSI

Here's a few concepts I've been studying.  A TPG (Target Portal Group) is basically a method of allowing the server to communicate to your storage system via iSCSI on multiple interfaces and multiple connections on a single session.  This means you can enable MPIO by having multiple TPGs.  Here's the dummy breakdown:
- Each interface (virtual or real) can only be part of one TPG
- Each TPG can have multiple interfaces
- Each session can have multiple connections
- Each connection can only be in one session.
- Each session can only communicate through one TPG

A great use of this is when a server has multiple virtual OS's and therefore needs multiple connections to the same storage system.  If you have multiple paths, you need ALUA.

When understanding ALUA (Asymmetric Logical Unit Access), it's helpful to know it's also called Target Port Group Support.  Basically it's a protocol for determining the best path from the server to the LUN (hence the LU in ALUA).  This protocol is standardized to work with any vendor's iSCSI hardware.

ISNS is basically DNS for iSCSI, but a little smarter in that it also understands TPG's and helps systems find each other that way.

More data: http://hd.kvsconsulting.us/netappdoc/801docs/html/ontap/bsag/GUID-FF148B6E-6CCB-48CC-9547-7D063A904B40.html

SAN protocols for dummies

Just doing a brush up on the basic storage concepts. Noobs will appreciate the simplification in this post, experts will likely find it too oversimplified. More detail can be found in the links or more recent posts!

For reference, the OSI model:
7. Application
6. Presentation
5. Session
4. Transport
3. Network
2. Datalink
1. Physical

Quick hits*1:
SCSI, SATA, FC, and SAS are layer 1 and 2 protocols. iSCSI is a layer 5 protocol. Ethernet is a layer 2 protocol. FCP is a layer 1, 2, and above protocol. FCoE is a layer 1-6 protocol.

Photo Courtesy of FCoE.ru

Local protocols: these are how the CPU communicates to the hard drives.  They are all layer 1 and 2 protocols specifying the hardware and electronic signals needed to send data between the drive and the CPU.  This has trended toward serial protocols (away from parallel) for performance and cost reasons.

1. SCSI (Small Computer System Interface): a high performance parallel standard that specifies hardware level communication over a local BUS.

2. SATA (Serial Advanced Technology Attachment): Slow, inexpensive.  Used mainly for unimportant, low change data.

3. Fibre Channel: Fast and expensive, this serial protocol is commonly used in enterprise SANs.

4. SAS (Serial attached SCSI): The SCSI protocol was modified to take advantage of cost and speed improvements in serial technology.  This is also commonly found in enterprise SAN's.

5. FATA (Fibre Attached Technology Adapted): Slow, inexpensive.  Really is SATA wrapped in a FC interface to gain from shelf technology.   This is less used than the other types.


Network Protocols: This is how a server can communicate over your SAN to its storage, essentially virtualizes the relationship between a computer and its local drives, allowing your server to think a virtual drive in a datacenter somewhere is actually directly plugged into it.

1. iSCSI:  Protocol simulates the SCSI protocol by wrapping it in ethernet-friendly packets.
Advantage: Works over existing ethernet networks (if given enough bandwidth).
Disadvantage: Some risks involved with having all your traffic on the same cables. Theoretically high overhead since it's higher up on the stack.

2. Fibre Channel Protocol: protocol that requires special fibre cabling and an entire alternate network to support communication.
Advantage: Can be fast, separates traffic and enhances stability.
Disadvantage: Expensive, requires special cabling.

3. Fibre Channel over Ethernet: FCoE. Protocol simulates the FC protocol by wrapping it in ethernet-friendly packets.
Advantage: Can be very fast for cheaper than FC.
Disadvantage: Theoretically more overhead. Doesn't separate traffic.


Sources:
You can find a great performance discussion here:
http://jmichelmetz.wordpress.com/2010/03/24/fcoe-vs-iscsi-the-cagefight-performance

*1: FCOE Discussion:
http://www.fcoe.ru/index.php?option=com_content&task=view&id=296&Itemid=65&lang=english

FC Discussion
http://bit.ly/dQkn8h