NetApp Insights: Usable Capacity

I saw some documentation given to a customer that estimated that for 144 2TB SATA disks (294.9TB), the customer could expect 170TB usable. It also said for 288 450GB SAS disks (126.5TB) they should expect 91TB usable. That's a big loss from a client's perspective.

I've previously developed a calculator to make it easy to plan your RAID Groups and aggregates, but now I want to use that to take a closer look at where all that space actually goes. A NetApp PSC expert told me the general rule is for FC disks 70% of raw is usable, and for SAS/SATA you take off another 10-12%. But let's see if we can dig into that.

• Computers measure base 2, but drive manufacturers measure base 10.  This means if your drive is labeled 1GB, it's actually 1000MB, not 1024MB.  
• The fuzziest part: drive manufacturers reserve between 7% and 15% on each disk.  Some of this is for parity, a lot of this is to account for failed sectors.  I've observed 2TB SATA drives reporting 1.69TB or less for a loss of 13.5%, I'll use that for these calculations.
• You lose some space due to WAFL/disk asymmetry.  The basic idea is that a 4KB block doesn't fit neatly into the disk sectors, so there's some waste.  Some of this is taken into account by the manufacturer's reserve, so I can't quantify this in our calculations.
• You lose some space to right-sizing.  Since each drive manufacturer's 2TB disk is a slightly different size, ONTAP right-sizes all disks to the lowest common denominator to avoid incompatibly sized disks later.  I can't find any data on how much space you lose to this process.
• For every RAID Group, you lose 2 disks to parity/double parity.
• You need to account for spares obviously.
• WAFL requires 10% of the usable space to run the file system.

So for our two scenarios above, here's what we find:
Scenario 1
288 450GB SAS Drives
Spare drives: 8
Parity drives: 28

Credit: me!

Scenario 2
144 2TB SATA Drives
Spare drives: 6
Parity drives: 20

Credit: me!

Analysis:

• You lose a consistent 15% because of the drive manufacturer whether you use EMC or NetApp or any other vendor.
• To accomplish NetApp's goal of data protection (spares, parity, WAFL striping), you lose another 18-23%.
• When you factor in backups and snapshots, you'll lose even more space.
• One bright side is that using NetApp's dedup and efficient snapshot technologies, you can end up regaining this lost space.

Notice I'm still a considerable way away from the estimates given to the customer: 7.8% low for the 450GB system and 5% high for the 2TB system.  There's still some gaps in my numbers here, I would definitely appreciate any tips!

NetApp Training Brain Dump: Snapshots

The concept here is that a snapshot can become as large as the original dataset in the volume (100%).  Remember that the space occupied by data in the volume is the sum of the existing LUNS/Qtrees and any snapshots that exist in that volume.  Empty space in the volume is ignored by snapshots.


Here's the important background idea: WAFL does not update-in-place when existing data is changed.  This means that for a normal LUN that has no snapshots, when data changes, it is written to a new location (total space occupied increases) and then the old data is deleted (total space occupied goes back to pre-change levels).

Illustration: If in a LUN with 6GB of data a 4KB block is changed, the sum total of space occupied by data rises to 6GB + 4KB, then back to 6GB as the out of date 4KB block is deleted and reclaimed.  WAFL handles this so quickly that your LUN effectively does not increase in size.  This is a great advantage for WAFL because update in place can cause data corruption.

This concept is essential to understanding how snapshots work in ONTAP.  Let's go back to our 6GB LUN with a 4KB change: WAFL writes the new 4KB data to new, unoccupied space and the snapshot is left occupying the space that would be otherwise deleted.  So as data changes, it is not actually the snapshot that is allocated more space, but its existence means that the space that could be reclaimed is now solely assigned to the snapshot.  So any data that is only assigned to the snapshot is considered occupied by the snapshot.  In this example, the snapshot would be considered to be 4KB in size.


If you're a visual learner like me, checking out this diagram will help you picture the concept.  


The size taken up by the snapshot increases in concert with the changes to the original LUN: 500MB of changes to the original LUN means that the snapshot will grow from 0 to 500MB in size.  For 20GB volume that has 6GB of data (including LUNs and other snapshots), the next snapshot can grow as large as 6GB, making the sum total of the original data and the new snapshot 12GB.  


You can find commands to control snapshots here.