Storage reclaim can be a major painpoint for organizations running traditional three tier architecture. With companies trying to deal with an ever increasing storage footprint, many of them turn to manual storage reclaim procedures to get storage capacity back after files or virtual disk images are deleted. These problems are exhuberated if you are running vSphere as your Hypervisor on traditional VMFS formatted LUN’s, as any Storage vMotion or Snapshot operation’s results in space not automatically being reclaimed from the source LUN’s. An example of the manual procedures required in running reclaim on traditional infrastructure running vSphere 6 is outlined below.
- If EnableBlockDelete advanced feature is not enabled on the ESXi host to identify disks as thin, block zeroing software must be run at the guest level using software such as sdelete. This will create I/O overhead so should be run during a maintenance window
- Next, after the storage is reclaimed at the guest level, the VMFS LUN must be reclaimed by running esxcli storage vmfs unmap on each LUN manually via the esxi command line. Due to performance overheads, this task is not scheduled automatically.
- Finally, the storage needs to be re-claimed at the array level by executing a command to reclaim zero pages. On some arrays this is automated, but not all.
As you can see this is a manual three step process that creates operational overhead. Whilst these procedures could potentially be automated, I don’t really see the point in automating operations on legacy infrastructure when you instead have the options today of replacing your infrastructure with smarter software-defined solutions that automate themselves. This provides one less thing for your system administrators or automation team to worry about.
Earlier, I blogged about SCSI unmap capabilities using Acropolis Block Services on Acropolis Base Software ver 4.7 HERE. The good new’s is this functionality is also built into Nutanix’s own free Hypervisor, AHV. This works by the virtual disks issuing SCSI unmap’s directly from the guest level through to the Nutanix Distributed Storage Fabric.
As you can see below, using acropolis base software less than version 4.7 shows the disks as Standard Hard Disks in Windows 2012 R2. The reason for this is previously the unmap commands were not translated from the guest OS down to the virtual disk layer.
After you upgrade to Acropolis 4.7, all disks are shown as Thin Provisioned in disk optimizer as it now supports unmap. For the disks to show as thin, the VM first requires a reboot after acropolis base software is updated to 4.7. If a disk is hot-added to the VM before reboot, the new disk will show as thin. Either way, the change is transparent.
To show SCSI UNMAP in action on Acropolis Hypervisor, I tested across three Jetstress Exchange 2013 VM’s full of 4.7TB of Databases
After the databases were deleted, Windows converted the file delete operation into a corresponding SCSI UNMAP request as per the following MSDN article. https://msdn.microsoft.com/windows/hardware/drivers/storage/thin-provisioning
To hurry it up, I also manually ran an optimize operation using the optimize drives utility to reclaim space. Whilst file delete worked, I noticed the process was slightly faster using disk optimizer as I assume file delete operations queues SCSI unmap’s in the background.
After optimizer was run you can see the used storage is back to a couple of GB’s per VM.
This is just another way Nutanix makes everyday operations invisible, as your admins no longer need to worry about manually reclaiming storage. It just works. As there are no LUN’s, you also don’t need to worry about doing a LUN level reclaim either.
Together with space saving measures such as Compression, De-Duplication, Erasure coding and now SCSI unmap, this allows you to make the most out of the usable capacity on your Nutanix Infrastructure without wasting capacity due to dead space.
Special thanks to Josh Odgers for letting me delete files on his test Exchange Server VM’s to tryout the unmap capabilities available in Acropolis Hypervisor.