vSAN Thin vs Thick Provisioning
I’ve recently had several conversations around whether leveraging thin or thick provisioning within a vSAN datastore is necessary. Although the default vSAN Storage Policy leverages thin provisioning, it is important to understand why it is the default and the recommended best practice.
Thin vs Thick Provisioning
Let’s start with how vSAN defines a thin or thick provisioned disk. Within a vSAN Storage Policy there is a setting called ‘Object Space Reservation’ (OSR). When the OSR value is equal to 0, it is considered a thin disk and when it is set to 100, it is considered a thick disk. A vSphere cluster backed by a vSAN datastore can have multiple vSAN Storage Policies and different VMs within a cluster can be assigned both a thin or thick provisioned storage policy – it is not an either/or situation. It is also valid to assign a percentage of 1 to 99 for the OSR value, I’m at a loss why that would be necessary.
From a design perspective, the primary factor for choosing a thin or thick provisioned disk is capacity management.
Versions of vSAN prior to version 6.7 Patch 01 did require thick provisioning to be used with Oracle RAC VMs in order to support multi-writer mode. (VMware KB 2121181)
When considering what vSAN features to enable or leverage within a vSphere SDDC environment, the following questions should be used to drive design decisions:
- What percentage of over-provisioning for memory will be allowed?
- Will VMs have a memory reservation? If so, how much memory will be reserved per cluster/site?
- What percentage of over-provisioning for storage capacity will be allowed in each cluster/site?
- What vSAN Storage Policy, along with FTT settings, will be leveraged in each cluster/site?
- What is the total storage capacity required by the planned VM deployments within each cluster/site?
- What percentage of growth (quarterly or yearly) is expected within each cluster/site?
- How many snapshots, per VM, are anticipated to exist within each cluster/site?
- Will vSAN encryption be leveraged now in the future?
From a vSAN perspective, the default policy and best practice for all vSAN datastore is to always leverage thin provisioning.
There are several technical reasons behind the recommended best practice. The ones I find most relevant to customers are the following.
- All VM snapshots, including those leveraged as part of a backup solution, inherit the vSAN Storage Policy settings. Therefore, if the vSAN Storage Policy assigned to a VM with an OSR=100 value, all snapshots will also be thick provisioned. As such, the number of snapshots will need to be included in all capacity calculations to determine the required number of physical ESXi nodes required and the individual disk capacity per node.
- The VM swap disk can be thick or thin provisioned and inherits the storage policy attributes. As of vSAN 6.7, the default setting for the swap disk is to be thin provisioned and can be managed through the ESXi Advanced Setting “SwapThickProvisionedDisabled” host setting. The swap disk size is based on the amount of memory allocated to the VM minus the memory reservation. Again, these two settings together have potential impact to capacity calculations and management. (VMware VirtualBlocks Blog Article)
- In much older storage arrays, database VMs were often created using a thick provisioned disk. The creation of a thick provisioned disk eliminated the need for a new block to be allocated and zeroed out at the time of a IO write. In a vSAN datastore, all writes happen at the caching tier within the disk group and are immediately acknowledged once the block is written to the cache disk. The replication of the object to the capacity tier, based on the storage policy settings, occur in the backend and is transparent to the VM. As such, in a vSAN datastore there is no longer a performance difference between a thin or thick provisioned disk.
Deduplication/Compression Considerations
Another key design factor that will influence the use of thin or thick provisioned disks, will be the inclusion of vSAN deduplication/compression within each cluster. When a thick provisioned vSAN Storage Policy is assigned to VMs in a cluster with deduplication/compression enabled, the storage policy will take precedence and prevent the 4K blocks within a single vSAN disk group from actually being deduplicated.
“Deduplication occurs when data is destaged nearline from the cache tier to the capacity tier of an all-flash disk group. The deduplication algorithm utilizes a 4K fixed-block size and is performed within each disk group independently. In other words, redundant copies of a block within the same disk group are reduced to one copy, but redundant blocks across multiple disk groups are not deduplicated. The deduplication at the disk group level by vSAN using a 4K block size helps provide a good balance between space efficiency and performance…An object that is assigned a storage policy containing an object space reservation rule of 100 percent is analyzed for deduplication, but no space savings are realized because capacity for the entire object is reserved.” (https://bit.ly/38wpe0r)
Typically, deduplication/compression is enabled on a vSAN datastore because the environment needs to be as efficient as possible from a storage capacity standpoint. The use of thick provisioning is in direct contrast to that design goal, as thick provisioning all VMs within a vSAN datastore implicitly means you are not actually allowing vSAN to deduplicate or compress the data on the disk.
Final Thoughts
vSAN has been designed with resiliency and performance in mind, thus eliminating the perceived performance benefits from thick provisioning. The existence of thick provisioning in a vSAN Storage Policy is much like older vSphere features like flash cache reservations — allowed for legacy reasons, but neither are necessary from a performance perspective any longer.
Understanding how vSAN is designed and how it operates within large-scale environments, the only reason to leverage thick provisioning would be if (organizationally) your company is unable to properly manage capacity. That being said, I would strongly encourage you to leverage thin provisioning with all of the vSAN datastores in your environments. Where possible, leverage vRealize Operations within your environments, as it will yield itself to performing solid capacity calculations for initial deployment and growth over time, mitigating any perceived risk from leveraging thin provisioning.
