vSAN has come a long way since it was first introduced and has been serving millions of environments worldwide. However, since its introduction, disk technologies and the transport mechanisms that power them, have all changed dramatically in their architecture, capabilities, capacity, and speed. The availability of such devices and the need for performance for demanding workloads drove VMware to rethink how vSAN operates but without fundamentally changing the way customers interact with it.
vSAN 8 is the result of all that effort and brings a completely new way to take advantage of capable hardware to deliver the performance the modern workloads demand.
I’ll leave a deeper dive into the technology for a later post and cover the most important aspects of this release that caught my attention in this one.
vSAN Express Storage Architecture (ESA)
VMware has fundamentally rearchitected how vSAN works with the new generation of devices while keeping the operational aspects for the user the same or as close as possible.
That change has unleashed a lot of performance and capacity enhancements, some of which will be covered in the following sections.
ESA is an optional new capability that you can enable on vSphere 8 deployments i.e. you can keep running the traditional vSAN setup if you want as ESA requires compatible/supported hardware to function. Look out for vSAN Ready Nodes that are already capable in your new hardware purchases to future-proof your hardware.
Comparison of vSAN Architectures
VMware is calling the architecture currently in use: Original Storage Architecture (OSA). With that in mind, how do the two architectures compare?
With OSA, we’ve become accustomed to having a cache disk per disk group that all the writes primarily go to, before being flushed out to the capacity drives. To increase performance and resilience, one adds more disk groups and the debate on whether to add more disks or disk groups, always generates some passionate discussion. Being around for a while, it supports a wide range of hardware too so the hardware compatibility list (HCL) for OSA is long!
In comparison, with ESA, forget about cache devices and disk groups – it’s all single-tier from here! ESA relies on NVMe-based flash devices and all of them go into a storage pool. The rest is the familiar vSAN software and logic magic that enables flexible data placement, thereby providing flexible configuration, resulting in the performance and resilience levels required.
Of course, the transition to this architecture will be gradual as more supported hosts become available. Once in place and added to a vCenter 8 controlled cluster, it’s simply a matter of Storage vMotioning to the new cluster.
Log-Structured File System
When I wrote about VMware Flex Cloud Storage a few months ago, I mentioned the Log-Structured File System (LFS) briefly. Remember that name as it will feature a lot in vSphere 8 announcements and is the main technology behind most of the storage innovations this time.
The new capabilities of ESA are also built upon LFS. There are still performance/capacity parts within the system, called “Performance Leg” and “Capacity Leg”, respectively. As you would expect, the Performance Leg is responsible for making temporary fast writes to a durable log, with the metadata and sending a fast write acknowledgement. Then the Capacity Leg kicks in and writes large full stripes efficiently to the storage which minimises I/O amplification and reduces the number of write operations in general.
The architecture is designed so that each device in a datastore store all three components i.e. the Performance and Capacity logs as well as the metadata components. That means all devices are claimed into a storage pool and participate in the performance and capacity levels independently, therefore, the concept of disk groups is eliminated.
This is significant as this change means that the failure domain now effectively shrinks down to a single disk. As the data stored on it (depending on the storage policy) is stored redundantly, it removes some of the failure scenarios that occurred due to the failure of a disk group.
Optimised Data Handling
So how is vSAN 8 able to improve upon how data is written to the disks? The answer becomes obvious once you consider where the compression, encryption, and checksum i.e. data services are handled in OSA.
As those operations are handled later in the write cycle, it results in inefficiencies in I/O operations in an OSA architecture. ESA, in comparison, takes care of all those services at data ingestion, which enables parallelised and efficient full stripe writes to the capacity space.
As you can tell, it reduces the amount of data that needs to be sent to other hosts, reducing the load on the network but also reduces the number of CPU cycles the destinations host needs to spend before writing that data to the disk – as the data services have already been executed on the data component.
Adaptive RAID-5
Administrators of smaller clusters are often torn between the resilience of RAID1 and the space efficiency of RAID 5 (or 6). vSphere 8 brings the good news that they may not have to anymore!
In vSphere 8, there are two new configurations of RAID5, and the secret is in how the data is distributed between the hosts. In this scheme, the storage policy automatically adjusts the data placement scheme (it may take about 24 hours before switching to it) to either a “4+1” or “2 + 1” scheme, depending on the number of hosts in the cluster.
That allows administrators to go for RAID5 with fewer hosts and without compromising on the data resilience of the storage.
Reduced Encryption Overheads
As mentioned earlier, in vSAN 8, the data services have been moved higher up the stack so they only need to occur once. That is no different for encryption services too.
In OSA, the data needs to be decrypted and encrypted back again, to apply compression and deduplication etc. Performing those operations earlier in the write cycle means that this extra step does not need to occur, resulting in minimising CPU and network impact. At the same time, it reduces I/O amplification as well.
Snapshots!
Due to the complete architectural change of vSAN, VMware had the chance to review and improve on how snapshots are taken and consolidated. While the real performance improvements will start coming out soon after to see from our own eyes, I can’t wait to see them already!
With vSAN 8, VMware has made improvements in how snapshots are consolidated. Snapshots have always relied on redo logs and the performance degradation of VMs as the number of snapshots increases is caused by them. They are also responsible for the performance limitations of most VADP backup solutions too.
For ESA, the performance levels have changed dramatically and according to VMware, they’ve seen tens or even a hundred times performance improvements in terms of snapshot consolidation, making the impact of snapshots on VM performance, negligible. That is an enormous achievement and should result in greater possibilities when it comes to backup and restore options.
As a side benefit (as shown in the slide), now one can track how much storage a snapshot is consuming – a capability that was sorely missed before.
No Improvements for Original Storage Architecture (OSA)?
Well, indeed there’s something for your beloved original vSAN architecture as well. There’s the ability now to increase the cache tier up to 1.6 TB, as compared to the 600 GB that is the current limit. That is almost 3 times as much cache as is currently possible.
It must be enabled manually and only applies to all-flash configurations but it’s a welcome change and will improve performance and reduce I/O demand on the systems in general as the cache will be able to hold larger working sets.
In my experience, most hosts already contain 1.6 TB cache disks (or above) so enabling it once the hosts are upgraded to vSphere 8, seems like a free performance upgrade that one should not miss.
Conclusion
I was expecting Data Processing Units (DSUs) to feature in vSAN 8 enhancements as well, but we’ll have to wait a bit for those it seems. However, as someone who is quite interested in storage and its performance, I am still excited about these changes in vSAN and hope that supporting hardware starts shipping soon.
There are so many other enhancements that I wanted to cover but there are space and time limitations today. I do plan to cover them as soon as I get the chance. In the meantime, enjoy reading more about these features as more in-depth details are released during VMware Explore.
Leave A Comment