VMware vSphere Metro Storage Cluster (vMSC) is a specific storage configuration that is commonly referred to as stretched storage clusters or metro storage clusters. These configurations are usually implemented in environments where disaster and downtime avoidance is a key requirement. This recommended practice document was developed to provide additional insight and information for the operation of a vMSC infrastructure in conjunction with VMware vSphere. This paper explains how vSphere handles specific failure scenarios, and it discusses various design considerations and operational procedures. For detailed information about storage implementations, refer to the documentation provided by the appropriate VMware storage partner.
Note that initially, vMSC storage configurations had to go through a mandatory certification program. As of vSphere 6.0 this is no longer needed. vMSC configurations are now fully partner supported and can be found on the vmware.com website under PVSP (Partner Verified and Supported Products). Before purchasing, designing or implementing please consult the PVSP listing to ensure the partner has filed for PVSP and has tested with the correct vSphere versions. ( )The vMSC listings typically also provide a link to the specifics of the implementation by the partner. As an example, the PVSP Listing for EMC VPLEX provides the following link: . This link provides all tested scenarios and supported components with EMC VPLEX.
This document is intended for individuals with a technical background who design, deploy, or manage a vSphere Metro Storage Cluster infrastructure. This includes but is not limited to technical consultants, infrastructure architects, IT managers, implementation engineers, partner engineers, sales engineers, and customer staff. This solution brief is not intended to replace or override existing certified designs for vSphere Metro Storage Cluster solutions; it instead is meant to supplement the knowledge and provide additional information.
A VMware vSphere Metro Storage Cluster configuration is a specific storage configuration that combines replication with array-based clustering. These solutions are typically deployed in environments where the distance between data centers is limited, often metropolitan or campus environments.
The primary benefit of a stretched cluster model is that it enables fully active and workload-balanced data centers to be used to their full potential and it allows for extremely fast recovery in the event of a host or even full site failure. The capability of a stretched cluster to provide this active balancing of resources should always be the primary design and implementation goal. Although often associated with disaster recovery, vMSC infrastructures are not recommended as primary solutions for pure disaster recovery.
This document does not explain the difference between disaster recovery and a downtime- or disaster-avoidance solution. For more details on this distinction, refer to Stretched Clusters and VMware vCenter Site Recovery Manager: Understanding the Options and Goals, located here:
The question we typically get is if there is a minimum license edition of vSphere required to create a vSphere Metro Storage Cluster. The answer to that question is no. You can create a stretched cluster with any edition, however, if you have a requirement for automated workload balancing from either a CPU or storage perspective then the minimum required license level is vSphere Enterprise Plus, as this license includes vSphere DRS and Storage DRS.
The storage subsystem for a vMSC must be able to be read from and write to both locations simultaneously. All disk writes are committed synchronously at both locations to ensure that data is always consistent regardless of the location from which it is being read. This storage architecture requires significant bandwidth and very low latency between the sites in the cluster. Increased distances or latencies cause delays in writing to disk and a dramatic decline in performance. They also preclude successful vMotion migration between cluster nodes that reside in different locations.
vMSC solutions are classified into two distinct types. These categories are based on a fundamental difference in how hosts access storage. It is important to understand the different types of stretched storage solutions because this influences design considerations. The two types are:
With uniform host access configuration, hosts in data center A and data center B have access to the storage systems in both data centers. In effect, the storage area network is stretched between the sites, and all hosts can access all LUNs. NetApp MetroCluster is an example of uniform storage. In this configuration, read/write access to a LUN takes place on one of the two arrays, and a synchronous mirror is maintained in a hidden, read-only state on the second array. For example, if a LUN containing a datastore is read/write on the array in data center A, all vSphere hosts access that datastore via the array in data center A. For vSphere hosts in data center A, this is local access. vSphere hosts in data center B that are running VMs hosted on this datastore send read/write traffic across the network between data centers. In case of an outage or an operator-controlled shift of control of the LUN to data center B, all vSphere hosts continue to detect the identical LUN being presented, but it is now being accessed via the array in data center B.
Our examples use uniform storage because these configurations are currently the most commonly deployed. Many of the design considerations, however, also apply to non-uniform configurations. We point out exceptions when this is not the case.
In this section, we describe the basic architecture referenced in this document. We also discuss the basic configuration and performance of the various vSphere features. For an in-depth explanation of each feature, refer to the vSphere 6.5 Availability Guide and the vSphere 6.5 Resource Management Guide. We make specific recommendations based on VMware best practices and provide operational guidance where applicable. It is explained in our failure scenarios how these best practices prevent or limit downtime.
Eight LUNs are depicted in Figure 3. Four of these are accessed through the virtual IP address active on the iSCSI storage system in the Frimley data center; four are accessed through the virtual IP address active on the iSCSI storage system in the Bluefin data center.
The vSphere cluster is connected to a stretched storage system in a fabric configuration with a uniform device access model. This means that every host in the cluster is connected to both storage heads. Each of the heads is connected to two switches, which are connected to two similar switches in the secondary location. For any given LUN, one of the two storage heads present the LUN as read/write via iSCSI. The other storage head maintains the replicated, read-only copy that is effectively hidden from the vSphere hosts.
Our focus in this document is on vSphere HA, vSphere DRS, and vSphere Storage DRS in relation to stretched cluster environments. Design and operational considerations regarding vSphere are commonly overlooked and underestimated. Much emphasis has traditionally been placed on the storage layer, but little attention has been applied to how workloads are provisioned and managed.
One of the key drivers for using a stretched cluster is workload balance and disaster avoidance. How do we ensure that our environment is properly balanced without impacting availability or severely increasing the operational expenditure? How do we build the requirements into our provisioning process and validate periodically that we still meet them? Ignoring these requirements makes the environment confusing to administrate and less predictable during the various failure scenarios for which it should be of help.
Each of these three vSphere features has very specific configuration requirements and can enhance environment resiliency and workload availability. Architectural recommendations based on our findings during the testing of the various failure scenarios are given throughout this section.
Our environment has four hosts and a uniform stretched storage solution. A full site failure is one scenario that must be taken into account in a resilient architecture, alongside host failures, network failures and different types of storage failures. vSphere HA is crucial in any environment to provide a certain level of availability, but even more so in a vMSC configuration. VMware recommends enabling vSphere HA and recommend thoroughly reading the following guidelines for an optimal vSphere HA configuration for vMSC based infrastructures.
VMware recommends enabling vSphere HA Admission Control. Workload availability is the primary driver for most stretched cluster environments, so providing sufficient capacity for a full site failure is recommended. Such hosts are equally divided across both sites. To ensure that all workloads can be restarted by vSphere HA on just one site, configuring the admission control policy to 50 percent for both memory and CPU is recommended.
VMware recommends using a percentage-based policy because it offers the most flexibility and reduces operational overhead. Even when new hosts are introduced to the environment, there is no need to change the percentage and no risk of a skewed consolidation ratio due to the possible use of VM-level reservations. For more details about admission control policies and the associated algorithms, refer to the vSphere 6.5 Availability Guide.
Additionally, as of vSphere 6.5, it is also possible to specify how much Performance Degradation you are willing to tolerate for your workloads. By default, this setting is configured to 100%. You can change this to your liking, and should be based on your SLA with the business. As this setting is new, we will briefly explain how it works by looking at an example.
c80f0f1006