I have been working on a VSAN design and deploy engagement for one of our customers, and we went up having some limitations and decisions that have to be taken into consideration especially with the device selection for ESXi boot, the storage controller, and the operating mode of that controller (RAID vs. Passthrough).
I thought that writing such a post can help others being in the same situation.
To start, I will list the available hardware that we have:
– DELL PowerEdge R730 Servers
– PERC R730 RAID Controllers
– Dual 16GB Sd-Card modules for IDSDM
– 8 Flash disks per server participating in All-Flash VSAN disk group
Consideration 01 – ESXi Boot Device
You can configure IDSDM from Bios in mirrored mode and install ESXi on SD-Card. See logs and coredumps redirection requirement in “Consideration 02” for this option.
You can install ESXi on RAID1 disks. However you have to take into consideration the restrictions mentioned in the below KB article about mixing VSAN disks with other VMFS volumes: https://kb.vmware.com/s/article/2136374
You can think of another controller per server acting in RAID mode for RAID1 ESXi disks in addition to the PERC R730 controller acting in HBA mode for the VSAN disks.
Consideration 02 – Logs and Coredumps
If you install ESXi on SD-Card, make sure you redirect logs to a syslog server (such as vRealize log insight) which is outside VSAN environment and coredumps to a coredump collector outside VSAN as well.
If you plan to use locally attached disks to store logs or coredumps, then consider these limitations:
– First of all, you are wasting more disk slots per server as less slots will be available for using more VSAN disks and diskgroups in the future If you are planning to scale up in terms of storage.
– Second, It is not supported to have VSAN disks co-exist with Non-VSAN VMFS volumes on this controller as per the below KB article:
https://kb.vmware.com/s/article/2136374
If you are planning to install ESXi on locally attached RAID1 disks to store logs/coredumps locally, again there are limitations and restrictions as per the KB article: https://kb.vmware.com/s/article/2136374
Consideration 03 – Storage Controller Mode
The PERC R730 RAID Controller can operate in either one of two modes:
– RAID mode
– HBA mode
You can’t have two disks mirrored in RAID1 for ESXi partition and another disks participating in VSAN in passthrough mode at the same time.
You can operate PERC R730 controller in HBA (Pass-through) mode for VSAN disks leaving ESXi installation on SD-Card modules.
If you configure PERC R730 controller in RAID mode for the sake of having ESXi installed on RAID1 locally-attached disks, you have to check the KB article https://kb.vmware.com/s/article/2136374 for the restrictions of having VSAN disks with Non-VSAN VMFS volumes on this controller.
RAID0 vs. Passthrough
Virtual SAN supports storage controllers in two modes, either pass-through or RAID0 mode. One of the big considerations when choosing a storage controller for Virtual SAN is whether it supports pass-through mode, RAID0 mode, or both.
RAID0
– RAID0 mode is utilized by creating a single drive RAID0 set via the storage controller software, and then presenting this to Virtual SAN.
– When utilizing RAID0 mode, the storage controller cache should be disabled (this is configurable on some, but not all storage controllers) to ensure the storage controller cache does not conflict with the cache of the SSD drives which is controlled by Virtual SAN.
– RAID0 mode will require interaction with the storage controller software to manage the addition and removal of drives. If you want to add a disk or replace a faulty one, you have to reboot the host and add the new disk to the RAID0 volume to be able to add it to the disk group. This means no hot-replace capability.
– If you are to use RAID 0, a custom claim rule may be required to set an SSD drive with the SSD flag.
Pass-Through
– Individual disks can be hot-replaced without the need to enter them into some RAID configuration. One thing to keep in mind. The disk cannot have any partition information.
The pass-through mode allows the hypervisor to have greater control over the drives in the capacity tier group.
– Virtual SAN has complete control of the local SSDs and HDDs attached to the storage controller.
VSAN recommendation is to use pass-through mode whenever possible.
H730 Vs. HBA330 Storage Controllers
What I would advise is configuring hosts with HBA330 controllers which are better suited for vSAN (H730 were designed for RAID not passthrough) and thus more reliable, to the point that Dells official recommendation is to not use H730P for vSAN and use HBA330 instead:
https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=34853
“While this controller will continue to be supported on vSAN for the full service life, DellEMC strongly recommends using the HBA330 pass-through controller for vSAN deployments. The HBA330 is specifically designed for Software Defined Storage applications like vSAN, and does not introduce the unnecessary RAID processing overhead involved with the H730 PERC RAID family of controllers.”
Mohamad Alhussein
(1 votes, average: 5.00 out of 5)
Loading…
