NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Switches for Small Storage Spaces Direct Cluster Hyper converged environment
Moving from Dedicated Direct Server to a 3 Node Hyperconverged Storage Spaces Direct solution. I need to implement a Storage Spaces Direct Cluster of the 3 nodes. From my review I believe we will ...
BrendonHolt wrote:From my review I believe we will need to switch to a RoCE Switch with RDMA enabled on the Intel LAN Cards (Intel 710 4 Port).
There are no RoCE switches per se. RoCE can operate in various modes and variants, RoCEv1 was a pure L2 protocol, RoCE is a L3 protocol using UDP packets. This does start from so called Resilient RoCE (plain standard switches without any flow control, RoCE over a "lossy" network) and goes up in various levels of of levels in Lossless RoCEv2 with global flow control, port flow control, QoS, PFC, Aplication Priority, and RoCE congestion control (ECN) on L3 networks.
BrendonHolt wrote:The XS716 is not stackable, so 1.) do we need to move to new switches or can we purchase another XS716T and bind them.
Complete a different story - both stacking and interconnecting two or more switches does add a bottleneck in the stacking link(s) resp the interconnection link(s). You won't get an NtoN full wirespeed between the two switches (unless stacking/interconnect port would be e.g. two 100G ones (for two XS716T).
BrendonHolt wrote:Does this support RoCE, I believe answer may be no.
See above again. Mellanox has put up some very informative reading on RoCE -> RoCE Architecture and Design
BrendonHolt wrote:Can we do RoCE on the M4300 Series 10GBe Switches. Very frustrated because only one forum thread spoke of this, but never did anyone answer with yes or no. I MUST BELIEVE that Netgear can give a answer if it works with Storage Spaces Direct Cluster and an Intel 710 T4 Adapter.
Depends on the support and feature level intended/required. No experience with the big managed Netgear switches - we're using switches with a similar feature (and much faster ports than 10G copper) like the Netgear M4500 models for such infrastructures, which include features like:
Data Center Bridge (DCB)
• Enhanced Transmission Selection (ETS, IEEE 802.1Qaz)
• Priority Flow Control (PFC, IEEE 802.1Qbb)
• Application Priority (IEEE 802.1Qaz)
Data Center Bridge Exchange (DCBX, IEEE802.1Qaz)
• CEE 1.01 support
• IEEE version support
BrendonHolt wrote:Does the M4300 work for Microsoft Hyper Converged Environment for Storage Spaces Direct Cluster? Limitations?
LaurentMa Does Netgear have the opportunity to bring up some FAQ or better configuration examples and switch proposals for these platforms becoming much popular in 2020? There is more but just Converged AV 8-)
That 4500 Router is a beast.
My research is leaning towards IWARP, the speed limitations from Stacking or Binding still exist, but this is disk write and I/O is limited. My primary concern is uptime, that is why we were moving to S2D. That said, I really APPRECIATE your very thorough responses.
What are your thoughts about IWARP, much more Switch Independent and seems to get r done.
My current thoughts:
1. RDMA in a Microsoft S2D Cluster wil be worth the extra costs. It is a small cluster, 3 or 4 nodes, All SSD DIsks, will be Hypervisor 2019 Server Data Center with 512GB RAM Each Node, Supermicro with two E5's 16 Core. So plenty of local power, but will be a bit taxed because going with 3 way mirror.
2. Loved the 4500MX but it is out of budget. Currently have a XS716T that has been working great up until this point. If I can connect another XS716T and they can be crossconnected at 20GBe I would probably never have I/O issue. Which brings up two seperate questions:
a.) Would the XS716T be good enough for IWARP with Chelsio Boards. It would be limited by the interlink. Have Intel in there now X710-T4, will move those over for VM's and Management functions.
b.) Would it be better to get two M4300-24X or 16X. We have two LACP configured 48 Port Netgears in a Stack now, 1Gbe for Virtuals.
c.) What is the maximume stacking speed on the M4300 Series Switch.
3. I have to have two switches in case one goes down, therefore, I believe the interlink is very important.
Side Note: I am moving from RAID 10 DIRECT DAS on the Servers with Replication, this solution is very fast and has not failed. ALL SSD Storage. It replicates with another server.
My partner makes an Excellent Point, with the Replication we can be up quickly, and all this complex configureations is simply to avoid DOWNTIME, something we have not had to this point with our simple but powerful environment..
@LaurentMa Does Netgear have the opportunity to bring up some FAQ or better configuration examples and switch proposals for these platforms becoming much popular in 2020? There is more but just Converged AV 8-)
100% Agree on this, it will be a move many are looking at. From the pecpective of the Vendor some use case or schematics or technical FAQ/Documentation on this would most likely lead to more sales and understanding. Frankly, many people will figure out it is very difficult and simply abandon.
If it works, simple explanation/documentation followed up by links to more details on CLI, etc... for Configution could prove very valuable.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
