ReadyNAS Pro unresponsive on 4.2.21 - first time in years.

Further update: I had to open a case with Netgear support. Eventually, the shipped me a replacement ReadyNAS Pro Business. About 17 days after I installed the replacement ReadyNAS Pro 6 and moved my disks from the “defective” unit to the “replacement” unit, the same problem recurred. Last Saturday, I noticed that my Squeezebox Boom and Squeezebox Radio lost connection to the Squeezebox server (or whatever it’s now called since it has been renamed so many times). I tried to ping my ReadyNAS and got nothing in response. When I pressed the front-panel button on the ReadyNAS Pro, the LED screen responded to the button press. So, I figured I would leave it alone for one more day to see if the problem goes away or gets worse. The following day (Sunday), the ReadyNAS was not responding to pings, either, but it also stopped responding to the front-panel button presses.

So, I logged into the ESXi server (which is using the ReadNAS as its iSCSI targets) and it still still saw the iSCSI volumes located on the ReadyNAS. The ESXi server is connected to the ReadyNAS via a direct CAT6 cable (iSCSI NIC2 <—> ReadyNAS NIC2). Moreover, I was still able to ping VMs running on the ISCSI server, which are physically located on LUNs provisioned on the ReadyNAS. I have one Windows 7 VM, which was responding to pings, but I could not RDP into it. There are several Cisco Unified Communications (RedHat based VMs) that are LUNs on the ReadyNAS, which were also responding to pings, but I could not SSH into them. Finally, I had to power down the ESXi server and then I pushed and held the front-panel button on the ReadyNAS until it turned off.

When I turned the ReadyNAS back on, it started a syncing process, which lasted several hours. When I powered up the ESXi server, it took it a long time to boot up. It had happened before (after I upgraded to RAIDiator 4.2.21). The VM whose LUN was provisioned last on the ReadyNAS causes the ESXi server to hang for a long time (close to 10 minutes) before it boots all the way up and is accessible via the vSphere Client. Basically, the ESXi server cannot establish a connection to this last LUN, which happens to be my Windows 7 VM. Connections to other LUNs on the ReadyNAS are fine. In fact, after I received the replacement ReadyNAS, I deleted the Windows 7 LUN, and then created a new LUN and installed Windows 7 anew. Still, this is the VM running off the LUN that was provisioned last on the ReadyNAS, and it manifests the same behavior as it did before I recreated this LUN and before I replaced the ReadyNAS. The only way to get the ESXi server to connect to this LUN is to go to the ReadyNAS FrontView, disable this LUN, then re-enable this LUN, and then to go to the vSphere Client, and rescan the LUNs. Then, the ESXi server finds the datastore on this LUN and the Windows 7 VM shows up in the list of VMs. At this point, I can manually start this VM and all is good until the next reboot of the ESXi server, which occurs when my ReadyNAS Pro freezes - generally within 14-20 days after another reboot.

So, my strong suspicion is that my ReadyNAS freezes are caused by the iSCSI protocol. One of the symptoms of a problem is that my ESXi server cannot automatically start the VM that uses the LUN created last on the ReadyNAS because it cannot connect to the datastore on this LUN upon initial boot. This seems to point to some sort of incompatibility between ESXi 5 and RAIDiator 4.2.21. Since the hardware has now been replaced, everything points to the software issue. Moreover, the communication between the ESXi server and the ReadyNAS occurs only via a direct CAT6 cable connecting the two servers’ NIC2s. I bound iSCSI to NIC2 on the ESXi server, and therefore, the source address with which ESXi communicates with the ReadyNAS is its NIC2’s IP address. Therefore, the ReadyNAS only knows of the ESXi server’s IP on its NIC2 and because of its internal routing table, it only sends packets to the ESXi server out of its own NIC2. When the ReadyNAS becomes unresponsive, not only do I lose communication to the ReadyNAS NIC1 - the IP that the rest of my network sees (which is accessible via an ethernet switch), but also the communication based on the ReadyNAS NIC2 (direct cable connection to the iSCSI server) is also affected. This rules out any issues on my LAN.

I’m not sure how many users of the ReadyNAS Pro Business use it as an iSCSI target. I’m pretty sure that I’m running the ReadyNAS in a pretty complex environment - more complex than most users. I believe there’s actually a bug in the RAIDiator software - probably a memory leak - related to iSCSI, which makes the ReadyNAS completely unresponsive within 12-16 days.