NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
mcoverdale
Jun 15, 2011Aspirant
Vmware ISCSI Disconnecting
Good Afternoon,
I have a standalone vanilla ESX host 10.0.3.5 connecting directly via ISCSI to Netgear NVX storage 10.0.3.1. (Seperate network)
I just can't make the ISCSI connection stable, it continually temporarily drops the ISCSI connection before automatically re-connecting a few seconds or minutes later.
I have logged multiple calls with VMWARE and they assure me the Netgear NVX device is faulty?
Things I have tried to resolve this issue with no success:
NVX Firmware Upgrade (Raidiator now 4.2.17)
Replaced the entire ESX Host with another DL380 G5 server
Tried different ISCSI ethernet cables Cat 5, 5e, 6, 6e
Tried enabling "Access Control" on NVX with servers IQN
Tried adding Netgear GS108 gig switch between the ESX server and Netgear NVX
I have a standalone vanilla ESX host 10.0.3.5 connecting directly via ISCSI to Netgear NVX storage 10.0.3.1. (Seperate network)
I just can't make the ISCSI connection stable, it continually temporarily drops the ISCSI connection before automatically re-connecting a few seconds or minutes later.
I have logged multiple calls with VMWARE and they assure me the Netgear NVX device is faulty?
Things I have tried to resolve this issue with no success:
NVX Firmware Upgrade (Raidiator now 4.2.17)
Replaced the entire ESX Host with another DL380 G5 server
Tried different ISCSI ethernet cables Cat 5, 5e, 6, 6e
Tried enabling "Access Control" on NVX with servers IQN
Tried adding Netgear GS108 gig switch between the ESX server and Netgear NVX
17 Replies
Replies have been turned off for this discussion
- mcoverdaleAspirantP.s It's a Netgear NVX Business edition with ISCSI support.
- mcoverdaleAspirantP.s.s. More things I've tried:
Disabled Jumbo Packets.
Set all VMware vswitches and windows servers(guest OS) to use MTU size 9000 (Default) - mdgm-ntgrNETGEAR Employee RetiredI would suggest you open a tech support case (see Online Submission link in my sig) and post your case number (preferably editing the title of the thread i.e. the title of the first post). Between tech support and the forum you should get good help to discover the cause of your problem.
Please don't use Cat5 cabling. That's old junk unless you're not interested in using gigabit ethernet. Cat5e is the oldest ethernet cable you should consider using.
An MTU of 9000 sounds like what you'd have with Jumbo Frames on, not off. An MTU of 1500 sounds more like what one would expect with Jumbo Frames disabled.
Edit: You're using 4k sector disks, but your disks are not 4k sector aligned. This situation means you're likely experiencing terrible write performance to the NAS which is likely the cause of your problems. There was a Technical Bulletin about this a while back: http://www.readynas.com/?p=3690
First I would suggest you backup your data.
Then:
Two options:
Option i
1. Run the "Disk Test" boot option: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu#nvx_nvx_pioneer_edition
2. If all disks pass the disk tests then hot-remove (remove while NAS is on) one disk, wait a minute or two. Then hot-add the disk again. The disk should be wiped and re-added to your array
3. Once your volume status is "redundant" again, repeat for the next disk
Option ii
1. Backup your data and verifying the backup is successful
2. Do a System > Config Backup (keep the zip file that is downloaded. If it is extracted automatically you may find the zip file in the Recycle Bin)
3. Do a factory reset, either via System > Update > Factory Default or via the boot menu: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu#nvx_nvx_pioneer_edition
4. Restore Config Backup
5. Restore data from backup
Option ii is better. This will give you native EXT4 meaning you will be able to create iSCSI targets >2TB. I think VMWare may be limited to targets of 2TB in size (not sure about this, you should look into this), but if there is a limit currently this may change in the future.
Once you've got 4k sector alignment if you still have issues further troubleshooting can be done. - mcoverdaleAspirantThanks for the quick response, I decided to go out drinking after my latest "Access Control" attempt failed, I meant to say earlier I set all the MTU packet limits to 1500 not 9000.
I have second blank identical Netgear NVX business ready to take over, and the exact same WD drives so I assume I'll hit the same 4k alignment error, how do align a blank Netgear NVX business to use these drives?
I can easily use VEEAM to replicate the VM across to the new NVX. - mcoverdaleAspirantReading your post again, all I have to do is factory reset my (Blank) NVX to fix the 4k error?
- mdgm-ntgrNETGEAR Employee Retired1. Update to the latest firmware (http://www.readynas.com/RAIDiator_x86_4_2_17_Notes). You will need at least one disk installed to do this. So if you haven't installed any disks yet, install one disk and turn the NAS on.
2. You will need to reboot when prompted to install the firmware update. Wait for the firmware update to complete then verify that it was successful.
3. If all disks are installed, in Frontview do a System > Update > Factory Default. Otherwise power down, install all disks and do a factory reset using the boot menu: http://www.readynas.com/kb/faq/boot/how_do_i_use_the_boot_menu#nvx_nvx_pioneer_edition
If the new NAS unit came with 4.2.12 or later (the old one came with 4.2.9 or was last factory reset on that firmware as you can see from initrd.log) then a factory reset (hot-swapping of the disks would take much longer and is certainly not recommended if you haven't loaded any data onto the unit yet) may not be necessary, but it's still nice to do a factory reset after updating to the latest firmware when doing a new setup, to get a nice clean setup on the latest firmware.
Edit: If the NVX is already on 4.2.12 or later then yes a factory reset is all that would be needed to get 4k sector alignment. You should update to the latest firmware if you haven't already first.
One more thing. This should be pretty obvious, but make sure you can mount the iSCSI targets fine on the new NAS before factory resetting the old one. There is a config file from the config backup, /etc/ietd.conf that contains the information that Frontview uses to mount the targets. So you'll need to put the iSCSI files in /c/.iscsi (.iscsi is a hidden folder) on the new NAS (you can move the files when logged in to CIFS as "admin" if Veeam can't backup to there) and ensure the /etc/ietd.conf is on the new NAS. You could do a System > Config backup on the old NAS, unzip it, delete the other files apart from /etc/ietd.conf, create a zip of the "etc" folder (important that it is done this way so files are restored correctly), then restore that zip file on the new NAS. - mcoverdaleAspirantSo I haved "Factory Default" my (blank) Netgear NVX from the boot menu.
I'm unsure on how to read the partition.log to check disk alignment, can someone please assist me to confirm the disks are now 4K sector aligned from the logs? (Disk 4 is currently Resync)
Log Files here:
http://www.unionsnsw.com/Netgear/ - mcoverdaleAspirantPerformance is now awesome!
I can now assume the disks are now 4K aligned.
Servers now boot up twice as quick before.
Big thanks mdgm. - mdgm-ntgrNETGEAR Employee RetiredYes the disks are 4k sector aligned. Hopefully things should be working fine now with no disconnects. If you still have problems please open a tech support case.
If you haven't already you could try enabling jumbo frames again if you wish. - mcoverdaleAspirantI've experienced a wonderful six months with my ReadyNAS NVX Business Edition but:
Now the iScsi disconnects are back with a vengeance, I updated the NVX to the latest firmware from 4.2.17 to 4.2.19 but nothing has worked.
I've installed no new servers or made any network changes of any kind but the Readynas NVX iSCSI drops just come out of nowhere again.
If anybody has any ideas please let me know? This is an absolute business critical device. (Tech Case has been logged)
P.S. iScsi is connected by a direct Ethernet cable connected to DL380 G5, so no chance of any other network traffic interfering.
Related Content
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!