I have an NVX that has given me some FSCK problems lately. It started with an expansion months ago that caused FSCK failures, so all the data was backed up, the NVX reset and rebuilt, and then all the data restored.Subsequently, new online FSCK failues were found. Rebooting with FSCK enabled took the data volume offline and tech support was unable to fix the problem. I managed to mount the volume read-only and back the data up and reinitialize again.In my latest attempts to reset and rebuild the box, I decided to be extra paranoid. After the initial 4T volume was built, I was informed that a volume expansion would take place on the next reboot (expected). So I decided to force an FSCK on reboot. That one came back clean. After the additional 1TB of space was added to the volume (Bringing the volume to a 5099GB size), I was informed again that a volume expansion would take place on the next reboot (expected). I forced an FSCK on reboot and rebooted the NVX. Now it's come back with errors and the data volume is offline. FSCK errors with a memory allocation failure. I'm somewhat convinced that something in 4.2.22 is misbehaving when expanding the volume, corrupting large numbers of inodes to the point that it exhausts all memory on the system.Also note that I've run the boot menu's Test Disks without any issues.Anyone have any ideas on how to resolve this? The fact that an empty data volume is coming back with FSCK errors is rather impressive.

I have NVX which has been fine for years. A hard drive started to cause problems (error rate increase). I fitted a new hard drive and the volume was expanded. I started to get problems with the NVX hanging on some directories. A volume check resulted in a huge number of errors listed. Now the volume won't mount. Netgear support just say the volume won't mount and offered to assist with factory resetting.95% sure there is a software bug.5.5TB lost. Just about to see if I can mount the drive.How did you mount in read only mode?

gyroscopes: please contact netgear tech support. meanwhile, please do not do anything to the unit.

gyroscopes:I mounted the volume read-only after tech support gave up (and didn't offer to mount it read-only either). To do it, I just enabled root ssh, logged in, and ranmount -o ro /dev/c/cNow it's been a week since I posted this topic and opened my tech support case and I have yet to see a technician do anything other than ask me to upload logs or put the devince in tech support mode. I'm tempted to just close the case and try rebuilding on my own with an older version of the firmware to see if maybe 4.2.20 can expand without destroying the filesystem.

For anyone lurking. We're now up the the point of collecting data for developers to look at. We've performed yet another factory reset, gathering logs after the unit builds the initial volume, right after a reboot w/fsck, and then right after the first expansion + reboot w/fsck (this is where fsck fails).The last bit of sanity checks being performed at the moment is a series of memory tests just to make sure the memory is not the problem. Of course, if there was a memory problem, I would have expected it to surface during the 4TB initial volume build and not only during the 1TB expansion.As you can see, this has taken a long time to get to this point. I normally would have just given up and tried an older firmware to get this running again, but if there is an actual problem in the 4.2.22 firmware, it would be nice to find and have fixed in a later release.

Now I'm up to developers looking into the issue. Hopefully they can figure this out.

FSCK Fails after expansion on NVX case# 20304759

22 Replies

Replies have been turned off for this discussion

gyroscopes
Aspirant
Jan 14, 2013
I have NVX which has been fine for years. A hard drive started to cause problems (error rate increase). I fitted a new hard drive and the volume was expanded. I started to get problems with the NVX hanging on some directories. A volume check resulted in a huge number of errors listed. Now the volume won't mount. Netgear support just say the volume won't mount and offered to assist with factory resetting.
95% sure there is a software bug.

5.5TB lost. Just about to see if I can mount the drive.

How did you mount in read only mode?
Jedi_Knight
Tutor
Jan 14, 2013
gyroscopes: please contact netgear tech support. meanwhile, please do not do anything to the unit.
horshack
Aspirant
Jan 15, 2013
gyroscopes:

I mounted the volume read-only after tech support gave up (and didn't offer to mount it read-only either). To do it, I just enabled root ssh, logged in, and ran

mount -o ro /dev/c/c

Now it's been a week since I posted this topic and opened my tech support case and I have yet to see a technician do anything other than ask me to upload logs or put the devince in tech support mode. I'm tempted to just close the case and try rebuilding on my own with an older version of the firmware to see if maybe 4.2.20 can expand without destroying the filesystem.
horshack
Aspirant
Jan 24, 2013
For anyone lurking. We're now up the the point of collecting data for developers to look at. We've performed yet another factory reset, gathering logs after the unit builds the initial volume, right after a reboot w/fsck, and then right after the first expansion + reboot w/fsck (this is where fsck fails).

The last bit of sanity checks being performed at the moment is a series of memory tests just to make sure the memory is not the problem. Of course, if there was a memory problem, I would have expected it to surface during the 4TB initial volume build and not only during the 1TB expansion.

As you can see, this has taken a long time to get to this point. I normally would have just given up and tried an older firmware to get this running again, but if there is an actual problem in the 4.2.22 firmware, it would be nice to find and have fixed in a later release.
horshack
Aspirant
Jan 29, 2013
Now I'm up to developers looking into the issue. Hopefully they can figure this out.
JohnnyB11
Aspirant
Feb 04, 2013
Please keep us informed. There are more people facing this problem, see the links in http://www.readynas.com/forum/viewtopic.php?t=69366
horshack
Aspirant
Feb 05, 2013
Support says, "Developers found that initial file system scan make file system corrupted."

I'm not really convinced. It doesn't sync up with all the bad behavior before I started debugging. My best guess is it's either a bug in FSCK or it's a bug in the expansion process/tools that are corrupting inodes.
horshack
Aspirant
Feb 05, 2013
As I predicted, a full rebuild including waiting for all the expansions to complete sill fail FSCK. Once again waiting on support.
JohnnyB11
Aspirant
Feb 05, 2013
Here too: Even a factory default and all the expansions killed the volume.
horshack
Aspirant
Feb 07, 2013
Going through a different permutation now. Started with just 2 disks, and then added a 3rd. It's restriping now. Will try an FSCK after the restripe is done and then again after it expands and we'll see what that shows.

For now, devs are trying to reproduce the bad behavior in the lab.