Yesterday evening, my shares became inaccessible. FrontView seems to be going nuts. We had just finished copying some files to the NVX and everything was working as normal, but when I tried to view the logs there were none. When I try "download all logs," FrontView goes away and I just get a blank HTML page with plain text, "Empty No Support."
I'm running Radiator 4.2.13, which I installed almost a week ago and have had no problems until now.
If I try to view the shares listing, about 50% of the time it says "no shares" and 50% of the time it shows all my shares, but each time the protocols available are different and the settings are never right. If I go into "Standard File Protocols," each time it loads they are different. CIFS and FTP - the two I use - are turning on and off. Regardless of what is shown in either of these views, I can see a CIFS listing of shares through Windows Explorer but cannot access any of them and I cannot connect to the device via FTP at all.
Occasionally, fans or temperature sensors that don't exist on the NVX pop up with warnings (usually 0RPM and -18F). My volume (XRaid2, 2 disks) is the only thing that has remained constant, at 86% full, so I believe my data is still there, just hidden under some malfunctioning software/firmware.
I tried to manually re-enable CIFS and redefine share permissions (there's only one read/write user), but no changes I try to make in FrontView seem to have any effect. I can't even shutdown or restart through FrontView; I have to double-press the power button on the unit, though it does gracefully shutdown.
I waited on hold last night for Support, but eventually just got a voice mail and so far I've not gotten a callback or a reply to my case I submitted online. I'm pretty upset about the data; though my backups are only a few days old, it would be a huge inconvenience for me to rebuild the device, not to mention how nervous this makes me regarding my data in the future, as I've only had the device 3 weeks!
I tried starting it again, out of desperation, while keeping a close eye on it to make sure it didn't start a resync or restripe. It came right up, but this seemed to have no effect; FrontView is still unpredictable and my changes are not having any effect. So I've shut it down and unplugged it (it's on a power schedule) to preserve it as-is until I get some suggestions!
I was finally able to contact support last night and get this resolved.
There is a defect in recent versions of the firmware where - under certain circumstances - if you try to backup to an external USB drive that is not mounted, it will actually write the backup data to the OS partition. The OS partition is 4GB and the OS takes roughly 400MB, so if you are running a backup of more than about 3.6GB, like I was, it will fill the OS partition and create the exact behavior I outlined above. FrontView acts up, and all your shares become inaccessible via any protocols.
The only solution for this is to telnet into the box and remove the extra data that has clogged up your OS partition. Support can do this for you remotely by having you boot to Support mode, if you are still within your phone support window. If you have had the product more than 90 days, I guess explain the problem to the person responsible for routing your call, and I would hope Netgear would do the right thing since it's a known bug in the firmware.
They are looking into resolving this, so hopefully it won't be a problem much longer, but I wanted to make two important points in this post for future readers:
1) Obviously, if you are experiencing this problem, I hope this will help you know what the problem is and what to do. I did the right thing by shutting down the box and waiting for support to help me resolve the problem at the OS level. All the data shares are stored in other partition, and your data should not be affected by this problem.
and
2) This problem occurred because I have two external USB drives I use for my backups, in a rotation. I plug them into separate USB ports, and they cannot each be mounted with the same share name, so there is no way - I verified this through support - to have a single backup job for each share that can write to either external drive. This means that for my six shares, I have twelve jobs. I wanted to use the backup button, and as discovered in 2006 on these boards, the workaround is to setup all your jobs for the backup button, and THEORETICALLY, the ones trying to write to the unmounted USB as the destination should just fail and the others will run. The glitch in the firmware now means that this workaround is risky and could fill your OS partition inadvertantly and cause your ReadyNas to act up!!!
***UNTIL THIS BUG IS CONFIRMED FIXED, YOU SHOULD NOT RUN A BACKUP JOB - MANUALLY OR VIA THE BACKUP BUTTON - THAT USES AN EXTERNAL USB DRIVE THAT IS NOT MOUNTED AS A DESTINATION***
I hope this post is able to help some people out, and keep you calmer than I was for 24 hours!
The very bad thing is, that this behavior is still the same in the 2012 (!) latest firmwares!!!
I had the same situation on the weekend. My external backup hdd died while the Readynas Ultra 6 was backing up. So the mountpoint switched somehow over to the frontview system os partition and filled it to the end. I didn't knew that, at this time. Frontview was acting like crazy, shares were not able to contact, but iSCSI worked normally. I couldn't do anything, because it was weekend and the device is way older than 90 days.
So I tried to figure it out on my own. With the help of this board I found out, that it has something to do with less space on the system partition. I switched the device to telnet mode and connected to the readynas. But in that mode, nothing is mounted. So I figured out how to mount the RAID5 volume (with mdadm). And voila! I could see the partitions and as you could imagine, the system os partiton was filled to 100%. I deleted the wrong placed folders from the backup job and rebooted the device. Everything was fine again!
I'm so glad, that I was able to fix this. I had such a bad weekend, cause I had no access to my NAS and the backup HDD died also. And the other backups were older than a year. I have also data from my customers on that device. I thought, I could close my business...
And i've installed EnableRootSSH for the next time 🙂
It is very bad form to revive a thread that's been dead for a few years. Open a new one in future and refer to an old thread that you consider relevant via a link (if you consider it would be helpful).
m.o.desaster wrote:
I couldn't do anything, because it was weekend and the device is way older than 90 days.
You can still use online support. It may take longer to get a response, but that's the supported way to do things. Online support is free though if they need to remotely login to your NAS a fee may be involved.
m.o.desaster wrote:
So I tried to figure it out on my own. With the help of this board I found out, that it has something to do with less space on the system partition. I switched the device to telnet mode and connected to the readynas. But in that mode, nothing is mounted. So I figured out how to mount the RAID5 volume (with mdadm). And voila! I could see the partitions and as you could imagine, the system os partiton was filled to 100%. I deleted the wrong placed folders from the backup job and rebooted the device. Everything was fine again!
When the OS partition gets full files on the OS partition can get corrupted. When support examines a problem like this they would check for such corruption and if they find any address the problem.
m.o.desaster wrote:
I'm so glad, that I was able to fix this. I had such a bad weekend, cause I had no access to my NAS and the backup HDD died also. And the other backups were older than a year. I have also data from my customers on that device. I thought, I could close my business...
Glad you fixed the problem (or appear to have fixed it) in your case. However if you do something like this and something goes wrong it would probably lead to support being denied.
m.o.desaster wrote:
And i've installed EnableRootSSH for the next time 🙂
Sometimes when the OS partition gets full even the SSH service stops so that may/may not help in a situation like this, though I would presume you're unlikely to run into this issue again. A USB disk failing in the middle of a backup job isn't going to happen very often.
It's a good idea to keep an eye on how full the 4GB OS partition is. You can do this via SSH using
# df -h
or by looking at disk_usage.log in the logs (Status > logs > Download all logs).
