We have ReadyNAS 3312 (RR 3312 w/ 12 bay, Firmware: 6.10.3) that has been throwing volume alert message reading: "The volume data encountered an error and was made read-only. It is recommended to backup your data." when large amounts of data have been copied onto it. This has happened three times in the past few weeks and occured twice in one-day. Following the last alert, we left it in read-only mode until we could figure out the issue, which we have so far been unsuccessful at doing.
The following error message was in the kernel.log:
The other two times it has gone down the kernal reads the same. It seems to indicate there is no space left, but in actuality there is ~60 TB free. We haven't been able to find the right solution from the forums so far. Would anyone know how to handle this?
When the OS puts the volume in read-only mode, it is trying to protect the volume from further damage. Because it's read-only, the available space will show as zero, though if it really did fill up, that could be the root cause of the issue.
DO NOT reboot the NAS. It may come back up with a dead volume.
DO NOT attempt to make the volume read/write. It likely won't work, and you'll just be un-doing the protection and risking further damage to the voluime, including possible total loss.
DO do what it told you to do -- backup your files now. And backup your NAS configuration if it'll be painful to re-create. Hopefully, you have most of the files already backed up. Because some files may also now be corrupt, don't destroy your old one if at all possible.
There is something critically wrong with the volume that only destroying it and re-building it is likely going to solve. Netgear paid support may be able to do something, but it's doubtful. But you'll want that backup even before they try. If there is only one volume, a factory default is the best solution. But before you do that, you should also check to see if any of the drives has a problem that's the root cause of the issue you are encountering, and replace any that need it.
We don't anticipate that the drive filled up, the data being moved over was on the order of GB when it has happened. So far, none of the drives have indicated any errors.
How do we determine the root cause of the volume error? We prefer not to proceed with the rebuild until we know the root cause, and can ensure it won't happen again in the future. We do have backups of all the data, but there is so much data involved it takes a month to restore from the backups and validate the accuracy of the restoration. This ReadyNAS is less than a year old, so hopefully we don't have to completely rebuild ever year.
Thank you for the response! I greatly appreciate it.
Logs in the downloaded .zip file are a good place to look to see if there is a hard drive that's giving you problems. Unfortunately, even new drives can fail.
I'm assuming that in a use case like yours, you have an UPS that is monitored by the NAS,. so a power interruption during a write operation isn't a possibility.
Hmm. It doesn't look like there are any disk errors recorded. The file repeats the following 12 times as expected (given the Channel and serial number change).
Well, then I can't point to anything that I think could have caused it, though i suppose there could still be an issue with the NAS hardware. I can say that it's atypical and I don't think you should be concerned you'll be encountering it periodically. I've used a variety of ReadyNAS over the years, including my current use of legacy rack-mount systems converted to OS6, and I have only seen this once. And that was mostly my fault -- the eSATA cable between an RN516 and EDA500 was loose, causing a lot of data issues. (I say mostly my fault because the choice of connector for eSATA is just a really bad engineering decision -- not Netgear's, mind you, but by whoever designed the interface standard. )
Occurrences of similar issues reported in this forum typically have an identifiable cause (usually a failing drive with at least ATA errors or lack of proper UPS protection).
Since I knew what caused it and that it was corrected, I thought I could save the volume. That's where I got the experience to say don't even try. No matter what I did, the volume was read-only again within minutes. Since my corrupt volume was on the EDA, I could just destroy and re-create it and not touch the primary volume. I feel your pain in the data restoration -- the EDA500 is extremely slow due to the eSATA interface, and I had 5x6TB in it in RAID5. But it was mostly archival data, so that probably made it a little less painful than yours will be if you are without current working data for any period of time.
The volume rebuild process will also be a good gauge of whether you do have a hardware issue, so check the logs after that completes.
We were doubting it was hardware, but still a small possibilty. Thank you for advice and all of the help. We will try the rebuild and post back here if we get something figured out. Oddly enough, I have a friend has seen this error thrown on their other two ReadyNAS servers, but has never been able figure out why it happened on their end, only that its periodically occured.
A quick update if you are curious: We have a friend well-versed in Linux that has been trying to help us come up with a solution. We found a problem that sounds very similar to the one we are having on the btrfs site.
The problem discussed on this open-source group (Seach for 'When you haven't hit the "usual" problem')
