× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: Volume suddenly read-only

EKroboter
Apprentice

Volume suddenly read-only

All of a sudden and out of nowhere, my volume has become read-only. The system does not show any errors and no disk malfunction.

What the hell? I'm updating all my backups now, but what should I do next? reboot? reset?

Model: RN51600|ReadyNAS 516 6-Bay Diskless
Message 1 of 11
StephenB
Guru

Re: Volume suddenly read-only

If you are only looking at the logs in the web ui, then you should download the log zip file and look in there for errors.

 

But getting the backup taken care is the right priority.

Message 2 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

Did exactly that. Run all bakcup jobs and update my offsite backup as well. Rebooted and the message of read-only no longer appears. It took a long time to calculate usage and free space. The last thing I did before this issue was un-assign and delete an iSCSI LUN. It was about 1.5 TB of data. Could the suddent deletion of that much data cause a problem?

 

The sahres are up but most of the files seem to corrupted. Cannot open any single file. This is a tremendous screw up on the NAS side.

Message 3 of 11
StephenB
Guru

Re: Volume suddenly read-only


@EKroboter wrote:

 

The shares are up but most of the files seem to corrupted. Cannot open any single file. This is a tremendous screw up on the NAS side.


If your backup is complete, I suggest doing a factory default (I wouldn't trust the volume integrity given the history).

Message 4 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

PDFs, images and video files are opening just fine. Office documents and spreadsheets are not. Noone can open them, but the backups seem to be just fine. I'm suspecting a really strange file system corruption here.

Also, the usage calculation does not seem to be correct. I should have 12 TB of usable space in RAID 5 (using five 3 TB drives), but this is what I'm seeing (see attached).

Message 5 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

Ok, this is getting more and more difficult to pinpoint. Mac clients can open every file. Windows PCs cannot. 

Message 6 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

And now it's back to Read-Only mode. What a nightmare.

Message 7 of 11
Sandshark
Sensei

Re: Volume suddenly read-only

I, too have a volume that went read-only.  A reboot clears it till a write operation occurs, then it goes read-only again.  Interrestingly, the BTRFS readonly flag is set to false.

 

I happen to know what caused it on mine.  The volume is in an EDA500 equivalent eSATA chassis and the cable came loose (it's the older shallow kind, not the newer, deeper one).  Though the connection is OK now, the damage has apparently been done.  Since this volume is fully backed up and consists of archival data rarely accessed, I decided to try a few things.  But it sounds like you may not have tiime to wait for any answers I find.

 

I thought a scrub should fix the issue, so I started one.  It self-aborted after 4 minutes.  But initiating a scrub from the GUI also initiates a re-sync, and I have a few hours left for that to complete.  Note that the GUI still says it's scrubbing, but SSH tells a different story.  That the re-sync process seems to be doing fine tells me that the eSATA connection is now OK.  But re-sync on an eSATA volume is even slower than a normal one, and so I wait.

 

Once the re-sync completes, I'm going to try a scrub from SSH so it won't also do another re-sync.  If it then works, maybe the re-sync really was needed.  If it still doesn't work, then I wasted a lot of time when I could have been destroying the volume, re-creating it, and restoring data from backup.  While @StephenB  suggests a fctopry default, that's not the appropriate action for an external volume, but may be the only total solution for a single chassis system to insure the OS and swap partitions also don't have issues.  But after all this, I'll know, and I can share my experience.

 

Message 8 of 11
Sandshark
Sensei

Re: Volume suddenly read-only

Re-sync completed and re-try at scrub self-aborted after 38 seconds.  No reason given, no errors reported.  So a little Googling suggested I start with the -B flag so it doesn't background it, and it then reported it aborted due to read-only file system.  Which is odd, because btrfs property get /eda2 (my volume is eda2)  returns ro=false.and the same is true of both sub-volumes.  So I have more to learn about how a BTRFS filesystem is marked read-only.

 

All remaining tools appear to only work on an unmounted filesystem, and ReadyNASOS doesn't seem to want me to do that, so it's destroy and re-build for me.

 

But in my Googling, I did learn that a too-full volume can lead to it becoming read-only..  Could that be your issue?  But that, of course, puts you in a catch-22, since you can't delete files from a too-full read-only volume.  And doing a full restore from backup is going to put you right back where you were (or close, there may be a better data/metadata division that gets you by for a while).

Message 9 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

Thank you very much for your input. In my case, I believe the file system was corrupted somehow during the deletion of the iSCSI LUN. It shouldn't have, there is no reason for it. 

Unfortunately I don't have time to run test or even study the subject. I have to get this up and running as we have 45 employees depending on files in here. I schedule downtime for today and have to get this thing working again, so a system reset and restore will be faster (relatively, I have to restore 3 TB of data through USB which will take more than 10 hours).

Message 10 of 11
EKroboter
Apprentice

Re: Volume suddenly read-only

Not it in my case. More than 70% free space. The problem arose one hour after deleting the LUN. Can quite put my finger on the cause of the problem.

Message 11 of 11
Top Contributors
Discussion stats
  • 10 replies
  • 3091 views
  • 0 kudos
  • 3 in conversation
Announcements