Re: NAS Does not see 3 previously populated volumes. RN104

PFD314 · ‎2018-07-16

Last night my ReadyNAS worked fine (Firmware 6.9.3).
Its got 3x3TB drives installed.
This morning my desktop and laptop couldn't connect to the mapped drives.
I was able to access the admin interface.

It shows the 3 populated volumes as Red, Inactive volumes.

My last backup is about a year old, so I really need to find a way to recover from this failure.

Logs Below:
Mon Jul 16 2018 12:56:56
System: ReadyNASOS background service started.
Mon Jul 16 2018 9:31:55
System: The system is shutting down.
Mon Jul 16 2018 9:30:41
Volume: Disk test failed to start for volume Test5.
Mon Jul 16 2018 9:28:34
System: Alert message failed to send.
Mon Jul 16 2018 9:28:34
Disk: Disk in channel 1 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 9:27:10
System: ReadyNASOS background service started.
Mon Jul 16 2018 9:23:44
System: The system is rebooting.
Mon Jul 16 2018 9:16:24
Disk: Disk in channel 3 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 9:14:42
System: Alert message failed to send.
Mon Jul 16 2018 9:14:42
Disk: Disk in channel 1 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 9:13:00
System: ReadyNASOS background service started.
Mon Jul 16 2018 9:09:00
System: The system is shutting down.
Mon Jul 16 2018 9:08:24
Disk: Disk in channel 1 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 9:06:08
Disk: Disk Model:ST33000651NS Serial:Z294T04S was removed from Channel 1 of the head unit.
Mon Jul 16 2018 9:06:07
Disk: Disk Model:ST33000651NS Serial:Z294T04S was added to Channel 1 of the head unit.
Mon Jul 16 2018 9:04:02
Disk: Disk Model:ST33000651NS Serial:Z294T04S was added to Channel 1 of the head unit.
Mon Jul 16 2018 9:03:55
Disk: Disk Model:ST33000651NS Serial:Z294T04S was removed from Channel 1 of the head unit.
Mon Jul 16 2018 8:26:14
Disk: Disk in channel 3 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 8:24:02
System: Alert message failed to send.
Mon Jul 16 2018 8:24:02
Disk: Disk in channel 1 (Internal) changed state from RESYNC to ONLINE.
Mon Jul 16 2018 8:22:08
System: ReadyNASOS background service started.
Mon Jul 16 2018 8:21:48
System: ReadyNASOS service or process was restarted.
Mon Jul 16 2018 3:56:57
Volume: Volume Test5 is resynced.
Mon Jul 16 2018 3:56:46
Disk: Disk in channel 2 (Internal) changed state from ONLINE to FAILED.
Mon Jul 16 2018 3:56:45
Disk: Detected increasing ATA error count: [626] on disk 2 (Internal) [ST3000DM001-9YN166, Z1F15T34] 32 times in the past 30 days. This condition often indicates an impending failure. Be prepared to replace this disk to maintain data redundancy.
....
Mon Jul 16 2018 1:51:37
Disk: Detected high uncorrectable error count: [48] on disk 2 (Internal) [ST3000DM001-9YN166, Z1F15T34]. This condition often indicates an impending failure. Be prepared to replace this disk to maintain data redundancy.
Mon Jul 16 2018 1:00:07
Volume: Volume Test5 is Degraded.
Sun Jul 15 2018 2:20:27
Volume: Defragmentation complete for volume Test5.
Sun Jul 15 2018 2:00:01
Volume: Defragmentation started for volume Test5.
Sun Jul 15 2018 1:00:38
Volume: Volume Test5 is Degraded.
Sat Jul 14 2018 23:16:39
System: ReadyNASOS background service started.
Sat Jul 14 2018 23:16:21
System: Alert message failed to send.
Sat Jul 14 2018 23:16:21
Volume: Resyncing started for Volume Test5.
Sat Jul 14 2018 23:16:21
Volume: Volume Test5 is Degraded.
Sat Jul 14 2018 23:16:18
System: ReadyNASOS service or process was restarted.
Sat Jul 14 2018 16:04:13
Volume: Resyncing started for Volume Test5.
Sat Jul 14 2018 16:04:08
Disk: Disk Model:ST33000651NS Serial:Z294T04S was added to Channel 1 of the head unit.
Sat Jul 14 2018 16:04:01
Disk: System will be shut down in 30 minutes because of disk failure.
Sat Jul 14 2018 16:03:58
System: Alert message failed to send.
Sat Jul 14 2018 16:03:58
Volume: Volume Test5 health changed from Redundant to Degraded.
Sat Jul 14 2018 16:03:58
Disk: Disk Model:ST33000651NS Serial:Z294T04S was removed from Channel 1 of the head unit.
Sat Jul 14 2018 14:55:31
System: ReadyNASOS background service started.
Sat Jul 14 2018 14:55:09
System: ReadyNASOS service or process was restarted.
Thu Jul 12 2018 17:15:51
System: ReadyNASOS background service started.
Thu Jul 12 2018 17:15:31
System: ReadyNASOS service or process was restarted.
Sun Jul 8 2018 2:01:35
Volume: Balance complete for volume Test5.
Sun Jul 8 2018 2:00:12
Volume: Balance started for volume Test5.
Sun Jul 1 2018 9:09:00
Volume: Disk test completed for volume Test5.
Sun Jul 1 2018 2:00:04
Volume: Disk test started for volume Test5.
Sun Jun 24 2018 11:48:24
Volume: Scrub completed for volume Test5'.
Fri Jun 22 2018 2:00:16
Volume: Scrub started for volume Test5.
Fri Jun 15 2018 2:27:13
Volume: Defragmentation complete for volume Test5.
Fri Jun 15 2018 2:00:16
System: Alert message failed to send.
Fri Jun 15 2018 2:00:16
Volume: Defragmentation started for volume Test5.
Fri Jun 8 2018 20:02:58
System: ReadyNASOS background service started.
Fri Jun 8 2018 19:20:03
System: The system is shutting down.
Fri Jun 8 2018 2:01:22
Volume: Balance complete for volume Test5.
Fri Jun 8 2018 2:00:17
Volume: Balance started for volume Test5.
Tue Jun 5 2018 11:42:40

PFD314 · ‎2018-07-16

Here is a pic of the web interface.
Its also configured as RAID5.

StephenB · ‎2018-07-16

You only had one RAID-5 volume (Test5). Perhaps you were confusing volumes and shares in your title?

There are several causes for this. Perhaps PM @mdgm-ntgr and ask him if he'd be willing to analyze your logs.

PFD314 · ‎2018-07-16

Yes, Only 1 Raid5 named "Test5".

PFD314 · ‎2018-07-17

I potentially made the problem worse. It looked like my RN104 had created a phantom volume: Test5-0 in addition to the normal volume I used Test-5

I destroyed volume 5-0 hoping it would let me redefine the disks as the original volume.

The log confirms "Volume: Volume Test5-0 was deleted from the system."

But the status of the systems shows the Volume Test5-0

StephenB · ‎2018-07-17

You might consider paid support (my.netgear.com), though it might require data recovery.

Of course if you have the data backed up, then you could do a factory reset, reconfigure the NAS, and then restore the data.

PFD314 · ‎2018-07-17

If the drives are still good; and it was a temporary glitch, shouldn't I be able to reconfigure the volume and save the data?

StephenB · ‎2018-07-17

If you have ssh enabled, you could force the RAID array to mount. If the array is out of sync, then the file system could be corrupted (even if the disks are healthy).

mdgm-ntgr · ‎2018-07-17

Please send in the full logs zip (see the Sending Logs link in my sig)

mdgm-ntgr · ‎2018-07-23

You should contact support. There are costs involved with data recovery attempts which may be completely unsuccessful.

One of the disks is clearly failing badly (see what you posted above and also smart_history.log).

systemd-journal.log shows that the NAS can't start the RAID (2/3 disks considered failed in md127).