My NAS has recently started to drop connection to several HDDs. "Disk removal detected [disk 2]", "Disk removal detected [disk1]". Aprox 4 hours later the drive is rediscovered and rebuilding starts.
I had some growing errors on disk 2, but then also disk 1 started to disconnect. I have bought a new HDD to replace disk 2, but it doesn't seems like the NAS is recognizing it. Can it be a faulty connector?
Right now it shows disk 1, 3 and 4 as OK. Disk 1 and 3 is 2TB, 2 and 4 is 3 TB.
RAIDiator 5.3.13.
Examplelog:
Mar 4 17:01:48 NAS-Server RAIDiator: System is up. Mar 4 17:13:53 NAS-Server RAIDiator: Ignoring hot-add event on /dev/sdb Mar 4 17:13:53 NAS-Server RAIDiator: Disk removal detected. [Disk 2] Mar 4 17:13:53 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 17:13:54 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 17:13:59 NAS-Server RAIDiator: Disk removal detected. [Disk 1] Mar 4 17:13:59 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 17:14:00 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 17:14:09 NAS-Server RAIDiator: Disk removal detected. [Disk 1] Mar 4 17:14:09 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 17:14:10 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 20:39:15 NAS-Server RAIDiator: New disk detected. If multiple disks have been added, they will be processed one at a time. Please do not remove any added disk(s) during this time. [Disk 1] Mar 4 20:39:18 NAS-Server RAIDiator: A new disk was added to the ReadyNAS. If multiple disks have been added, they will be processed one at a time. Please do not remove any added disk(s) during this time. Mar 4 20:39:29 NAS-Server RAIDiator: Disk removal detected. [Disk 1] Mar 4 20:39:29 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 4 20:39:30 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 5 01:06:06 NAS-Server RAIDiator: New disk detected. If multiple disks have been added, they will be processed one at a time. Please do not remove any added disk(s) during this time. [Disk 2] Mar 5 01:06:57 NAS-Server RAIDiator: A new disk was added to the ReadyNAS. If multiple disks have been added, they will be processed one at a time. Please do not remove any added disk(s) during this time. Mar 5 01:07:07 NAS-Server RAIDiator: Disk removal detected. [Disk 2] Mar 5 01:07:07 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible. Mar 5 01:07:08 NAS-Server RAIDiator: A disk was removed from the ReadyNAS. One or more RAID volumes are currently unprotected, and an additional disk failure or removal may result in data loss. Please add a replacement disk as soon as possible.
Model: ReadyNAS RND4000v2|ReadyNAS NV+ v2 Chassis only
This looks like either a disk issue or a board issue. I can take a look at the logs if you like. If so, download the full log set and throw them up on a Google link or similar and PM me the link.
Also, now would be a good time to backup the data if you don't have a backup already.
The situation is not good. I can see you have been using two different size disks (likely 2x2TB 2x3TB?). This in itself is fine but it creates two raids on top of each other.
In the below raid output we would expect to see md0 (OS raid), md1 (swap raid), md2 (data raid 1) and md3 (data raid 2). md2 is completely missing and all the other raids are degraded.
md1 : active raid1 sda2[4] 524276 blocks super 1.2 [4/1] [U___]
md0 : active raid1 sda1[4] 4193268 blocks super 1.2 [3/1] [U__]
unused devices: <none>
The data volume cannot mount as a result of the missing md2 and you only see the OS volume information (information for volume "C", the data volume, is missing). At this point, access to the data is lost. You will be able to see some shares appear as folders but the content is not there. Neither can you write to the volume.
Filesystem Size Used Avail Use% Mounted on /dev/md0 4.0G 763M 3.1G 20% / tmpfs 16K 0 16K 0% /USB
This situation most certainly occurred because of the disks dropping in and out of the raids causing double or even triple disk failures in the array. Your disks look fine from a healthcheck point-of-view. I can see you inserted a newer one though but that won't do much as the data raids are already in trouble. I don't see the disks in the past actually being bad. One of the disks had some errors on it, but only 2 ATA errors - you replaced that one.
The 3 others appear fine yet still, we see disks dropping in and out.
3!!Thu Feb 21 17:28:58 CET 2019!!root!!Disk removal detected. [Disk 1] 3!!Thu Feb 21 17:35:36 CET 2019!!root!!Disk removal detected. [Disk 2] 3!!Fri Feb 22 06:47:39 CET 2019!!root!!Disk removal detected. [Disk 2] 3!!Fri Feb 22 06:47:39 CET 2019!!root!!Disk removal detected. [Disk 3]
The above happens over and over and this will surely have been messing up your data raids and in turn your volume. I suspect that perhaps the back-plane might be bad given that you didn't perform these actions.
Do you have a backup of the data? I think data recovery is possible here, granted that we have at least 3 healthy disks that seems to be have been part of the raid before it went south. However, it would be wise to let NETGEAR do that part if you really need the data back. It will cost a data recovery contract which is a couple hundred bucks or something along those lines.
If you have a backup or don't need the data then a factory default would probably be the best option. However, I would be concerned myself to use the NAS as this issue here is possibly chassis related. I would advise that you test each disk with a tool from manufacturer to ensure the disks are fully OK. At that point you can start suspecting the chassis might be faulty.
Can you tell if the drives are dropping out when they spin up? If so, it could be a power supply issue. Main chassis issue is just as likely, though, I think.
I don't necessarily see the spindown at the time of disks dropping out. However, there is indeed a lot of spin up/down in the logs. Like this:
Feb 21 18:37:02 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 18:42:05 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 18:44:30 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 18:49:33 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 18:51:58 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 18:57:01 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 18:59:26 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 19:04:29 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 19:06:53 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 19:11:56 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 19:14:21 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Feb 21 19:19:24 NAS-Server noflushd[1934]: Spinning down disks.
Feb 21 19:21:49 NAS-Server noflushd[1934]: Disks spinning up after 2 minutes.
Furthermore, diving a little deeper I can see a few times where this is reported.
Feb 22 06:39:34 NAS-Server kernel: sd 0:0:1:0: timing out command, waited 180s
Feb 22 06:39:34 NAS-Server kernel: sd 0:0:1:0: timing out command, waited 180s
Feb 22 06:39:34 NAS-Server kernel: sd 0:0:1:0: [sdb] Unhandled error code
Feb 22 06:39:34 NAS-Server kernel: sd 0:0:1:0: [sdb] Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
Feb 22 06:39:34 NAS-Server kernel: end_request: I/O error, dev sdb, sector 0
Feb 22 06:39:34 NAS-Server kernel: __ratelimit: 32 callbacks suppressed
Feb 22 06:39:34 NAS-Server kernel: Buffer I/O error on device sdb, logical block 0
Feb 22 06:39:34 NAS-Server kernel: Buffer I/O error on device sdb, logical block 1
Feb 22 06:39:34 NAS-Server kernel: Buffer I/O error on device sdb, logical block 2
Feb 22 06:39:34 NAS-Server kernel: Buffer I/O error on device sdb, logical block 3
I think this could be a SATA connectivity issue though, also given that the smart-test passes. It is worth checking all disks fully with manufacturer software IMO. If they come out clean it is probably chassis issue.
connecting the disks to a linux PC and attempting to recover data on your own
connecting the disks to a PC and using a RAID recovery tool such as r-studio: https://www.r-tt.com/
purchasing a used chassis, and attempting to migrate the disks to it.
None of these are sure-proof, and most will cost you some money. The first three require the least technical skill on your part. Since your NAS is end-of-life (and becoming less compatible with Windows over time), I'd suggest getting a new ReadyNAS and contracting with Netgear to temporarily mount the data.
Then if it turns out the NV+ v2 chassis is ok after all, you could repurpose it as a backup NAS.
OK, the R-Drive solution appeals to the geek in me, but there is of course the question of time available, and also storage space available. Best value for money and time for me, might be to buy a new NAS and attempt to migrate the old drives. As You say the system is quite old. I need to consider these options more thoroughly.
Once again, I'm grateful for all valuable insight!
I’m looking at used or new NAS to migrate to. I’m currently looking at the new ReadyNAS 214 RN21400, and in my area the only used NAS is the Ultra 4 series, which I think is too old. Is there other models suitable for my migration?
Unfortunately Netgear doesn't provide paid support for used ReadyNAS, and the warranty isn't transferable. So Netgear won't help you migrate disks from an NV+ to an Ultra. If you want their help, you need to go with a new OS 6 model.
The RN214 is faster than the ultra. The RN424 is worth considering, though it is more expensive than the RN214.
Allright, I’m gonna buy the 214. But while ordering I started to wonder, do I need to buy it fully stacked with new HDD’s or can I use my «damaged» drives from the NV+?
do I need to buy it fully stacked with new HDD’s or can I use my «damaged» drives from the NV+?
Test the old ones in a PC with lifeguard. After the non-destructive long test completes, do the write zeros test also. If both of those pass, then you could re-use the old drives.
OK, but if I want to try to migrate the data from the old NAS to the new one? (With help from Netgear support) I just wondered if the recovery process needs to copy the data from the old drives to the new NAS, or if I can just put them right into the new NAS and start the recovery from there.
You will need to first install the old drives and copy the data to another device, as the old volume will be read-only. Then, you can either factory default with the old drives or insert new ones and copy the data back to the NAS with a new volume in the format of the new OS.
