- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Data Volume Lost after Disk Replace #25483643
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Data Volume Lost after Disk Replace #25483643
Hello ReadyNAS Community,
after a disk failure (disk 3), I replaced this disk. A volume resynchronization was started automatically. However, this resynchronisation failed with no obvious errors and now the entire data volume is unavailable. Can anyone please provide any help as to how to try to rebuid this volume or make it available again?
Regards
Alex
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
I don't suppose you have a backup of the data on the volume?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Hi,
I still have access via HTTP GUI and SSH. I don't know how good the logs will be, but I will provide then nonetheless. I'm not sure how tho, can't seem to find an attachment button around here.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
@vandermerwe wrote:I don't suppose you have a backup of the data on the volume?
Hi,
well no. It's quite a lot of data, was around 5 TB (I had 4 x 2 TB drives), so not actually a lot of space to back it up twice. The NAS was supposed to be the "redundant" backup 😉
So I'm open to any suggestions.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Sending logs is described here, you just download them from the nas then email them in:
http://kb.netgear.com/app/answers/detail/a_id/21543
There are many scenarios that a redundant volume does not offer protection against, therefore backups are essential.
What model readynas is it?
I think you need to ask Netgear support to look to look at this, open a support case and be prepared to pay.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
@vandermerwe wrote:Sending logs is described here, you just download them from the nas then email them in:
http://kb.netgear.com/app/answers/detail/a_id/21543
There are many scenarios that a redundant volume does not offer protection against, therefore backups are essential.
What model readynas is it?
I think you need to ask Netgear support to look to look at this, open a support case and be prepared to pay.
Thanks for this. I have sent the logs. I have already contacted Netgear, but as you probably know / assume they are not very eager to help. At the moment they are looking into the case. However, I assume that the Netgear implementation of RAID is based on existing technolgoies, and that there is maybe at least one or two rebuild or salvage commands I can try. I was hoping I could maybe find an answer here.
The model is RNS-104, was bought diskless, and has 4 drives in it.
Thanks in advance for your time,
Alex
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Hi azerva,
I looked at some information on the logs you have sent and on my understanding, it appears that Drive 4 may have also failed. To confirm this, please download disk diagnostic tools from Western Digital (WD lifeguard) or Seagate's(Seatools) webiste and check your drives. My hopes are still high that your data will be recovered.
I'll look forward to your response.
Thanks,
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Your logs show that disk 4 has been giving errors for quite some time.
It would have been good to have been asked before there was a big problem whether the errors were a cause of concern.
When you replace a failed disk, the resulting resync that puts heavy stress on all disks can finish off another disk that is failing.
ATA Error Count: 90 Reallocated Sectors: 696 Reallocation Events: 696 Spin Retry Count: 0 End-to-End Errors: 0 Command Timeouts: 0 Current Pending Sector Count: 48 Uncorrectable Sector Count: 48 Temperature: 47 Start/Stop Count: 1554 Power-On Hours: 14041 Power Cycle Count: 232 Load Cycle Count: 1555
Those ATA errors, reallocated sectors, current pending sectors and uncorrectable sector counts considered together indicate very clearly that this disk is bad.
An attempt to clone this disk e.g. using dd_rescue may be necessary.
This looks like a data recovery situation.
You may need to consider purchasing data recovery services. Support could explain the cost and the scope of what is covered.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Hi mdgm,
thanks for your reply. I remember only getting one e-mail notification from the system regarding disk 4, somehow I was not aware that there were so many errors reported by the disk already, otherwise I would have mentioned it. Also, disk 4 is online now, that's what I find a little strange. When disk 3 failed a week ago, it was really offline, i.e. red and could not start. I even tried putting in a usb docking port and it didnt boot up. Disk 4 however "seems" to be fine now. This is why I still have some hope left that there is something I can do.
Fri Jul 24 2015 2:15:52 Disk: Disk in channel '4' (Internal) changed state from ONLINE to FAILED. Fri Jul 24 2015 2:15:49 Disk: Disk in channel '3' (Internal) changed state from RESYNC to ONLINE. Fri Jul 24 2015 2:15:46 Volume: The rebuild operation has finished on volume 'data'. However, the volume is still degraded.
By loooking at the timestamps of the events, it would seem that the resync finished just seconds (02:15:46) before disk 4 failed (02:15:52), but I guess this is a reporting error and the events probably happened the other way around, and the "failure" of disk 4 caused the resync to fail and it was just reported the other way around. What do you think, is this a correct assumption? Otherwise, If the resync did finish before disk 4 failed, then I would expect to be able to use the device like I did after disk 3 failed, i.e. volume in a degraded state with disk 4 offline.
All in all it seems that disks 3 and 4 were of poor quality, since both were bought at the same time and seem to have failed at almost the same time, even before warranty ran out.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
@JohnRo wrote:Hi azerva,
I looked at some information on the logs you have sent and on my understanding, it appears that Drive 4 may have also failed. To confirm this, please download disk diagnostic tools from Western Digital (WD lifeguard) or Seagate's(Seatools) webiste and check your drives. My hopes are still high that your data will be recovered.
I'll look forward to your response.
Thanks,
Hi JohnRo,
I connected disk 4 to my PC and I can see it using SeaTools. Which Check / Repair should I try to do?
Thanks in advance.
Alex
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
The problem with the disk caused the resync to fail.
In the status log and elsewhere I see a number of mentions of errors dating back some months.
Before replacing a disk that has failed it is advisable to check to see if any other disks have problems as well as consider where you are regarding backups.
For new disk purchases I would go with e.g. WD RED disks or SeaGate NAS disks.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
@mdgm wrote:The problem with the disk caused the resync to fail.
In the status log and elsewhere I see a number of mentions of errors dating back some months.
Before replacing a disk that has failed it is advisable to check to see if any other disks have problems as well as consider where you are regarding backups.
For new disk purchases I would go with e.g. WD RED disks or SeaGate NAS disks.
Hi,
thanks for your reply. I am also pretty sure that this is what happened. However, and I'd really like to come back to this, unlike disk 3 which failed completly, disk 4 is still accessible (hooked up to my PC right now) and I would like some advice or opinions about what I could at least try before giving up on the data. I have the Seagate Tools installed (since its a Seagate Disk), a SHORT DST Test failed. The tool recommends to use the DOS version to try and repair the bad sectors.
Should I give this a go or does anyone have any better ideas?
Thanks
Alex
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
No.
Wait for Netgear support to see if data recovery is an option. If they haven't offered it, ask them.
If you start altering the data on the disks then it is likely you will make data recovery more difficult. It is possible that cloning disks and trying to reassemble the raid structure may be what is needed, but leave this to someone with expertise, and be prepared to pay for it.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace
Hello azerva,
I replied to your PM. Like what mdgm and vandermerwe advised you to do, contact support and leave this to the experts.
The data you have in that disk 4 cannot be retrieved without the other 2 original disks. Put the disk back to the NAS bay where you removed it and have the system logs ready as you will have to give that to the support center.
You may need to prepare a spare disk to clone disk 4. But let support check first if it is still possible before getting one (if you do not a have spare disk and will have to buy).
here's what mdgm replied yesterday to this topic.
Those ATA errors, reallocated sectors, current pending sectors and uncorrectable sector counts considered together indicate very clearly that this disk is bad. An attempt to clone this disk e.g. using dd_rescue may be necessary. This looks like a data recovery situation. You may need to consider purchasing data recovery services. Support could explain the cost and the scope of what is covered.
Regards,
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace #25483643
Hi guys,
update on the whole situation. Netgear Support recommended cloning the disk with ddrescue. I did this, the process finished with 4 errors. Below is the log file.
# Command line: ddrescue -fv /dev/sdb /dev/sda ddrescue.log # current_pos current_status 0x14419EC7C00 + # pos size status 0x00000000 0x139A33C4000 + 0x139A33C4000 0x00001000 - 0x139A33C5000 0x00004000 + 0x139A33C9000 0x00001000 - 0x139A33CA000 0xA76AF6000 + 0x14419EC0000 0x00001000 - 0x14419EC1000 0x00005000 + 0x14419EC6000 0x00002000 - 0x14419EC8000 0x8DA724E000 +
If I am reading this correctly, there are 4 relatively small areas (designated by the - at the end which are bad). I tried to start ReadyNAS with disks 1, 2 and the cloned disk 4, however this didn't work.
Are there any additional steps I need to consider? Should I consider re-running ddrescue using the -r option for more retries? Or is there a command to try to re-detect the RAID volume?
Thanks for any hints.
Alex
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace #25483643
Are support not following up on this case?
Did they offer you data recovery services?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Data Volume Lost after Disk Replace #25483643
Hello azerva,
I strongly recommend not to touch anything but just follow thier instructions. I don't think booting up the NAS with the remaining disks along with the cloned one will rebuild the RAID and get you access the data right away, there are still few steps they will have to do once you have the disk cloned. Please contact the support center to continue with it.
Regards,