I am a systems engineer and I have never seen a software raid (madm) deteriorate so much and all alone. I have 4 WD red drives that have 3 months on them, seriously, I think the linux (improved) by netgear is destructive. I'd like to at least know what disk configuration they call X-raid on a nas 104 with 4*4TB disks. If I had that information, maybe I could fix it. If I don't find a solution, this is the last time in my life I buy a netgear product! and the last also in my work.
nobody has a solution, I have a friend who has a synology for 7 years without ever problem, me, on my readynas already 3 big problem in 2 years... whitout support. People lose all their data because of you, I think I will buy a synology, they are much better than you.
I just realized that the file system is btrfs... 😞 Could someone from the community give me the result of the command on a nas 104 pen in firmware version 6.9.3. Thank you in advance:
I dont' send my log in attanchement because .zip extension is prohibited , i don't test read only file system, how do I start read only?
I dont' send my log in attanchement because .zip extension is prohibited , i don't test read only file system, how do I start read only?
There is some privacy leakage if you post the logs publicly. That's why @mdgm-ntgr directly you to the "sending logs" link in his signature. It includes an email address for sending the logs.
Hi, thanks so much for your help. My problem happened when I overheated (I think). I could no longer connect (ssh and http). I restarted it, after restarting it only saw a full 10TB volume (then less than 1TB was really used). After a second restart, he sees 4 volumes. I'm here... Another thing I don't understand is that madm seems loaded while Netgear seems to use btrfs on its NAS... Here is the return of the command:
thanks by advance for you hlep. My problem is that i don't know the standard configuration for this nas. if I knew that, I would have a chance to fix it, I don't understand why nobody here can give me this informations..
Ok, thank for this work, isn't ok, it's a little best :
i use "--really-force" because --force have not effect, look the log :
# mdadm --assemble -force --scan mdadm: NOT forcing event count in /dev/sda5(2) from 4733 up to 4736 mdadm: You can use --really-force to do that (DANGEROUS) mdadm: /dev/md/data-2 assembled from 2 drives - not enough to start the array. mdadm: NOT forcing event count in /dev/sdc4(0) from 9888 up to 9891 mdadm: You can use --really-force to do that (DANGEROUS) mdadm: /dev/md/data-1 assembled from 2 drives - not enough to start the array. mdadm: NOT forcing event count in /dev/sdc3(1) from 27559 up to 27562 mdadm: You can use --really-force to do that (DANGEROUS) mdadm: /dev/md/data-0 assembled from 2 drives - not enough to start the array. mdadm: /dev/md/1_0 assembled from 1 drive - not enough to start the array. mdadm: No arrays found in config file or automatically
btrfs device scan Scanning for Btrfs filesystems (return nothing) btrfs fi show (return nothing) mount -o ro/dev/md127 /data mount: can't find LABEL=0e356676:data
So i try :
mdadm --assemble --really-force --scan
mdadm: forcing event count in /dev/sda5(2) from 4733 upto 4736 mdadm: forcing event count in /dev/sdc5(3) from 4733 upto 4736 mdadm: clearing FAULTY flag for device 3 in /dev/md/data-2 for /dev/sda5 mdadm: clearing FAULTY flag for device 1 in /dev/md/data-2 for /dev/sdc5 mdadm: Marking array /dev/md/data-2 as 'clean' mdadm: /dev/md/data-2 has been started with 4 drives. mdadm: forcing event count in /dev/sdc4(0) from 9888 upto 9891 mdadm: forcing event count in /dev/sda4(2) from 9888 upto 9891 mdadm: /dev/md/data-1 assembled from 4 drives - not enough to start the array. mdadm: forcing event count in /dev/sdc3(1) from 27559 upto 27562 mdadm: forcing event count in /dev/sda3(3) from 27559 upto 27562 mdadm: clearing FAULTY flag for device 1 in /dev/md/data-0 for /dev/sdc3 mdadm: clearing FAULTY flag for device 3 in /dev/md/data-0 for /dev/sda3 mdadm: Marking array /dev/md/data-0 as 'clean' mdadm: /dev/md/data-0 has been started with 4 drives. mdadm: /dev/md/1_0 assembled from 1 drive - not enough to start the array. mdadm: /dev/md/data-1 has been started with 4 drives. mdadm: /dev/md/1_0 assembled from 1 drive - not enough to start the array. mdadm: /dev/md/1_0 assembled from 1 drive - not enough to start the array.
root@Nas:~# btrfs device scan Scanning for Btrfs filesystems root@Nas:~# btrfs fi show Label: '0e356676:data' uuid: 111e17a0-275f-492f-8efe-763b2e680abe Total devices 3 FS bytes used 685.07GiB devid 1 size 5.44TiB used 689.02GiB path /dev/md126 devid 2 size 2.73TiB used 0.00B path /dev/md125 devid 3 size 2.73TiB used 0.00B path /dev/md127
mount -o ro/dev/md126 /data mount: wrong fs type, bad option, bad superblock on /dev/md125, missing codepage or helper program, or other error
In some cases useful info is found in syslog - try dmesg | tail or so.
dmesg : [206091.700507] md/raid:md127: device sdb5 operational as raid disk 0 [206091.700528] md/raid:md127: device sdc5 operational as raid disk 3 [206091.700537] md/raid:md127: device sda5 operational as raid disk 2 [206091.700544] md/raid:md127: device sdd5 operational as raid disk 1 [206091.702520] md/raid:md127: allocated 4294kB [206091.707565] md/raid:md127: raid level 5 active with 4 out of 4 devices, algorithm 2 [206091.707583] RAID conf printout: [206091.707590] --- level:5 rd:4 wd:4 [206091.707599] disk 0, o:1, dev:sdb5 [206091.707606] disk 1, o:1, dev:sdd5 [206091.707613] disk 2, o:1, dev:sda5 [206091.707620] disk 3, o:1, dev:sdc5 [206091.720371] md127: detected capacity change from 0 to 3000179490816 [206092.078846] BTRFS: device label 0e356676:data devid 3 transid 37485 /dev/md127 [206092.215249] BTRFS info (device md127): setting nodatasum [206092.215274] BTRFS info (device md127): has skinny extents [206092.269780] BTRFS error (device md127): failed to read the system array: -5 [206092.291085] BTRFS error (device md127): open_ctree failed [206092.347978] md: md126 stopped. [206092.421870] md: bind<sdb4> [206092.422458] md: bind<sda4> [206092.422979] md: bind<sdd4> [206092.423491] md: bind<sdc4> [206092.424578] md: md126 stopped. [206092.424612] md: unbind<sdc4> [206092.424637] md: export_rdev(sdc4) [206092.424815] md: unbind<sdd4> [206092.424834] md: export_rdev(sdd4) [206092.424957] md: unbind<sda4> [206092.424975] md: export_rdev(sda4) [206092.425093] md: unbind<sdb4> [206092.425111] md: export_rdev(sdb4) [206092.513348] md: md126 stopped. [206092.677339] md: bind<sdc3> [206092.677891] md: bind<sdb3> [206092.684506] md: bind<sda3> [206092.685045] md: bind<sdd3> [206092.720393] md/raid:md126: device sdd3 operational as raid disk 0 [206092.720414] md/raid:md126: device sda3 operational as raid disk 3 [206092.720422] md/raid:md126: device sdb3 operational as raid disk 2 [206092.720430] md/raid:md126: device sdc3 operational as raid disk 1 [206092.722088] md/raid:md126: allocated 4294kB [206092.731956] md/raid:md126: raid level 5 active with 4 out of 4 devices, algorithm 2 [206092.731973] RAID conf printout: [206092.731980] --- level:5 rd:4 wd:4 [206092.731988] disk 0, o:1, dev:sdd3 [206092.731995] disk 1, o:1, dev:sdc3 [206092.732003] disk 2, o:1, dev:sdb3 [206092.732010] disk 3, o:1, dev:sda3 [206092.753033] md126: detected capacity change from 0 to 5986298363904 [206093.258553] BTRFS: device label 0e356676:data devid 1 transid 37485 /dev/md126 [206093.382343] md: md125 stopped. [206093.420611] md: bind<sdc2> [206093.421723] md: md125 stopped. [206093.421756] md: unbind<sdc2> [206093.421780] md: export_rdev(sdc2) [206093.741556] md: md125 stopped. [206093.760535] md: bind<sdb4> [206093.761170] md: bind<sda4> [206093.765525] md: bind<sdd4> [206093.770133] md: bind<sdc4> [206093.793645] md/raid:md125: device sdc4 operational as raid disk 0 [206093.793664] md/raid:md125: device sdd4 operational as raid disk 3 [206093.793673] md/raid:md125: device sda4 operational as raid disk 2 [206093.793680] md/raid:md125: device sdb4 operational as raid disk 1 [206093.795355] md/raid:md125: allocated 4294kB [206093.795736] md/raid:md125: raid level 5 active with 4 out of 4 devices, algorithm 2 [206093.795747] RAID conf printout: [206093.795754] --- level:5 rd:4 wd:4 [206093.795761] disk 0, o:1, dev:sdc4 [206093.795768] disk 1, o:1, dev:sdb4 [206093.795776] disk 2, o:1, dev:sda4 [206093.795783] disk 3, o:1, dev:sdd4 [206093.820383] md125: detected capacity change from 0 to 3000179490816 [206094.353848] BTRFS: device label 0e356676:data devid 2 transid 37485 /dev/md125 [206094.487569] md: md124 stopped. [206094.510461] md: bind<sdc2> [206094.511563] md: md124 stopped. [206094.511600] md: unbind<sdc2> [206094.511625] md: export_rdev(sdc2) [206094.924525] md: md124 stopped. [206094.939646] md: bind<sdc2> [206094.941350] md: md124 stopped. [206094.941388] md: unbind<sdc2> [206094.941416] md: export_rdev(sdc2) [206118.617487] BTRFS info (device md125): setting nodatasum [206118.617518] BTRFS info (device md125): unrecognized mount option 'ro/dev/md127' [206118.630395] BTRFS error (device md125): open_ctree failed [206166.628712] BTRFS info (device md125): setting nodatasum [206166.628741] BTRFS info (device md125): unrecognized mount option 'ro/dev/md126' [206166.640391] BTRFS error (device md125): open_ctree failed
I write very rarely on my NAS, it allows me to save my family photos from time to time. My hard drives are only 3 months old, (WD Red 4TB) . I think like you, has a de-syncronization, I don't think I have a corrupted file. I don't know how to sync again, or mount a disk to get my data.
I can see from your logs that you used to use ST2000DM001 disks. Those disks were notoriously unreliable. You would have had problems using those disks in any system.
If you can access your data now I would backup that data, verify the data is O.K. then do a factory reset and restore your data from backup.
You'd get a clean single-layer array with your new 4TB disks as opposed to the triple layer array you have at the moment.
Your dmesg.log shows that 2 disks dropped out of your array, sda and sdc (which are disks 1 and 2 - or 0 and 1 if counting from 0 - as you can see in disk_info.log).
You may wish to check the health of these disks e.g. using WD Data LifeGuard Diagnostics.
@mdmg, hello and thank you for your answer. I removed the ST2000DM001 disk several months ago for the reasons you gave. The problem I have now happened after, I replaced the records several months ago. Right now I don't have access to the data. If I try to mount the discs I get the following back:
mount ro/dev/md127 /data/
mount: special device ro/dev/md127 does not exist
root@Nas:~# mount -o ro/dev/md127 /data
mount: wrong fs type, bad option, bad superblock on /dev/md125,
missing codepage or helper program, or other error
btrfs fi show Label: '0e356676:data' uuid: 111e17a0-275f-492f-8efe-763b2e680abe Total devices 3 FS bytes used 685.07GiB devid 1 size 5.44TiB used 689.02GiB path /dev/md125 devid 2 size 2.73TiB used 0.00B path /dev/md126 devid 3 size 2.73TiB used 0.00B path /dev/md127
StephenB My system is currently booted to read-only.
I really don't think one of my disks is degraded, because they are new, I think the raid configuration information is corrupted. I don't know how to fix this.
I really don't think one of my disks is degraded, because they are new, I think the raid configuration information is corrupted. I don't know how to fix this.
To clarify the terminology a bit: A RAID-5 volume is degraded when one disk is missing (or has failed). The data is still available, thanks to the RAID redundancy. Replacing the disk cures that problem.
I generally won't describe a disk as "degraded", because that creates some confusion with the degraded volume status. Instead I will say it is "failing" or "failed".
In your specific case, two of the disks lost writes from the NAS - that is why the event counters didn't match. Forcing the event counters allows the RAID array to be assembled. But there is damage/corruption to the file system that runs on the RAID array, due to the lost writes. How extensive that damage is depends on exactly what was lost. A small number of writes might damage a single file, or a folder - or could compromise the entire volume (for instance, corrupting the superblock).
So there likely will be some need for file system repair. It's possible that your mount method is causing the problem (I'm not seeing any btrfs commands before the mount). That is why I'm asking if you rebooted after the event counter forcing, as the system would issue the correct commands.
Normally I'd expect to see
# btrfs device scan
# btrfs fi show
# mount -o ro /dev/md127 /data
I'm not seeing the device scan, and the mount command appears to be missing the -o.
