I do not mind movies, music and stuff, but I just was replacing my workstation which I use to do my job stuff, so I have moved about 2 Tb of important data to the NAS, checked for consistency and destroyed the source...Well... it turns out now I have a problem...:(
Since it is responding to ping, and I can ssh as root to it, I was thinking... Maybe it is possible to reasamble raid using LVM or mdadm ? It is inactive, not destroyed or degraded after all... Problem is I do not know Netgear specific changes done to regular Linux...
Since it is responding to ping, and I can ssh as root to it, I was thinking... Maybe it is possible to reasamble raid using LVM or mdadm ? It is inactive, not destroyed or degraded after all...
The SMP error generally is a memory-related issue (for instance de-referencing a null pointer). So you might want to check the memory seating and run the boot-menu memory diagnostic. Another test would be to remove all the drives (labeling by slot), and doing a factory install to a test disk (that you don't mind reformatting). That would give you reasonable confirmation that the chassis is working normally.
OS-6 does use mdadm, but this is a btrfs error. That sugggests that the RAID array is assembled correctly, but there is file system corruption. Note the OS partition itself is also formatted as btrfs. You could try booting the system into tech support mode. From there, you can enter the commands below to mount the data volume as read-only. Note this assumes a healthy array, so you likely will get an error somewhere along the way.
# start_raids
# rnutil chroot
# btrfs device scan # btrfs fi show # mount -o -ro /dev/md127 /data
Many thanks for this... I'd never think about memory modules....
Well... so far I did the first part... ran memory test...
3 passes total (1 sucessful, 2 with error) - went strangly quick (3 passes took below 20 minutes together, 2x2gb PC2 modules, 4Gb in total)...
So I replaced memory modules (I put 2x1Gb PC2) and ran test... this time, 3 pasess took nearly 60 minutes (while it was just half of original capacity) and no errors...
So I left the new memory in and ran the device.
Still no joy - RAID inactive... but this time... RAIDar diagnostics says :
System
Volume data is inactive
Logs
No errors found.
So... looks like it is a bit better... More of that... admin page still not working (not even Neatgear logo or so - just regular page not found), but I can now see the shares in windows...I can't open them tough - getting windows error 0x80070035 - network path not found...
df -h tells me I have 4gb partition with 44% usage, but btrfs fi show tells me that 3,98gb of 4gb is used...
A full OS partition will create a variety of issues, and btrfs fi show is saying that it there is no free space.
What balance command are you using? Perhaps try
# btrfs balance start -dusage=0 //
That shouldn't require any working space. It only reclaims totally unused block-groups, so it might not be enough. However, it is possible that it will reclaim enough that you can do a full balance when it completes.
Note there might be corrupted configuration files on the NAS as a result of the full OS partition. So it might not totally fix your problem.
that went with no problem but said reclaimed 0 of 60 or 70 (can't remember exact number)
then i increased usage to 1 and that throw well known error...
I also did:
btrfs fi df
to see space used divided by system, data and metadata... but nothing strange there...the numbers are correct, and in total the space used is almost identical as what df -h says... From this it doesn't look like system or metadata is eating the free space...
My files are not visible in /data... and md127 is not curently mounted... I just find out that dev/md127 mountpoint is/data so this should be mine stuff... but not at this moment as mount dev/md127 /data shows....supprise....'no space......':(
I wonder how to find out where this disk space went... as usual methods like df are not realiable...
df isn't reliable for btrfs, which is why btrfs has it's own df command. But since that is giving the same information, it appears that there is some file system corruption in the OS partition.
Did you try mounting the data volume after booting in tech support mode as I described earlier?
If that is failing, and Netgear doesn't choose to take a look remotely, then you might need to do a factory reset (which will lose data). ReclaiMe might be another option to offload the data, though it would require you to find a way to connect all the disks to a Windows PC.
Well... btrfs fi df gives similar output than df -h, but that differs from what btrfs fi show says...
I did booting in tech mode, but somehow ssh connection is refused in this mode. In normal mode it works fine.
Tech mode gives me a 5 digit service number, but have no idea how to use it - do not know what software tech support uses... Normal ssh in this modedoesn't work for me...
In the meantime, I removed all disks, put a spare one, did factory reset... and voila - all works fine including admin page and samba shares... So chasis itself is probably ok.
In the meantime, I removed all disks, put a spare one, did factory reset... and voila - all works fine including admin page and samba shares... So chassis itself is probably ok.
All I'm doing at this moment is as root, via SSH... somehow the box refuses telnet connection.
I managed to remove some temp stuff, enough to run balancing... I was able to (step by step) achieve parameter usage=90... So at this moment:
# root@NAS:/# btrfs fi show # Label: '33eadc2d:root' uuid: a6c6dff0-165c-459e-bf3d-59ecfe3f43a2 Total devices 1 FS bytes used 1.51GiB devid 1 size 4.00GiB used 2.45GiB path /dev/md0
# Label: '33eadc2d:data' uuid: 88ad7d29-4753-49d9-896a-56a19732199a Total devices 1 FS bytes used 2.88TiB devid 1 size 7.26TiB used 2.89TiB path /dev/md127
This is far better than before... but still getting 'no space left on device' when trying to install stuff... and after reboot, I can't do the balancing anymore - says no space.... wierd...
I have not too much experience with RAID sollutions (better would be LVM 🙂 - can you tell me what, in working device, should be in fstab and mdabm.conf ? And also what should be expected output from mdadm --examine --scan
I was also trying to mount md127 by
# root@NAS:/dev# mount /dev/md127 /data # mount: wrong fs type, bad option, bad superblock on /dev/md127, # missing codepage or helper program, or other error # In some cases useful info is found in syslog - try # dmesg | tail or so. # root@NAS:/dev# dmesg | tail # [ 9460.518737] BTRFS warning (device md127): md127 checksum verify failed on 1441013137408 wanted 449FF305 found D94C67D0 level 1 # [ 9460.533652] BTRFS warning (device md127): md127 checksum verify failed on 1441013137408 wanted 449FF305 found D94C67D0 level 1 # [ 9460.533659] BTRFS error (device md127): failed to read block groups: -5 # [ 9460.534041] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.572370] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.593738] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.612037] BTRFS error (device md127): parent transid verify failed on 1441013628928 wanted 312635 found 305195 # [ 9460.619276] BTRFS error (device md127): parent transid verify failed on 1441013628928 wanted 312635 found 305195 # [ 9460.619280] BTRFS warning (device md127): failed to read tree root # [ 9460.632015] BTRFS error (device md127): open_ctree failed
All I'm doing at this moment is as root, via SSH... somehow the box refuses telnet connection.
I managed to remove some temp stuff, enough to run balancing... I was able to (step by step) achieve parameter usage=90... So at this moment:
# root@NAS:/# btrfs fi show # Label: '33eadc2d:root' uuid: a6c6dff0-165c-459e-bf3d-59ecfe3f43a2 Total devices 1 FS bytes used 1.51GiB devid 1 size 4.00GiB used 2.45GiB path /dev/md0
# Label: '33eadc2d:data' uuid: 88ad7d29-4753-49d9-896a-56a19732199a Total devices 1 FS bytes used 2.88TiB devid 1 size 7.26TiB used 2.89TiB path /dev/md127
This is far better than before... but still getting 'no space left on device' when trying to install stuff... and after reboot, I can't do the balancing anymore - says no space.... wierd...
I have not too much experience with RAID sollutions (better would be LVM 🙂 - can you tell me what, in working device, should be in fstab and mdabm.conf ? And also what should be expected output from mdadm --examine --scan
I was also trying to mount md127 by
# root@NAS:/dev# mount /dev/md127 /data # mount: wrong fs type, bad option, bad superblock on /dev/md127, # missing codepage or helper program, or other error # In some cases useful info is found in syslog - try # dmesg | tail or so. # root@NAS:/dev# dmesg | tail # [ 9460.518737] BTRFS warning (device md127): md127 checksum verify failed on 1441013137408 wanted 449FF305 found D94C67D0 level 1 # [ 9460.533652] BTRFS warning (device md127): md127 checksum verify failed on 1441013137408 wanted 449FF305 found D94C67D0 level 1 # [ 9460.533659] BTRFS error (device md127): failed to read block groups: -5 # [ 9460.534041] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.572370] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.593738] BTRFS error (device md127): failed to read block groups: -17 # [ 9460.612037] BTRFS error (device md127): parent transid verify failed on 1441013628928 wanted 312635 found 305195 # [ 9460.619276] BTRFS error (device md127): parent transid verify failed on 1441013628928 wanted 312635 found 305195 # [ 9460.619280] BTRFS warning (device md127): failed to read tree root # [ 9460.632015] BTRFS error (device md127): open_ctree failed
I have not too much experience with RAID sollutions (better would be LVM 🙂 - can you tell me what, in working device, should be in fstab and mdabm.conf ? And also what should be expected output from mdadm --examine --scan
and why my posts keep dissapear, just after I post them ??:(
and why my posts keep dissapear, just after I post them ??:(
The automatic spam filter is kicking in. Netgear mods are supposed to check that periodically, but sometimes they don't get to it. If it happens on future posts, you can PM me - I'm not a mod, but I do have the ability to release them.
root@NAS:~# mdadm -D /dev/md127 /dev/md127: Version : 1.2 Creation Time : Wed Nov 9 09:56:02 2016 Raid Level : raid5 Array Size : 17567012352 (16753.21 GiB 17988.62 GB) Used Dev Size : 5855670784 (5584.40 GiB 5996.21 GB) Raid Devices : 4 Total Devices : 4 Persistence : Superblock is persistent
Update Time : Thu Dec 20 07:08:12 2018 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0
Layout : left-symmetric Chunk Size : 64K
Name : 2fe72582:data-0 (local to host 2fe72582) UUID : a5c0aee6:27d8df34:a416f3ca:4092b6f4 Events : 4034
Number Major Minor RaidDevice State 0 8 3 0 active sync /dev/sda3 1 8 19 1 active sync /dev/sdb3 2 8 35 2 active sync /dev/sdc3 3 8 51 3 active sync /dev/sdd3
Hi 🙂 Thanks for this input... it looks the same as mine (to me at least 🙂
I managed to log in by telnet - I had to copy/paste the password - from keyboard somehow did not work...
Anyway... I entered your commands:
# start_raids
# rnutil chroot
# btrfs device scan
# btrfs fi show
# mount -o -ro /dev/md127 /data
I put my output to the file attached... looks like the md127 is not mounted... it is looking for data , not for data-0. Have no idea how to correct this...
Well.... that didn't do the job... same faut - wrong fs type bla bla bla...
I can see 2 problems - do not know if related...
When using telnet mode, after #start_raids it is starting 3 arrays (root, swap and data), and wants to mount the data array. That's generating error 'no such file or directory) - shown on jpg file in previous post...And mount returns wrong fs type....
In SSH mode however, commands like #btrfs restore...generates checksum error
checksum verify failed on 1441013137408 found D94C67D0 wanted 449FF305
Csum didn't match
Could not open root, trying backup super
The most frustrating is that partition is clean (so active), so the data probably are intact, just inaccessable...and I'm close to the limit of my possibilities to make this working...
What I wonder... is that why I can't mount it... ok - file system corrupted, data unreadable... but it should be possible to mount it...
That is why I'm still thinking... why #start_raids is reporting finding /dev/md/data-0 instead of data... And what is relation between /dev/md/1 /dev/md/0 /dev/md/data-0 with /dev/md1 md2 and md127 ...Why it messes up with 3 instances of the same partition (data, data-0, md127)??...
why #start_raids is reporting finding /dev/md/data-0 instead of data...
I think that's just the way Netgear creates the RAID array. My own system shows /dev/md/data-0 and that is mapped to md127, and mounted as data. So that part is not wrong.
I think that linux is unable to determine the file system correctly (perhaps consistent with the btrfs checksum error you saw).
Did you try using the -i (ignore errors) option with btrfs restore?
