Volume scan failed to run properly #26188803

Hallelujah!

I have now managed to get my data volume back up! Unfortunately, Netgear L2 tech did not manage and I had to do this by myself.

As promised I will give you the results. Note that I am not a trained IT-engineer, so I got this from Google search and dedication. If you decide to do this it is on your own risk - I take no responsibility that it will work on your system.

I had only 4 sata-ports on my PC, and had to install Linux on a USB in order to get all 4 disks connected. I wanted to use Knoppix Linux, but I am sure most Linux versions would do.

1. Get Linux installed on the computer:

I installed Knoppix Linux to a USB-stick. This was not trivial as I had a new Z170 motherboard and a regular USB-boot would not work, using Universial-USB-installer-1.9.6.3 or unetbootin-windows-613. I ended up attaching a sata DVD and burned a Knoppix DVD, booted from the DVD and installed Knoppix on the USB.

1. Removed the DVD-RW drive and attached all the 4 NAS-drives to the PC. Booted up in Knoppix (USB) and started to see if I could access the drives. I noticed that Knoppix displayed one mounted volume and a few volumes that was not mounted (these volumes turned out to be LVM physical volumes). I could access the files on the mounted RAID volume which turned out to be the NAS OS.

1. I performed several different commands after some Google search, being careful not to run anything that could make changes to the drive, in case I needed to perform some data recovery.

First I checked the partition tables on all four drives using gdisk:

knoppix@Microknoppix:~$ sudo gdisk /dev/sda

They were identical, not shown as I did this in four different windows.

Then I wanted to checked the raid setup and ran:

knoppix@Microknoppix:~$ sudo mdadm --detail --scan

ARRAY /dev/md/4 metadata=1.2 name=A021B7C18D0C:4 UUID=d6301b60:0ce2f767:558c574f:db007ccb

ARRAY /dev/md/1 metadata=1.2 name=A021B7C18D0C:1 UUID=d2791ec8:5adda84e:c7463c2e:c0f2016b

ARRAY /dev/md/0 metadata=1.2 name=A021B7C18D0C:0 UUID=a218f0a3:1b607e2e:953b087b:04ed9c99

INACTIVE-ARRAY /dev/md3 metadata=1.2 name=A021B7C18D0C:3 UUID=5aa62eb3:fa4e39b8:213486da:d587542d

ARRAY /dev/md/2 metadata=1.2 name=A021B7C18D0C:2 UUID=829ccffc:55683ba6:36bb7959:6eed3523

 From this I figured out there was an inactive array md3.

Then I used e2fsck to check the partition:

knoppix@Microknoppix:~$ e2fsck /dev/md3

e2fsck 1.42.13 (17-May-2015)

e2fsck: Invalid argument while trying to open /dev/md3

 

The superblock could not be read or does not describe a valid ext2/ext3/ext4

filesystem. If the device is valid and it really contains an ext2/ext3/ext4

filesystem (and not swap or ufs or something else), then the superblock

is corrupt, and you might try running e2fsck with an alternate superblock:

   e2fsck -b 8193 <device>

or

   e2fsck -b 32768 <device>

This made me think there was a problem with the superblocks on the partitions, that turned out not to be important. Searching and looking for answers I decided to stop the array and start it again:

knoppix@Microknoppix:~$ sudo mdadm --stop --scan

knoppix@Microknoppix:~$ sudo mdadm --assemble --scan

mdadm: /dev/md/4 has been started with 2 drives (out of 3).

mdadm: restoring critical section

mdadm: /dev/md/3 has been started with 4 drives.

mdadm: /dev/md/2 has been started with 4 drives.

mdadm: /dev/md/1 has been started with 4 drives.

mdadm: /dev/md/0 has been started with 4 drives.

mdadm: Found some drive for an array that is already active: /dev/md/4

mdadm: giving up.

Then used lvmdiskscan to see if I could see the volumes and if there was a problem with any of them :

knoppix@Microknoppix:~$ sudo lvmdiskscan

/run/lvm/lvmetad.socket: connect failed: No such file or directory

WARNING: Failed to connect to lvmetad. Falling back to internal scanning.

/dev/ram0 [       4.00 MiB]

/dev/md0   [       4.00 GiB]

/dev/ram1 [       4.00 MiB]

/dev/md1   [   1023.88 MiB]

/dev/ram2 [       4.00 MiB]

/dev/md2   [       4.08 TiB] LVM physical volume

/dev/ram3 [       4.00 MiB]

/dev/md3   [     931.50 GiB] LVM physical volume

/dev/ram4 [       4.00 MiB]

/dev/md4   [       3.64 TiB] LVM physical volume

/dev/ram5 [       4.00 MiB]

/dev/ram6 [       4.00 MiB]

/dev/ram7 [       4.00 MiB]

/dev/ram8 [       4.00 MiB]

/dev/ram9 [       4.00 MiB]

/dev/ram10 [       4.00 MiB]

/dev/ram11 [       4.00 MiB]

/dev/ram12 [       4.00 MiB]

/dev/ram13 [       4.00 MiB]

/dev/ram14 [       4.00 MiB]

/dev/ram15 [       4.00 MiB]

/dev/sde1 [       4.46 GiB]

/dev/sde2 [     24.82 GiB]

/dev/sdf1 [       4.46 GiB]

0 disks

21 partitions

0 LVM physical volume whole disks

3 LVM physical volumes

There was 3 volumes listed. Followed up with lvdisplay to see the logical volume:

knoppix@Microknoppix:~$ sudo lvdisplay

/run/lvm/lvmetad.socket: connect failed: No such file or directory

WARNING: Failed to connect to lvmetad. Falling back to internal scanning.

--- Logical volume ---

LV Path               /dev/c/c

LV Name               c

VG Name               c

LV UUID               DHaiSO-OE5j-wbTe-rW1L-Zh1L-DNFP-vbPjvA

LV Write Access       read/write

LV Creation host, time ,

LV Status             NOT available

LV Size               6.80 TiB

Current LE             111404

Segments               3

Allocation             inherit

Read ahead sectors     auto

From here I assumed the volume c was not available. Followed up with lvscan:

knoppix@Microknoppix:~$ sudo lvscan

/run/lvm/lvmetad.socket: connect failed: No such file or directory

WARNING: Failed to connect to lvmetad. Falling back to internal scanning.

inactive         '/dev/c/c' [6.80 TiB] inherit

Hmm. The data volume (c) was inactive. Now, I had previously tried to activate the array using mdadm --detail --scan. I searched the web further and came across this site/post that solved the case: http://pissedoffadmins.com/os/mount-unknown-filesystem-type-lvm2_member.html

knoppix@Microknoppix:~$ modprobe dm-mod

knoppix@Microknoppix:~$ sudo vgchange -ay

/run/lvm/lvmetad.socket: connect failed: No such file or directory

WARNING: Failed to connect to lvmetad. Falling back to internal scanning.

1 logical volume(s) in volume group "c" now active

Voila! The volume came up and I then managed to mount it! I put all the disks back in the Netgear NAS and it booted normally. I am now transferring files to the other backup Netgear NAS as we speak. I guess this will take a bit. Also the 4th disk is now resyncing.

Sat Jan 30 17:04:37 CET 2016 System is up.

Sat Jan 30 17:04:37 CET 2016 Volume C is approaching capacity: 88% used 878G available

Sun Jan 17 12:15:59 CET 2016 System is up.

Sun Jan 17 12:15:59 CET 2016 The paths for the shares listed below could not be found. Typically, this occurs when the ReadyNAS is unable to access the data volume. Squeezeboxserver Documents Video media Photos Music

Sun Jan 17 12:15:41 CET 2016 Volume scan failed to run properly.

 I hope this can be useful for others, including the L2 Netgear support, which in my opinion should have been able to address this issue in the first place. Not letting me go searching around the web for possible solutions. If I am able to figure this out (though I have a PhD in genetics, and have been around computers for 25 years) an engineer at Netgear definitely should have fixed this easily. This in my point qualify for a refund! Also, that Netgear does not log their service to provide proof/documentation of their work is surprising.

I am happy I figured it out, and hope this can be useful for someone else in a similar situation.