× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: ReadyNAS 426 stuck on booting..

Supercows
Guide

ReadyNAS 426 stuck on booting..

HI,

after axpanding a disk the rebuild was hanging on 0,28%. Now my readynas wont boot in normal mode.

If i boot in read only the log says that the firmware was upgraded. I cannot remind giving the command to do that on the same moment. What can I do? I stil can boot in readonly access. Normal boot gives a systemd-udev-settle.service when i press the power button. 

 

Oct 29, 2021 20:10:53
 
System: ReadyNASOS background service started.
Oct 29, 2021 20:01:28
 
System: The system is shutting down.
Oct 29, 2021 19:59:24
 
System: ReadyNASOS background service started.
Oct 29, 2021 19:59:24
 
System: Firmware was upgraded to 6.10.5.
Oct 29, 2021 18:41:59
 
System: The system is rebooting.
Oct 29, 2021 17:02:56
 
Disk: Disk in channel 3 (Internal) changed state from RESYNC to ONLINE.
Oct 29, 2021 16:59:34
 
Volume: Resyncing started for Volume data.
Oct 29, 2021 16:59:27
 
Disk: Disk Model:WDC WD60EFZX-68B3FN0 Serial:WD-C814B1SG was added to Channel 3 of the head unit.
Model: RN426|ReadyNAS 426 – High-performance Business Data Storage - 6-Bays
Message 1 of 11
Supercows
Guide

Re: ReadyNAS 426 stuck on booting..

Radair:

 

Successfully completed diagnostics

System

  • Disk 2 has 52 Reallocated Sectors
  • Disk 2 has 32 Reallocation Events
  • Disk 2 has 3 Current Pending Sectors
  • Volume data is inactive
  • Volume data-0 is inactive

Logs

  • 2021-10-29 20:10:35: md/raid:md127: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 20:03:14: md/raid:md127: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:59:03: md/raid:md126: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:54:27: md/raid:md126: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:46:44: md/raid:md127: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:37:18: md/raid:md126: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:27:00: md/raid:md127: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:12:53: md/raid:md126: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:09:24: md/raid:md127: raid level 5 active with 2 out of 3 devices, algorithm 2
  • 2021-10-29 19:05:14: md/raid:md126: raid level 5 active with 2 out of 3 devices, algorithm 2
Message 2 of 11
StephenB
Guru

Re: ReadyNAS 426 stuck on booting..

Not sure what happened with the firmware update, but the log does suggest that the resync had finished before the reboot/firmware install.

 

Do you have 3 disks in the NAS?

 

Are you able to access the data in read-only mode?  If so, I suggest backing it up (if you don't have an up-to-date backup already).

 

The errors on disk 2 are concerning.  It would be useful to download the log zip file (which RAIDar should be able to do), and then copy/paste the contents of mdstat.log.

 

 

Message 3 of 11
Supercows
Guide

Re: ReadyNAS 426 stuck on booting..

Thanks for your reply!

I have 3 disks in the readynas. I added 1 today.

 

Mdstat.log:

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid1 sdc2[2] sdb2[1] sda2[0]
523264 blocks super 1.2 [3/3] [UUU]

md126 : active (read-only) raid1 sda4[0] sdc4[1]
976619904 blocks super 1.2 [2/2] [UU]
resync=PENDING

md127 : active (read-only) raid5 sda3[3] sdb3[2]
2925416800 blocks super 1.2 level 5, 32k chunk, algorithm 2 [3/2] [UU_]

md0 : active raid1 sdc1[4] sda1[3] sdb1[2]
4190208 blocks super 1.2 [3/3] [UUU]

unused devices: <none>
/dev/md/0:
Version : 1.2
Creation Time : Wed Mar 16 08:41:11 2016
Raid Level : raid1
Array Size : 4190208 (4.00 GiB 4.29 GB)
Used Dev Size : 4190208 (4.00 GiB 4.29 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent

Update Time : Fri Oct 29 21:23:29 2021
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

Consistency Policy : unknown

Name : 0a4510d0:0 (local to host 0a4510d0)
UUID : 4858ca72:1e8ebe2a:dcbdda0a:4c08b023
Events : 1139

Number Major Minor RaidDevice State
3 8 1 0 active sync /dev/sda1
2 8 17 1 active sync /dev/sdb1
4 8 33 2 active sync /dev/sdc1
/dev/md/data-0:
Version : 1.2
Creation Time : Wed Mar 16 08:41:11 2016
Raid Level : raid5
Array Size : 2925416800 (2789.89 GiB 2995.63 GB)
Used Dev Size : 2925416800 (2789.89 GiB 2995.63 GB)
Raid Devices : 3
Total Devices : 2
Persistence : Superblock is persistent

Update Time : Fri Oct 29 20:03:16 2021
State : clean, degraded
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 32K

Consistency Policy : unknown

Delta Devices : 1, (2->3)

Name : 0a4510d0:data-0 (local to host 0a4510d0)
UUID : 35e58e71:f4df1a57:38243e16:1e8a5376
Events : 122055

Number Major Minor RaidDevice State
3 8 3 0 active sync /dev/sda3
2 8 19 1 active sync /dev/sdb3
- 0 0 2 removed
/dev/md/data-1:
Version : 1.2
Creation Time : Fri Oct 29 16:59:31 2021
Raid Level : raid1
Array Size : 976619904 (931.38 GiB 1000.06 GB)
Used Dev Size : 976619904 (931.38 GiB 1000.06 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

Update Time : Fri Oct 29 16:59:32 2021
State : clean, resyncing (PENDING)
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Consistency Policy : unknown

Name : 0a4510d0:data-1 (local to host 0a4510d0)
UUID : 135af482:d759e7bd:2ad1f59b:8ea3bb81
Events : 1

Number Major Minor RaidDevice State
0 8 4 0 active sync /dev/sda4
1 8 36 1 active sync /dev/sdc4

 

unfortunately I cannot access data in readonly mode. Can I do something on ssh to make that possible?

Is there something I can do to fix this?

 

Message 4 of 11
StephenB
Guru

Re: ReadyNAS 426 stuck on booting..

The resync pending isn't something I've seen before.

 

It is possible to set the volume to read/write with mdadm

mdadm --readwrite /dev/md127

But I really don't know what will happen if you try that.  It could make things worse.

 

What sizes are the disks? What disk did you add?  Was it smaller than the others?

 

 

 

 

 

Message 5 of 11
Sandshark
Sensei

Re: ReadyNAS 426 stuck on booting..

There is no way the added drive did a RAID sync of the data volume in two minutes.  The only things that could have completed sync in that time are the OS and swap partitions.  I have seen that erroneous message at that time before (and, I think, I reported it here).  It can make you erroneously believe that it is OK to go on with using the NAS as normal, but it's not.  Consequently, it appears the system was re-booted and did an OS update while a sync was taking place, which can be bad.  Resync pending is what you would see for md126 when md127 is still doing a sync.  You have these two because you at one point had smaller drives, so you have two stacked RAIDs.  But with something having gone wrong with the sync of md127 (because of the re-boot, I'm fairly certain), syncing has stopped on both.

 

You are seeing two volumes because BTRFS does not recognize that the two RAID layers go together.  So, it has mounted one as data-0 (the name of the first MDADM RAID layer) and the other as data.  But since neither is valid stand-alone, you have no access to the files.

 

While I recognize what you have, I've never seen them together, so also don't know your best next step beyond contacting Netgear support.  It's possible that making md127 read/write will start it re-syncing.  But then md126 will likely want to re-sync, and BTRFS may not recognize the file system properly until you re-boot after the both syncs complete.  Or maybe re-booting without the new drive is the best move.  md127 will come up as degraded, but maybe not read-only.

 

If you have no backup, so a factory reset is not an acceptable alternate if nothing works to fix it, you are best off leaving it to ReadyNAS support.

Message 6 of 11
StephenB
Guru

Re: ReadyNAS 426 stuck on booting..


@Sandshark wrote:

There is no way the added drive did a RAID sync of the data volume in two minutes. 

Agreed. I didn't look at the times as closely as I should have.

 

Normally with XRAID, md127 contains all the drives in the array.  md126 often contains a subset (the larger drives).

 

At this point

  • md126 only has sda and sdc (with nothing shown as missing, but the pending resync)
  • md127 only contains sda and sdb with one drive shown as missing -  which would be sdc.  

 

It is possible that the system would show you the files if you reboot w/o the newly inserted drives.  But if you try that, I suggest booting the NAS in read-only mode, in order to prevent any further damage. 

 

If it works, the next step would be to make a backup.  If disk 2 fails during the resync of the new drive (which is a real danger, given the errors), then you would lose data.

 

If it doesn't work (or you don't want to risk it), then I agree with @Sandshark that you should contact paid netgear support.

 

 

Message 7 of 11
Supercows
Guide

Re: ReadyNAS 426 stuck on booting..

Thanks guys.

 

How can I mount /dev/md126 or /dev/sda4 or /dev/sdc4

 

When i mount it i'll hoop to see my data back

 

I have the feeling that my raid 1 is still there :

root@nas:/data# mdadm --detail /dev/md126
/dev/md126:
Version : 1.2
Creation Time : Fri Oct 29 16:59:31 2021
Raid Level : raid1
Array Size : 976619904 (931.38 GiB 1000.06 GB)
Used Dev Size : 976619904 (931.38 GiB 1000.06 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

Update Time : Fri Oct 29 16:59:32 2021
State : clean, resyncing (PENDING)
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Consistency Policy : unknown

Name : 0a4510d0:data-1 (local to host 0a4510d0)
UUID : 135af482:d759e7bd:2ad1f59b:8ea3bb81
Events : 1

Number Major Minor RaidDevice State
0 8 4 0 active sync /dev/sda4
1 8 36 1 active sync /dev/sdc4
root@NAS:/data# mdadm --detail /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Wed Mar 16 08:41:11 2016
Raid Level : raid5
Array Size : 2925416800 (2789.89 GiB 2995.63 GB)
Used Dev Size : 2925416800 (2789.89 GiB 2995.63 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent

Update Time : Sat Oct 30 10:13:33 2021
State : clean, degraded
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 32K

Consistency Policy : unknown

Delta Devices : 1, (2->3)

Name : 0a4510d0:data-0 (local to host 0a4510d0)
UUID : 35e58e71:f4df1a57:38243e16:1e8a5376
Events : 122067

Number Major Minor RaidDevice State
3 8 3 0 active sync /dev/sda3
2 8 19 1 active sync /dev/sdb3
- 0 0 2 removed

4 8 35 - spare /dev/sdc3

 

root@NAS:/# mdadm --examine /dev/sda4
/dev/sda4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 135af482:d759e7bd:2ad1f59b:8ea3bb81
Name : 0a4510d0:data-1 (local to host 0a4510d0)
Creation Time : Fri Oct 29 16:59:31 2021
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 1953239824 (931.38 GiB 1000.06 GB)
Array Size : 976619904 (931.38 GiB 1000.06 GB)
Used Dev Size : 1953239808 (931.38 GiB 1000.06 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=263912 sectors, after=16 sectors
State : active
Device UUID : c18e41ce:3c26f7ba:f12d0834:d4c2a9de

Update Time : Fri Oct 29 16:59:32 2021
Bad Block Log : 512 entries available at offset 264 sectors
Checksum : 296c8871 - correct
Events : 1


Device Role : Active device 0
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
root@NAS:/# mdadm --examine /dev/sdb4
mdadm: cannot open /dev/sdb4: No such file or directory
root@NAS:/# mdadm --examine /dev/sdc4
/dev/sdc4:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 135af482:d759e7bd:2ad1f59b:8ea3bb81
Name : 0a4510d0:data-1 (local to host 0a4510d0)
Creation Time : Fri Oct 29 16:59:31 2021
Raid Level : raid1
Raid Devices : 2

Avail Dev Size : 1953239824 (931.38 GiB 1000.06 GB)
Array Size : 976619904 (931.38 GiB 1000.06 GB)
Used Dev Size : 1953239808 (931.38 GiB 1000.06 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=263912 sectors, after=16 sectors
State : active
Device UUID : 2327a496:26ca26dd:7553d4ef:d17510c6

Update Time : Fri Oct 29 16:59:32 2021
Bad Block Log : 512 entries available at offset 264 sectors
Checksum : b7319d3f - correct
Events : 1


Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)

 

root@NAS:/# fdisk -l /dev/sda

Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: A80170F7-1442-46A3-BAEE-5434D72E9CAC

Device Start End Sectors Size Type
/dev/sda1 64 8388671 8388608 4G Linux RAID
/dev/sda2 8388672 9437247 1048576 512M Linux RAID
/dev/sda3 9437248 5860533103 5851095856 2.7T Linux RAID
/dev/sda4 5860533104 7814037119 1953504016 931.5G Linux RAID

 

 

root@NAS:/# mdadm --assemble --scan --verbose
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md1
mdadm: no recogniseable superblock on /dev/md/data-1
mdadm: no recogniseable superblock on /dev/md/data-0
mdadm: no recogniseable superblock on /dev/md/0
mdadm: /dev/sdc4 is busy - skipping
mdadm: /dev/sdc3 is busy - skipping
mdadm: /dev/sdc2 is busy - skipping
mdadm: /dev/sdc1 is busy - skipping
mdadm: Cannot assemble mbr metadata on /dev/sdc
mdadm: /dev/sdb3 is busy - skipping
mdadm: /dev/sdb2 is busy - skipping
mdadm: /dev/sdb1 is busy - skipping
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: /dev/sda2 is busy - skipping
mdadm: /dev/sda1 is busy - skipping
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: No arrays found in config file or automatically

Message 8 of 11
Sandshark
Sensei

Re: ReadyNAS 426 stuck on booting..

I already explained what you already have.  md126 and md127 are already mounted (they show as active), though they are not properly sync'ed.  More importantly, the BTRFS file system that is supposed to encompass them both doesn't see them as being linked.  You can't just go mounting additional partitions that are supposed to be a part of an MDADM RAID and BTRFS volume independently.  If it were to work, you'd do more harm that good.

Message 9 of 11
Supercows
Guide

Re: ReadyNAS 426 stuck on booting..

OK, I hoped there was still some kind of hope.

 

I have my documents etc in a azure backup. Series and movies are gone. I am going to reset the system and start over. In the last 15 years and three readynas further never had issues like this before. I am very disappointed. It is what it is

 

Thanks for the help.

 

Message 10 of 11
StephenB
Guru

Re: ReadyNAS 426 stuck on booting..


@Supercows wrote:

OK, I hoped there was still some kind of hope.

 


Of course if you are planning a reset anyway, you could try booting up read-only w/o the new disk first.

 

Or use mdadm to set the two raid groups to read-write and see what happens.

Message 11 of 11
Top Contributors
Discussion stats
  • 10 replies
  • 1702 views
  • 0 kudos
  • 3 in conversation
Announcements