Re: Missing Storage Readynas 516

pgjscottieuk · ‎2017-03-13

I am running my 516 nas with 6 x 4TB hard drives = 24TB in XRAID RAID 5

My NAS tells me i have 10.11TB Free Space Used Data 4.33TB i thought 24TB's in Raid 5 would give me 20TB space

I started of with 2 drives then added two more some months later then realised the WD Red Pro i was using was at end of life wanted all to be the same so as i could source them added them one at a time

All the drives in place show and are working properly each shows a blue light confirming this am i wrong here is this correct

pgjscottieuk · ‎2017-03-14

After a Resync the volume has now expanded showing correct size 18.07TB instead of 14.43

View solution in original post

Retired_Member · ‎2017-03-13

If a disk is labeled as 4TB, it only has roughly 3.6 TiB. (read wikipedia here for the difference https://en.wikipedia.org/wiki/Tebibyte). And the nas shows you the lower on its admin webpage

Furthermore running your system in raid5 you are "loosing" 1 complete disk to have redundancy insurance. If one disk breaks you would be able to replace it without loosing everything.

So, your 6x4TB disks in raid5 gets us 5x3.6TiB, which is roughly 18 TiB diskspace overall for your system configuration.

As you are reporting only 14.5TiB in your system overall, there might be the problem, that not all disks are recognized to their full capacity.

You could do the following to find out what capacity the nas recognizes for each of your disks. On the admin webpage go to System/Volumes and hover your mouse for a few seconds over each of the blocks representing a disk. In the 7th row of each displayed text-box you will the see capacity for the concerned disk. Under normal circumstances I would expect for all 6 something like 3.6.

Retired_Member · ‎2017-03-13

After downloading the logfiles you might want to check btrfs.log to see how the partitions are truely layed out and you will see something like this

Label: 'xxx:data' uuid: yyy
Total devices 2 FS bytes used 7.59TiB
devid 1 size 5.44TiB used 3.70TiB path /dev/md127
devid 2 size 10.92TiB used 3.91TiB path /dev/md126

I have 4x6TB RN204, which in raid5 finally results into a volume of roughly 16.5TiB. Which is exactly that, what one would expect in this case.

pgjscottieuk · ‎2017-03-13

Hi yes in have checked all 6 are showing 3.6TB

pgjscottieuk · ‎2017-03-13

How do i download the logfiles

StephenB · ‎2017-03-13

If you purchased new between 1 June 2014 and 31 May 2016, then you have lifetime (free) chat support. If you have it, then you should use it - see my.netgear.com

@pgjscottieuk wrote:

How do i download the logfiles

There's a download control on the log page in the web ui. That will download the full log zipfile.

Your volume appears to be 4 TB short. There are three possibilities.

You could be running dual redundancy
The btrfs volume might not have expanded when the last disk was added (even if all disks are in the RAID array).
One disk might not be part of the active RAID array (despite the status shown in your screenshot)..

In the last case, one of the disks might be marked as a "spare" in the logs. Try looking through btrfs.log, mdstat.log and volume.log.

Retired_Member · ‎2017-03-13

On the admin webpage in the logfile tab you can download a zipfile containing 60+ logfiles amongst them btrfs.log.

jak0lantash · ‎2017-03-13

@Retired_Member

There is also an OS partition (4GB) and a swap partition (500MB I think) on every drive.

So, by order of importance:

1. RAID redundancy.

2. HDD manufacturer fantasy 1000 factor, implying difference between TB and TiB.

3. 4GB OS + 500MB Swap per HDD.

4. FS overhead.

@StephenB

I think you're right about the spare.

pgjscottieuk · ‎2017-03-14

Have looked through the files you have mentioned cant find anything suggesting a spare have attached thanks again for help

There did seem to be a spare at the end of the mdstat.log number 3 ?

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md127 : active raid5 sde3[0] sdc3[3](S) sda3[5] sdb3[4] sdd3[2] sdf3[1]
15608666624 blocks super 1.2 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]

md1 : active raid6 sdc2[0] sda2[5] sdb2[4] sdf2[3] sde2[2] sdd2[1]
2093056 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU]

md0 : active raid1 sde1[0] sda1[5] sdb1[4] sdc1[3] sdd1[2] sdf1[1]
4192192 blocks super 1.2 [6/6] [UUUUUU]

unused devices: <none>
/dev/md/0:
Version : 1.2
Creation Time : Thu Feb 19 15:00:27 2015
Raid Level : raid1
Array Size : 4192192 (4.00 GiB 4.29 GB)
Used Dev Size : 4192192 (4.00 GiB 4.29 GB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Mon Mar 13 21:05:10 2017
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Name : 7c6e3734:0 (local to host 7c6e3734)
UUID : c1aecbdd:0f2a5130:71ba0480:07a324f5
Events : 1139
Number Major Minor RaidDevice State
0 8 65 0 active sync /dev/sde1
1 8 81 1 active sync /dev/sdf1
2 8 49 2 active sync /dev/sdd1
3 8 33 3 active sync /dev/sdc1
4 8 17 4 active sync /dev/sdb1
5 8 1 5 active sync /dev/sda1
/dev/md/1:
Version : 1.2
Creation Time : Sat Sep 24 11:40:47 2016
Raid Level : raid6
Array Size : 2093056 (2044.00 MiB 2143.29 MB)
Used Dev Size : 523264 (511.00 MiB 535.82 MB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Thu Mar 9 17:35:17 2017
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : 7c6e3734:1 (local to host 7c6e3734)
UUID : 39ce8693:bda6cd12:2a906266:c669f6f5
Events : 19
Number Major Minor RaidDevice State
0 8 34 0 active sync /dev/sdc2
1 8 50 1 active sync /dev/sdd2
2 8 66 2 active sync /dev/sde2
3 8 82 3 active sync /dev/sdf2
4 8 18 4 active sync /dev/sdb2
5 8 2 5 active sync /dev/sda2
/dev/md/data-0:
Version : 1.2
Creation Time : Thu Feb 19 15:00:27 2015
Raid Level : raid5
Array Size : 15608666624 (14885.58 GiB 15983.27 GB)
Used Dev Size : 3902166656 (3721.40 GiB 3995.82 GB)
Raid Devices : 5
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Mon Mar 13 21:04:44 2017
State : clean
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
Name : 7c6e3734:data-0 (local to host 7c6e3734)
UUID : 2fb715d8:4a24e72b:57e364dc:cda58209
Events : 22927
Number Major Minor RaidDevice State
0 8 67 0 active sync /dev/sde3
1 8 83 1 active sync /dev/sdf3
2 8 51 2 active sync /dev/sdd3
4 8 19 3 active sync /dev/sdb3
5 8 3 4 active sync /dev/sda3
3 8 35 - spare /dev/sdc3

pgjscottieuk · ‎2017-03-14

Please find the btrfs.log

Label: '7c6e3734:root' uuid: 02c2df95-1651-4bf0-af83-582b5010b5ee
Total devices 1 FS bytes used 825.85MiB
devid 1 size 4.00GiB used 4.00GiB path /dev/md0
Label: '7c6e3734:data' uuid: afa8cfbe-ebc8-4c1f-8ab0-a1b1d851b14b
Total devices 1 FS bytes used 4.34TiB
devid 1 size 14.54TiB used 4.63TiB path /dev/md127
=== filesystem /data ===
Data, single: total=4.52TiB, used=4.33TiB
System, DUP: total=8.00MiB, used=532.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=57.50GiB, used=4.96GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=512.00MiB, used=0.00B
=== subvolume /data ===
ID 256 gen 114574 top level 5 path home
ID 259 gen 134390 top level 5 path .apps
ID 260 gen 7 top level 5 path .vault
ID 262 gen 128365 top level 5 path Pictures
ID 263 gen 134185 top level 5 path FILMS
ID 264 gen 128365 top level 5 path Music
ID 265 gen 121597 top level 5 path Documents
ID 266 gen 130260 top level 5 path Backup
ID 267 gen 133755 top level 5 path ._share
ID 268 gen 121600 top level 5 path .timemachine
ID 274 gen 121600 top level 266 path Backup/.snapshots
ID 276 gen 121595 top level 265 path Documents/.snapshots
ID 278 gen 121590 top level 264 path Music/.snapshots
ID 280 gen 121585 top level 262 path Pictures/.snapshots
ID 282 gen 133950 top level 263 path FILMS/.snapshots
ID 284 gen 59664 top level 256 path home/admin
ID 827 gen 38005 top level 5 path .purge
=== btrfs-show-super /dev/md0 /dev/md127
superblock: bytenr=65536, device=/dev/md0
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0xbd8c24b9 [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid 02c2df95-1651-4bf0-af83-582b5010b5ee
label 7c6e3734:root
generation 204967
root 33370112
sys_array_size 226
chunk_root_generation 108555
root_level 1
chunk_root 20971520
chunk_root_level 0
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 4292804608
bytes_used 865964032
sectorsize 4096
nodesize 4096
leafsize 4096
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x1
( MIXED_BACKREF )
cache_generation 2556
uuid_tree_generation 204967
dev_item.uuid 684bde8e-8778-4436-a921-f4f3422fccb7
dev_item.fsid 02c2df95-1651-4bf0-af83-582b5010b5ee [match]
dev_item.type 0
dev_item.total_bytes 4292804608
dev_item.bytes_used 4292804608
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0
superblock: bytenr=65536, device=/dev/md127
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0xafa88107 [match]
bytenr 65536
flags 0x1
( WRITTEN )
magic _BHRfS_M [match]
fsid afa8cfbe-ebc8-4c1f-8ab0-a1b1d851b14b
label 7c6e3734:data
generation 134390
root 5591031808
sys_array_size 226
chunk_root_generation 134153
root_level 2
chunk_root 20975616
chunk_root_level 2
log_root 0
log_root_transid 0
log_root_level 0
total_bytes 15983274622976
bytes_used 4768776925184
sectorsize 4096
nodesize 4096
leafsize 4096
stripesize 4096
root_dir 6
num_devices 1
compat_flags 0x0
compat_ro_flags 0x0
incompat_flags 0x1
( MIXED_BACKREF )
cache_generation 40
uuid_tree_generation 134390
dev_item.uuid b934a4af-c55d-4c00-8721-264456194c1b
dev_item.fsid afa8cfbe-ebc8-4c1f-8ab0-a1b1d851b14b [match]
dev_item.type 0
dev_item.total_bytes 15983274622976
dev_item.bytes_used 5091721478144
dev_item.io_align 4096
dev_item.io_width 4096
dev_item.sector_size 4096
dev_item.devid 1
dev_item.dev_group 0
dev_item.seek_speed 0
dev_item.bandwidth 0
dev_item.generation 0

mdgm-ntgr · ‎2017-03-14

Yes, one of the disks has been marked as a spare.

It's also clear that your volume was created on very old firmware.

pgjscottieuk · ‎2017-03-14

Hi hope you received the 5 digit code that you requested

You say it was set up on very old firmware the unit was purchased in Feb 2015 and i followed the setup procedure as instructed

I check on a regular basis the update button to try and make sure its as current as possible

mdgm-ntgr · ‎2017-03-14

A resync should be taking place now. Once that completes the volume should then expand automatically.

pgjscottieuk · ‎2017-03-14

Thank you as soon as it finnishes will update it has about 14 hours to go

Before coming on to the community site i had tried to find the Resync facilty as i had that train of thought

pgjscottieuk · ‎2017-03-14

After a Resync the volume has now expanded showing correct size 18.07TB instead of 14.43

Fishbones · ‎2017-03-17

Have the same issue ... replaced 3 x 1 TB disks with 2TB disks (total now 6 x 2TB). Volume shows 8.15TB instead of 10TB. How does one force a Resync?

jak0lantash · ‎2017-03-17

Please download the logs and paste the content of mdstat.log so i can double check. But 6x2TB in RAID5 showing 8.15TB of capacity seems correct.
You need to remove the capacity of one disk because of the RAID5, then count the difference between Terabytes and Tebibytes (wikipedia), then count that 4.5GB on each disk is used by the system, finally consider the filesystem overhead.

Fishbones · ‎2017-03-17

As requested....

rdcalculator indicates 9.06TiB

Thanks for the help.

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] 
md126 : active raid5 sda4[0] sde4[5](S) sdf4[6] sdd4[4] sdc4[2] sdb4[3]
      3906491648 blocks super 1.2 level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
      
md127 : active raid5 sda3[0] sdf3[9] sde3[8] sdd3[7] sdc3[2] sdb3[6]
      4859553920 blocks super 1.2 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
      
md1 : active raid6 sda2[0] sdf2[5] sde2[4] sdd2[3] sdc2[2] sdb2[1]
      2093056 blocks super 1.2 level 6, 512k chunk, algorithm 2 [6/6] [UUUUUU]
      
md0 : active raid1 sda1[0] sdf1[9] sde1[8] sdd1[7] sdc1[2] sdb1[6]
      4190208 blocks super 1.2 [6/6] [UUUUUU]
      
unused devices: <none>

jak0lantash · ‎2017-03-17

There is an issue with sde4, it's marked as spare and md126 is on 5 devices instead of 6.

Fishbones · ‎2017-03-17

OK. I presume sde is Disk 5... I'll replace the disk and see if it resyncs to full capacity.

jak0lantash · ‎2017-03-17

No no, the issue is not with the hard drive. It's not a physical issue. Sde4 is "just" a partition, that is a RAID member. You have to fix the RAID.

I'd grow md126 to 6 devices, and remove / add sde4 if necessary (using mdadm).

Fishbones · ‎2017-03-17

So somehow OS6 messed up the X-RAID expansion

Should there not be only a md126? Why the md127?

I'm not familiar with "mdadm" and RAID. So I'll probably have to backup (painfull) and do a factory reset.

Thanks for the tips.

BR.

jak0lantash · ‎2017-03-17

In your situation, it's normal that there is md127 and md126.

mdgm-ntgr · ‎2017-03-17

I can fix this easily. I've sent you a PM.

When you vertically expand your data volume (expand by adding higher capacity disks) it will create another RAID layer. So rather than just having a md127 you'll have a md126 as well.

mdgm-ntgr · ‎2017-03-17

It is resyncing.