526x terrible data rates - metadata dup is the problem #28424801

[mark_j_wagstaff]

I have replaced a Pro with a 526x and I am having terrible problems with it.   Amongst other issues, I have very slow data rates copying my Pro back-up onto the new NAS.  The machine can greing along with rates of 1 to 2 MB/s.  I raised a ticket with support; they took a look at the logs and advised the problem was fragmentation and metadata duplication.  Fragmentation makes no sense on a new system but metadata might.  I ran an share balance and it took seconds to complete.

I would be very grateful if you could look at the logs below and advise whether metadata is the problem.  Also, these logs are post balance and the metadata dup seems to be very high - ie the balance has not worked.

Thanks in advance for any advice, comments. 

=== filesystem /data ===

Data, single: total=6.44TiB, used=6.43TiB

System, DUP: total=32.00MiB, used=768.00KiB

Metadata, DUP: total=8.50GiB, used=7.67GiB

GlobalReserve, single: total=512.00MiB, used=0.00B

=== filesystem /data ===

Data, single: total=6.44TiB, used=6.43TiB

System, DUP: total=32.00MiB, used=768.00KiB

Metadata, DUP: total=8.50GiB, used=7.67GiB

GlobalReserve, single: total=512.00MiB, used=0.00B

---------------

data resilver 2017-05-09 22:52:44 2017-05-10 13:34:10 completed 

data balance 2017-05-15 17:34:32 2017-05-15 17:34:38 completed Done, had to relocate 1 out of 6229 chunks 

data balance 2017-05-16 20:11:39 2017-05-16 20:11:45 completed Done, had to relocate 3 out of 6562 chunks 

data defrag 2017-05-16 20:12:09

[mdgm-ntgr]

---

---

A balance won't do anything much if what's allocated is pretty much fully in use. Where a balance will do things is if much more is allocated than what's in use. 6.43TiB used out of 6.44TiB and 7.67GiB out of 8.50GiB indicate that the volume is already well balanced.

[mark_j_wagstaff]

Thank you for responding. Below is a comment from Netgear support. This states the metadata is excessive and should be MB only. Does this make sense to you?

=== filesystem /data ===
Data, single: total=4.50TiB, used=4.49TiB
System, DUP: total=8.00MiB, used=544.00KiB
Metadata, DUP: total=6.50GiB, used=5.47GiB <-- Used metadata is too big. That is normally in MiB only.
GlobalReserve, single: total=512.00MiB, used=0.00B

[mdgm-ntgr]

Well I'd probably expect less GiB than that for that size of a volume but if you're using a lot of snapshots (uses CoW at the point of time it is taken even if CoW is disabled on the share) and/or bit-rot protection (we link this to enabling/disabling CoW on the share even though the two are different) you can get quite a lot of metadata. Also if you have a huge amount of small files that would generate a lot of metadata as well. metadata is data about data. The more files you have, the more different versions of those files in snapshots etc. the more metadata will be required.

Data, single: total=14.46TiB, used=14.16TiB
System, DUP: total=8.00MiB, used=1.53MiB
Metadata, DUP: total=11.00GiB, used=8.93GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

The above is from one of my systems. You can see I have less than 1GiB of metadata per TiB but then I don't have any snapshots and have bit-rot protection disabled.

As for the network connection issues you could try a known working ethernet cable and a direct-connection between the NAS and your Mac and see if the performance is better doing that.

I believe support has mentioned on your case that there is a backend settings change we could try for SMB. This change should improve directory listing performance for SMB on Mac amongst other things. It's certainly worth a try.

[mark_j_wagstaff]

Sincerely thanks for the comprehensive reply. You have provided a straight forward description in language I can understand!

I have bit rot protection on all shares. Does bit rot protection have a significant overhead in terms of write to disk and metadata?