× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973

OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

How did you install unrar and those apps?
Message 751 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Sabnzbd, sickbeard et al are just downloads from their respective sources (github, sourcefourge). Unrar and git were installed via apt-get. I'm using the unrar from the non-free libs.
Message 752 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Ah so maybe there is an issue with the version of unrar or git you're using.

Does the add-on work any better for you than installing the apps manually?
Message 753 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

To be fair, I highly doubt that the original packages for unrar and git (which I also used on OS4) would cause this problem. They would be in debian sid if they were this wonky, not in a production ready distribution.

I'm also not sure which addon you mean? I hope not any of whocares' addons, since I refuse to pay 2.5 dollars for every single version of _free_ software.
Message 754 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Sometimes you have to pay to get quality. Is your time worth less to you than a few dollars?
Message 755 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

I don't mind paying for quality. What I do mind is paying for version 1, version 1.0.1, 1.0.2, 1.0.3 etc. Also, this setup worked just fine on OS4. The problems I have look exactly like a number of blogposts on the internet suggest: btrfs locking up the system for a few milliseconds, causing waitstates.
Message 756 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

I'm wondering if changing the mount option for the volume with noatime will help, as BTRFS' wiki suggests it could (Source). It's really rather sad to see the performance being so poor. At the moment, my expensive NAS is more a very large read-only drive rather than a functioning NAS.
Message 757 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Well, that worked to a point... I'm now 100% sure it's the file system that's causing issues.

This is a copy from a game folder in my steam library to the NAS. It is just shy of 6GB in size and contains a large number of small files, as well as a few large ones. I had top open in a putty window while copying.


Every section marked in red, the BTRFS processes were dominating in top. Not just the transaction process, also a flush process, the various endio processes and the md127 process. Once they started floating to the surface, copying would grind to a halt (either below 30KB/s or even complete stops). At those points, I/O wait would also reach 75% on some of the cores. As soon as these processes would terminate (or sleep, whatever), copy would resume and IO wait would go to 2 or 3%, which is normalish.

As I am writing this, I've also restarted my sabnzbd and wait states are below 2%, load on the system is below 3. That is, until btrfs rears it's ugly head again, at which point wait spikes up again. I do not yet dare to re-allow both post-processing (par2, unrar) and downloading at the same time, since the crippled filesystem simply can't cope, where the ext system in OS4 had no trouble at all.

Netgear, this has got to go. BTRFS is not production ready. Heck, you've got a bloody release candidate on these devices (0.20-rc1, btrfs --version for those interested). This system is a disaster. In the time it took me to write four lines, load has spiked up from 2 to 7, simply because ONE process is writing to disk and btrfs can't keep up. The filesystem is using over 7GB for meta data alone, and every time you do something on the device, it increases. Not only that, but it has a scheduled time of locking up your IO every damn 30 seconds (source). Why didn't you just pick ext4, which is a proven system and has much higher performance?

edit:
I'm still monitoring my downloads in sabnzbd. Download speeds are at ~10MB/s, which is slower than I had on OS4 (12.5 top, which is the limit I have on usenet). But whenever the btrfs processes show up, it's 1KB/s or less. The filesystem is killing performance. Apart from this download and my shell session with top, nothing else is running. Once these processes for the FS sleep, load drops to about 4. Once they wake, it's back up to 7. Heck, during the btrfs slowdown, a simple request to http takes 13 seconds (!!) where the norm is < 150ms.
Message 758 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

What version of par2, unrar etc. do you have installed? Have you tried compiling a newer version from source?

Whilst EXT4 is a good filesystem it does have its limitations. BTRFS provides some nice new features like unlimited snapshots (well limited only by free space on your volume) and better checking to verify that data is not corrupt.

BTRFS whilst not faster (at least at the moment) is a better filesystem than EXT4.

If you want to downgrade your Ultra back to 4.2.x check your PM inbox.
Message 759 of 1,275
mangrove
Apprentice

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

The sane thing to do would of course have been to offer a choice. 1: Proven, boring and stable. 2: Unproven, feature-rich and potentially unstable.

It's not too late to offer that choice for new volumes, of course. Come on, NTGR! Do something sane for 6.1.5! 😉

BTRFS was a stupid choice, but seeing that the whole of OS6 was released long before finished, using a beta file system is just icing on the stupidcake. 🙂
Message 760 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

root@MueR-NAS:/# aptitude show python
Package: python
State: installed
Automatically installed: no
Version: 2.7.3-4
Priority: standard
Section: python
Maintainer: Matthias Klose <doko@debian.org>
Architecture: all
Uncompressed Size: 674 k
Depends: python2.7 (>= 2.7.3-1~), python-minimal (= 2.7.3-4)
Suggests: python-doc (= 2.7.3-4), python-tk (= 2.7.3-4)
Conflicts: python-central (< 0.5.5)
Breaks: python-bz2 (< 1.1-8), python-csv (< 1.0-4), python-email (< 2.5.5-3), update-manager-core (< 0.200.5-2)
Replaces: python-dev (< 2.6.5-2)
Provides: python-ctypes, python-email, python-importlib, python-profiler, python-wsgiref

root@MueR-NAS:/# aptitude show unrar
Package: unrar
New: yes
State: installed
Automatically installed: no
Version: 1:4.1.4-1
Priority: optional
Section: non-free/utils
Maintainer: Martin Meredith <mez@debian.org>
Architecture: amd64
Uncompressed Size: 245 k
Depends: libc6 (>= 2.2.5), libgcc1 (>= 1:4.1.1), libstdc++6 (>= 4.1.1)
Conflicts: rar (<= 2.60-1)
Description: Unarchiver for .rar files (non-free version)
Unrar can extract files from .rar archives. If you want to create .rar archives, install package rar.
Homepage: http://www.rarlabs.com/

root@MueR-NAS:/# aptitude show par2
Package: par2
State: installed
Automatically installed: no
Version: 0.4-11
Priority: extra
Section: utils
Maintainer: Bart Martens <bartm@debian.org>
Architecture: amd64
Uncompressed Size: 279 k
Depends: libc6 (>= 2.2.5), libgcc1 (>= 1:4.1.1), libstdc++6 (>= 4.2.1)
Description: Parity Archive Volume Set, for checking and repair of files


Par2 isnt the problem either, the symptoms described above happen while par2/unrar are not even running. At present, it's only downloading to the NAS, either from usenet or from my local PC.

Whilst downgrading to OS4 would solve the performance issues, it would bring back the issues of being on a bloody old system. While I really, really dislike Netgear's decision to not support this OS on any older devices, the only thing other than downgrading I can do is try and locate the problem with as much accuracy as I can, so they might improve the software. In the meantime, I'll complain and moan loudly in hopes they actually hear me. There is a fundamental flaw in a filesystem that can't cope with a lot of disk activity. Heck, they even advice to turn on noatime on their own wiki, because they know performance is crap.

I upgraded to OS6 on a small private LAN-party the other week, since I knew my friends would also be bringing their NAS devices, and I could back up my files. I don't want to loose all my data and I don't have the space to store it locally. And as I said, as much as I would like to get rid of my problems, I would much rather suffer through it if there's a light at the end and I can drink a nice dram of whisky knowing I helped everyone out a little by pushing through.
Message 761 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

BTRFS did receive some major performance improvements in the Linux 3.12 kernel. Perhaps Netgear can investigate if adding the new kernel will improve things?
Message 762 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

That python release is from last year, as is the version of unrar you have installed. Not sure about par2 bit I think that's old as well. You may well find compiling the latest version from source gives you better results (might need some backporting to get things working properly). Whilst WhoCares? does charge for his add-ons he works hard to make his add-ons work well, often using newer versions of packages that are newer than what can be found in the apt-get repository for the Debian distro running on the NAS. In fact for legacy OSes he has to run newer packages than what are in the distro or do some backporting or more likely a bit of both. All this work takes time.
Message 763 of 1,275
MueR
Aspirant

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

mdgm wrote:
That python release is from last year, as is the version of unrar you have installed. Not sure about par2 bit I think that's old as well.

Par2 simply hasn't been updated since 0.4. For unrar, I pulled the latest from testing, it's now on 1:5.0.10-1. Python 2 hasn't been updated past 2.7.3-4 and quite frankly I'm not familiar enough with python to fix issues in the software I want to run if I upgrade to v3. Also, while older versions of packages may contribute a small amount to the problem, they cannot be the sole cause, since a simple FTP or SMB copy to the device results in the same performance issues whenever the btrfs processes wake up. Setting noatime on the volume helped reduce load and iowait by a large amount, but still it shows up (and already caused two complete lockups within two hours).

To further add proof that it's btrfs causing issues: the following commands on shell took 15 seconds each (btrfs-transact was active both times)
mkdir /data/addons/config/sabnzbd/postproc
chown bas:admins /data/addons/config/sabnzbd/postproc/

Now, you can't in all seriousness tell me that a mkdir on shell taking that long could be caused by a potentially outdated unrar/par2/python package. It simply does not happen. It's the btrfs processes. At the same time as this is taking place, I see SQL errors on sabnzbd with the message that the database is locked. File system access is completely locked by btrfs wanting to be clever and snapshot stuff. I did disable snapshots on all shares (and antivirus too), but I doubt that disables the feature in btrfs.

Tomorrow I should receive a new 4GB ram unit for the box, maybe some more memory will improve things. It's worth a shot anyway. The only other reason besides memory and FS I can think of is failing hardware. Without a comparative test by someone else with a similar device, it's hard to say either way.
Message 764 of 1,275
ATCIS
Tutor

ReadyNAS Pro 6 BTRFS Testing - Remote SSH Access

MueR wrote:

Tomorrow I should receive a new 4GB ram unit for the box, maybe some more memory will improve things. It's worth a shot anyway. The only other reason besides memory and FS I can think of is failing hardware. Without a comparative test by someone else with a similar device, it's hard to say either way.

Dear MueR,

In the interest of helping you and possibly the legacy ReadyNAS community at large, I would be willing to provide you with remote SSH access (and HTTPS GUI if needed) to a Pro 6 for testing purposes if you would be so inclined. If that would be of any interest to you, please PM me with any particulars with regard to Hard Drive configuration etc. to make your testing most fruitful.

Regards,
Message 765 of 1,275
mangrove
Apprentice

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

mangrove wrote:
How do you solve the performance problem with your own encryption? Larger blocksize?


Well, I guess... they did in some way. Compressed performance is ten times better than TrueCrypt, still measly but 25MB/s (write) on an iSCSI container on an encrypted volume (sequential writes). On an unencrypted volume, it is closer to 70MB/s. This looks normal in comparison to competing devices for an Atom platform.
Message 766 of 1,275
MueR
Aspirant

Re: ReadyNAS Pro 6 BTRFS Testing - Remote SSH Access

ATCIS wrote:
Dear MueR,

In the interest of helping you and possibly the legacy ReadyNAS community at large, I would be willing to provide you with remote SSH access (and HTTPS GUI if needed) to a Pro 6 for testing purposes if you would be so inclined. If that would be of any interest to you, please PM me with any particulars with regard to Hard Drive configuration etc. to make your testing most fruitful.

Regards,


Hey,

Thanks for the offer, that's great 🙂 I'm still working on several potential fixes myself, once I have the results of those I'll contact you to see if the problem can be replicated 🙂
Message 767 of 1,275
F_L_
Tutor

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

About to take the step over to OS6 (doing backup right now)

Just to be sure
If using the prep-file:
R4toR6_Prep_Addon.bin

There is no need for factory default in step 3?

OS6 in 3 Steps:
1. Download pre modified IMG, (see link above to see what has been modified)
2. Upgrade firmware in Frontview 4.2 (with above file)
3. Reboot & factory default
Message 768 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

The prep add-on will automatically trigger a factory default at the end of the update process so you don't need to trigger it manually. After installing the add-on, upload the firmware and reboot.
Message 769 of 1,275
ATCIS
Tutor

Re: ReadyNAS Pro 6 BTRFS Testing - Remote SSH Access

MueR wrote:

Hey,
Thanks for the offer, that's great 🙂 I'm still working on several potential fixes myself, once I have the results of those I'll contact you to see if the problem can be replicated 🙂

Hey MueR,

I just wanted to let you know that the Pro 6 is ready for you! Please check your PM's for details. . .

Happy Friday!
Message 770 of 1,275
IcyK
Tutor

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Does anyone know how to delete the snapshots by hand?
My complete drive is filled with snapshot it seems, but they cant's be deleted because it's a read only file system (????)


root testix /data/Pictures # du -h --max-depth=1
3.4G ./tomtom
1.1G ./phonecard
445G ./snapshot
450G .

root testix /data/Pictures # rm -rf snapshot
rm: cannot remove `c_2013_10_25__00_01_28/phonecard/20130901_121037.jpg': Read-only file system


I already disabled the snapshots in the GUI but that doesn't seem to help (no new snapshots are being made, but the old ones aren't deleted).
Message 771 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

You should be able to delete the snapshots manually using the GUI
Message 772 of 1,275
IcyK
Tutor

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

Unfortunately, that doesn't work. When I select 'delete', they won't get deleted. Only the ones from the last day of the month I can delete, the rest remains there (and eating up space...).
I now have snapshots from June 15 till August 15 (the day I disabled snapshotting) but I can't get rid of them...

It is quite annoying: there is about 975 GB of data on the drive but the whole 1.75 TB is reported full.
Message 773 of 1,275
mdgm-ntgr
NETGEAR Employee Retired

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

What version of ReadyNAS OS are you running?
Message 774 of 1,275
IcyK
Tutor

Re: OS6 now works on x86 Legacy WARNING: NO NTGR SUPPORT!

I'm running 6.0.6 with a custom 'boot menu' and a custom kernel to use a DVB-C receiver (HDHomerun), that's why I'm a bit reluctant to update.
And I haven't seen a reference to snapshots in the release notes of the newer versions.

/edit: I'm running it on an Ultra 2
Message 775 of 1,275
Top Contributors
Announcements