RN102 running OS Version 6.10.2-T49 (Beta 1) - ftp issues again

[ArchPrime]

Hi , I have a webhosting account that has cron jobs set up to each evening ftp transfer a current tar archive copy of each website I am currently working on to my own ReadyNAS device. 

For most websites this has been working fine for some time now, and contunues to work fine, but for the last couple of OS updates, I have noticed the largest archives (1.3GB) are struggling to get through - with the cron job every time reporting: 

"Warning: ftp_put(): Opening BINARY mode data connection ..... FTP upload has failed"

Any help much appreciated.  What could be causing ReadyNAS to struggle with larger files?

Filesizes have not really changed since before the isse started - only changes I can think of are in RN102  OS version installed.

[Sandshark]

Just a guess, but the memory-constrained RN100 series is going to start to have issues at some point as the OS grows in size, leaving less RAM for cache.

[Icewaterhot]

Hi @ArchPrime 

Can you show me how you configured ftp job? I tried FTP job upload file (2GB) and download file (2GB) are work fine, Can you check if it is a network interruption problem? 

[ArchPrime]

Hi, thank you very much for testing.

You have given me an idea - I wonder if maybe that my ftp speed is slightly lower, or file size is slightly larger than before, so that my existing cron jobs schedule cannot quite complete one ftp transfer from web host to NAS before the next cron job starts up - so maybe a new incoming ftp connection is enough to cause current ftp in progress to fail?

In support of this idea, I have in the past had incoming ftp transfers fail part way through, when ReadyNAS starts up some other backup process , conecting to and backing up a directory  on my local PC via wired LAN connection.

Maybe OS6 cannot deal gracefully with more than one sustained network transfer event at a time through its NIC?

For what it is worth, below is an example cron command on web host. 

/opt/php56/bin/php  /home/primen5/public_html/xcloner/cloner.cron.php PrimeXclonerBackup2.php

I had set about 5 of these to run each night, spaced out at one hour intervals. Each command is identical apart from particular php script to invoke.

A utility called Xcloner makes php script for each website , which cron invokes to archive then ftp a copy of archive to user set destination.

This particular example  relates to one of two ftp jobs that now fails every time (both >1GB)  - but the others still work fine.

[Icewaterhot]

If our backup job is doing,  it is not finished last time, it will wait for the last time to finish it and will do it next time.So stll check it on your logs,At the time of your problem, "journalctl -a" will see what goes wrong.

[ArchPrime]

Hi - but if another process that is not a backup driven by OS6  (like a 2GB ftp file incoming)  tries to use the ReadyNAS NIC at the same time as backup is running, can this cause a problem for the incoming FTP, or for the backup?

[ArchPrime]

OK, I tested my hypothesis by spacing the cron jobs to two hours apart, but still received the same ftp error with the first job in the queue:

 <b>Backup file: /home/primen5/public_html/prime/administrator/backups/prime.tar</b><br /> <b>Starting ftp transfer:</b><br /> Connected to Normal <b>101.100.159.59:50000 Mode: Passive</b><br />
Warning: ftp_put(): Opening BINARY mode data connection for /PrimeBackupNAS/Websites Backup/prime.tar in /home/primen5/public_html/xcloner/cloner.cron.php on line 265 <b style='color:red'>FTP upload has failed for file /home/primen5/public_html/prime/administrator/backups/prime.tar ! Stopping ....<br /></b>

[ArchPrime]

Hi, I dowloaded ReadyNAS logs, but see no log named "journalctl -a" amongst them.

I am  pasting a link below to a zipfile containing all logfiles in the hope that one of them is relevent (this forum won't allow zip files as attachments)

<redacted link>

The most recent ftp error was reported by the cron job at the web host on Mon 9/09/2019 11:23 PM (New Zealand time)

These log files were downloaded shortly after that notification.

[StephenB]

You shouldn't post log zip files publicly - there are some privacy implications.  I've redacted the link for you.

---

@ArchPrime wrote:

Hi, I dowloaded ReadyNAS logs, but see no log named "journalctl -a" amongst them.

---

Of course not.  Journalctl is a program that runs on the NAS - you need to run that via ssh.

[ArchPrime]

Hi thanks Stephen

Are any of those non SSH logs likely to be relevent?  If so, I am happy to send the link privately to anyone willing to look.

I don't know anything about SSH - I understood  that it was not supported for use by the average ReadyNAS user, and that using it may render me ineligible for official support? (I am out of warrenty already, so do not want to add another reason to discourage Netgear developers or other helpful people from taking an interest)

[StephenB]

Kernel.log is flooded with rx overruns

Sep 09 22:29:23 PrimeNAS kernel: mvneta d0074000.ethernet eth0: bad rx status 0e830000 (overrun error), size=455
Sep 09 22:29:23 PrimeNAS kernel: mvneta d0074000.ethernet eth0: bad rx status 0e830000 (overrun error), size=15
Sep 09 22:29:30 PrimeNAS kernel: mvneta d0074000.ethernet eth0: bad rx status 0e830000 (overrun error), size=664
...

I am suspecting that your RN102 just can't keep up.

[StephenB]

---

@ArchPrime wrote:

I don't know anything about SSH - I understood  that it was not supported for use by the average ReadyNAS user, and that using it may render me ineligible for official support? 

---

You are on your own if you use it, and Netgear reserves the right to deny support if there's evidence that you did damage to the OS with it.

But it's ok to enable it and use it to troubleshoot, etc.  it's enabled on all my own NAS.

---

@ArchPrime wrote:

Are any of those non SSH logs likely to be relevent?  If so, I am happy to send the link privately to anyone willing to look.

---

I suggest PMing @Icewaterhot since he has already expressed interest.

As I noted you are getting a lot of rx overruns on the NIC - which means that the NAS isn't able to keep up with the inbound traffic.

[ArchPrime]

Hi, thanks for that.

Is it likely more RAM would help?  I have an option on an old ReadyNAS Ultra 6 which has been upgraded to 4GB ram.

Also, is it possible that something in recent OS6 releases has opened the faucet too wide for RN102 hardware?  Can the OS be made to throttle  back according to known hardware available?

[ArchPrime]

In orrder to acces that log I have enabled SSH, and tried to log in using Putty application from Windows 10.

Every time I try to log in using the correct NAS IP address, administraor username and password, Putty exits with error "Remote side unexpectedly closed network connection".

Before I beat my head in trying to understand the subtlties of SSH, is "journalctl -a" log likely to give better info on my ftp issue than already gleaned from standard system logs originally posted (showing rx errors)?

[StephenB]

Did you enable ssh in the web ui (system->settings->services)?  After it's enabled, log is as root using the NAS admin password.

There is some info on using journalctl here: https://www.linode.com/docs/quick-answers/linux/how-to-use-journalctl/

If I have an idea on what I'm looking for, I find it useful to pipe journalctl with grep:

# journalctl --no-pager -r | grep -i "search term" 

The -r option reverses the default presentation (showing newest messages first).

Putty works fine, but if you like you can also enter ssh root@nas-ip-address from the windows search bar.

[Icewaterhot]

Thanks to stephenB for explaining the details to the customer.

Hi @ArchPrime  

You can send me the log, as follow:

https://kb.netgear.com/21543/How-do-I-send-all-logs-to-ReadyNAS-Community-moderators

[StephenB]

---

@ArchPrime wrote:

Is it likely more RAM would help?  I have an option on an old ReadyNAS Ultra 6 which has been upgraded to 4GB ram.

Also, is it possible that something in recent OS6 releases has opened the faucet too wide for RN102 hardware?  Can the OS be made to throttle  back according to known hardware available?

---

More RAM might make a difference - and I suspect the ultra 6 would have a faster CPU.  You could convert the Ultra to run OS6 btw.

Netgear does drop some features on the Arm platforms (and in 6.10.x they turned off AntiVirus on the RN100s).  There's not much you can do beyond that, other than turn off services you don't use (and of course uninstall apps you don't need).

[ArchPrime]

Thanks Stephen.

Will try both strategies and report results, for anyone else striking similar issues...

In the first strategy (reducing resource load on RN102) uninstalling apps is easy enough, but In terms of switching off services, what else can/should I switch off?

I currently have no options enabled under the 'cloud' tab in OS6

I currently have no NAS instigated backups scheduled (only the incoming ftp backups initiated at web host)

Is there any benefit in reducing the various network sharing modes on offer (eg DNLA, RSync etc)? If so, which of these can I generaaly switch off safely?  Most of these I wouldn't recognize the use case for.

I basically need to accept incoming FTP,  to detect /mount/share readynas drives from Windows 10, and from Android devices so these platforms can back up to RN02, or occasionally retrive a file from there , including when devices are in remote a location  - and that is about it. 

No particular need for media streaming , or for Mac or linux sytems to have access ReadyNAS.

Cheers

Paul

[StephenB]

---

@ArchPrime wrote:

I basically need to accept incoming FTP,  to detect /mount/share readynas drives from Windows 10, and from Android devices so these platforms can back up to RN02, or occasionally retrive a file from there , including when devices are in remote a location  - and that is about it. 

---

All you need for that are SMB, FTP, and HTTP/HTTPS.  Perhaps SSH for troubleshooting - it doesn't add anything to the processing load or memory footprint when you aren't using it.  Rsync is handy for incremental backup to other linux systems (and if the server you are backing up uses Linux, then you might see if that works better for you than FTP).  If this is over-the-internet you can (and should) use FTPS or Rsync-over-SSH.

Turning off "Quota" on the volume settings wheel will also reduce the memory/processing footprint for the BTRFS file system.  You won't see a full disk usage breakdown from the shares page - but you can re-enable quota briefly if you need that.