NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
Readynas 104 won't boot. Error 354 out_of_memory. After upgrade to 6.4.1 #26323717
installed 6.4.1 from 6.2.5 yesterday. All went ok. Did all the 'calculating'. worked fine.
failed overnight with out_of_memory error 354. now won't boot.
tried reinstallOS. No luck.
tota...
We have the same problem with one of our NAS here at work, so it’s not the only one.
So good to hear there might be workarounds and that it doesn’t seem to be a HW (memory) problem. Found it already a bit too coincidental that the mem failure would’ve just happened after fw upgrade. So prolly will save us RMA hassle.
Luckily the NAS was only rarely used until now for some less important backups. But it will normally get also another purpose in the very near future so would like to get it fully working again soon.
Symptoms are (almost exactly) the same as described above:
After fw upgrade from 6.2.5 to 6.4.1 system became unusable, normal boot never finishes and results in an out of memory error 354 after half an hour or so. Only hard power off (unplug) possible.
RAIDar sees the NAS booting "System starting up...", and then after a while "Management service is offline", and then after about 20 min it disappears and then shortly after also the OOM 354 error is displayed on the NAS.
During the period where it shows "Management service is offline" it shows the option buttons: Admin Page, Browse, Locate, Download Logs, Diagnostics
All of the buttons seem to work ok for a while, except for Admin Page:
Not able to access admin page: After supplying user/pw "Connecting to ReadyNAS Admin Page..." and then after a while "ReadyNAS Admin Page is offline."
Can perform locate (LEDs on), shares seem to be accessible for a short while and I can download logs a few times and Diagnostics are ok.
Tried several things:
Mem test displays 00:00:01 immediately after launching it and staying there forever. Hard reboot needed (power unplug). Don’t know if this display is pattern #, test time or error #.
Tried USB OS (re)install and OS reinstall from Boot Menu, didn’t help.
Tried the boot without network cable, as above, but has the same OOM result. So this is slightly different from above.
What does kinda work is booting in "Volume read only" mode from boot menu.
In this mode the NAS seems to boot and work normal, except for the fact you can’t write to it or delete anything of course and that the calculation (of used/free space) doesn’t seem to end.
Was able to do full backup (1.5T) and later a full CRC check of the backed up data on NAS. NAS was on for about a week like this and no problems.
Also scrub, balance and defrag don’t give errors, altho wonder how this can really work on a RO volume.
Yesterday tried to flash 6.4.2 RC1 but same problem still exists. Except that the error # now also is 390 instead of 354.
Volume RO is still the only mode where we can still do something.
There are no quota on the shares, so cant disable that. :(
Config:
RN10400
2 x 4TB (X-raid Raid-1) 60% Free
very basic config:
Services: SMB, HTTP, HTTPS (AV disabled)
no apps
no quota
besides admin only one extra user.
3 shares: one with about 1.5T data with snapshots and bit rot enabled. Only 36 snapshots, so not really a large number. Permissions: admin + user.
2 test ones, currently empty, (snapshots disabled, bit rot enabled). Permissions: anonymous.
LifeVibes wrote:We have the same problem with one of our NAS here at work, so it’s not the only one.
So good to hear there might be workarounds and that it doesn’t seem to be a HW (memory) problem. Found it already a bit too coincidental that the mem failure would’ve just happened after fw upgrade. So prolly will save us RMA hassle.
Luckily the NAS was only rarely used until now for some less important backups. But it will normally get also another purpose in the very near future so would like to get it fully working again soon.
Symptoms are (almost exactly) the same as described above:
After fw upgrade from 6.2.5 to 6.4.1 system became unusable, normal boot never finishes and results in an out of memory error 354 after half an hour or so. Only hard power off (unplug) possible.
RAIDar sees the NAS booting "System starting up...", and then after a while "Management service is offline", and then after about 20 min it disappears and then shortly after also the OOM 354 error is displayed on the NAS.
During the period where it shows "Management service is offline" it shows the option buttons: Admin Page, Browse, Locate, Download Logs, Diagnostics
All of the buttons seem to work ok for a while, except for Admin Page:
Not able to access admin page: After supplying user/pw "Connecting to ReadyNAS Admin Page..." and then after a while "ReadyNAS Admin Page is offline."
Can perform locate (LEDs on), shares seem to be accessible for a short while and I can download logs a few times and Diagnostics are ok.
Tried several things:
Mem test displays 00:00:01 immediately after launching it and staying there forever. Hard reboot needed (power unplug). Don’t know if this display is pattern #, test time or error #.
Tried USB OS (re)install and OS reinstall from Boot Menu, didn’t help.Tried the boot without network cable, as above, but has the same OOM result. So this is slightly different from above.
What does kinda work is booting in "Volume read only" mode from boot menu.
In this mode the NAS seems to boot and work normal, except for the fact you can’t write to it or delete anything of course and that the calculation (of used/free space) doesn’t seem to end.
Was able to do full backup (1.5T) and later a full CRC check of the backed up data on NAS. NAS was on for about a week like this and no problems.
Also scrub, balance and defrag don’t give errors, altho wonder how this can really work on a RO volume.
Yesterday tried to flash 6.4.2 RC1 but same problem still exists. Except that the error # now also is 390 instead of 354.
Volume RO is still the only mode where we can still do something.
There are no quota on the shares, so cant disable that. :(
Config:
RN10400
2 x 4TB (X-raid Raid-1) 60% Free
very basic config:
Services: SMB, HTTP, HTTPS (AV disabled)
no apps
no quota
besides admin only one extra user.
3 shares: one with about 1.5T data with snapshots and bit rot enabled. Only 36 snapshots, so not really a large number. Permissions: admin + user.
2 test ones, currently empty, (snapshots disabled, bit rot enabled). Permissions: anonymous.
Your situation is identical to mine so I suspect something else has been done that's not been detailed. Unless it's specificly this special version of 642+quotas off?
This problem appears to happen on the 104 in some cases when the filesystem is quite full after updating from 6.2.x.
Even if you don't have quotas enabled for any shares we do use quotas still to calculate how much space is used by snapshots for instance.
There is likely a combination of things using up memory on affected systems.
Having run into this problem if you boot into read-only mode and backup your data, do a factory default (wipes all data, settings, everything) and restore your data from backup the system should be running fine with quotas enabled.
If you run into this if you prefer we can disable quotas from the backend so that the NAS will boot in normal mode.
This problem appears to happen on the 104 in some cases when the filesystem is quite full after updating from 6.2.x.I guess quite full is quite relative, as ours still has 60% free...
Even if you don't have quotas enabled for any shares we do use quotas still to calculate how much space is used by snapshots for instance.
yeah, I guessed there probably wouldve also been some internal quotas.
There is likely a combination of things using up memory on affected systems.
Now the tricky part is to find out the combination. But seeing that it seems mostly quite basic configs (min services, no apps, no AV etc), and the fact that it boots ok in RO mode and that in normal mode there seems a small window where it seems quite ok, except for the Admin Page which might be another clue, before it breaks down, I guess reduces the # combos.
If there is something we can do, let us know. Already have a looot of logs we could send :smileylol:
Having run into this problem if you boot into read-only mode and backup your data, do a factory default (wipes all data, settings, everything) and restore your data from backup the system should be running fine with quotas enabled.
That wouldve been my ultimate last resort option and was already preparing for it by making a new backup. On ext4 target this time as previous was on NTFS to be able to read under Windows. Just hope the backup is a bit quicker this time, as it took about 2.5 days for backing up 1.5TB :smileyfrustrated:
Is it also ok to backup/restore the configuration, or better to redo the config from scratch after the factory default?
Will the restore also restore the snapshots etc?
And then hope that by restoring the backup the memory combo isnt triggered again. :)
If you run into this if you prefer we can disable quotas from the backend so that the NAS will boot in normal mode.
Think we prefer to start from scratch then, cause who knows what the quota disable might cause in the future :P ;)
Update:
My system is stable. Support recommended a fresh install would be good. I have a stand-by NAS which is a mirror of the main machine (which I used during the outage) and once that's sync'd (and backed up to external drives too!) I'll do a wipe/restore.
The one and only thing that's not good is read and write speed. I've gone from a comfortable 30-40MBytes/s read/write to 8.5 write, 27 Read. Which is actually worse than my old NV+ which I use as a backup.
I'm going to do a restart and see if that improves but I doubt it will.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
