ReadyNAS 3220 Locking up after upgrading to 6.2

[kplatte]

Logs sent.
FYI NAS did not lockup last night during backups.

[mdgm-ntgr]

It still looks like you have a huge amount of metadata.

Lowering the thread count may help. With a high NFS thread count you quickly get a lot of fragmentation.

[kplatte]

NAS has not locked up for 2 night in a row now. Still running 6.3.3 beta
Well we had to set thread count to 8 or some are backups would not start since were not enough threads to mount NFS share.

[kplatte]

Maybe it was the bad hard drives causing lockups over night. 4 nights now all good. We will know for sure if it makes it over the weekend since we run some different backups.

[kplatte]

Made it to Sunday night backups then it locked up again.

[mdgm-ntgr]

You have a lot of fragmentation and it appears your volume was created on 6.1.4 or earlier.

I would suggest you backup your data, do a factory reset (wipes all data, settings, everything) and restore your data from backup.

You should also try and find a way to stagger your backups so that you can lower that thread count.

[kplatte]

Yeah working on getting another drive big enough to copy files off the NAS before we can try a factory reset

[kplatte]

Just another update. No lockups since Sunday night. I did lower thread count to 2.
We changed backup script to mount and unmount NFS share before and after each backup so that maybe helping with the lower thread count working.

We have 2 backups that run at 10pm those our NAS to NAS copies. We have 2 backups that run at 2am every night (Server to NAS) and thats when NAS would lock up. But we have disabled one of them for past few weeks and only running the more important backup.

Will see if makes it over the weekend with thread count set to 2.

[kplatte]

Update:
NAS is now only locked up on Sunday mornings so far past 2 weeks.
Will see how this week goes.

[mdgm-ntgr]

Any pattern as to what happens Sunday mornings which may explain the lockups?

[kplatte]

Looks like we found a job that runs Sunday mornings at 4am that rotated are weekly backups on the NAS only keeping the 5 latest backups. We changed the time that job runs to see if that may of been causing lockups on Sunday backups.

[kplatte]

No luck. NAS still locking up over weekends.
I see a new beta6 will install that and see if that helps.

[kplatte]

Updated firmware to 6.3.3-T137 Monday. No luck ups yet. Will see how it does over the weekend.

[Jedi_Knight]

Any luck?

[kplatte]

Locked up last night. Red health light was on this morning this time.

[mdgm-ntgr]

Always at the same time on Sunday?

[kplatte]

nope different times. last week like 4am last night 2:30am

[mdgm-ntgr]

What's running at that time on the weekend? Is there an unusually high load at that time?

[kplatte]

I rebooted the NAS and it is showing our volume is degraded again and is doing a rebuild on it now.

[mdgm-ntgr]

Can you send in a fresh set of logs?

[kplatte]

Latest log file sent.

[mdgm-ntgr]

Your smart_history.log indicates a failing disk.

I still would recommend you backup your data (if you haven't already), do a factory reset (wipes all data, settings, everything) with good disks in place and restore your data from backup.

[mdgm-ntgr]

Let's see if the setting change I just made helps.

[kplatte]

NAS has locked up past 2 nights now.
Probably due to the bad drive. Getting more errors in log to replace drive now after rebooting.

[kplatte]

Locked up again last night. Replacement hard drive comes in today.