readynasd stuck at high cpu-load, web-backend not working

[DanRoth]

Trying go get an afp-share with snaphots announced as a timemachine-service apparently I messed up something.

This is what happens:

• Soon after start, readynasd will take up all cpu, the web-backend will not connect.
• Services are still running.
• I can ssh into the machine, it is not even that slow.
• /fwbroker web access is working. 

This is what I did so far:

• Upgrade the firmware using /fwbroker to 6.6.1

This is what I get from logs /status pages etc:

root@myreadynas:~# systemctl status readynasd.service

readynasd.service - ReadyNAS System Daemon

   Loaded: loaded (/lib/systemd/system/readynasd.service; enabled; vendor preset: enabled)

   Active: activating (start) since Mon 2017-02-27 12:13:28 CET; 1min 29s ago

Main PID: 2181 (readynasd)

   Status: "Importing DB"

   CGroup: /system.slice/readynasd.service

           └─2181 /usr/sbin/readynasd -v 3 -t

Feb 27 12:13:28 myreadynas systemd[1]: Starting ReadyNAS System Daemon...

Feb 27 12:13:28 myreadynas readynasd[2181]: readynasd log started

Feb 27 12:13:28 myreadynas readynasd[2181]: readynasd started. (restarted=0)

Feb 27 12:13:28 myreadynas readynasd[2181]: DB (main) schema version: 23 ==> 23

Feb 27 12:13:29 myreadynas readynasd[2181]: DB (queue) schema version: new ==> 0

Feb 27 12:13:29 myreadynas readynasd[2181]: Cannot load/etc/frontview/samba/usermap.conf

root@myreadynas:~# /usr/sbin/readynasd -v 5 -t -l -

[17/02/27 12:34:46] 3657:3657 readynasd.c:1016 () readynasd log started

[17/02/27 12:34:46] 3657:3657 context.c:177 (debug) NEW GLOBAL_CONTEXT: db.sq3=0xabc420

[17/02/27 12:34:46] 3657:3657 context.c:213 (debug) NEW GLOBAL_CONTEXT: event.sq3=0xac6718

[17/02/27 12:34:46] 3657:3657 context.c:216 (debug) NEW GLOBAL_CONTEXT: globalctx=0xabbea8

[17/02/27 12:34:46] 3657:3657 readynasd.c:1206 () readynasd started. (restarted=0)

root@myreadynas:~# /usr/sbin/readynasd -v 4 -t -l -

[17/02/27 12:35:09] 3663:3663 readynasd.c:1016 () readynasd log started

[17/02/27 12:35:09] 3663:3663 readynasd.c:1206 () readynasd started. (restarted=0)

[17/02/27 12:35:29] 3663:3663 db_access.c:152 (error) Error create table (db)

[17/02/27 12:35:29] 3663:3663 readynasd.c:1248 (error) dba create failed

root@myreadynas:~# df -h

Filesystem     Size Used Avail Use% Mounted on

udev             10M 4.0K   10M   1% /dev

/dev/md0       4.0G 639M 3.1G 17% /

tmpfs           249M     0 249M   0% /dev/shm

tmpfs           249M 396K 249M   1% /run

tmpfs           125M 704K 124M   1% /run/lock

tmpfs           249M     0 249M   0% /sys/fs/cgroup

/dev/md127     1.9T 977G 880G 53% /data

/dev/md127     1.9T 977G 880G 53% /home

/dev/md127     1.9T 977G 880G 53% /apps

Any help is highly appreciated. 

[DanRoth · Accepted solution]

Back in the game! 

"OS reinstall" did the trick.

Apparently the system could now handle the stuck process and even apply a lingering update. It now reports 6.6.1 and is working normally. 

Thank you a lot for the fast help!

[Retired_Member]

Switch off AV service. It might be overwhelmed by that. 6.6.1 is known for problems with AV. Wait for fw 6.7 to enable it again.

[DanRoth]

I never knowingly activated any AV (Antivirus?) service. Is it possibly running anyway? How does it relate to readynasd? Could you tell me the name of the AV service binary. As I said, I cannot use the web backend.

[Retired_Member]

Yes Antivrus. Unfortunately I do not know how to do this using ssh. Community, is there anybody else who might have an idea on this.

Well, you could also try different browsers to check, whether not being able to reach the admin website is a browser issue.

[StephenB]

---

@DanRoth wrote:

Trying go get an afp-share with snaphots announced as a timemachine-service apparently I messed up something.

Feb 27 12:13:29 myreadynas readynasd[2181]: Cannot load/etc/frontview/samba/usermap.conf

---

Did you change anything (including .conf files) with ssh?  If so, can you reverse those changes?

[DanRoth]

---

@StephenB wrote:

Did you change anything (including .conf files) with ssh?  If so, can you reverse those changes?

---

I will try that.

Right now I have different poblem: a "shutdown -r now" not only restarted the system but apparently sent the attached UPS (APC Back-UPS 650) into a frenzy-state switching into battery mode and then cutting off the power in mid boot. Now the device is bricked with a blinking power button.

That is one of the weirdest things I have seen for a long time. There is no doubt that the shutdown sequence triggered something in the UPS but I have no idea what and how.

Anyway this is way beyond the OP.

[StephenB]

You might need to back up the NAS and rebuild it from scratch.

[DanRoth]

---

@StephenB wrote:

You might need to back up the NAS and rebuild it from scratch.

---

My idea was to try a Bootmenu > "OS reinstall" with one of the RAID 1 disks, leaving the second disk as a spare backup. That way I maybe get access to the disk content again and could so a backup over the network.

It is not so easy to mount the Raid disks directly to another machine, there is some mdadm stuff necessary. 

[DanRoth · Accepted solution]

Back in the game! 

"OS reinstall" did the trick.

Apparently the system could now handle the stuck process and even apply a lingering update. It now reports 6.6.1 and is working normally. 

Thank you a lot for the fast help!

[StephenB]

---

@DanRoth wrote:

Back in the game! 

"OS reinstall" did the trick.

---

I was worried that it might not be enough, but clearly it was.

Great news, thx for following up.