This topic has been marked solved and closed to new posts due to inactivity. We hope you'll join the conversation by posting to an open topic or starting a new one.
Detected high command timeouts, short spin downs and where is readynas OS installed please?
Hi all,
Started getting these emails recently
Detected increasing command timeouts errors: [25] on disk 2 (Internal) [ST4000VN000-1H4168, Z3018KBN] 21 times in the past 30 days. This condition often indicates an impending failure. Be prepared to replace this disk to maintain data redundancy.
TLDr
1) Should I be worried about the high command timeout warnings?
2) Where is the Readynas OS stored, on installed disks or on the user installed ones?
Setup info: - Readynas RN102 - 6.9.3
- No installed apps but isc-dhcp-server and bind installed through ssh.
- Spindown set to 25 min - JBOD, no snapshots
I'm also getting some other warnings in journalctl
ug 27 12:38:41 NETNAS noflushd[18778]: Spinning up disk 1 (/dev/sdb) after 0:01:33.
Aug 27 12:38:41 NETNAS noflushd[18778]: Spinning up disk 2 (/dev/sdc) after 0:01:30.
Aug 27 12:39:55 NETNAS kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Aug 27 12:39:55 NETNAS kernel: ata2.00: failed command: SMART
Aug 27 12:39:55 NETNAS kernel: ata2.00: cmd b0/d0:01:00:4f:c2/00:00:00:00:00/00 tag 9 pio 512 in
res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Aug 27 15:38:43 NETNAS noflushd[18778]: Spinning up disk 1 (/dev/sdb) after 0:00:47.
Aug 27 15:38:44 NETNAS noflushd[18778]: Spinning up disk 2 (/dev/sdc) after 0:00:46.
Aug 27 15:38:44 NETNAS systemd[1]: Started Radar Update.
Aug 27 15:39:01 NETNAS kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Aug 27 15:39:01 NETNAS kernel: ata2.00: failed command: SMART
Aug 27 15:39:01 NETNAS kernel: ata2.00: cmd b0/d0:01:00:4f:c2/00:00:00:00:00/00 tag 17 pio 512 in
res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Aug 27 15:39:01 NETNAS kernel: ata2.00: status: { DRDY }
Aug 27 15:39:01 NETNAS kernel: ata2: hard resetting link
Aug 27 15:39:01 NETNAS kernel: ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Aug 27 15:39:01 NETNAS kernel: ata2.00: configured for UDMA/133
Aug 27 15:39:01 NETNAS kernel: ata2: EH complete
Aug 27 15:39:18 NETNAS readynasd[2334]: Detected increasing command timeouts on disk 2 [25, 26]
Aug 27 15:39:18 NETNAS readynasd[2334]: Detected high command timeouts: [26] on disk 2 (Internal) [ST4000VN000-1H4168, Z3018KBN]. This condition often indicates an impending failure. Be prepared to replace this disk to maintain data redundancy..
Aug 27 16:36:22 NETNAS noflushd[18778]: Spinning up disk 1 (/dev/sdb) after 0:02:53.
Aug 27 16:36:22 NETNAS noflushd[18778]: Spinning up disk 2 (/dev/sdc) after 0:02:51.
Aug 27 16:37:33 NETNAS kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Aug 27 16:37:33 NETNAS kernel: ata2.00: failed command: SMART
Aug 27 16:37:33 NETNAS kernel: ata2.00: cmd b0/d0:01:00:4f:c2/00:00:00:00:00/00 tag 2 pio 512 in
res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Aug 27 16:37:33 NETNAS kernel: ata2.00: status: { DRDY }
Aug 27 16:37:33 NETNAS kernel: ata2: hard resetting link
Aug 27 16:37:33 NETNAS kernel: ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Aug 27 16:37:33 NETNAS kernel: ata2.00: configured for UDMA/133
Aug 27 16:37:33 NETNAS kernel: ata2: EH complete
Aug 27 16:37:50 NETNAS readynasd[2334]: Detected increasing command timeouts on disk 2 [26, 27]
Aug 27 16:37:50 NETNAS readynasd[2334]: Detected high command timeouts: [27] on disk 2 (Internal) [ST4000VN000-1H4168, Z3018KBN]. This condition often indicates an impending failure. Be prepared to replace this disk to maintain data redundancy.
Aug 27 16:37:52 NETNAS msmtpq[31120]: mail for [ -C /etc/msmtprc email@replaced.com --timeout=60 ] : send was successful
Aug 27 16:37:52 NETNAS readynasd[2334]: Current (temp, start_stop_cnt, power_on_hrs, power_cycle_cnt, load_cycle_cnt) = (35, 25776, 32977, 232, 25784)
Spindowns I'm noticing that my spin down time is very short
Aug 27 14:23:27 NETNAS noflushd[18778]: Spinning down disk 2 (/dev/sdc).
Aug 27 14:24:45 NETNAS noflushd[18778]: Spinning up disk 1 (/dev/sdb) after 0:01:17.
Aug 27 14:25:04 NETNAS noflushd[18778]: Spinning up disk 2 (/dev/sdc) after 0:01:35.
Aug 27 14:55:04 NETNAS noflushd[18778]: Spinning down disk 1 (/dev/sdb).
Aug 27 14:55:07 NETNAS noflushd[18778]: Spinning down disk 2 (/dev/sdc).
Aug 27 15:00:15 NETNAS noflushd[18778]: Spinning up disk 1 (/dev/sdb) after 0:05:08.
Aug 27 15:00:16 NETNAS noflushd[18778]: Spinning up disk 2 (/dev/sdc) after 0:05:06.
I'd expect this for Disk 2 which has scripts writing to it, but not Disk1, or are both Disks always spun up/ down at the same time?
Readynas OS I was thinking I could move some scripts, DHCP lease files/ logs to a usb stick to try to reduce the amount Disk2 is accessed but that led me me to wonder where the OS is installed?
I'm not quite understanding the disk layout and was hoping someone could help interpret it for me. Am I close? My best guess is that - There's no inherent/ internal storage for the readynas without Disks installed?! This is what's confusing me, how does it boot first time? Disks are installed with btfrs file system
- A 4Gb partition is created on each disk with RAID for the OS and mirrored? Hence my spindowns not lasting due DHCP leases, or similar files being written? - I'm wondering whether these disk timouts are therefore not a problem... it's just the OS trying to write to disk and having to wait for the disk to spin up but registering that as a delay? - Shares are created with /dev/md* but not mirrored
/dev/md0:
Version : 1.2
Creation Time : Sat Dec 24 16:08:02 2016
Raid Level : raid1
Array Size : 4190208 (4.00 GiB 4.29 GB)
Used Dev Size : 4190208 (4.00 GiB 4.29 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Mon Aug 27 18:39:46 2018
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : 0e35e4f0:0 (local to host 0e35e4f0)
UUID : c0f20fb2:52af152f:7b54b5bb:2b846f3c
Events : 89151
Number Major Minor RaidDevice State
3 8 17 0 active sync /dev/sdb1
2 8 33 1 active sync /dev/sdc1
2) Where is the Readynas OS stored, on installed disks or on the user installed ones?
Not sure what you mean by "installed disks" vs "user-installed disks". The only disks are the two you installed. The OS is stored on both of them (on a 4 GB RAID-1 volume - /dev/md0).
2) Where is the Readynas OS stored, on installed disks or on the user installed ones?
Not sure what you mean by "installed disks" vs "user-installed disks". The only disks are the two you installed. The OS is stored on both of them (on a 4 GB RAID-1 volume - /dev/md0).
Re: Detected high command timeouts, short spin downs and where is readynas OS installed please?
Thanks Stephen,
1) No timeouts since I turned off the disk spin down on 28th. The disk is getting the same level of access from the scripts during that time so it's likely related to an error being logged specifically when spinning up (?) That's helpful now though, I'll start moving files that change somewhere else and see whether I can get then to properly spin down for more than a couple of minutes and whether I then get the errors when they spin back up
2) Thanks, sorry for the lack of clarity there. I was wondering whether it had some small internal memory/ disk installed.
So... being really stupid. How is the OS installed on first boot? Let's say I put two new disks in there, where is the image/ bootstrap held to install onto those?
Re: Detected high command timeouts, short spin downs and where is readynas OS installed please?
On initial install (or a factory reset), the drives are formatted and the OS is installed from flash memory. When/if the system boots up in safe mode it is also running from flash. Otherwise it boots from the disks.
Re: Detected high command timeouts, short spin downs and where is readynas OS installed please?
Have you come across some document explaining the boot procedure? In other systems I use I'd expect to see the root partition booted through /etc/fstab or the
initramfs (cat /proc/cmdline)
Reason I'm asking is I was thinking of moving the var folder in the hope of spinning down disks, although I guess I could just move the logs as that seems the most accessed (already moved DHCP leases)
