- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
RN104 Poor network performance after upgrade to firmware 6.9.3
Hi
Recently had to update to FW 6.9.3.
The ReadyNAS is now extremely slow, network wise. Browsing from Windows Explorer seems to have to wake the drive up, and when I try and copy files to it I don't get any consistent speed, it moves up and down from 0kb/s to 10MB/s (ReadyNAS is located on a 100MB network)
Can't see any network settings to confirm duplex or anything, but worked fine on earlier FW versions (went from 6.2.0 to 6.5.x to 6.9.3)
The NAS is basicaly unusable at the minute
Any ideas?
Thanks
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
Have you checked network_settings.log for errors?
You may wish to get a gigabit switch and see if connecting the NAS to that and connecting the switch to your router works better.
Or maybe replace your router with one with gigabit ports.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
It’s nothing to do with the port. I only have 100mb ports in the house, it’s inconsistent network after the firmware update
I rebooted the NAS and it seeems to work fine for a short period then degrades however I survived to power it off from the plug to reboot. If I try reboot from the admin site it sits at rebooting forever and doesn’t come back
This firmware definitely has bugs
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
It's in the logs zip file that you can download. You can send in your logs if you like (see the Sending Logs link in my sig).
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
Hi
Thanks, I appreciate that - I'll send the logs over now
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
The logs are flooded with I/O errors for disk 4 (see e.g. systemd-journal.log). It looks like that disk is bad. smart_history.log shows that the ATA error count on that disk has recently climbed to 6.
Disk 3 has a current pending sector count of 1 though that was from way back in June 2015.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
Hi
Thanks for that notice, I'll have a look at replacing disk 4. Could the I/O errors be due to bad network throughput hampering disk performance?
The network performance and the network errors I get when copying to it started after the 6.9.3 update
I did notice that it seems to be trying to connect to my network at 1000 half duplex, when it's connected to a 100mb switch.
Is there a way to set the card speed and duplex to stop the mismatch that auto detect is doing?
Also, as I said, the system is super fast straight after a powerdown and reboot, but then degrades over a short period of time (not sure how long, but looks to be over a 4-6 hour period - rebooted last night around 11pm and it was great straight after, checked this morning and it's slow again)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
The disk failure and the firmware update would be a coincidence.
eth1 is reporting 100Mb/s Full Duplex in network_settings.log. The info showing for eth0 is not relevant as you're not using that network port.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
So do you think that the performance issues are down to drive 4 looking to have read errors and nothing to do with the firmware?
If I pulled Drive 4 and left it as a single disk waiting to rebuild the array that would then confirm if the performance improved - but if I then put Drive 4 back in would it detect it as a newdrive and rebuild the array (while I purchased a new drive)
Do the tag#4 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE relate to potential drive failures?
When I look at the volume log I can see 1 pending sector count for sdb, but nothing for sda and both are passing the latest self test
I don't want to have to buy a new drive if that's not the cause, as you can appreciate. Happy to get one if it's pending a failure, but if that still doesn't fix the issue, I'm back to square one with a NAS that no longer performs properly
Really appreciate the help on this
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
smart_history.log shows that the current pending sector on sdb was years ago.
The ata errors on sda are much more recent.
The errors point either to a bad disk or much less likely to a hardware problem with the chassis. You may wish to test the disk using WD Data LifeGuard Diagnostics.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
Just done some reading and not sure that ata_errors are a worry point just yet, as they seem to coincide with a reboot
https://kb.netgear.com/19392/ATA-errors-increasing-on-disk-s-in-ReadyNAS
Still pretty sure that the fw is the cause of my network performance and availability issues.
I don't suppose you saw anything else in the logs that show any other types of issues, especially building after 11pm last night, which is when I did a last power off and power on?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
@Synn wrote:
Just done some reading and not sure that ata_errors are a worry point just yet, as they seem to coincide with a reboot
https://kb.netgear.com/19392/ATA-errors-increasing-on-disk-s-in-ReadyNAS
Perhaps run a disk test from the volume settings wheel (system->volume page). Download the logs before and after the test, and then see if anything changes.
A failing disk will certainly hurt file transfer speed (as will removing one).
Did you also look at the ethernet stats in network_settings.log?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
I tried the WD tool and it wouldn't detect the NAS so will look at your alternative
Regarding the network stats, I did - and I can see 12801 dropped RX packets on eth1, as an example
No idea how to fix these as not sure if it's the NIC in the NAS that might be faulty now, or some other reason
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
@Synn wrote:
I tried the WD tool and it wouldn't detect the NAS so will look at your alternative
Diagnostics need direct access to the disk itself. You have to remove the drive, and connect it to the PC (either with SATA or via a USB adapter/dock).
@Synn wrote:
Regarding the network stats, I did - and I can see 12801 dropped RX packets on eth1, as an example
That is worth pursuing. However it could be a side effect (if the system can't write to the RAID array fast enough, eventually it will have to drop packets).
What about errors, frame, overruns, and carrier collisions? You can cut/paste the stats and post here if you want us to take a look. This is also a case where downloading the logs, doing some tests (e.g, file transfers), and then downloading the logs again can be useful.
You can also test basic connectivity from a PC. Run the command prompt (CMD) and enter ping nas-ip-address -n 10000 Use the real nas-ip-address. That will ping the NAS 10000 times, and give you a summary at the end. (You can also use smaller counts of course).
BTW, is there a reason you still are running fast ethernet?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
These are the network stats
Link encap:Ethernet HWaddr e8:fc:af:e5:46:37
inet addr:192.168.1.243 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fdaa:bbcc:ddee:0:eafc:afff:fee5:4637/64 Scope:Global
inet6 addr: fe80::eafc:afff:fee5:4637/64 Scope:Link
inet6 addr: 2a00:23c5:915:4a00:eafc:afff:fee5:4637/64 Scope:Global
UP BROADCAST RUNNING MULTICAST DYNAMIC MTU:1500 Metric:1
RX packets:3768537 errors:0 dropped:12801 overruns:0 frame:0
TX packets:1507920 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:532
RX bytes:376942335 (359.4 MiB) TX bytes:28520415460 (26.5 GiB)
Interrupt:27
No errors or overruns
Running fast ethernet as my home router only supported 100mb network ports, so I had 100mb network ports put in the house a long time ago. The switch that the NAS connects to is only 100mb.
100mb is fine for my needs, but it needs to work and not fluctuate
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
@Synn wrote:
No errors or overruns
Yes, so there's nothing definitively pointing to the NIC card or the switch.
But I am seeing that you have ipv6 enabled on the NAS. Can you try disabling that and retesting?
@Synn wrote:
Running fast ethernet as my home router only supported 100mb network ports, so I had 100mb network ports put in the house a long time ago. The switch that the NAS connects to is only 100mb.
Clearly the NAS should keep up with fast ethernet, so this is a sidebar.
But if your cabling is cat5 and connects all 4 twisted pairs end-to-end, then you should be able to use gigabit (assuming the distance isn't too far). And if the NAS is directly connected to the switch, then there would be a performance benefit if the switch is gigabit, even if you set the house ports to fast ethernet. For one thing, modern wifi is often faster than fast ethernet. Another aspect is that multiple devices can access the NAS simultaneously.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
Disabling that now, but it's always been like this and didn't have an issue before these firmware updates
I did have to go from 6.2.0 to 6.5.x to 6.9.3
If I had confidence that it was disk 4 on it's way out I'd just replace it, but I'm not 100% on that
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
The combination of the ATA errors and the I/O error flood is pretty convincing to me that there's a hardware problem of some kind whether it's the disk or something else.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
@Synn wrote:
Disabling that now, but it's always been like this and didn't have an issue before these firmware updates
I understand your thinking on the firmware update theory. But other people running 6.9.3 aren't seeing this.
BTW, if this is enabled on your router (as it appears to be), you might need to disable it there as well as the NAS. Disabling in the router alone isn't enough, as the PCs might use the NAS link-local address.
What apps and services do you have enabled on the NAS? Do you have quota enabled in the volume settings wheel?
@Synn wrote:
If I had confidence that it was disk 4 on it's way out I'd just replace it, but I'm not 100% on that
Can you post the SMART stats (and ATA count) for that drive here?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
I don't have any apps installed on the NAS and have the following services turned on
SMB
AFP
iTunes
ReadyDLNA
uPNP
HTTP
HTTPS
Here's the smart_history log
time model serial realloc_sect realloc_evnt spin_retry_cnt ioedc cmd_timeouts pending_sect uncorrectable_err ata_errors ------------------- -------------------- -------------------- ------------ ------------ -------------- ---------- ------------ ------------ ----------------- ---------- 2015-05-31 19:11:52 WDC WD40EFRX-68WT0N0 WD-WCC4YYYYYYYY 0 0 0 -1 -1 0 0 0 2015-05-31 19:11:53 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 0 2015-06-02 15:14:09 WDC WD40EFRX-68WT0N0 WD-WCC4YYYYYYYY 0 0 0 -1 -1 1 0 0 2018-08-13 16:13:13 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 3 2018-08-13 16:19:03 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 4 2018-08-13 16:39:37 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 5 2018-08-15 15:35:23 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 6
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
time model serial realloc_sect realloc_evnt spin_retry_cnt ioedc cmd_timeouts pending_sect uncorrectable_err ata_errors ------------------- -------------------- -------------------- ------------ ------------ -------------- ---------- ------------ ------------ ----------------- ---------- 2018-08-13 16:13:13 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 3 2018-08-13 16:19:03 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 4 2018-08-13 16:39:37 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 5 2018-08-15 15:35:23 WDC WD40EFRX-68WT0N0 WD-WCC4XXXXXXXX 0 0 0 -1 -1 0 0 6
It's not just thte ATA error count (6 is still pretty low), but the I/O errors in systemd-journal.log, kernel.log and dmesg.log:
Aug 22 08:45:33 ReadyNAS kernel: blk_update_request: I/O error, dev sda, sector 27152759 Aug 22 08:45:33 ReadyNAS kernel: md/raid1:md127: sda3: rescheduling sector 17452544 Aug 22 08:45:33 ReadyNAS kernel: ata1: EH complete Aug 22 08:45:46 ReadyNAS kernel: md/raid1:md127: redirecting sector 17452544 to other mirror: sda3 Aug 22 08:45:53 ReadyNAS kernel: ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 Aug 22 08:45:53 ReadyNAS kernel: ata1.00: irq_stat 0x40000001 Aug 22 08:45:53 ReadyNAS kernel: ata1.00: failed command: READ DMA EXT Aug 22 08:45:53 ReadyNAS kernel: ata1.00: cmd 25/00:40:40:4e:9e/00:05:01:00:00/e0 tag 0 dma 688128 in res 51/40:df:a0:51:9e/00:01:01:00:00/e0 Emask 0x9 (media error) Aug 22 08:45:53 ReadyNAS kernel: ata1.00: status: { DRDY ERR } Aug 22 08:45:53 ReadyNAS kernel: ata1.00: error: { UNC } Aug 22 08:45:53 ReadyNAS kernel: ata1.00: configured for UDMA/133 Aug 22 08:45:53 ReadyNAS kernel: sd 0:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE Aug 22 08:45:53 ReadyNAS kernel: sd 0:0:0:0: [sda] tag#0 Sense Key : Medium Error [current] [descriptor] Aug 22 08:45:53 ReadyNAS kernel: sd 0:0:0:0: [sda] tag#0 Add. Sense: Unrecovered read error - auto reallocate failed Aug 22 08:45:53 ReadyNAS kernel: sd 0:0:0:0: [sda] tag#0 CDB: Read(16) 88 00 00 00 00 00 01 9e 4e 40 00 00 05 40 00 00 Aug 22 08:45:53 ReadyNAS kernel: blk_update_request: I/O error, dev sda, sector 27152800 Aug 22 08:45:53 ReadyNAS kernel: md/raid1:md127: sda3: rescheduling sector 17452544 Aug 22 08:45:53 ReadyNAS kernel: ata1: EH complete
kernel.log is flooded with these errors so much that the section in the logs zip covers a period of time that's only just over 100 minutes. Seeing a flood of messages about I/O errors immediately suggests disk failure as the most likely situation and if it's not that the next most likely would be some other form of hardware failure.
Judging a failed/failing disk is subjective and you have to take different things into account: the SMART count, when the SMART count rose and by how much, any log flooding with messages about the disk, what problems are being seen that may be symptoms of the failure etc.
It's not necessarily the case that the NAS will give an alert that the disk is failing/failed when that is the case, as the point of failure can vary and is often a subjective call. Obviously if the quick self test indicated that the disk has failed then that's very clear, but a disk can pass a quick self test and still be bad.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
As this NAS is literally 3 years old, would it be worth me getting a 6 month Gearhead support engagement to cover the rest of the chassis? (assuming that it effectively provides an extended warranty)
It's not overly clear what the Gearhead contract does.
I'll look at getting a new disk based on this advice
Thanks 🙂
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
@mdgm-ntgr wrote:
kernel.log is flooded with these errors so much that the section in the logs zip covers a period of time that's only just over 100 minutes. Seeing a flood of messages about I/O errors immediately suggests disk failure as the most likely situation and if it's not that the next most likely would be some other form of hardware failure.
I agree. SMART doesn't catch everything, and these errors clearly point to either the disk or (less likely) the SATA interface in the NAS.
The warranty on the NAS is three years (not transferrable) - you might check to see if you are covered.
Running lifeguard on the disk is the best way to test it.
If you have a spare disk (whether on the HCL or not), you can test the slot by
(a) removing all disks, labeling them by slot
(b) insert the spare disk into slot 1 and do a factory install from the boot menu. Check to see that disk is working properly by doing extended file transfers - tens or (better) hundreds of gigabytes.
(c) power down the NAS and move the disk into slot 4. Reboot and retest the transfers. Also download the logs, and look in kernel.log for the I/O errors.
A riskier way is to do the above test with disk 4. If the i/o errors occur in slot 1, then it's the disk. if they happen only in slot 4, then it's the system. If they completely go away, then there's something very weird happening.
But doing the test with disk 4 risks data loss, since you will lose data redundancy on the RAID array. If you have a full backup of the data, then you could of course take the risk.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: RN104 Poor network performance after upgrade to firmware 6.9.3
No warranty 😞
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content