and what could that be? As I said, the NAS is on a different IP address so all my network shares point to a different IP.
Might not be linked to network activity at all. There are background processes that kick in from time to time. If you use indexing, that will get updated periodically, and of course AV (if enabled) will run scans from time to time. Snapshot processing, and other btrfs stuff might also be kicking in.
Do we know where the temperature sensor is? Could that just be the NIC IC warming up a bit because it's being used and the temp sensor happens to be nearby?
The one that's spiking appears to be the CPU temp, not the system temp. The CPU sensor is part of the chip. In many
(Sorry not sure how to "break" the quote line...)
The temp spike is 100% linked to me plugging the ethernet cable to the NAS. I've taken notes of when I plug and unplug and the time matches the temperature spikes.
The graph colours are misleading when they overlap. It's the system temp (see attached).
But, clearly, the CPU temp is rising and falling before the system temp, and likely being it's main driver. And as @StephenB noted, there appaers to be NIC activity when it does so. Are you running ant-virus, ReadyCloud, and/or any other app that might be trying (and failing) to "phone home" on a regular basis?
Ignore the rise in CPU temp at 10am. That's me opening the patio door to get some fresh air in and then the heating starting. 🙂
If you check 2pm and 4.30pm temps, there is no CPU temp increase. Maybe a few degrees which I would should be the consequence of running the GUI on my browser.
That, in my opinion, does not explain a 10C+ system temperature rise.
No, I literally have nothing running on this NAS. Only file services.
Attached another example: that was me unplugging the ethernet for like 5 minutes and plugging it back again. CPU temp actually goes down a bit (I closed the GUI) but the system temp goes down 10C in a few minutes and up 10C again in another few minutes.
Again, this might be a red hearring but I find it weird. I will check with my thermal camera to see if there's anything weird happening on the mainboard.
But most importantly, the NAS has been up 3 days and 14 hours since I've disconnected it from the main network. As the NAS was never disappearing when being heavily used, I think this removes the "hardware failure" out of the equation.
Poor cooling could be a part of the equation, though maybe only because a component has become more sensitive to temperature than it should be. I don't remember if one of the heat sinks on the Pro6 motherboard is on the NIC. But there are some "stick-on" ones that can get knocked off when you service the power supply. Any chance that happened? A chip with residual thermal pad for heat sync attachment but missing the sink is going to heat faster than just a bare one, and much more than one with the sink.
When you hooked the NAS up differently, did you also move it to where it might cool differently? One thing some ignore is that there needs to be sufficient space behind the unit to reduce back pressure on the fan and keep heat from building up locally.
What you are experiencing looks mostly like hysteresis in the fan profile. Especially on the "quiet" setting, the profile allows the fan speed to stay low and the temperature to rise until a threshold is reached. Then, it boosts the fan speed to cool it, but does so such that it drops lower than the point at which it would be increased again, so as not to constantly go back and forth between the settings. And if your replacement for the PSU power supply has a lower CFM ratting than the original, that could affect how well the standard profile controls temp since the NAS can't control that fan.
I don't remember exactly how my Pro6 units ran WRT that, and I know I always used the "Balanced" setting and all had the stock fans or an exact replacement. I really wish Netgear would expose the fan control parameters more in the GUI. Especially on legacy units that have a CPU fan (newer units don't have one), I find that the temperature can be allowed to get rather high before the speed is increased, causing the speed boost increase to be a larger step than would have been necessary had it done so earlier. The configuration for newer units often contain a hysteresis setting, where legacy units don't. But you can only edit them via SSH and they get replaced at every OS update if you do. The segment below is from the RN3220.conf profile. Granted a rack-mount unit, but I have added a similar statement to my legacy RN4200's profile and the result calmed the speed switching down when it starts to reach the max temperature, though that is always driven by the CPU temp on mine. Of course, the 4200 is also a rack-mount unit, so behavior may differ on a desktop model.
label temp1 "System"
set temp1_max 80
set temp1_max_hyst 72
Try changing the cooling profile (Cool/Balanced/Quiet) and see if it changes the behavior more to your liking. If that doesn't work, try adding a hysteresis statement to the configuration file. I don't know if the Netgear fan control looks at the difference between the max and max_hyst and uses that hysteresis all the time or if it just affects when it approaches max. The profile for the Pro6. is /etc/frontview/sensors/NV6.conf. Or you can replace the symbolic link at ./etc/sensors.d/system.conf with a modified copy of the profile itself and not touch the default one.
The NAS normally lives in a cabinet which doesn't have a door. Because the back of the NAS was getting a bit on the warm side, I installed a fan at the back of the cabinet which blows air OUT of the cabinet itself. Right now it's sitting on the floor, so definitely better airflow but I feel it's never been a problem anyways.
NAS is normally on Balanced. When I plug/unplug the ethernet, I don't hear the fans spinning down or anything changing. That said, I noticed the PSU fan is blowing inwards which doesn't make much sense but I must have checked when I replaced it. Anyways, the PSU airflow is independent from the NAS. The air is sucked in at the back and blown out at the bottom of the NAS. There is a very small vent on a side inside the NAS but I doubt that could make a big difference. That said, I replaced that fan months ago.
Heatsink: anything is possible but I'd say no. I can check though. Still, I'd be curious to know where the System temperature sensor is.
I can try playing with the fan controls but I honestly don't think it's a fan issue. I've carefully listened and "felt" the fans when I unplugged the RJ45 (as I had the same idea, fans slowing down) but I didn't feel anything. The temp is going up 10C in a few minutes so I'd expect the fans to change their speeds by a lot to justify that change in temperature. I'll measure the temperature of the exhausted air to see if there is any change when I plug/unplug the RJ45.
In the end, the NAS is staying up even when connected to my main desktop (still no internet). That is good news. This whole temperature thing is just a bell ringing for attention, it might be a bug or something unrelated or totally normal. Who knows.
I got the thermal camera out. And indeed there is an IC under a heatsink which immediately warms up as soon as the RJ45 is plugged. I can see it warms up some of the adjacent PCB so I guess that's the cause of our temp spike.
However.
This only happens when the bottom ethernet port is used - which I don't normally use. If I plug the network cable in the top one, the IC does not warm up.
We could conclude that that ethernet port is faulty but... it works! I have a feeling this is normal behaviour to be honest.
But to be on the safe side I'll configure my usual ethernet port and use that. After all when the box "crashes" the first thing that happens is that it goes offline. A faulty network IC could the the cause of all of this.
I think I see the other heat sink in your photo -- it's in the rear, left corner of the NAS (upper left in your photo). And I suspect you'll find that one heats up when you use the other port. That other one is the one that's easy to knock loose when servicing the power supply, so make sure it seems to be solidly attached.
I really don't see anything warming up when I plug the other port, I did look for that. But for now I shall assume that what I see is normal and that somehow the NIC controller of the bottom port is closer to where the temperature sensor of the mainboard is.
I've switched to the upper port, which is the one I normally use. Let's see if the NAS stays online. If it does, I'll move to the next step: link my shared folder to the new IP and use it - just not on the main network.
Not sure how much the community is interested to hear an update but the NAS has been up for 6 days now with no issues. Next step is to set my Desktop to use the NAS as previously but on the second NIC I have on my system - hence the NAS is only reachable by one computer.
This is weird though. Clearly not an HW issue but I really don't have anything running on that box so it shouldn't make any difference if it's online or not. We shall see.
This is weird though. Clearly not an HW issue but I really don't have anything running on that box so it shouldn't make any difference if it's online or not. We shall see.
Do you have any ports forwarded to the NAS in the router? Maybe if uPNP is enabled in the router, you could try disabling it.
I do, as I said I have FTP forwarded from the internet and my ISP gives me a fixed IP address. The port is not the standard one so it should be safe but you never know.
I need to educate myself on uPNP, I know it's there but I don't know what it does. What are your suspects on uPNP?
I do, as I said I have FTP forwarded from the internet and my ISP gives me a fixed IP address. The port is not the standard one so it should be safe but you never know.
Maybe try connecting the NAS tn the full network again, after you disable the port forwarding, and disable uPNP if it is enabled.
Something seems to be causing the NAS to crash, and it would be good to know if it is a device connected to your network, or something outside.
I need to educate myself on uPNP, I know it's there but I don't know what it does. What are your suspects on uPNP?
uPNP (universal plug and play) allows any device on your network to ask the router to open a port. Convenient in some ways, but also dangerous. Best to disable it if you don't really need it.
Sure, if the NAS works now it's back on my small network, actively used as a NAS by my desktop only, the next step would be to put it back where it was but with no internet access whatsoever, including external access. Good to know about uPNP, which I can disable.
Another update: NAS is still up (10 days now) and been on my desktop network (isolated by the main network) for 4 days now. It works.
The next step is to put it back where it was, but with port forwarding disabled.
Good progress though. From "you need a new one" to "it's been up 10 days" 🙂 These things are still pretty expensive!
I was looking into TrueNAS and I was told the 8GB requirements has been recently moved to 16GB - which feels silly for a NAS. Even 8GB feels quite a lot if you only intend to use it as a file server.
Well, the NAS is back where it was, main network with internet access. Only difference is that I disabled the port forwarding and Universal PnP.
It works. It's been online for 18 days in total - including the initial tests off network.
If this works, that's great! But if it does, it might indicate a firmware issue. I wouldn't expect a NAS to crash because it's being accessed from the internet. I appreciate Netgear does not seem to be very busy developing for the ReadyNAS (and my NAS is unsupported) but is there a way to forward this to their engineers? I have a feeling this might not be just my problem, it's just that not many people have port forwarding on their network I suppose 🙂
Now, I can VPN on my network but every now and then I used the FTP to give people access to large files stored on my NAS. Is there a safe alternative than opening ports on my router?
I'll update again in a while. For now, thanks to everybody for helping me.
If this works, that's great! But if it does, it might indicate a firmware issue. I wouldn't expect a NAS to crash because it's being accessed from the internet.
Obviously it shouldn't crash, but hacking attempts could result in crashes.
Well, the NAS is back where it was, main network with internet access. Only difference is that I disabled the port forwarding and Universal PnP.
uPNP is pretty dangerous, because if any device on your network is compromised, it can tell your router to forward traffic to any other device - making it much easier for your entire network to become compromised. I don't know why vendors put it their router firmware to begin with.
Now, I can VPN on my network but every now and then I used the FTP to give people access to large files stored on my NAS. Is there a safe alternative than opening ports on my router?
A safer method would be to use a cloud repository for what you want to share, though if the files are large that wouldn't be free. Then copy things that you want to share into the cloud repository.
Not sure how you are set up now. But if you want to use FTP, you can set up the NAS to require FTPS ("Enable Force FTPS" in system->settings->services->FTP). Limit the passive port range (the NAS default is far too big). Generally I've found that 4 passive ports per simultaneous connections is enough. So if you are sharing with a couple of people you'd need to forward only 5 ports total. Best to use dynamic/private ports (any set of five in the range 49152-65535). If you are sharing specific files, then don't leave the ports forwarded all the time.
That will somewhat complicate the sharing - you'd need to create a user account/password for shared access, and change the password from time to time. And you'd need to provide more instructions on how to set up their FTP client. For instance giving them
the non-standard connection port.
the username/password
telling them to use "require explicit FTP over TLS" (FileZilla) or ""TLS/SSL explicit encryption" (WinSCP).
Well from my point of view an OS should be solid enough to be able to withstand a hack attempt without crashing.
I'll keep Upnp disabled, I read online it was safe nowadays but I see your point.
Online services for sharing are good but that implies me sending the file there and the other person downloading it. It takes more time. Also, I might not feel comfortable using those services for certain more sensitive data.
I don't have force SFTP currently so that's a start. I only have 5 passive ports enabled. And yes, I've always given those info to those who needed to access my NAS.
I'm not sure I want to test this now, given the headache I've been through! 🙂 Thanks for your input though.
Ok, I made the cable. I plugged it into the NAS - it had disappeared again - and I had a black screen, but the monitor reported a signal, 640x480@67hz.
Nothing would show up so at some point I issued CTRL-ALT-DEL twice and the box rebooted within seconds.
The BIOS appeared on screen so I know the adaptor is correct. How can I display a terminal on screen when it's loaded? The monitor should be able to handle weird resolutions.
The CTRL-ALT-DEL trick per se is a life changer at least I don't have to power cycle the box - even though it rebooted very quickly but it cannot be worse than a power switch. That also tells me - correct me if I am mistaken - that the core of the OS was still working? Had it been fully frozen it wouldn't have responded to the keyboard.
You saw no messages during boot at all? AFAIK, you have to have SSH enabled to get a terminal once the OS is fully booted, but you should see some messages from the boot process. The BIOS will respond to the keyboard, but I don't know if CTL-ALT-DEL is something it understands. But I would expect it to display some sort of error message if it can't see the boot code on the flash.
I see the BIOS and the early stages of boot process of SYSLINUX 3.31.
Then black screen - black screen, NOT no signal.
SSH is enabled.
Well, a modern OS should initiate a software reboot if CTRL-ALT-DEL is issues. I wouldn't expect the BIOS to take over and forcefully reboot the system. But this is an old system and also non-standard. Still, a forced reboot is better than powe cycling the whole thing!
The NAS keeps crashing as a couple of months ago...
This time no response from CTRL-ALT-DEL. In fact, plugging the keyboard does not light up any of the LEDs and CAPS-LOCK does not work. It did work earlier on when the NAS was gone from the network.
I power cycled and got into the BIOS, not sure why. There I found a clue. Only 4HDD were being detected! The first one - the oldest - was not.
I saved and exit, got in the BIOS again and all 5 drives were there.
I'm wondering whether that old HDD is dying? There is no evidence of that though. SMART is clean. But it could be one of those weird faults the system cannot handle and just crashes. It wouldn't be the first time I see that.
Ideas? I have a feeling that if the drive is the culprit, it would pass any tests I throw at it until it stops working altogether.
