I have a ReadyNAS 214 4 Bay Unit and running Version 6.10.3.
On September 3, I was not able to access the unit from my browser. When I checked the unit, the lights were blinking and there was no obvious issues. My plan was to power down the unit gracefully and then power it up. However, when I pressed, and held, the power button, I was not prompted by the unit, asking if I wanted to power down. I continued to hold down the power button without any success. The only think I could do was to pull the plug and luckily the unit came back online and everything was working as usual.
I looked in the log and found these entries before I pulled the plug:
Sep 03, 2020 04:59:38 PM System: ReadyNASOS background service started. Sep 03, 2020 04:59:35 PM System: ReadyNASOS service or process (enclosure_monit) was restarted.
Sep 02, 2020 04:52:49 PM System: ReadyNASOS background service started. Sep 02, 2020 04:52:46 PM System: ReadyNASOS service or process (enclosure_monit) was restarted.
Since September 3, there have been four more occasions where the unit locks up and I have had to pull the plug to get it back online. The latest being November 16. In looking at the entries in the log, these are the types of entries I am seeing:
Oct 06, 2020 09:29:44 AM System: ReadyNASOS background service started. Oct 06, 2020 09:29:42 AM System: ReadyNASOS service or process (enclosure_monit) was restarted.
Oct 05, 2020 09:45:04 AM System: ReadyNASOS background service started. Oct 05, 2020 09:44:55 AM System: ReadyNASOS service or process (readynasd,continuous_snap) was restarted.
Oct 03, 2020 09:44:08 AM System: ReadyNASOS background service started. Oct 03, 2020 09:43:59 AM System: ReadyNASOS service or process (readynasd,enclosure_monit) was restarted
Sep 29, 2020 09:35:06 AM System: ReadyNASOS background service started. Sep 29, 2020 09:35:04 AM System: ReadyNASOS service or process (enclosure_monit) was restarted.
Sep 28, 2020 09:07:10 AM System: ReadyNASOS background service started. Sep 28, 2020 09:07:06 AM System: ReadyNASOS service or process (sysadm) was restarted.
Sep 24, 2020 10:55:43 AM System: ReadyNASOS background service started. Sep 24, 2020 10:55:42 AM System: ReadyNASOS service or process (enclosure_monit) was restarted.
I am not seeing anything special about the time of these notices, as they have occured at all times.
Like I said, I have had this unit for years and have not seen these errors before starting September 2/3.
Thanks very much for the response. Per your note, I downloaded the logs and received the zip file. When I opened the System.log file, it only had 535 bytes of information which was several lines from today only, i.e., less than one day and nothing else. My UI says I have over 3800 entries in the log. Is there something special I need to do to get the complete log downloaded? Also, the Kernal.log file was empty. I can upload the zip file, or other individual files, if you think that would help.
It looks like those logs have already rotated, so they're no longer on the NAS. You could perhaps look in disk-info.log, and make sure the SMART stats for your disks are ok.
I suggest capturing the logs again - either the next time you forcibly reboot, or perhaps if you see the restart messages in the log.
Don't post the full logs here - it's ok to cut/paste part of them them, but there is some privacy leakage if you post them all.
If the log really started on 11/22, where is the data?
I looned at the disk-info.log and everything looked good as does the UI dashboard.
Per your note, I will download the log files the next time I see an issue but given what I see above, it looks like the system.log is really not capturing the information as indicated in the first entry.
Today I discovered my NAS had locked up again, i.e., when I tried to access it from my browser, no response. Also, pushing the power button did not prompt me asking if I wanted to shut down, so I had to pull the plug again, and luckily, it came back up, all fine. The time of the power down and restart is about 09:54:31, I believe.
When the system came back up and I was able to access my browser, I checked the logs and saw that my sync job that runs twice a week to a remove Netgear NAS, worked fine and there were these entries in the log:
Nov 26, 2020 09:57:10 AM
System: ReadyNASOS background service started.
Nov 26, 2020 09:57:08 AM
System: ReadyNASOS service or process (readynasd,snapshot_monito) was restarted.
Nov 24, 2020 05:50:08 AM
System: Antivirus scanner definition file was updated to 59.25997.
Nov 23, 2020 05:59:56 AM
System: ReadyNASOS background service started.
Nov 23, 2020 05:59:34 AM
System: Antivirus scanner definition file update failed due to download failure. Check your Internet connection.
Nov 23, 2020 05:59:55 AM
System: ReadyNASOS service or process (sysadm) was restarted.
You can see that the system was restarted at 05:59, for some reason and there were issues with the anti-virus file but a new virus file was downloaded.
Per you direction, I was able to download all of the log files and looked in the system.log. Luckily there was enough entries in the log that started at about 08:27 and continued to 10:06 and the reboot was 09:56:40, see below
Nov 26 09:54:31 MajorTom connmand[2005]: ntp: adjust (slew): +0.001440 sec -- Reboot -- Nov 26 09:56:40 MajorTom mdadm[1978]: NewArray event detected on md device /dev/md0
In looking at the entries prior to the reboot, I am seeing what appears to be attempted breaches of the device. I am seeing several IP addresses I don't recognize, For example:
Nov 26 08:27:23 MajorTom sshd[29754]: Failed password for root from 112.85.42.183 port 45794 ssh2
Nov 26 08:27:27 MajorTom sshd[29760]: Failed password for invalid user adm from 31.184.199.114 port 64548 ssh2
Nov 26 08:27:33 MajorTom sshd[29764]: reverse mapping checking getaddrinfo for 89-105-113-254.fluidata.co.uk [89.105.113.254] failed - POSSIBLE BREAK-IN ATTEMPT!
The log says that the NAS anti-virus file was updated at 05:50 and my NAS is behind my Google WiFi router.
The examples above are just a sampling of the IP addresses I don't recognize, i.e., there are others. Not sure if the system is locking up due to the number of attempts, or not.
All that said, not sure if this points us to the root cause, or not, but it is very disturbing to me. If these are attempted break-ins, is there a way in the NAS to only accept IP requests from one place, i.e., my remote identical NAS where I exchange information? Are there other safeguards I can put in place?
The system file is about 570K so can provide that, if it would help. I can also send to you privately, if that is preferred. I can also provide an extract, if that would be better.
Thanks very much for your thoughts and Happy Thanksgiving!
Thanks for the additional information. Following your guidance, I checked with Google Wi-Fi and there is no DMZ in my router. I also began looking at the information in the system.log file and saw what appeared to be many attempted breaches using port 22, i.e., I saw several IP addresses that were concening.
As I indicated earlier, I have an identical ReadyNAS unit at another location and was syncing the information between the two units, i.e., off-site storage. I was using rsync with SSH, which I believe was part of the issue, with port 22 open.
For the moment, I have both units shut down and am copying the information from each unit to an external drive. I will be doing a factory reset on each box to make sure that there is nothing on the box that may have been installed by someone unknown to me.
Once the factory reset is done, I want to recreate my shares, reload the files on each unit, and then set up a process to sync the two boxes in a secure way. I want to lock down both units so that my local box will only honor the IP address of my remote box and my remote box will only honor the IP address of my local box. I found this help article on Netgear which I think is what I want to do:
Personally, I believe a VPN is the best way to accomplish this. My VPN of choice is ZeroTier, which must be loaded on each device that will access the VPN, and there are other options. Many routers support OpenVPN, which is a good choice if you want all devices on one end to be able to access all on the other.
First, I agree with @Sandshark that using a VPN is the best way. It would prevent those hacking attempts from reaching the NAS.
That said, you are already using rsync over ssh, which is encrypted/secure. The hacking attempts were using ordinary ssh (not rsync), and they were able to reach the NAS because you had port 22 forwarded in your router. If you stick with rsync-over-ssh then I'd suggest using a non-standard port instead of port 22. You could then forward that non-standard port to port 22 in the appropriate router. That won't guarantee there will be no more hacking attempts, but it will lower the odds. And of course you will want to use a strong admin password on the NAS.
One other thing you could do is to disable password authentication in the ssh settings - that would prevent any login attempts using ssh. However, you would then need to enable it in the web ui before you could access the NAS using ssh locally.
I want to lock down both units so that my local box will only honor the IP address of my remote box and my remote box will only honor the IP address of my local box. I found this help article on Netgear which I think is what I want to do:
The destination NAS will be seeing the WAN IP address of your source NAS's router - and that will change whenever your ISP decides to refresh your router's WAN address. So this idea won't work with your setup. Plus it isn't needed. You already are using an ssh key to manage access. Your hacking attempts were not trying to penetrate rsync - they were just taking advantage of the open ssh port.
Thanks Stephen and Sandshark for your great input. I am still reviewing and exploring ZeroTier as an option.
You mention continuing to use the rsync ssh but use a port other than 22. Do you have any thoughts/recommendations on what would be a good one to use? Then in my router, I would set up forwarding from External Port xxx to Internal Port 22. Is that correct?
