This topic has been marked solved and closed to new posts due to inactivity. We hope you'll join the conversation by posting to an open topic or starting a new one.
I accidently Closed the post so I am reposting this due to my Nightmare-
ReadyNas516, Latest Firmware/software- 6 disk NAS with 5 12TB Seagate
Just Got done replacing 2 drives to get completely rid of my disks that had command timeouts, all was well with no command timeouts or errors whatsoever. Then came a power outage!! After the NAS rebooted, it was unable to mount the drives and showed them in red, it also said it could not mount the drives, and did not know what type of raid this was.
I then rebooted the unit again, and it was able to mount the drives, but now I HAD 15 COMMAND TIMEOUTS on one disk, and none on the rest. It said it mounted home, but the area was not empty.
At this point, I am going to replace the disk with the 15 timeouts (now about 3-4 day later, I have another disk with 1 command timeout.)
The first question, is this somewhat normal or not?
The second question, when I replace the disk, do I have to replace the disk in the same slot, I remove it from? In other words, pull out disk 2 andd replace it in another open spot, say disk 6, or must it return to the bay it came from.
I have done a disk test.
Hopefully I will replace my bad disk with the 15 command timeouts, and and hopefully the disk with 1 command timeout will not increment.
Thanks for all the help in advance-
Jeff
Sandshark-
It is best to re-insert in the same slot, but you don't have to.
The command timeouts are not normal. It is possible you just got some bad drives, but that behavior does raise an alarm about a potential problem with the NAS itself, possibly the power supply.
Netgear does not offer repair service out of warranty, so that's not an option. If it were the power supply, I'd expect that you would see at least some warnings in the log about one or more voltage being out of spec unless it is just at spin-up, before the OS is loaded and can look for such things.
Do you happen to know if the timeouts you had on the other drives were also coincident with a power cycle?
What specific model(s) of Seagate drives do you have installed?
The Drives I have are 12TB Iron Wolf ST12000VNZ008, clueless on the other info.
Getting Ready to change out the 15 Timeout Drive, I am going to power off the nas first,
reboot, then when I am back up check timeouts, and probably at that point change out the drive. The timeouts have stayed the same for about 3 days. However the NAS seems quite slow.
Stephen - I guess the goal is to delete this admin directory?? Will this fix my problem?
Isn't the dir required, or will it build a new one?
Let me try again to explain this.
/home is a linux mount point. The system normally will remount /data/home onto /home. This is done with the same command you used to mount / onto /mnt.
/home itself is supposed to be empty. The NAS software has an error check for that, and in your case the error is triggering.
The only way to make the error go away is to delete all the files and folders in the original /home. Once the system is running, the remount of /data/home means you can't just go into /home to do that. I had you mount / onto /mnt, because /mnt will doesn't include the remounts.
The error is only a concern because of the space taken up by the spurious files and subfolders. It sounds like that is neglible in your case, but it would be good to clean it up.
So the system won't rebuild the folder, and it isn't needed. The real folder is /data/home/admin, and that won't be affected by the deletion in /mnt/home. Deleting /home/admin when the system is running would be a big mistake, as that is the same as /data/home/admin.
I belive it is possible that the admin directory cant be deleted until we delete the hidden dir, and the hidden file-just my best guess.
I don't think so, but I'm not certain what the problem actually is. I am wondering if it's actually a btrfs subvolume (which would be very odd, but conceivable). Let's wait until the resync is finished, and then pick it up again.
Disk 2 has 19 Command Timeouts Disk 4 has 1 Command Timeouts
Are these the same bays that had the timeouts before?
You might try swapping disk 1 and disk 2, and see if the problem moves with the disk, or if it stays with the bay. Maybe label them as you move them, so you can put the disk back later (w/o losing track).
Stephen- I was quite nervous about doing this but I did, and here are the results: (never changed positions before this)
With Disk 2 in #1 Position --------------------------- Disk 1- 19 Command TimeOuTs (No Change) Disk 2- 8 Command TimeOuts (this had been previously 0) Disk 3- 0 Command Timeouts (No Change) Disk 4- 1 Command Timeouts (No Change) Disk 5- 0 Command Timeouts (No Change)
I had seemed to notice from the past, that drive 2 always seemed to have the highest count, however this unit, stays on 24/7 and usually does not power down. So I never noticed drive incrementing. So The Command timeouts stayed with Drive Bay 2. (or caused incrementation)
That Being said-Tell me what you think should be my next step? My best guess- only a guess- Put Disk 2 in the Drive Bay that is open for drive 6?
Should this be done while running? Again this is only my guess. Jeff
For the record, I was only going to have 5 Disks in the Raid
It does sound like bay 2 has issues. Power down and move the drive in bay 2 to bay 6. A problem on one slot can affect others when that slot is populated, but not normally when it's not. So, hopefully, this gets you back stable. It's hard to say if this will propagate to other slots without knowing the root cause, which we will likely never know. But you should probably consider that there may be a new NAS in your future.
The Volume Failed to Mount, all drives red- I will try a reboot, The Reboot worked
and the volume mounted- Command timeouts, remained the same.
Successfully completed diagnostics System Disk 1 has 19 Command Timeouts Disk 4 has 1 Command Timeouts Disk 6 has 8 Command Timeouts Logs 2021-04-04 20:54:43: home.mount: Directory /home to mount over is not empty, mounting anyway.
How do I fix this issue, does this have to do with my share directories, being in the wrong place??
Two of my Directories, Future and Documents are NOT inside of the home Directory-
I thought this would be OK, and I do have access to them. If I were to delete these directories, and then recreate them later inside the Home Direcctory whould this end this
issue of:
2021-04-04 20:54:43: home.mount: Directory /home to mount over is not empty, mounting anyway.
Two of my Directories, Future and Documents are NOT inside of the home Directory-
If I were to delete these directories, and then recreate them later inside the Home Direcctory whould this end this
issue of:
2021-04-04 20:54:43: home.mount: Directory /home to mount over is not empty, mounting anyway.
...
None of my Directories are in Home, they are now all in Data
No. That will not fix the error.
But how are you creating these directories? Are they shares created from the NAS web ui?
Home folders are special, and they are always associated with a NAS user account. Ordinary shares are never put in home, and you shouldn't be creating folders in home manually (or creating your folders manually in /data either). It sounds like you are doing that, instead of creating shares from the Web UI. If so, you shouldn't be doing that.
2021-04-04 20:54:43: home.mount: Directory /home to mount over is not empty, mounting anyway.
To fix this issue you need to first enable ssh on your NAS - which you do in the system->services->settings.
Next you log into the NAS using ssh. You use root as the username and the NAS admin password. Don't log in as admin.
On a Windows 10 PC, you do this by entering
ssh root@nas-ip-address
into the windows seach bar (using the real NAS IP address). The system will ask for confirmation on the NAS certificate the first time you do this, and of course you need to accept. On a Mac, you enter the same command from terminal.
Once logged in, you need to enter the following commands:
mount --bind / /mnt
cd /mnt/home
rm -rf *
cd //
umount /mnt
Be careful when typing these commands, as you can do damage if you type them incorrectly. If you see any errors, let us know what they are before you proceed further. The rm -rf * command is the one that deletes the files in the /home mount point. Note these files are invisible to you - the /home that holds your home folders is actually /data/home, not /home.
The Volume Failed to Mount, all drives red- I will try a reboot, The Reboot worked
and the volume mounted- Command timeouts, remained the same.
Successfully completed diagnostics System Disk 1 has 19 Command Timeouts Disk 4 has 1 Command Timeouts Disk 6 has 8 Command Timeouts
It's good that the reboot worked, but it's odd that the volume didn't mount.
I often stress the need for a backup plan here - and I think it is particularly important for you to maintain good backups. We don't know exactly what's going on, and I think there is some risk of data loss. And of course, keep an eye on the command timeouts.
Thanks for all your help, as well as Sansharks help. At this point I have rebooted the NAS many times, and the NAS has had no problems with Command Timeouts holding steady. It also seems that the drive is mounting. However I do see this:
Successfully completed diagnostics System Disk 1 has 19 Command Timeouts Disk 4 has 1 Command Timeouts Disk 6 has 8 Command Timeouts Logs 2021-04-06 12:38:45: home.mount: Directory /home to mount over is not empty, mounting anyway.
I think that I will wait a day or so and replace the drive with 19 timeouts, with the new one.
as I already set up a return.
I did try the ssh root thing. However I was not sucessful with it.
I was not sure if I was supposed to log out after enabling ssh, I tried it both ways.
The Last time I tried- I enabled ssh while logged in as admin, I then logged out. I then brought up a command prompt in win10. I then typed in
ssh root@192.155.0.27
this is what I got- these are not real ips
Microsoft Windows [Version 10.0.19041.867] (c) 2020 Microsoft Corporation. All rights reserved.
I did not check it this last time, however the first time, that it did allow access it had a message, and I clicked ok. At that point, I got out of it because I was doing something, and was unable to continue. I have left it with no password autentication, (not checked) however this is what I am getting now.
This is from the command prompt (thru win10)
When I turned on the ssh I did this from my admin login.
I had to return the one drive that had 19 Timeouts, so I replaced it, and have started the resyncing process, Thanks again for all your help, I will wait to get this done, and hopefully you can guide me thru the ssh process that you had talked about. I would have waited, till we were done but they wanted the drive returned by tommorrow. Thanks again
In a dos type based system, i could quirey it by typing dir, to see what dir I am in,
and then move to the directory I require. it seems that this is similar.
Yup, it's Linux. You can Google command examples. Where you are should be in the prompt. To see what is contained where you are use ls, or ls -all for more details. Do make sure you are where you think you are (/mnt/home) so you don't delete anything you shouldn't, and make sure you use the forward slash and not the DOS backslash.
You did connect as root, not admin, correct?
It seems you are on the right track, as there should be nothing to delete in home when mounted in that way.
Try rm -rfv *, adding the "verbose" option and see if it gives any more information.
I can see the network on my pc so this is what I see:
If I goto admin folder, there is a hidden folder called .ssh, (I knew the period means hidden)
then inside of that folder there is a file named: ssh_authorized_keys. The file size of that
file is 0KB.
This is seen through the eyes of my network with the admin login, it shows always, the way my system is configured. (this is under my win 10 network area)
I am currently resyncing with ssh off I belive, very slow to even get into the admin of the network interface, so I will wait to finish the resync, before doing this.
I belive it is possible that the admin directory cant be deleted until we delete the hidden dir, and the hidden file-just my best guess.
Stephen - I guess the goal is to delete this admin directory?? Will this fix my problem?
Isn't the dir required, or will it build a new one?
I will follow your instructions as soon as the drive is done resyncing, everything slows to a crawl, just trying to login while resyncing (been looking at the entry page for about 5 minutes, just trying to log in.) so I wait !! (some of my networked drives are mapped as well) I was able to log in but wait, wait, and wait! Trying to see if I turned off ssh curently, not sure at this point.
Stephen - I guess the goal is to delete this admin directory?? Will this fix my problem?
Isn't the dir required, or will it build a new one?
Let me try again to explain this.
/home is a linux mount point. The system normally will remount /data/home onto /home. This is done with the same command you used to mount / onto /mnt.
/home itself is supposed to be empty. The NAS software has an error check for that, and in your case the error is triggering.
The only way to make the error go away is to delete all the files and folders in the original /home. Once the system is running, the remount of /data/home means you can't just go into /home to do that. I had you mount / onto /mnt, because /mnt will doesn't include the remounts.
The error is only a concern because of the space taken up by the spurious files and subfolders. It sounds like that is neglible in your case, but it would be good to clean it up.
So the system won't rebuild the folder, and it isn't needed. The real folder is /data/home/admin, and that won't be affected by the deletion in /mnt/home. Deleting /home/admin when the system is running would be a big mistake, as that is the same as /data/home/admin.
I belive it is possible that the admin directory cant be deleted until we delete the hidden dir, and the hidden file-just my best guess.
I don't think so, but I'm not certain what the problem actually is. I am wondering if it's actually a btrfs subvolume (which would be very odd, but conceivable). Let's wait until the resync is finished, and then pick it up again.
I don't think so, but I'm not certain what the problem actually is. I am wondering if it's actually a btrfs subvolume (which would be very odd, but conceivable). Let's wait until the resync is finished, and then pick it up again.
I think that when he originally went in and enabled SSH that the /data/home folder was not mounted at /home, so the OS created an actual admin folder (or maybe subvolume) in the real /home instead of the mounted one. Since user shares are normally subvolumes, it does make sense that maybe it is a subvolume, and that's what makes rm incapable of deleting it. But he also says there is still a hidden file in the folder that the rm didn't delete, so does the rm with those flags not delete invisible files and that's the real issue? Maybe add the -f option to force removal of the invisible file that maybe has something else keeping it from being deleted?
