- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Re: Readynas 424 SSH service not starting (no error)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Readynas 424 SSH service not starting (no error)
Hi all, I updated from 6.10.3 -> 6.10.4 today, against my better judgement (I've had issues with updating firmware in the past) and again, I'm having issues, the firmware update screwed with mono, so I had to SSH in to fix it, I got it working and rebooted my NAS, unfortunately more problems arose, at which point I had no SSH access, whenever I try to enable it under "services" it just doesn't do anything, it doesn't turn on, there's no message box, no logs, nothing.
I tried an OS reinstall using the boot option, to no avail, I have a feeling it's something to do with OS storage being full but I've no idea how to clear it, I could telnet in using the Netgear tech support boot option but didn't really know where to go from there
Any help would be greatly appreciated, thank you.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Are you able to download the log zip file from the web ui?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hi, yeah, I am, I managed to download the log file, is there somewhere I should upload it? From what I can tell it thinks it's starting fine, and when I try to enable something else it realises it's actually disabled and says so in the log, and as I found out last night, it looks like all services have this issue, not just SSH.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Killerjerick wrote:
Hi, yeah, I am, I managed to download the log file, is there somewhere I should upload it?
You don't want to post the full log zip publicly. You can request that the mods review it for you though. You upload it to a cloud service (dropbox, google drive, etc), and then send them ( @JohnCM_S or @Marc_V ) a private message (PM) asking them to check it. Includes a download link to the log zip, and a link to this forum thread.
You send a PM using the envelope icon in the upper right of the forum page.
@Killerjerick wrote:
I have a feeling it's something to do with OS storage being full
Look in volume.log (in the log zip). Scroll down, and you will find a section like this:
=== df -h === Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 703M 3.0G 19% / tmpfs 1.9G 0 1.9G 0% /dev/shm ...
The /dev/md0 line tells you how full the OS partition is. Generally it is less than 25% full.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hi Stephen, I attempted to contact JohnCM, but haven't heard back, can only assume he's busy. This is what I see in the volume.log
=== df -h === Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 2.6G 1.2G 70% / tmpfs 980M 20K 980M 1% /dev/shm tmpfs 980M 3.8M 976M 1% /run tmpfs 490M 1.4M 489M 1% /run/lock tmpfs 980M 0 980M 0% /sys/fs/cgroup
I'm guessing that I was incorrect, doesn't look like the root dir is full, although it is more full than it should be. Is there anywhere else I can poke around to see if I can find something that doesn't add up?
Thank you 🙂
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
You could look in system.log and kernel.log for errors around the time you tried starting ssh from system->settings.
Also, systemctl-status.log should show that ssh.service is running (or not). Systemctl-journal.log is probably also worth a look.
BTW, if you use the @ (e.g., @JohnCM_S ), then forum user (including mods) should get a notification of some kind. That might help get attention.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hey @StephenB It's been over a month and no response from either mod, I assume they are both too busy right now, what's my next course of action? Ideally I'd love to not shell out the $$ for netgear support, but is there anything else I can do from my end? I've checked both of the logs you suggested and nothing jumps out at me, but my experience in linux in particular is lacking, I've checked and I seem to be able to install apps from the netgear "upload" option, but if my problem stems from an incorrect install from a particular app, I'm not able to uninstall it, the only thing that I did around the time of the issues was attempt to install a full version of FFMPEG, which is why in my case I assumed I had accidentally installed it to the OS partition, but that seems to not be the case, judging from log files.
I'm hesitant to attempt to install ffmpeg again using the upload function as I'm positive that'll mess something up.
Thanks in advance.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
I am thinking browser cache to be honest. Did you try and clear all browser history and cache and then try to enable SSH from the Web UI again after that? The browser cache often messes with me after updates to newer firmwares.
This issue sounds like something I have seen a bunch of times, TBH.
You can go in over Tech-Support mode and re-enable SSH if all comes to all but start with attacking the browser cache and let us know (even try different browsers as well). If you hit a dead-end let me know I will throw you some commands for Tech Support mode.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hey @rn_enthusiast I indded did try that, including using Edge, Chrome, Firefox and my phone with no luck, thanks anyway.
Those commands would be wonderful, I successfully telnetted into the nas with the netgear tech support mode, but I didn't know where to go from there, and stopped myself short of causing irreversible damage. My limited knowledge has gotten me into a pickle once before but I managed to solve that myself (just linux permission based stuff).
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hi again
The below is at your own risk. Making mistakes can have consequences, as you know yourself 🙂 If you see errors or unexpected behaviour then stop and think before proceeding. This should be the steps to enable SSH, if my memory serves me well.
Once you are in over tech support mode, you can start the OS raid manually:
mdadm --assemble /dev/md0 /dev/sd[a-z]1
You should see md0 running now:
cat /proc/mdstat
Hereafter, mount the OS volume:
mount /dev/md0 /sysroot
You should see it mounted to /sysroot. You can check with:
mount
Now, use "vi" to change the default services file in order to enable SSH.
vi /sysroot/etc/default/services
Find these 5 lines in the file and change them to the same values as per below.
SSH=1 SSHPORT=22 SSH_UI_ENABLE=1 REMOTE_ACCESS_SSH_PORT=0 SSH_PASSWORD_AUTHENTICATION=1
Save the changes and exit the "vi" editor.
Un-mount the OS partition
umount /sysroot
Stop the OS raid:
mdadm --stop /dev/md*
Check that the raid is stopped:
cat /proc/mdstat
Flush memory to disk and reboot the NAS:
sync
reboot -fn
Cheers
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Killerjerick wrote:the only thing that I did around the time of the issues was attempt to install a full version of FFMPEG, which is why in my case I assumed I had accidentally installed it to the OS partition, but that seems to not be the case, judging from log files.
Are you saying you didn't successfully install FFMPEG at all, or that you think you didn't install it to the OS partition. Because if you did successfully install it via SSH and a .deb package, it most definitely would have installed to the OS partition. While that alone shouldn't fill your OS partition, it could be pointing to it as workspace, which could become problematic if you've actually used it.
But another possibility is that this installation's dependancies installed an update to a package already in the NAS with one that's incompatible with the Netgear portion or over-wrote a configuration file that it needs. While an OS re-install will, I believe, restore most Debian packages that are a part of the OS to their proper version, it doesn't over-write all configuration files because it doesn't want to mess with your properly installed apps.
So while you are in support mode, it would be good to still check that the OS partition isn't too full.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
As well as editing the default services file you may also need to create a symlink whilst the OS partition is mounted (if it doesn't already exist):
ln -s /lib/systemd/system/ssh.service /sysroot/etc/systemd/system/multi-user.target.wants/ssh.service
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
You might also need to chroot in order to get the commands you want to use.
Instead of starting with the mdadm-assemble command suggested by @rn_enthusiast you could start with
# rnutil chroot
That should start RAID, mount the OS partition, and chroot in one step.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Going into the chroot is not needed for rn_enthusiast's commands or the one I mentioned. You can do it if you want, once you chroot into the sysroot you would remove references to sysroot in any of the commands until you exit the chroot again.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
You are changing a config file. No need for chroot to do that 🙂 It would just complicate matters as you need to exit out of it again, etc.
Also, the rnutil chroot command will start the data raid. There isn't really a need for that.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Hi @rn_enthusiast thanks again for all your help, unfortunately it looks like my problem is worse than originally thought, your first command fails every time with the error
"mdadm: No super block found on /dev/sdc1 (expected magic a92b4efc, got 07020701)
mdadm: no RAID superblock on /dev/sdc1
mdadm: /dev/sdc1 has no superblock - assembly aborted"
is there something else I should be doing? Sorry for the late reply, I can only really do this after college between 9-11pm AEST (Timezones suck)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Sandshark wrote:
@Killerjerick wrote:the only thing that I did around the time of the issues was attempt to install a full version of FFMPEG, which is why in my case I assumed I had accidentally installed it to the OS partition, but that seems to not be the case, judging from log files.
Are you saying you didn't successfully install FFMPEG at all, or that you think you didn't install it to the OS partition. Because if you did successfully install it via SSH and a .deb package, it most definitely would have installed to the OS partition. While that alone shouldn't fill your OS partition, it could be pointing to it as workspace, which could become problematic if you've actually used it.
I didn't successfully install FFMPEG at all, although I can't exactly check that without SSH (I believe) I'll definitely check the disk space if I manage to successfully mount the drive.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Have you made a backup of the NAS?
It might be more direct to just do a factory default and start over.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
Unfortunately, there's no feasible way to backup 12tb of collected media, is there no way to just reinstall the os while keeping the data?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
From TS mode, what does this give you?
cat /proc/partitions
And what happens if you run:
rnutil chroot
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
various partitions including
sda
sda1
sda2
sda3
sdb
sdb1
sdb2
sdb3
sdc
sdc1
md0
md1
md127
rnutil chroot seems to throw me into "root@<serialnumber>:/#"
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Killerjerick wrote:
Unfortunately, there's no feasible way to backup 12tb of collected media...
There is, and I do back up my own NAS (which has a similar amount of data).
I back up to other ReadyNAS, but you can purchase a 16 TB USB drive for ~$300 USD. For instance, https://www.bestbuy.com/site/wd-easystore-16tb-external-usb-3-0-hard-drive-black/6427998.p?skuId=642...
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Killerjerick wrote:
rnutil chroot seems to throw me into "root@<serialnumber>:/#"
I think @rn_enthusiast is wondering if that command mounted md0 (the OS partition).
You can easily check that by running mount.
If it is mounted, then cat /proc/mdstat would give you more information on what is going on with sdc.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
@Killerjerick wrote:Hi @rn_enthusiast thanks again for all your help, unfortunately it looks like my problem is worse than originally thought, your first command fails every time
This is why givng instructions on how to deal with problems like this can be problematic. There are a bunch of other commands that we would nearly always run when checking a system and we get used to seeing problems like this.
No superblock is bad, but the 4GB root volume uses raid-1 so one disk isn't too bad. It does warrant further investigation to get an idea as to whether there's a problem with that disk. For starters
smartctl --all /dev/sdc
Would tell you the SMART stats from that disk.
You can also examine the raid device underneath the 4GB root volume using e.g.
for i in /dev/sd[a-z]1; do echo $i; mdadm -E $i | egrep "Creat|Update|State|Raid"; done;
or
mdadm --examine /dev/sd[a-z]1 | egrep -i 'sd|event|uuid|recovery|super|role|update'
@Killerjerick wrote:
Sorry for the late reply, I can only really do this after college between 9-11pm AEST (Timezones suck)
Different time zones is expected with an international community.
@Killerjerick wrote:rnutil chroot seems to throw me into "root@<serialnumber>:/#"
That's what you expect to see when you chroot in. If you are in the chroot instead of
vi /sysroot/etc/default/services
You would do e.g.
vi /etc/default/services
You should still look for the lines @rn_enthusiast mentioned, bearing in mind that as above some of the commands are slightly different when you are inside the chroot.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 424 SSH service not starting (no error)
smartctl --all /dev/sdc
throws
/dev/sdc: Unknown USB bridge [0x13fe:0x5200 (0x100)] Please specify device type with the -d option. Use smartctl -h to get a usage summary
while
for i in /dev/sd[a-z]1; do echo $i; mdadm -E $i | egrep "Creat|Update|State|Raid"; done;
throws
Creation Time : Mon Sep 23 18:19:13 2019 Raid Level : raid1 Raid Devices : 2 State : clean Update Time : Tue Mar 2 20:56:40 2021 Array State : AA ('A' == active, '.' == missing, 'R' == replacing) /dev/sdb1 Creation Time : Mon Sep 23 18:19:13 2019 Raid Level : raid1 Raid Devices : 2 State : clean Update Time : Tue Mar 2 20:56:40 2021 Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
Thank you for bearing with me thus far 🙂