Re: Service management offline

Naveenkumar001 · ‎2025-02-19

hi ..Please help me on this am getting Service management is offine .unable to access my admin page .but i able to download logs.

Am using Readynas 3312. I have data in this .I need that data too..what troubleshoot procedures ii can try. kindly suggest ..spend ur valuable time.

StephenB · ‎2025-02-21

@Naveenkumar001 wrote:
>>> Is SSH enabled on the NAS?

Yes

I suggest accessing the NAS with ssh, and then check that your data is there (ls -als /data).

The logs show that the volume is mounted, but I am seeing unexplained BTRFS timeouts:

Feb 20 10:53:40 DHTL-TP1-NAS kernel: INFO: task btrfs-transacti:4250 blocked for more than 200 seconds.
Feb 20 10:53:40 DHTL-TP1-NAS kernel:       Tainted: P           O    4.1.30.x86_64.1 #1
Feb 20 10:53:40 DHTL-TP1-NAS kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 20 10:53:40 DHTL-TP1-NAS kernel: btrfs-transacti D ffff88021321bd68     0  4250      2 0x00000000
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffff88021321bd68 ffffffff88e2c4a0 ffff880265f08fc0 ffff88026fc147c0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffffffff88e2c8d8 ffff88026fc147c0 ffff88021321c000 ffff880265d8f1f0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffff880265d8f1f0 0000000000000000 ffff88026644b370 ffff88021321bd88
Feb 20 10:53:40 DHTL-TP1-NAS kernel: Call Trace:
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88a4f1e2>] schedule+0x32/0x80
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e62df>] wait_current_trans.isra.34+0x9f/0x100
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff8809c8c0>] ? add_wait_queue+0x60/0x60
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e7e58>] start_transaction+0x2c8/0x550
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e8142>] btrfs_attach_transaction+0x12/0x20
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e4d0b>] transaction_kthread+0x7b/0x240
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e4c90>] ? open_ctree+0x24b0/0x24b0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082bb4>] kthread+0xc4/0xe0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082af0>] ? kthread_freezable_should_stop+0x60/0x60
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88a523d2>] ret_from_fork+0x42/0x70
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082af0>] ? kthread_freezable_should_stop+0x60/0x60

On SMB access, I am seeing a lot of errors that look like this:

[2025/02/20 16:12:33.035542,  2] ../source3/auth/auth.c:315(auth_check_ntlm_password)
  check_ntlm_password:  Authentication for user [IMW] -> [IMW] FAILED with error NT_STATUS_NO_SUCH_USER

Is IMW an account that should exist on the NAS?

You are running very old firmware (6.6.0 - released in 2016). You appeared to make an attempt to update to 6.10.10 on 19 Feb

Feb 19 2025 17:04:49 INFO:UPLOAD_IMAGE: _filename_=ReadyNASOS-6.10.10-x86_64.zip _filesize_=90385981
Feb 19 2025 17:04:50 INFO:GET_REMOTE_INFO: name=ReadyNASOS version=6.10.10 size=90783744 md5=b2f2ee74b9bbf269d449490b24b370fe arch=x86_64
Feb 19 2025 17:04:50 INFO: arch=x86_64 personality= sysinfo->personality=1
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: Checking file for md5... 
Feb 19 2025 17:04:50 INFO:CHECK_MD5_SUM: local=b2f2ee74b9bbf269d449490b24b370fe remote=b2f2ee74b9bbf269d449490b24b370fe
Feb 19 2025 17:04:50 INFO:CHECK_MD5_SUM: MD5 OK! Create link in /etc/.flash_update -> .flash_update_file
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: Write info to .flash_update_info... 
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: SUCCESS

After that you appear to have downgraded back to 6.6.0

Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: _filename_=ReadyNASOS-6.6.0-x86_64.img _filesize_=63207424
Feb 19 2025 17:29:10 INFO:GET_REMOTE_INFO: name=ReadyNASOS version=6.6.0 size=63207424 md5=aec396425a099a22e0fc4f43bbe6ac69 arch=x86_64
Feb 19 2025 17:29:10 INFO: arch=x86_64 personality= sysinfo->personality=1
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: Checking file for md5... 
Feb 19 2025 17:29:10 INFO:CHECK_MD5_SUM: local=aec396425a099a22e0fc4f43bbe6ac69 remote=aec396425a099a22e0fc4f43bbe6ac69
Feb 19 2025 17:29:10 INFO:CHECK_MD5_SUM: MD5 OK! Create link in /etc/.flash_update -> .flash_update_file
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: Write info to .flash_update_info... 
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: SUCCESS

Can you give more details on what you were doing here?

View solution in original post

StephenB · ‎2025-02-19

@Naveenkumar001 wrote:

hi ..Please help me on this am getting Service management is offine .unable to access my admin page .but i able to download logs.

Did you try running the diagnostic from RAIDar?

Are you seeing any clues in the log zip?

Naveenkumar001 · ‎2025-02-19

On diagnostic. it's said some sector count in HDD 5. So we removed that 5 no HDD and restarted again ..No Improvement.

Did nt find any clues in Logs..Can u say which log file majorly i shud check for finding clues.

Naveenkumar001 · ‎2025-02-19

2025-02-20 10:30:36: md/raid:md127: raid level 6 active with 11 out of 12 devices, algorithm 2

Am getting this in Diagonastic screen of raidar.

StephenB · ‎2025-02-20

What happens when you try to access the data

Naveenkumar001 · ‎2025-02-20

I can able to reach the data of folder...but the final folder loading many hours..and showing empty...Atleast if able to copy paste data....it more good....

StephenB · ‎2025-02-20

@Naveenkumar001 wrote:
I can able to reach the data of folder...but the final folder loading many hours..and showing empty...Atleast if able to copy paste data....it more good....

If you want me to take a look at the logs, then upload the full log zip to cloud storage. Then send me a PM (private message) using the envelope icon at the upper right of the forum page that includes a download link. Make sure the link permissions are set so anyone with the link can download the zip.

Sandshark · ‎2025-02-20

Removing that drive was a really bad idea. It is highly unlikely that could have been the cause of your issue, and you've now reduced your redundancy. If you used the default XRAID configuration, then you have RAID6, so you do still have some redundancy.

Does RAIDar now show the volume is re-syncing?

That kind of problem is often caused by a corrupted Samba configuration file. Unfortunately, most of them are not replaced by an OS re-install because they are specific to your configuration. It can be because of an over-full OS partition, but the fact that you can get the logs via RAIDar makes that less likely to be the cause.

Did you take any action immediately before this happened (like change any share access)? Did it happen on a power cycle?

Is SSH enabled on the NAS?

Naveenkumar001 · ‎2025-02-20

The same logs been shared. please kindly suggest further procedures to be recover my data.

Naveenkumar001 · ‎2025-02-20

Does RAIDar now show the volume is re-syncing?

Yes its showing the hard disk resync sign for extra harddrive ..which is re inserted again. But after some hours. total NAS is gon offline. not showing in raidar too..No ping of its IP also.

Hence after restart. Shows service management is offline. and repeats.

Did you take any action immediately before this happened (like change any share access)? Did it happen on a power cycle?

No root cause . Last scedule an Backup Job for NAS. that too stopped before NAS goes to offline.. its seems no track of intial prob.

Is SSH enabled on the NAS?

Yes

StephenB · ‎2025-02-21

@Naveenkumar001 wrote:
>>> Is SSH enabled on the NAS?

Yes

I suggest accessing the NAS with ssh, and then check that your data is there (ls -als /data).

The logs show that the volume is mounted, but I am seeing unexplained BTRFS timeouts:

Feb 20 10:53:40 DHTL-TP1-NAS kernel: INFO: task btrfs-transacti:4250 blocked for more than 200 seconds.
Feb 20 10:53:40 DHTL-TP1-NAS kernel:       Tainted: P           O    4.1.30.x86_64.1 #1
Feb 20 10:53:40 DHTL-TP1-NAS kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 20 10:53:40 DHTL-TP1-NAS kernel: btrfs-transacti D ffff88021321bd68     0  4250      2 0x00000000
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffff88021321bd68 ffffffff88e2c4a0 ffff880265f08fc0 ffff88026fc147c0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffffffff88e2c8d8 ffff88026fc147c0 ffff88021321c000 ffff880265d8f1f0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  ffff880265d8f1f0 0000000000000000 ffff88026644b370 ffff88021321bd88
Feb 20 10:53:40 DHTL-TP1-NAS kernel: Call Trace:
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88a4f1e2>] schedule+0x32/0x80
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e62df>] wait_current_trans.isra.34+0x9f/0x100
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff8809c8c0>] ? add_wait_queue+0x60/0x60
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e7e58>] start_transaction+0x2c8/0x550
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e8142>] btrfs_attach_transaction+0x12/0x20
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e4d0b>] transaction_kthread+0x7b/0x240
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff882e4c90>] ? open_ctree+0x24b0/0x24b0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082bb4>] kthread+0xc4/0xe0
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082af0>] ? kthread_freezable_should_stop+0x60/0x60
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88a523d2>] ret_from_fork+0x42/0x70
Feb 20 10:53:40 DHTL-TP1-NAS kernel:  [<ffffffff88082af0>] ? kthread_freezable_should_stop+0x60/0x60

On SMB access, I am seeing a lot of errors that look like this:

[2025/02/20 16:12:33.035542,  2] ../source3/auth/auth.c:315(auth_check_ntlm_password)
  check_ntlm_password:  Authentication for user [IMW] -> [IMW] FAILED with error NT_STATUS_NO_SUCH_USER

Is IMW an account that should exist on the NAS?

You are running very old firmware (6.6.0 - released in 2016). You appeared to make an attempt to update to 6.10.10 on 19 Feb

Feb 19 2025 17:04:49 INFO:UPLOAD_IMAGE: _filename_=ReadyNASOS-6.10.10-x86_64.zip _filesize_=90385981
Feb 19 2025 17:04:50 INFO:GET_REMOTE_INFO: name=ReadyNASOS version=6.10.10 size=90783744 md5=b2f2ee74b9bbf269d449490b24b370fe arch=x86_64
Feb 19 2025 17:04:50 INFO: arch=x86_64 personality= sysinfo->personality=1
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: Checking file for md5... 
Feb 19 2025 17:04:50 INFO:CHECK_MD5_SUM: local=b2f2ee74b9bbf269d449490b24b370fe remote=b2f2ee74b9bbf269d449490b24b370fe
Feb 19 2025 17:04:50 INFO:CHECK_MD5_SUM: MD5 OK! Create link in /etc/.flash_update -> .flash_update_file
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: Write info to .flash_update_info... 
Feb 19 2025 17:04:50 INFO:UPLOAD_IMAGE: SUCCESS

After that you appear to have downgraded back to 6.6.0

Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: _filename_=ReadyNASOS-6.6.0-x86_64.img _filesize_=63207424
Feb 19 2025 17:29:10 INFO:GET_REMOTE_INFO: name=ReadyNASOS version=6.6.0 size=63207424 md5=aec396425a099a22e0fc4f43bbe6ac69 arch=x86_64
Feb 19 2025 17:29:10 INFO: arch=x86_64 personality= sysinfo->personality=1
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: Checking file for md5... 
Feb 19 2025 17:29:10 INFO:CHECK_MD5_SUM: local=aec396425a099a22e0fc4f43bbe6ac69 remote=aec396425a099a22e0fc4f43bbe6ac69
Feb 19 2025 17:29:10 INFO:CHECK_MD5_SUM: MD5 OK! Create link in /etc/.flash_update -> .flash_update_file
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: Write info to .flash_update_info... 
Feb 19 2025 17:29:10 INFO:UPLOAD_IMAGE: SUCCESS

Can you give more details on what you were doing here?

Naveenkumar001 · ‎2025-02-21

Nope....Sry ..SSH disabled.....and tried to access the NAS with only 1 drive ...which is removed in starting hdd5 and reinsert again to check the nas...there by I able to access the NAS without my volume raid 6.
And while enabling ssh..it's popup of volume to be present for enabling SSH.

Data is present..I able to see the all folders by accessing through Run \\ip command

Same time I tried of updating NAS firmware without volume.but the iso file not accepted the latest firmware. And tried with same version also..that package also not took. So dropped the scene.

IMW is not the account in NAS. Is the PC account where we access the NAS through IP.

Now NAS is running in 10 drive out of 12. Currently in Non redundant state .
Note : No drive misplaced in it place.

Nas status in Raidar : Management service is offline .
So How to login my Nas with my volume .So atleast I can take copy of my datas. And I do reset the device with new hdds.
Please kindly guide me.

StephenB · ‎2025-02-22

@Naveenkumar001 wrote:

Data is present..I able to see the all folders by accessing through Run \\ip command

So How to login my Nas with my volume .So at least I can take copy of my datas.

Copy the data over the network using SMB.

If you use "private" home folders, you can get those files also over SMB using the NAS admin credentials. For instance, open CMD and enter

net use * /d /y
net use z: \\nas-ip-address\data /u:admin nas-admin-password

will mount the full data volume (using the real NAS IP address and admin password of course).

Once the data is safe, we can move on the the other questions.