Reply

ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

Pajuno
Aspirant

ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

The device ReadyNas 2312 was working perfectly for months until we had numerious of unexpected shutdown. Before the cunexpected shutdown we have only use up to 50% of the storage space (12 X 10TB) and to our suprise we found out that 90% of the storage space has been used up already and the LUN created was no more available  also iSCSi initiator was failing to connect from the server end.

 

We replaced the disk and performed a factory reset on the ReadyNas 2312 device and we left it to resync for 60 hours using RAID 6 default configuration.

 

After the resync was successful, we found out that it has also used up 90% of the storage disk (without any iSCSi connection) and telling us we have 10% storage, LUN could not be found and also iSCSi inintiator failing to connect. Another thing we observed was that the (12 X 10TB) are healthy and the machine always resync itself after a shutdown.

 

Things done so far.

1. Replaced existing hard drives (12 x10TB) and formatted them: Still using up 90% (within 20 minutes) without any transfer of data from our connecting server. LUN cannot be found and iSCSi initiator keeps failing to connect. Resync on every startup which takes 60 hours.

2. Factory reset the machine with the drives (12 x10TB) intact: Still using up 90%(within 20 minutes) without any transfer of data from our connecting server. LUN cannot be found and iSCSi initiator keeps failing to connect. Resync on every startup which takes 60 hours.

3. Updated firmware of the machine with the drives(12 x10TB) intact: Still using up 90% (within 20 minutes) without any transfer of data from our connecting server. LUN cannot be found and iSCSi initiator keeps failing to connect. Resync on every startup which takes 60 hours.

4. OS reinstall with the drives (12 x10TB) intact: Still using up 90% (within 20 minutes) without any transfer of data from our connecting server. LUN cannot be found and iSCSi initiator keeps failing to connect. Resync on every startup which takes 60 hours.

 

Question

What is the way forward?

Message 1 of 17
StephenB
Guru

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

You might want to try paid netgear support (via my.netgear.com).  Something very strange seems to be happening on your NAS.

 

Have you downloaded the full log zip, and looked through it?  Possibly enable connection auditing, and see if you can track down where the storage is going?

 

You could also access the NAS via ssh, and look at the full data volume directly - using linux tools to analyze where the space is going.

Message 2 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

How do I go about it using Linux. I have a Linux operating system but will it connect since iSCSi if failing to connect? Kindly advise.

Thanks
Message 3 of 17
Sandshark
Sensei

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

The NAS operatring system is Linux, and you access it via SSH, which you have to turn on in the GUI.  When accessing via SSH, log in as root with the admin password.

 

What method are you using to determine that 90% of the drive space is used up?  Your NAS isn't fast enough to do that in such a short period of time unless it is simply reserving the space for iSCSI.

Message 4 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

Kindly check the log

 

[20/03/06 19:07:48 PST] notice:volume:LOGMSG_RESILVERSTARTED_VOLUME Resyncing started for Volume data.
[20/03/06 19:07:49 PST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/03/06 19:07:49 PST] notice:system:LOGMSG_FIRMWARE_UPGRADED Firmware was upgraded to 6.8.0.
[20/03/06 19:08:39 PST] info:system:LOGMSG_SET_ALERTCONTACT Alert contact information was saved.
[20/03/06 19:08:39 PST] info:system:LOGMSG_SET_ALERTCONFIG Alert settings are updated.
[20/03/06 19:08:56 PST] info:system:LOGMSG_READYNASD_ABORTED ReadyNASOS service or process (readynasd) was restarted.
[20/03/06 19:08:56 PST] notice:volume:LOGMSG_RESILVERSTARTED_VOLUME Resyncing started for Volume data.
[20/03/06 19:08:57 PST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/03/07 04:16:21 CET] notice:system:LOGMSG_DISABLE_NTP NTP is disabled.
[20/03/07 04:27:07 CET] warning:volume:LOGMSG_VOLUME_USAGE_CRITICAL Less than 10% of volume data's capacity is free. Performance on volume data is degraded. To improve performance, you must add capacity.
[20/03/09 03:43:26 CET] notice:volume:LOGMSG_RESILVERCOMPLETE_VOLUME Volume data is resynced.
[20/05/29 17:14:16 CEST] info:system:LOGMSG_READYNASD_ABORTED_NOINFO ReadyNASOS service or process was restarted.
[20/05/29 17:14:17 CEST] warning:volume:LOGMSG_VOLUME_USAGE_CRITICAL Less than 10% of volume data's capacity is free. Performance on volume data is degraded. To improve performance, you must add capacity.
[20/05/29 17:14:18 CEST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/06/15 22:22:44 CEST] notice:system:LOGMSG_SYSTEM_REBOOT The system is rebooting.
[20/06/15 22:23:45 CEST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/06/16 00:44:33 CEST] info:share:LOGMSG_ADD_LUN LUN test was added.
[20/06/16 00:59:26 CEST] info:system:LOGMSG_READYNASD_ABORTED ReadyNASOS service or process (readynasd) was restarted.
[20/06/16 00:59:27 CEST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/06/16 01:00:28 CEST] info:share:LOGMSG_ADD_LUN LUN TES2 was added.
[20/06/16 01:04:02 CEST] info:share:LOGMSG_DELETE_LUN LUN test was deleted.
[20/06/16 01:07:27 CEST] info:share:LOGMSG_DELETE_LUN LUN TES2 was deleted.
[20/06/16 01:13:28 CEST] info:share:LOGMSG_ADD_LUN LUN PAJUNO was added.
[20/06/16 01:17:36 CEST] warning:volume:LOGMSG_VOLUME_USAGE_CRITICAL Less than 10% of volume data's capacity is free. Performance on volume data is degraded. To improve performance, you must add capacity.
[20/06/16 01:20:36 CEST] info:share:LOGMSG_DELETE_LUN LUN PAJUNO was deleted.
[20/06/16 01:22:26 CEST] notice:system:LOGMSG_SYSTEM_REBOOT The system is rebooting.
[20/06/16 01:23:29 CEST] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
[20/06/16 01:30:32 CEST] info:share:LOGMSG_DELETE_LUN LUN pajunospace was deleted.
[20/06/16 01:31:00 CEST] info:share:LOGMSG_ADD_LUN LUN PAJUNO was added.

Message 5 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

Kindly find systemd-joirnal

 

-- Logs begin at Fri 2020-05-29 17:13:42 CEST, end at Tue 2020-06-16 20:53:48 CEST. --
Jun 16 19:10:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 19:10:18 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 19:10:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 19:10:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 19:10:43 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:45:38 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:38 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:45:38 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:43 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:45:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:43 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:45:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:48 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:45:48 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:48 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:45:48 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:53 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:45:53 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:53 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:45:53 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:58 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:45:58 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:45:58 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:45:58 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:03 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:03 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:03 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:03 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:08 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:08 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:08 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:08 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:13 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:13 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:13 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:13 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:18 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:18 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:23 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:23 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:23 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:23 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:28 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:28 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:28 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:28 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:33 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:33 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:33 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:33 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:38 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:38 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:38 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:38 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:43 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:43 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:48 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:48 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:48 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:48 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:53 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:53 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:53 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:53 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:58 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:46:58 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:46:58 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:46:58 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:03 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:03 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:03 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:03 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:08 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:08 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:08 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:08 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:13 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:13 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:13 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:13 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:18 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:18 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:18 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:23 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:23 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:23 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:23 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:28 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:28 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:28 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:47:28 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:33 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:pajuno:79e259f1:tes2group in Storage Node
Jun 16 20:47:33 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:47:33 PAJUNO kernel: Unable to locate Target IQN: iqn.1994-11.com.netgear:patrick:49cf3115:pajuno in Storage Node
Jun 16 20:53:43 PAJUNO kernel: iSCSI Login negotiation failed.
Jun 16 20:53:47 PAJUNO dbus[4419]: [system] Activating service name='org.opensuse.Snapper' (using servicehelper)
Jun 16 20:53:47 PAJUNO dbus[4419]: [system] Successfully activated service 'org.opensuse.Snapper'
Jun 16 20:53:48 PAJUNO readynasd[4583]: Error: Connection failure: Connection refused
Jun 16 20:53:48 PAJUNO systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 15557 (sysctl)

Message 6 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

The shutdown triggered this message before the crash

 

- Logs begin at Fri 2020-05-29 17:13:42 CEST, end at Tue 2020-06-16 20:53:48 CEST. --
May 29 17:14:09 PAJUNO readynasd[4591]: readynasd log started
May 29 17:14:10 PAJUNO readynasd[4591]: readynasd started. (restarted=0)
May 29 17:14:16 PAJUNO readynasd[4591]: Last run started at 1583550536 was aborted
May 29 17:14:16 PAJUNO readynasd[4591]: Detect abnormal shutdown of readynasd and child without core
May 29 17:14:16 PAJUNO readynasd[4591]: ReadyNASOS service or process was restarted.
May 29 17:14:17 PAJUNO readynasd[4591]: Using fan mode: cool
May 29 17:14:17 PAJUNO readynasd[4591]: Less than 10% of volume data's capacity is free. Performance on volume data is degraded. To improve performance, you must add capacity.
May 29 17:14:18 PAJUNO readynasd[4591]: [createNetworkPortals] Updating iSCSI network portals:
May 29 17:14:18 PAJUNO readynasd[4591]: ResetFailedUnit: GDBus.Error:org.freedesktop.systemd1.NoSuchUnit: Unit radar.timer is not loaded.
May 29 17:14:18 PAJUNO readynasd[4591]: ReadyNASOS background service started.
May 29 17:27:44 PAJUNO readynasd[4591]: ERROR: Radar: Maximum iteration exceeded
Jun 15 22:09:22 PAJUNO readynasd[4591]: Could not connect to Replisync socket!
Jun 15 22:09:22 PAJUNO readynasd[4591]: Could not connect to Replisync socket!
Jun 15 22:09:22 PAJUNO readynasd[4591]: get_replisync_job_status() failed!

Message 7 of 17
StephenB
Guru

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

If you are willing to try another factory reset, can you isolate the NAS from the servers that are trying to use it first?  

 

For instance, put it on it's own vlan, with just the management PC.  Then do the reset (and don't recreate the LUNs right away).

 

Then see if the space goes up the same way.

Message 8 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

It works fine without creating a LUN. Once the LUN has been created, it revert to the same issue. Do you think I should try downgrade the firmware to an older version that is compatible with the device?
Message 9 of 17
Sandshark
Sensei

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

Are you using thin or thick provisioning, and what size LUN are you creating?  It souds as if you are creating a thick provisioned LUN that takes most of the available space.

Message 10 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

I'm using thick which equals 83TB. I'm using 81TB thick provision.
Message 11 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

12 x10TB with RAID 6 configuration.
Message 12 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

LUN disappears and the iSCSi connection fails
Message 13 of 17
Sandshark
Sensei

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

OK, so you are creating an iSCSI LUN that uses 90% of the drive space and are wondering why 90% gets used up?  That doesn't leave the NAS much overhead, witch is the likely reason for the connection issue.

 

I can only guess that you previously used thin provisioning, which will grow as used, not thick, which immediately uses up the allocated space.

Message 14 of 17
Pajuno
Aspirant

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

This is the fourth set of drives I'm changing and I definitely know that I used thick LUN provision. Even in the scene where we use up %90 of storage space, is it supposed to cancel every connection to access files because I have never seen a device denying file access when full.
Message 15 of 17
Sandshark
Sensei

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space

The drives are clearly not the issue.  Have you tried leaving the NAS more headroom -- creating a smaller LUN?  You certainly need to try comething different.  Doing the same thing over and over is going to get you the same result over and over.  If it didn't. even when you don't like that result, I'd be more worried.

 

BTRFS does require headroom.  How much when the space is taken up by an essentially empty iSCSI LUN, I have no idea.  Is it "supposed" to deny access?   Well, I'm sure it's not a designed-in feature.  But it certainly could be a limitation.  Your log shows you are receiving a warning that you are apparently ignoring.

 

Assuming you have upgraded the NAS OS at some point since first put in service, there may be something about the newer version that requires more headroom when the LUN is created.

 

I still don't understand why you are "surprised" that 90% of the storage is being used up when you are performing an operation that is designed to do just that.  A thick-provisioned LUN reserves the full amount of allocated space when it is created, so it is "used up" from the OS's point of view.  If that was not previously the case, then you are doing something different now than before, and my guesses are the LUN size or thin vs. thick provisioning.  If the OS was previously showing only 50% use when you had a similarly sized LUN, then it was not thick provisioned.  And based on your experience with the thick provisioned one, a thin provisioned one might run into a similar issue if you were to let it become full, so keeping it smaller is likely a good idea.

Message 16 of 17
StephenB
Guru

Re: ReadyNas 2312 resync after shutdown and automatically uses 90% storage space


@Sandshark wrote:

I still don't understand why you are "surprised" that 90% of the storage is being used up when you are performing an operation that is designed to do just that.  


The 12x10TB RAID-6 array overall should have 100 TB of storage - ~90.95 TiB.  A thick LUN of 83 TiB should end up using 91% of the capacity.  So I agree that part is expected.

 

Also, that it makes sense to try this with a smaller LUN.  Of course if the 83 TiB size is needed, then the volume would need to be expanded (perhaps adding some 14 TB or 16 TB drives into the mix).

Message 17 of 17
Top Contributors
Discussion stats
  • 16 replies
  • 1202 views
  • 1 kudo
  • 3 in conversation
Announcements