Where you see 'Backup: Saved backup job' that is where I edited the source/destination to another backup folder on the remote PC, and at that time I started the backup, which helpfully doesn't get logged.
2020 Aug 26 12:15:15 - Backup: Saved backup job MePC D VMs.
2020 Aug 26 12:57:31 - Volume: Data tier migration started for volume RN316A.
2020 Aug 26 16:05:09 - Backup: Successfully completed backup job MePC D VMs.
2020 Aug 26 16:12:07 - Backup: Saved backup job MePC D VMs.
2020 Aug 26 19:17:23 - Volume: The data on the highest tier is exceeding its capacity for volume RN316A. Data will use the lower tier volume and performance will be impacted.
2020 Aug 26 19:55:58 - Backup: Error backup job MePC D VMs: Failure during copy.
2020 Aug 26 21:00:00 - Volume: Lower tier is now full and the data on the highest tier is not able to migrate the data for volume RN316A. Please upgrade the lower tier capacity to allow migration to continue.
2020 Aug 26 21:57:33 - Volume: Data tier migration started for volume RN316A.
2020 Aug 27 00:35:49 - Volume: Data tier migration completed for volume RN316A.
When I looked at the volume, hovering over the pie-chart to envoke to pop-up*, it showed 9GB (giga) as RG2 Unllocated, but at the top it showed 8TB unallocated. After doing a disk check (Aug 29th-30th), I then did a balance (30th, took 15 min.), which changed RG2 Unallocated to TB's.
Yes I should Balance more often, but why & what did it run out of?
If the 'lower tier is full & the data on the highest tier is not able to migrate' what is it doing 26th 21:57 to 27th 00:35?
Does/should the migration run at higher priority (CPU & IO) to try to be ahead of other activity?
(* is the data in the pop-up available elsewhere, some where that doesn't dissapear every few seconds when the progress bar refreshes?)
2020 Aug 26 21:57:33 - Volume: Data tier migration started for volume RN316A.
2020 Aug 27 00:35:49 - Volume: Data tier migration completed for volume RN316A.
2020 Aug 27 03:00:01 - Volume: Data tier migration started for volume RN316A.
2020 Aug 27 03:00:06 - Volume: Data tier migration completed for volume RN316A.
2020 Aug 29 03:00:01 - Volume: Data tier migration started for volume RN316A.
2020 Aug 29 03:00:06 - Volume: Data tier migration completed for volume RN316A.
2020 Aug 29 17:10:08 - Volume: Disk test started for volume RN316A.
2020 Aug 30 09:49:34 - Volume: Disk test completed for volume RN316A.
2020 Aug 30 12:38:11 - Volume: Balance started for volume RN316A.
2020 Aug 30 12:54:19 - Volume: Balance complete for volume RN316A.
2020 Aug 30 12:56:17 - Volume: Defragmentation started for volume RN316A.
2020 Aug 30 18:29:01 - Backup: Successfully completed backup job MePC D VMs.
2020 Aug 31 02:00:01 - Volume: Failed to start data migration for volume RN316A because the volume is already running a defrag.
2020 Aug 31 08:47:55 - Backup: Saved backup job MePC D VMs.
2020 Aug 31 15:06:23 - Backup: Successfully completed backup job MePC D VMs.
2020 Aug 31 15:11:51 - Backup: Saved backup job MePC D VMs.
2020 Aug 31 21:35:24 - Backup: Successfully completed backup job MePC D VMs.
2020 Sep 01 02:00:02 - Volume: Failed to start data migration for volume RN316A because the volume is already running a defrag.
2020 Sep 02 02:00:01 - Volume: Failed to start data migration for volume RN316A because the volume is already running a defrag.
2020 Sep 02 08:10:01 - Volume: Defragmentation complete for volume RN316A.
2020 Sep 02 08:17:37 - Volume: The data on the highest tier is exceeding its capacity for volume RN316A. Data will use the lower tier volume and performance will be impacted.
2020 Sep 02 08:17:42 - Volume: Data tier migration started for volume RN316A.
2020 Sep 02 12:23:26 - Backup: Saved backup job MePC D VMs.
2020 Sep 02 12:32:36 - Volume: Data tier migration completed for volume RN316A.
2020 Sep 02 15:29:14 - Backup: Successfully completed backup job MePC D VMs.
2020 Sep 03 02:00:01 - Volume: Data tier migration started for volume RN316A.
2020 Sep 03 03:36:05 - Volume: Data tier migration completed for volume RN316A.
2020 Sep 04 02:00:01 - Volume: Data tier migration started for volume RN316A.
2020 Sep 04 02:00:06 - Volume: Data tier migration completed for volume RN316A.
After the Balance made TB's available, shortly thereafter I reran the failed backup job.
I then started Defrag. It would be really nice if the Volume info showed Defrag progress and the ability to pause or stop it.
Even better, other tasks pause it for you & restart it. Or it runs for a specified time so you could schedule 'chunks' amoung other maintenance activity.
Part 2B.
2020 Sep 04 02:00:01 - Volume: Data tier migration started for volume RN316A.
2020 Sep 04 02:00:06 - Volume: Data tier migration completed for volume RN316A.
2020 Sep 04 09:14:32 - Volume: Balance started for volume RN316A.
2020 Sep 04 10:57:35 - Volume: Balance complete for volume RN316A.
2020 Sep 04 11:41:25 - Volume: Defragmentation started for volume RN316A.
2020 Sep 05 02:00:02 - Volume: Failed to start data migration for volume RN316A because the volume is already running a defrag.
2020 Sep 06 01:00:02 - Volume: Failed to start balance for volume RN316A because the volume is already running a defrag.
2020 Sep 06 01:00:07 - Volume: Balance failed to start for volume RN316A.
2020 Sep 06 02:00:02 - Volume: Failed to start data migration for volume RN316A because the volume is already running a defrag.
2020 Sep 06 07:05:07 - Volume: Defragmentation complete for volume RN316A.
2020 Sep 06 07:07:54 - Volume: The data on the highest tier is exceeding its capacity for volume RN316A. Data will use the lower tier volume and performance will be impacted.
2020 Sep 06 07:07:57 - Volume: Data tier migration started for volume RN316A.
2020 Sep 06 11:17:45 - Volume: Data tier migration completed for volume RN316A.
02:00:06 The high tier is empty.
07:05:07 Defrag complete.
Then it spends 4 hours migrating.
So it seems Defrag fills the high tier. I presume migration then fragments it again.
Yes I should Balance more often, but why & what did it run out of?
It looks like it ran out of unallocated space. So the mitigation here is to run the balance more regularly (perhaps set up a schedule in the volume settings). I run it every three months on my own systems.
@Michael_Oz wrote: It would be really nice if the Volume info showed Defrag progress and the ability to pause or stop it.
Agreed. Though this is another case where running it more frequently might help. The last defrag on my volume (similar size to yours) took about 15 minutes. Though of course the amount of fragmentation does depend on the usage.
There is an auto defrag option for each share, which might be worth a try - it might not take as long. Though scheduling it in the maintenance settings would give you control over when it occurs.
So it seems Defrag fills the high tier. I presume migration then fragments it again.
Isn't that beside the point.
As should be clear, I'm not using ReadyTier myself, so I have no direct experience.
But I don't think the Defrag is "filling the high tier". Instead, I think the defrag is only being done on the low tier, and that the NAS is postponing migration until the defrag is done. In the meantime, the system is still caching in the high tier, so it's usage will increase.
I added in the dates to your two log quotes, to point out that the first entry doesn't seem that relevant to me. It doesn't seem unlikely that the SSD tier would be full after 2 days. It does seem a bit odd that the "tier full" message happens right after the defrag completes. That might be a side effect of the migration process being restarted.
4 hours to migrate 500 GB of data does seem like a long time (~35 MB/sec), so it does sound like it isn't just copying files. What mechanical drive models are you using?
It is possible that migration does increase fragmentation - if it works on a block level, then it likely does, especially if snapshots are turned on.
Maybe @JohnCM_S or @Marc_V could provide a bit more information on how the data tiering works?
