Discussion stats
  • 5 replies
  • 1117 views
  • 1 kudo
  • 3 in conversation
Announcements

Top Contributors
Reply
Highlighted
Aspirant

Full data loss after latest upgrade

So, it happened with me. I have RN 204 with two 4TB WD red installed.

Recetly I've logged into admin panel and the device offered me a minor upgrade, from  one of latest versions (6.8.smth from what I can remember to 6.9.1). So, I've agreed.

 

After restart the device has provisioned my disks as they were empty. LUKS header has been lost and all the data gone.

 

After some investigations I've reproduced the issue and the scenario is quiet trivial:

 

  1. In version 6.9.1 the kernel thinks that logical sector size is 4096 and physical sector size is 4096
  2. In all the previous versions it was 512 for logical and 4096 for physical
  3. After reboot to 6.9.1 the kernel failed to validate GPT signature and reported disks as empty
  4. BEAUTIFUL netgear software re-provisioned disks and overwrote beginning of the data partition. Together with mdadm and LUKS headers

Thank you, thank you guys for your amazing products. Thank you, thank you.

Model: RN204|ReadyNAS204
Message 1 of 6
Highlighted
NETGEAR Employee Retired

Re: Full data loss after latest upgrade

Support for disks with 4KB logical sectors was added way back in 6.2.0 and 6.3.0. 6.3.0 was the first ever firmware for the RN204.

 

Did you migrate disks from a RN102 or the RN104 to the RN204?

 

Do you have any logs zip saved from the NAS from any time before the firmware update? If so can you send one of those in (see the Sending Logs link in my sig)?

 

Do you have logs from the current state of the system?

Is your backup up to date?

 

Message 2 of 6
Highlighted
Aspirant

Re: Full data loss after latest upgrade

> Support for disks with 4KB logical sectors was added way back in 6.2.0 and 6.3.0. 6.3.0 was the first ever firmware for the RN204

 

I'm not a netgear developer so I don't know all the shiny history of your firmware versions. From what I can get after my post-mortem investigations - originally disks were partitioned for 512/4096 and after upgrade they were repartitioned for 4096/4096

 

> Did you migrate disks from a RN102 or the RN104 to the RN204?

Nope, this was my first and last NG device. I've bought two 4TB drives and your mangificient device.

 

> Do you have any logs zip saved from the NAS from any time before the firmware update? If so can you send one of those in (see the Sending Logs link in my sig)?

Why may I need to save logs in case everything worked?

 

> Do you have logs from the current state of the system?

Uhu, I may dump em.

 

> Is your backup up to date?

Unfortunately 2 months old. So, why don't you guys put a BIG RED STICKER onto your devices saying:

 

ALWAYS HAVE UP TO DATE BACKUPS BECAUSE WE LOVE TO APPLY DESTRUCTIVE OPERATIONS AND DON'T TEST OUR FIRMWARE PROPERLY

Message 3 of 6
Highlighted
NETGEAR Employee Retired

Re: Full data loss after latest upgrade

So you've always been running firmware with support for 4k logical sectors then.

 

My request was just in case you had. It is best practice to download logs both before and after firmware updates and to make sure that your regular backup is up to date before updating the firmware where possible. See ReadyNAS OS 6: Firmware Upgrade Guide and Tips

 

I don't recall any other reports of this.

 

In our experience we do find that when firmware updates are blamed for data loss that the root cause turns out to be something quite different.

Message 4 of 6
Highlighted
NETGEAR Expert

Re: Full data loss after latest upgrade

We'd like to investigate, but we don't have much real information here.  I've got a RN204 sitting right next to me with 4TB WD Red drives, and I just upgraded from 6.8.1 (4.4.88 kernel) to 6.9.1 (4.4.98 kernel) with no issue.

 

You mentioned that you were able to reproduce it.  How?  What, precisely, did you do to reproduce it?

Message 5 of 6
Highlighted
Aspirant

Re: Full data loss after latest upgrade

I had another 4tb disk which I've added into array long time ago and then pulled it back. So I've pulled out erased disk, installed old one, upgraded and got it erased. Obviously OS version was not 6.8, that disk has been removed out of array in october/2016

Message 6 of 6