R8000 Firmware v1.4.1.30_1.2.26 won't restore config from previous firmware

[rowets]

Hi guys

I've just upgraded to the latest version of the firmware for my R8000. There was lots of messing about (Netgear servers must have been busy or something) but after the firmware was applied, it looked like it had done a factory reset.

So I went back to a backup of the config that I had taken just before the upgrade and tried to restore that.

I didn't get any errors, but the restore finishes & seems to have no effect - the original settings are not imported.

The issue is that I have around 60 fixed ip addresses setup which is going to be a nightmare to reinsert manually.

Please don't tell me that the old backup of the config when running the old firmware (only a few versions older) is not compatible.

Hope you can help

Trevor

[rowets · Accepted solution]

So my accepted solution (NG want me to provide one), is to no longer use NG for saving lots of config as they can trash my data whenever they want and expect me to type it in again - not doing that!

Instead I'm using a Raspberry PI as a DHCP server and disabling this feature on the NG router.

That way I can properly back up the settings.

[IrvSp]

That firmware version is not for the R8000, but either an R8000P or R7900P.

Usually if one does hit a problem, a RESET and manual entry of the settings is advisable. It really depends on what was changed?

Is it possible you got a corrupted d/l?

---

@rowets wrote:

There was lots of messing about (Netgear servers must have been busy or something) but after the firmware was applied, it looked like it had done a factory reset.

---

It is normal for the router to re-boot as it is internally installing the firmware. I'll assume you did the install via a browser Genie? Don't understand the part above about the NG servers... normally it is a fast d/l and then the router starts updating. Not sure what you meant by 'messing about' but it is possible you had gotten a bad d/l. I'd try getting it again via SUPPORT and try again. I'd advise against using a backup if you did indeed lose all settings.

[rowets]

Yes, it was a problem selecting the correct model from the web pages, I have a R8000P (sorry about that), I'm in the UK and it wouldn't let me select the exact model from the web page

When trying to connect to N/G it just sat there for ages trying to download the firmware, but it got there in the end

I don't think it's a corrupted download as the router is working perfectly.

The reboot I understand, I've that many times in the past on firmware upgrades

But after this firmware upgrade, it went into a complete reset of the router, eg, new network names, new passwords, and all the fixed ip addresses that I'd set up and all the other settings that I can't remember.

Is it expected behaviour that this firmware will completely reset the the router and the config files created on previous firmwares will not work

[IrvSp]

---

@rowets wrote:

But after this firmware upgrade, it went into a complete reset of the router, eg, new network names, new passwords, and all the fixed ip addresses that I'd set up and all the other settings that I can't remember.

Is it expected behaviour that this firmware will completely reset the the router and the config files created on previous firmwares will not work

---

Sorry, I can't comment on this specific router, but I've owned my NG routers and others as well and I've never seen this?

I'd guess it might be possible due to the problems getting the file down, could have been a timing issue even, I do not know?

I guess since you are at 'square one' with the router you might want to go back a level, set a few settings, like SSID names and then upgrade again and see if they hold? If they do, it might have been something odd happened? Not knowing what actually changed code wise in the firmware it would be hard to tell if it could/would wipeout settings or not?

Since it is working, you might want to turn off AUTO-UPDATE if the firmware supports it. Of course, sometime in the future, a security fix might be made you'd want to update at that time.

[rowets]

I've always turned off auto update, so that I can control the updates myself.

I must have had about 7 or 8 NG routers in the past (running the R8000P and two repeaters at the moment) and I've never seen this sort of thing in the past.

I'm really hoping this isn't going to be the way that NG will be working in the future.

Since starting this post I've been manually entering the fixed ip addresses again and I'm still only half way thru!

I have no idea what other settings I've lost 😞

If this happens again, I'm moving away from N/G as this is unacceptable to wipe the router and backups not work.

If I could manually upload a csv of the entries then that would help, but there is only the backup restore feature and the files are encoded, so I can't even manually add the entries.

Thanks for your suggestions

Trev

[rowets]

Just for anybody who might get caught like i did, this is the statement in the netgear release notes, which is absolutely shocking

1. Write down all the settings which you changed from the default values, since you may need to reenter them manually.

This is exactly what happened to me after upgrading to the latest firmware

I really can't believe that NGs backup and restore strategy consists of Pen and Paper 

I'm really disappointed that NG cannot merge settings going forward between firmware upgrades

Reentering around 60 reserved ip addresses took several hours, something i am not prepared to do again and NG should realise this approach is unacceptable in this day and age.

This strategy is riduclous, I like NG products but I'm not manually entering all the data again after a firmware upgrade.

I don't care about most of the 'latest and greatest' features, I looked at that when I bought the router in the first place.

However, something as basic as having backups that always work on later versions should be something that is always supported on any product.

Now that I know this is how NG work, I will never buy another NG product whilst they expect users to use Pen and Paper as a backup strategy!

This is what NG expect of their users, so beware of firmware upgrades - you have been warned!

[IrvSp]

That message had been in ALL the firmware at one time. Also had said at one time you must use a wired connection, now it is suggested.

Now for the R8000 it does still say that, still, I never had to do that, nor on the old R7000 I had. It does say 'may' so it isn't a sure thing you will lose everything?

I'm just wondering if it was to the long time it took to get the firmware down? Maybe the router internally did some resetting to see if it could improve the d/l? Like I said, I've not seen this complaint before (doesn't mean it doesn't happen) and in all the years I've had various NG routers it never happened to me.

You might not have read older README's, but at one time you were told you had to RESET and manually re-enter settings after updating ... which over time became the 'warning' that you get now.

A long time ago I took screenshots of all my pages I made changes to and keep them for reference just in case.

Why you lost the settings is a mystery?

I too wish the settings save were human readable. Either CSV, TXT, XLS, or HTML. I suspect the reason is so there is no way to alter them as loading in an altered set might cause some problems, especially if you don't how to change it?

[IrvSp]

@rowets, for what it is worth department, older R7000 firmware updates, like V1.03.24 also had that statement, but a contradictory statement at the top as well:

"Note: Please remember to do factory default after firmware upgrade. Enter "Backup Settings" in the web GUI and click "Erase" button of "Revert to factory default settings". "

That had been on many (the above was from 2 years ago).

I also don't know what other router manufacturers state or can/could have happen after updates, nor 3rd party firmware makers.

Knowing how software applications are created and compiled sometimes there are good reasons for forcing erasing NVRAM and manually re-entering settings. If this is the case for NG firmware I just don't know, but I'm sure this could happen at some point.

[rowets]

I get that possible wiping of the settings can sometimes be required - I work in software and I know what issues an upgrade can cause.

But not having the backups being able to be restored after a firmware upgrade, that just shouldn't happen.

Everything else is working perfectly so the  download is not corrupted.

There are possibly hundreds of settings that can be made in the router, and having to type them in all again is a nightmare. 

Don't get me wrong, I like NG stuff, been using it for years but now, with lots of devices internet enabled around the house means that more people will be having many more devices. And the old days of manually reentering should no longer been expected of users.

So backups from a slightly older firmware version should ALWAYS work in the new firmware, in my view!

[IrvSp]

"So backups from a slightly older firmware version should ALWAYS work in the new firmware, in my view!"

I think the problem is the 'saved' settings? I don't know how they are structured nor where in the RAM it is saved and how? It is a 64KB file and pure hex basically. Is it read in and processed by a hex byte(s) and then written to a specific RAM location or just read and written consecutively a specific RAM location? There is 'source code' available, but I did get it once and it was hard to read, but I didn't try very hard either. I'm sure the information is in the source.

Even if it was location dependent, you'd still need to clear NVRAM in case something got moved? Like I said, I've not seen any firmware clear NVRAM as happened to you.

Did you try reloaded a saved configuration, just curious?

I have used both LinkSys and ASUS routers, and I don't recall having update problems either? Don't know the situation now though?

In any event, NG has been doing this 'forever' it seems. I doubt they will change... and with the proliferation of wireless IoT's this is becoming a larger issue than a few years ago. Maybe they will wake-up and handle this.

[antinode]

> A long time ago I took screenshots of all my pages I made changes to
> and keep them for reference just in case.

   Rather than take a picture of a web page, I'd ask the browser to save
the HTML (which I'd expect to be smaller, and from which copy+paste is
more practical).  But, either way, the display may omit useful
information.  For example, on an R7000 (V1.0.9.6_1.2.19), the ADVANCED >
Advanced Setup > Port Forwarding / Port Triggering report does not show
the protocol(s): TCP/UDP.

> I too wish the settings save were human readable. [...]

   For some models, the D7000[v1], for example, it is.  However, a
checksum may make it difficult to do any editing.

      https://community.netgear.com/t5/x/x/m-p/1376433
      https://community.netgear.com/t5/x/x/m-p/1395902

   Note that the D7000v1 firmware is sufficiently stagnant that over at
least seven firmware versions, I've never needed to reset anything after
a firmware update.  (Of course, multiple old bugs persist, too.)

[rowets]

What you are saying makes perfect sense.

The whole story was as follows;

1. I backed up the Settings

2. I applied the new firmware, which reset the router back to factory settings

3. Being happy that I'd just done a backup, I just set a couple of parameters, eg, the router pw and the network names and then restored from the backup that I had taken before the upgrade

4. When the router restarted after the restore, it had gone back to factory settings, and wiped the few settings that I had applied manually. None of the fixed ip addresses, static routes, port forwarding or anything I'd setup and was present in the backup was restored.

5. No matter how many times I tried, it did the same thing.

Taking a straight memory dump of the settings is a lazy way of doing a backup. In the past this may have been acceptable but not these days.

I never assumed it would be straight memory backup, i thought it was an encrypted file which is why i couldn't read it 

The settings should be dumped to say a JSON file or something so that they can be easily restored when new entries are added in the firmware

To get everything working took over 4 hours of manually inputting all of the data that I could remember I'd configured.

You don't get this with other devices, why should a router be different.

With my phone, I don't have to print out all the settings whenever there is an upgrade to the firmware/operating system, the settings are either preserved or previous backups can be restored.

This should be the same, preserve the data on upgrade or provide a mechanism to transition the data to the new firmware

As you say, with IOT, this is going to be more and more of an issue. This is certainly the reason that I have a complicated configuration (currently using Alexa, Hue, Smartthings, security cameras, the list goes on!)

Before IOT, I had a very simple configuration, but now it's really something that can't be easily manually input

Thanks for your reply, now i understand what NG have done

[rowets · Accepted solution]

So my accepted solution (NG want me to provide one), is to no longer use NG for saving lots of config as they can trash my data whenever they want and expect me to type it in again - not doing that!

Instead I'm using a Raspberry PI as a DHCP server and disabling this feature on the NG router.

That way I can properly back up the settings.

[IrvSp]

Using a different DHCP server you can backup is not the complete solution. There are many more settings (Wireless, LAN, Ports, USB, etc.) that need to be saved. Some of those like the ports could be extensive.

[rowets]

Exactly, but I have around 60 fixed ip addresses that I'm not going to type in again. 

The other settings I have, like port forwarding and the network names/passwords can be backed up using screenshots/pen & paper.

Whilst NG have a backup solution that is not reliable (which defeats the point of a back solution),  I need to look elsewhere to back up the fixed ip addresses.

It's not until people raise the issue that NG will look at the 'old' backup solution they provide and update it to modern day environments where fixed ips solve lots of prblems, eg, where people have Alexa, Hue, Smartthings etc

I can't find anywhere other than here to raise the issue!

[IrvSp]

---

@rowets wrote:

It's not until people raise the issue that NG will look at the 'old' backup solution they provide and update it to modern day environments where fixed ips solve lots of prblems, eg, where people have Alexa, Hue, Smartthings etc

I can't find anywhere other than here to raise the issue!

---

Well, here is not going to have that happen. Burried in the bottom of a problem a moderator might not even look at is not a good place. However, here is, https://community.netgear.com/t5/Idea-Exchange-For-Home/idb-p/idea-exchange-for-home. No guarantee that will work either but it is just possible someone who can take this up the ladder would.

The main problem in my estimation is the code itself and how created. I'd think each new router started with the old router code and followed that path. I've seen bad code and good code, and designs as well. In the cases here, I suspect it is where and how the data (settings) are stored. I suspect right after each other at a specific RAM location however that RAM location might/could change on every new release, hence the need to reset (clears NVRAM) and re-enter. Now if the new code in the firmware uses data area at the END, that might not be a problem. Somewhere inside of the old data for a location, trouble... and restoring old save configurations would just not work. Matter of fact, add even 1 byte to the executable code and if the data wasn't structured to a specific address but was directly after the executable code, trouble then would happen and a clear of NVRAM and a re-enter of all data required. Even saving in some 'readable' format might not work. First you might be able to edit it, but I'd hope it was check-summed so that couldn't happen. What happens when a new setting was added but the old settings didn't have it? Hopefully the code using it would use a default then, but who knows.

Not sure how other devices do it? Maybe they have dedicated RAM in the device so that the first thing the update would do is write the settings/data to it, install the new code, and then copy back the settings to the proper place? All part of the design.

Not sure what other vendors or 3rd party firmware creators do? If you look at http://192.168.1.1/debug.htm on your router you can see some data such as RAM used. The R8000 seems to have plenty of space left. At least mine does.

[rowets]

Thank you so, so much for the URL, the only places that I could find always required me to pay money to NG so I could contact them (outside of the 90 days etc).

I totally get what you are saying about even adding 1 byte to the size of the executable would screw up the backup feature between firmware versions.

With the product I work on, settings are saved and restored using JSON (something else could work equally well). The restore process is more intelligent and picks out the fields that the code knows about.

If a new field is added with a new code drop, the new code will not find the new field in the backup when doing a restore from old versions, so a sensible defualt is used instead.

This way, all old backups will work on new code versions, and any new values have sensible defaults.

This is a fairly simple pattern, and with all the clever things that the router already does, this should be childs play to implement.

Thanks again for your reply

[IrvSp]

---

@rowets wrote:

This is a fairly simple pattern, and with all the clever things that the router already does, this should be childs play to implement.

---

I don't think it would be 'that simple'. Maybe for brand new routers,  but not for older ones. They'd have to 'test' the old save file first, and it can be from any f/w version. Also you assume the folks doing the coding are able to do this. If a 3rd party it might be a function of how much they get paid? These folks can't 'code right' it seems. Always something breaking. Now there are complaints about Smart Connect, and others have some problems. Old problems seem to drag on and/or get worse.

There are 3 parts that are the problem here. Project Management seems to be a big part of it. They don't seem to have a handle on all the problems that exist. Support that does get the problem reports. Do they actually tell the code people and project management what needs to be fixed and HOW they recreated it? Lastly the coders. Are they competent? Can they fix the code AND verify it was fixed? I don't think so. And of course one major problem is NG's Project Management doesn't see to be 'here', nor true support people. No one seems to pickup problems, and when they do see one, they 'endorse' you to Support and open a case for you. A lot of good that has done me, I've got one on SMB 1 needing to be used to see shares (security exposue using SMB 1) since 3/31/2018. I've demonstrated the problem with screen captures and detailed info to recreate. They have CLOSED it at least 4 times on me. Last time I was communicating directly with the Taiwan support team and at one point 3 different ID's were on my PC. They saw the problem and the main contact told me he doesn't understand why SMB 1 is required. They then closed it again. I've contacted forum monitors and they are working to re-open it... Confidence I have of this being fixed, almost ZERO...

[rowets]

...well in my experience backups taken before V1.4.1.30_1.2.26 don't work with V1.4.1.30_1.2.26 anyway, so maybe now would be a good time to implement an improved back/restore process.

Interesting the other points you raise re Project Management etc. I've worked for big Teleco companies and what you say is so true. I've seen them focusing just on 'the bottom line sales' and outsourcing everything they can - and overall the product looses it's quality due to the high staff turnover at the outsource company. The outsource staff are always chasing the bigger salaries so they never hang around too long - the staff have told me this is simply their culture!

So as an end customer, you are dealing with a company whose staff are mainly outsorced/contractors who don't care.

This is what I'm starting to see from a customer perspective with NG

I'll add a post to the suggestions area, and thanks for the heads up on anything happening!