- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
NVRAM bug - FOUND IT!!
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
NVRAM bug - FOUND IT!!
I have posted for some time about various issues with corrupt config parameters (NVRAM). If nothing else, I am a really stubborn geek with 50+ years of experience and I don't like unexplained failures. Computing should be a predictable thing!! I wasn't going to let this bug beat me.
Long story short - the script below shows a 100% failure that corrupts NVRAM. The issue is a software error (maybe hardware assisted) if the size of NVRAM goes over 0xFFFF. Not clear if this is the max physical size of NVRAM or a software bug with 16-bit ariithmatic but, either way, it is a serious software bug. You might say "Why do I care? My config can't ever be that big!" You would be surprised. In addition to normal information, NVRAM also contains information about every device your router sees. There is no delete/garbage collection. NVRAM grows to that size and then you have a problem but don't even know it. Eventually, the corruption will cause a noticeable problem but that may be a long time after the actual failure event or when you try to reload your config after a disaster.
NG has open source for some of the Orbi software but this part is proprietary so I can't work on a fix. (would be happy to do that for FREE if NG would provide the source).
100% failure scrupt follows. Exist in every version of firmware I have tried.
Scott
--------
= PuTTY log 2021.10.22 11:20:37 =~=~=~=~=~=~=~=~=~=~=~=
< load my production NVRAM>
root@RBR50:/# nvram show | grep size:
size: 65364 bytes (168108 left)
root@RBR50:/# nvram set junk1=1234567890
root@RBR50:/# nvram show | grep size:
size: 65381 bytes (168091 left)
root@RBR50:/# nvram set junk2=1234567890
root@RBR50:/# nvram set junk3=1234567890
root@RBR50:/# nvram set junk4=1234567890
root@RBR50:/# nvram set junk5=1234567890
root@RBR50:/# nvram show | grep size:
size: 65449 bytes (168023 left)
root@RBR50:/# nvram set junk6=1234567890
root@RBR50:/# nvram show | grep size:
size: 65466 bytes (168006 left)
root@RBR50:/# nvram set junk7=1234567890
root@RBR50:/# nvram show | grep size:
size: 65483 bytes (167989 left)
root@RBR50:/# nvram set junk8=1234567890
root@RBR50:/# nvram show | grep size:
size: 65500 bytes (167972 left)
root@RBR50:/# nvram set junk9=12345
root@RBR50:/# nvram show | grep size:
size: 65513 bytes (167959 left) <size = 0xFFE9, OK>
root@RBR50:/# nvram set junk10=12345
root@RBR50:/# nvram show | grep size:
size: 12 bytes (233460 left) <NVRAM toasted>
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
I would contact a forum moderator and get this information to them:
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
Congratulations. My Orbi matches your exactly used plus free = 233,472. Looks like a classic case of using an 'int' where it should be a 'long int'.
Is this a V1 or V2 Orbi?
How many devices are represented in the Orbi tables (LAN reservation, previous but not connected, etc.?)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
Tested on V1.
54 active connected devices and 56 items in the previous but not connected per WebUI.. Looks like there are 198 items in the NVRAM "access_control" table.
Scott
@CrimpOn wrote:Congratulations. My Orbi matches your exactly used plus free = 233,472. Looks like a classic case of using an 'int' where it should be a 'long int'.
Is this a V1 or V2 Orbi?
How many devices are represented in the Orbi tables (LAN reservation, previous but not connected, etc.?)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
@FURRYe38 wrote:I would contact a forum moderator and get this information to them:
@Blanca_O @DarrenM @ChristineT
Done.
Scott
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
@SLK-Purdue wrote:
54 active connected devices and 56 items in the previous but not connected per WebUI.. Looks like there are 198 items in the NVRAM "access_control" table.
My access_control list has 77 items in it. Looking at the Access Control web page
37 Active devices
36 Allowed but not connected
1 Blocked but not connected
----
74 Total. I think the missing 3 are three entries for ETH0. 74+3 = 77.
I would expect you to have 54+56+3 = 113, not 198.
I do not have Access Control enabled, so there is no particular difficulty in deleting "allowed but not connected' devices because they will simply authenticate the next time they show up. But, where did those extra 85 entries in your system come from?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
My bad - access_control table has 105 entries. Counting error. I think that matches current totals as you did.
Thanks,
Scott
@CrimpOn wrote:
@SLK-Purdue wrote:54 active connected devices and 56 items in the previous but not connected per WebUI.. Looks like there are 198 items in the NVRAM "access_control" table.
My access_control list has 77 items in it. Looking at the Access Control web page
37 Active devices
36 Allowed but not connected
1 Blocked but not connected
----
74 Total. I think the missing 3 are three entries for ETH0. 74+3 = 77.
I would expect you to have 54+56+3 = 113, not 198.
I do not have Access Control enabled, so there is no particular difficulty in deleting "allowed but not connected' devices because they will simply authenticate the next time they show up. But, where did those extra 85 entries in your system come from?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
My ARM assembly language skills aren't as good as some other hardware architectures but, based on a disassembly of NVRAM, it sure looks like that code has a hard coded size constant of 0x10000. That would explain the bug. No clue on the actually allocated size of NVRAM in the EEPROM but I am guessing the buffer size is the real issue. Would likely be an easy fix if NVRAM source code were part of the NG "open source" but, sadly, it isn't.
NG - why keep this a secret? There is nothing in using EEPROM for NVRAM that is your secret intellectual property. Everybody does that. I understand not wanting to release the secret sauce behind for mesh algorithm but that not in reading and writing NVRAM.
AGAIN - this is a VERY serious problem for all users. Send me the source code and I will fix this FOR FREE and send you back code and give you all rights. You're getting a consultant with 50+ years of systems experience that bills at $500/hr when I work for money.
Scott
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
Wondering if this issue is recent or how far it goes back in FW versions...
Wondering if also this is only or Orbi AC or does this effect more products globally.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
@FURRYe38 wrote:Wondering if this issue is recent or how far it goes back in FW versions...
Wondering if also this is only or Orbi AC or does this effect more products globally.
If this were part of the open source code, we would know. Sadly, it isn't
Since this is not something that would likely be introduced in an update, I would guess bases on experience that this is a day 1 bug.
Scott
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
Well maybe however the NVRAM and configuration section maybe something that is used across multiple product lines. Similar code base can be used across different products. Like the UI, basically the same format and feature listings, though some areas change per product lines. Hopefully this is only effected Orbi AC then.
Hopefully NG will take a look and get it corrected. Thanks for your deep dive man. Wish NG had you on as a dev.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: NVRAM bug - FOUND IT!!
@FURRYe38 wrote:Well maybe however the NVRAM and configuration section maybe something that is used across multiple product lines. Similar code base can be used across different products. Like the UI, basically the same format and feature listings, though some areas change per product lines. Hopefully this is only effected Orbi AC then.
Hopefully NG will take a look and get it corrected. Thanks for your deep dive man. Wish NG had you on as a dev.
Been pretty clear that I am available and the price is right.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
• What is the difference between WiFi 6 and WiFi 7?
• Yes! WiFi 7 is backwards compatible with other Wifi devices? Learn more