Customising the firmware for the DGND3700v2

[neil_townsend]

So, it is possible to compile the stock firmware for the DGND3700v2 and have it working on the device. (See here and here.)

I would love to modify it, and there are lots of things that are possible. However, it makes sese to (a) start small and (b) use the existing framework as much as possible.

The interaction between the webpages and the system appears to be primarily done by an executable, setup.cgi, in /usr/sbin. The challenge is that I can't find any source code for this execuatble or any hints about how it works. So if I want to add an etra parameter (say an extra ADSL config option or a different DDNS provider), there is no obvious place to break in.

Does anyone know where to get the source code for setup.cgi or have any ideas about how the system retains config, applies it etc?

[cilynx]

Hi Neil --

I'm working on one of these for a buddy of mine -- specifically, he'd like it to support local DNS, which should be an easy enough addition if we can reverse-engineer the existing management system.  I don't have any insight into setup.cgi yet.  Binwalk doesn't tell me much other than that there's some HTML files stuffed into the executable. 

I did put together a little Makefile that pretty cleanly orchestrates both the toolchain and the image build:

https://github.com/cilynx/DGND3700v2

Have you made any headway reverse engineering the management system?  I just ordered a used one of these things off of eBay, so I have one to brick / take apart w/o screwing over my buddy.  Once it shows up, I'll start getting deeper into it.  Besides augmenting the default firmware, I'm interested to see if we can get OpenWRT working on these things as well. 

[neil_townsend]

Hi,

    Great to hear from you. I've made quite a bit of progress:

 - it has a modified minihttp which takes every request and turns it into a cgi request which is passed on to setup.cgi

 - that does everything: it saves any updates, triggers actions and returns the next web page.

 - the webpages are stored in standard harmless with added variables which setup.cgi renders when sending the page out.

 - all of the process of note are effectively controlled by /usr/sbin/rc_apps, accessed via a number of symbolic links.

    What I have done:

 - written a bit of code to intervene between minihttp and setup.cgi which can be configured to catch and handle a subset of requests.

 - replaced the dynamic dns with a different solution.

 - provided much more adsl control

 - working on proper port forwarding

 - working on improved DNS and DHCP

     I will put up what I've done so far on GitHub and post a link, very happy to collaborate. If we could get openwrt that would be great, but that may be beyond my capabilities.

[cilynx]

Your progress sounds outstanding.  You might even have already solved the problem I'm hoping to solve.  As for OpenWRT, I've already ported OpenWRT to one unsupported router (https://github.com/cilynx/Candyhouse-Linux) and being that very similar units to the DGND3700v2 are already supported, I think it's totally doable.  From some googling around, it looks like the primary hurdles are going to be the NAND flash (GPL drivers exist, but are not available in the OpenWRT build tree) and the ADSL modem (which I know nothing about, but is obviously supported in the Netgear firmware).

Looking forward to seeing and learning from your work!

[neil_townsend]

OK, my love hate relationship with git continues. Somehow I nearly lost have my mods in the process of putting them in a repo. However, got them back.

I've put everything I've done so far in:

https://github.com/ntadmin/DGND3700v2

It would seem sensible to emrge your Makefile with the top level one there, but I wanted you to get a look at the (slightly arcane) approach I've taken - it's all a bit 'chicken and egg' and 'let's beat up a Makefile because we can' to be considered good and maintainable code. I've put some headline information in the top README.

The really brutally horrible bit is the parsing syntax I've put in place to describe the links between the variables in the .htm files and the variables in nvram which they read from (and write to). It works, but it's not pretty.

[neil_townsend]

ps - not obvious from the code: you can remove noip from the Netgear compile list (Sources/apps/Makefile)

[neil_townsend]

>  it looks like the primary hurdles are going to be the NAND flash

Not sure exactly what this means, as this is the first time I've ever done this kind of thing. However, if it helps, the Netgear source comes with some code for accessing the NVRAM variable store. That source results in a library and an executable for doing that. The code may have hints for doing more than that:

    <Netgear base>/Source/apps/nvram

[neil_townsend]

> the ADSL modem (which I know nothing about, but is obviously supported in the Netgear firmware).

A couple things I've spottted so far.

• In the target.tar.bz2 root filesystem that it uses as a base, /usr/etc/adsl contains a file called adsl_phy.bin. I suspect that this is key adsl code, not sure how to find out.
• As is fairly normal from my lmited investigations, adsl config is controlled by /usr/sbin/[a,x]dslct

There may also be some information in richud's work on the v1 (http://www.richud.com/wiki/DGND3700_V1_Transmission_Firmware) - although a different DSL chipset.

[neil_townsend]

If you've found this thread and want the headlines:

1. Close to a working first version with a few changes (DNS and DHCP migrated to dnsmasq; Good ADSL control; Better DDNS support and proper port-forwarding). The code is at https://github.com/ntadmin/DGND3700v2,

2. The bigger dream would be to get openWRT working on it; @cilynx has suggested that it could bedone using a similar approach to the one here: https://github.com/cilynx/Candyhouse-Linux 

[cilynx]

My eBay router is out for delivery, so I should be able to start playing with things over the weekend.  I'll fork your repo and send you a PR to get us on the same page.

[neil_townsend]

Cool - I'm pretty busy Fri-Sun, so please don';t take lack of response as lack of interst. Wouyld ge great tpo get on the same page. I've put some work into the repo, I'm hoping that, combined with your makefile, it provides a working 'build from scratch' improved firmware. The issues and the README in the top directory give a pretty good summary of the state of play.

[neil_townsend]

Quite a few more updates done to the repo, the usual tweak when you go from 'it works on my device' to 'does ti work if I don't hold its hand ...'

[neil_townsend]

First beta release now available: https://github.com/ntadmin/DGND3700v2/releases/tag/NEWT16B

[cilynx]

Awesome work, Neil.  I've been putting some effort into simplifying and streamlining updates to the stock firmware.  Basically, I'm looking to make the changes that I want to make with as little impact to the rest of the firmware as possible.  I have dnsmasq in place and happy now.  Next steps are to rip out the useless dnrd and udhcpd apps, then build a fully-featured busybox to make life easier and dropbear for modern access.  I've tested dropbear and it does work in general.  I need to build a newer version of busybox to get base64 support so I can store the host key in nvram to persist over reboots.  However, it's now after 4am and I have to work tomorrow.

Cheers!

https://github.com/cilynx/DGND3700v2/releases/tag/0.1

[neil_townsend]

Likewise, great work! Sounds like we are on very similar paths, I'm just slightly keener to add to the web interface. I am also very keen to get an up to date busybox in place with dropbox, once I've got ad-blocking and proper port forwarding working. However, if you crack them first, do ytou mind if I simply integrate your work into my github?

[cilynx]

I tried updated busybox to 1.19 to pick up the base64 applet, but something went wrong and bricked my test router.  None of the usual tricks worked, so I had to cobble on a serial port, spin up sc_tftp and reflash back to a good state from there.  Instead of digging too far into what broke with 1.19, I decided to focus on what the netgear-shipped 1.13 could do for me if reconfigured.  Well, it turns out 1.13 has uuencode/uudecode, which works for my needs.  I just published a new 0.2 release which has dropbear working out-of-the-box.

As for sharing code, please do take and reuse anything you find valuable.  I'm definitely basing my work off of yours, so there's no reason for you not to do the same.  The busybox and dropbear configuration tweaks are both in the Makefile, and you'll probably find rc_dropbear valuable as well, particularly the nvram / uuencode handling.

[neil_townsend]

For busybox you'll need at least this patch:

http://lists.busybox.net/pipermail/busybox/attachments/20151110/ce523411/attachment.obj

[neil_townsend]

Didn't spot your comment about bricking - the patch I noted is a compilation issue. I'm putting 1.24.2 onto the router to test it (over the next couple of days) but as '/usr/sbin/busybox2' to see if it seems to run before putting it on as the main busybox. Out of interest, did the trick here not work to recover the bricked router?

[cilynx]

When it bricked, it stopped using 192.168.0.1 in any way.  If I just turned it on, the power led alternated between red and amber -- ~7 seconds red, ~12 seconds amber.  At the moment it turned red, it would answer a ping or two on 192.168.1.1, but would never respond to either http or tftp no matter how it was timed.  If I held down the reset button while booting, it would alternate between the power button lit up red and one of the link buttons lit up green every second and in that mode, I couldn't get it to respond to anything on any address.  TFTP worked as expected once I started 'sc_tftp y' from the CFE.

[neil_townsend]

Wow! Seriously bricked. Impressive. Impressive recovery as well.

[neil_townsend]

@cilynx Been looking at your dropbear additions, and I had a couple of questions:

 - Looking at the code, does it actually store and retrieve or does it end up making a new key each boot?

 - How do you handle creating a user to login to? telnetd has been 'configured' (by modifing the busybox code for /usr/sbin/login) to use /etc/htpasswd as the user/password list. How did you handle the lack of a user-password in /etc/passwd?

Thanks,

[cilynx]

If the key file is in /etc/dropbear, the start function uses that. If there is no key file (/etc is reset to stock on reboot), it looks in nvram and uudecodes the key file if it's there. If it doesn't find the key in nvram, it generates a new one. The stop function uuencodes the key file and saves it in nvram.

If you look in the dropbear_configured target in the Makefile, you'll see where I add the root password to /usr/etc/passwd (which becomes /etc/passwd). To set your own password and have it stick over reboot, the rc_dropbear would need to proxy /etc/passwd in nvram, using the same method as the key file.

[neil_townsend]

Cool, thanks, got it. I'll look at seeing if I can mod /etc/passwd on the fly so that if the user updates the password from the web interface it changes in sync ...

[cilynx]

Turns out Netgear already stores the web admin password in plaintext (security ftw!) in nvram.  I added chpasswd to busybox and a couple lines to rc_dropbear and cut DGND3700v2-cilynx-0.3.img.

[neil_townsend]

Good work! Couple of things that link to this and a previous discussion:

1. The plaintext password is in /etc/htpasswd for all to see, you don't even need to access nvram.
2. The busybox code for /usr/sbin/login is modded in the Netgear distribution to read this file for login authentication rather than /etc/passwd. I wonder what else they have modded in busybox which might explain the issue you had changing the version ...