The web server should run out of xinetd, and other issues
The default web server in openwrt is small, but it is single threaded, and thus unsuitable as a general purpose web server. After evaluating apache (too big), ngnx (too unfamiliar), and lighttpd, I went with lighttpd as the default web server for both the on-board documentation and the configuation interface.
This has several advantages (fully threaded web server, http 1.1 support, well supported in the field, flv streaming, etc), and a few disadvantages that can be addressed in the next release.
0) To get better privilege separation, I ran the outward facing webserver under a non-root uid. The configuration web server however, runs as root, and it would be better to merely have the luci cgi it talks to having those privs, and run luci via fastcgi. This would also allow having only one - non-root-priv web server running, thus saving memory there as well.
1) As the router's web servers are unused 99.99% of the time, they could be fired off via xinetd, and expire after a few minutes. Lighttpd has an old 'wontfix' bug for inetd support, here:
http://redmine.lighttpd.net/issues/1584 - but adding a 'expire idle time' variable and related support for inetd appears to be fairly easy.
2) Somewhat related to the above, lighttpd wakes up once a second for no good reason.
I'm not married to lighttpd, but it does work well even on this limited device, and scales enormously.