"God is not external to anyone, but is present with all things, though they are ignorant that he is so."
Here my little rant and praise place, where the daily experiences of my programming work are expressed.
I publish them with the idea that others might find it useful and benefit from it.
Check also other posts
with other tags.
Windows XP for a UNIX Man
last edited 2009/10/30 11:34 (*
Dell Latitude C600 Laptop
I got an older Dell Latitude C600
(2003), with 1440x1050 pixels large 14.1" LCD screen, still functional, but 256MB RAM only (upgraded to 384MB). Kubuntu 9.04 didn't work on it, the screen was unreadable, the drivers for the ATI Mobility M3 Video Accelerator
seem not working for the rather odd resolution (update: found HOW TO: Fix video problem on the Dell Latitude C600/C500
). In order to still use the machine, I installed Windows XP SP2 - for me as Open Source evangelist a difficult task compromising my life long attitude to avoid Windows by Microsoft.
Here are the most useful packages to make Windows XP useable:
- Cygwin.org , most UNIX commands natively compiled like useful programs: screen, ssh, scp, tar etc.
- Terminator, descent terminal emulator, requires python and ruby from Cygwin (dropped its use and rather run sshd and use putty)
- VirtualWin , virtual screens for Windows XP (alternatively also Dexpot.de , or parts of XP Power Tools )
- VirtualBox.org , virtual machine, installing Ubuntu Mini distribution on it to have a pure UNIX
- and the usual programs: Mozilla Firefox , Mozilla Thunderbird , Opera.com , OpenOffice.org , GIMP,org , Inkscape.org
- SMplayer, plays DVDs and most video formats
- Putty, ssh client
- Skype.com , to voice/video chat
- Flash Plugin, to view streamed video in webpages
Windows XP on a Dell Latitude C600 (1440x1050)
Since this laptop has only USB 1.1, the 1TB USB HD My Book Essential
(CHF 100, MediaMarkt.ch 2009/9) is slow, about 200-500KB/s transfer rate. The USB webcam Genius Eye 312
(CHF 20, MediaMarkt.ch 2009/9) works, but the USB wireless adapter Netgear WG111v3
(CHF 30, MediaMarkt.ch 2009/9) isn't working properly, every 30Min I'm cut off, even the access point is close by and good signal strength is indicated - the typical low quality of network drivers from all the "cheap" adapters like Netgear.
Anyway, I try to make the best out of the situation to use Windows XP
with this machine:
- 1GHz Pentium III
- 384MB RAM (256MB+128MB)
- IDE 20GB local disk
- USB 1TB external disk
- USB 54Mbps wifi adapter
- USB Webcam 640x480/30fps
total cost CHF 150 (apprx. Euro 100), plus a laptop bag (CHF 30, MediaMarkt.ch 2009/9)
I have to say, there are a few things which please me to use XP:
- USB devices can be plugged in, and icons show up, and I can explore the content, like for USB sticks and other devices, hassle free.
- Webcam works right away with Skype
- GUI is responsive, basic features are intuitive
- Easy to change fonts to be antialiasing
- Widerange of apps available, but I'm rather reluctant to download from unknown companies or individuals
- Rebooting the machine, and often low resolution of 1024x768 shows up, all desktop icons are rearranged - and have to change resolution again to 1440x1050 and arrange the desktop icons again
- Annoying "Auto Play" feature for new USB devices, even I turned it off it's ignored
- Unstable wireless, very annoying
- I tried to NAT between the wifi and the ethernet LAN, followed the tutorials but still didn't work, little debugging information (no /var/log/*)
- External USB disk turns off when machine goes into sleep/hibernation (which is good), when it awakes, the USB disk is considered newly plugged in
Western Digital 1TB My Book Essential
I use the laptop for an off-grid setting where I will have limited electricity, like from a small solar panel, and therefore an older laptop is just right for this application.
The 1TB My Book Essential from Western Digital for just roughly CHF 100 / Euro 68 was a good option to carry my data of the past 20 years with me, formated under XP with NTFS (r/w also under Linux and MacOS-X) - but there was no way to copy all my data via USB 1.1 and 200-500KB/s only. So I attached it to a desktop with Kubuntu 9.04 with USB 2.0 ports, and it worked right away (ntfsprogs was already installed), after plugin in the /dev/sdb was listed, and I mounted it via
mkdir /mnt/mybook1tb/; mount /dev/sdb1/ /mnt/mybook1tb/
and then I started to copy all my data from the server which exports all data to the desktop via NFS, on the USB disk using piped tar like this
tar cf - dir1 dir2 ... | (cd /mnt/mybook1tb/; tar xfv -)
and this copy session I started within a screen session.
The transfer rate is about 10GB/h or 2.7MB/s, which is about 4-10x faster than with USB 1.1; so the transfer of 900GB takes 90hrs or 3.75 days.
Since I still do pretty much web development, and switched from Apache to Lighttpd I was pleased to see lighttpd ported to Cygwin as well.
Now I needed a small dns server to resolve my local web-sites like
http://test.local, and MaraDNS.org compiled just fine.
Note: the loopback device 127.0.0.1 isn't up by default, you actually need to install it as a device; e.g. when you are off-line without any network connection this might be needed so your local DNS is considered, as there is no way to define a default DNS lookup at Windows XP like at UNIX with /etc/resolv.conf - when you have a network connection, then the DNS entries of that network is considered.
I run my own little CMS as a Perl CGI and it required a module Image::Size, which I installed like this
perl -MCPAN -e 'install Image::Size'
the first attempt failed, as it was quite memory hungry to fetch and compile it, and the 'child_fork' emulation using longjmp failed; after quitting some other apps it worked, and the module was in. Impressive to see Perl install compiled modules on a Non-UNIX system like Windows XP with Cygwin; needless to say this is also due the GCC which is so widerange supported.
X11 is also ported, and in order to use X11 apps mixed with native XP apps, start
X -multiwindow &
and export DISPLAY=:0.0
so other apps know which display to use.
- Most UNIX commands using Cygwin incl. xterm
- Local DNS using MaraDNS.org , locally compiled
- Lightweight webserver using Lighttpd.net , precompiled in Cygwin
Install the init.d structure from the Cygwin package, and adding in /etc/rc.d/rc.local a few lines to start up DNS, HTTPD and X11.
Chicken Egg problem caught me, I was using a bare laptop, and tried to get Kubuntu on it, failed at the first attempt; installed XP and installed the stuff I need, and discovered there is a solution for the X11 config. After a few hours I concluded following:
- X11 with 1400x1050 works, dedicated xorg.conf required (probing doesn't work)
- Webcam works under Linux with a new driver and dedicated compiling
- Netgear Wifi works under Linux with a new driver and dedicated compiling (see solution )
which is the "usual" additional work to get peripherie working, also something I'm not willing to invest anymore.
I tried Kubuntu 9.04 on various systems, and I'm rather disappointed of the unfinished piece of software, and I'm tired to configure simple networking by editing 2-3 files, instead by a handy one page GUI. So, I wait til Kubuntu 9.x has matured, and give it a try with a Live System, and then if I get full resolution, and decent GUI for network administration incl. WPA Wifi settings, I gonna switch again to Linux.
While trying to resolve the annoying wifi connectivity loss, I discovered that it did matter a lot of the orientation of the USB wifi adapter: I attached the stick with "klett" on the back of the laptop screen, first vertically, and then turned it horizontally and the signal strength of the access point increased significantly. I also discovered my older Netgear WG602v2 AP was much better than the brand new D-Link DIR-301, very odd - but what remained was occasionally wifi network drops and reconnection, definitely Netgear WG111v3 network driver issue.
Rebirth of FastCGI
last edited 2009/04/16 08:57 (*
Since I switched from Apache.org
, as mentioned in How To Save 300MB RAM
, I got to an old buddy I almost forgot: FastCGI.
exit(); # probably never reached
It's an internal (tcp or named socket) based sub-server process which handles the requests. Why should this be anything better than Apache with mod_php or mod_perl?
Here some numbers, Intel P4 2.4GHz with linux-2.6.27-7-generic, running lighttpd/1.4.19:
- perl-cgi: 18 requests/s or 55ms / request
- perl-fcgi (via named socket): 240 requests/s or 4.1ms / request
which makes about 13x speedup! It can be explained that the fcgi perl is already started and immediately handles the request - whereas a pure cgi has to load perl and then the cgi code as well.
In order to use fcgi with lighttpd see at ModFastCGI , be sure you don't use the lighty-enable-mod as it only works for php, enable it by hand in your lighttpd.conf:
- add "mod_fastcgi" into server.modules = ()
- add fastcgi.server definition within your server definition
I coded a fcgi test so after 100 or 1000 requests the while() loop would be exited - but it caused a few "500 internal error" at the client end: 1 error in 10,000-20,000 requests. In other words, to quit a fcgi causes internal errors within lighttpd for a busy web-site (@ 200 requests/s). Just a few errors you might say, but if you run something important I recommend you rather give /etc/init.d/lighttpd restart instead quitting the fcgi itself. Why? I do like to reset the fcgi because nobody is perfect, and perl might have or my fcgi code have memory leaks, and if you let it run for days and millions of requests the smallest memory leak will show up - that's why I personally prefer to reset a fcgi once a day.
See also FastCGI.com
How To Save 300MB RAM
last edited 2009/04/15 18:12 (*
Also known as "Goodbye Apache & Bind, Welcome Lighttpd & NSD" (GABWLN)
I started to develop webpages at the time when Gopher was available, and Mosaic as first graphical browser was released.
At the time the NCSA web-server was 'patched' and so the Apache.org took off. I developed most of
my sites either using perl CGIs or mod_perl via handler() and some sophisticated converters to deliver
HTML pages at the end. In other words, I was hoocked up to Apache early and I used it intensively for
almost everything done for the web, no other options considered - but there is always a first time so to speak,
this is happening now.
The memory usage has been a concern for me the last years - in a nutshell: I kept using a 1GHz Celeron with 1GB RAM for the last 4 years,
and noticed with every upgrade doing the same tasks, like browsing the web and programming, required up-to-date programs which themselves
became more bloated, used memory and became slower . . . and nobody complained. How so? Most people upgrade to faster machine to do the same
tasks with more bloated software having the same user experience in regards of responsiveness - the programs aren't faster, they are actually
getting slower, just the machines we use get faster - not a good development. Anywyay.
I had a bit outdated Linux/Debian based web-server with Apache 1.3.x and finally had to upgrade to Apache 2.x and
so the memory footprint was increased significantly, over 190MB for the first process, yet, the subprocesses much less, but still.
Anyway, I was looking for an alternative, and I found Lighttpd.net , and now I'm dropping Apache.org . . . goodbye.
Lighttpd is a single process application, using 5MB memory - it supports CGI, even FastCGI (which I will look at again),
virtual hosts, URL rewrite, and other "must-have" features I require - and I made a few tests, and most features I do need are
there. So, I give it a try on my main server and see how it behaves.
Further, I noticed named (bind9) and nscd, its dns cache, using up 80-90MB alone - incredible, just being a primary for a couple of
domains I run? This is inacceptable - so I was looking for an alternative, and I found NSD .
I was able to use most zone-files from bind, so the switch was roughly done quickly - the nsd uses 7MB memory . . .
Update 2009/03/11: under Debian/(K)Ubuntu, package 'nsd' is version 2, in order to use up-to-date nsd version 3, install package nsd3 - in case you wonder why
certain settings aren't possible
Real progress for me is: do more with less.
As said, unfortunately people do not realize the majority of programs, like web-browsers or text-processors,
are using more memory, and are slower - and just due the upgrade to new hardware with faster CPUs and more memory this isn't
noticed. A good example for such backwardness in software development is Microsoft with Vista, and it deserved to fail so miserable on the
market - it was a good sign people didn't buy into advertising but paid close attention to the unbiased reviews.
Update: If require to have recursive name resolving, e.g. serving a local domain plus resolving public domains, you may choose MaraDNS.org as NSD doesn't resolve recursively (yet).
Check also other posts with other tags.