Welcome  ·  Travel  ·  Mobility  ·  Living  ·  Computer  ·  Art  ·  Spirituality


Computer
Computer Diary


    "Yes, we can."

    Barack Obama
    4th November 2008, Chicago, USA

    Here my little rant and praise place, where the daily experiences of my programming work are expressed. I publish them with the idea that others might find it useful and benefit from it.

    Tag <GeoLite>

    Check also other posts with other tags.

    2009/02/23
    SQL vs GREP with 230K lines (12MB) GeoLite
    last edited 2009/04/23 06:17

    I like to index all my texts (articles, emails) according 'geonames', a database of locations, for that purpose I found a CSV from MaxMind.com: GeoliteCity , and started to create a database with DBD::SQLite, and finally made this comparison.

    The dataset is 12MB, with 235,000 lines:

    % wc GeoLiteCity-Location.csv 
      235422  277043 12450133 GeoLiteCity-Location.csv
    

    % time grep \"Marseille\" GeoLiteCity-Location.csv
    49739,"FR","B8","Marseille","",43.3000,5.4000,,
    0.045u 0.090s 0:00.13 100.0%    105+1040k 0+0io 0pf+0w
    

    vs perl with DBD::SQLite, whereas the geonames.db is 14MB in size, and used in a script with

    select city,long,lat from cities where city == 'New York'
    

    where as an index was made for city column, then the command line call:

    % time ./geotag Marseille
    Marseille:
            Marseille, B8, FR: 43.3, 5.4
    0.271u 0.062s 0:00.33 100.0%    10+2054k 0+0io 0pf+0w
    

    The computing time is grep 0.090 vs sql 0.062, which is 1/3 faster, but user time was 6 times longer, which is explainable by the overhead to load perl and the required modules.

    Update: Geonames.org seems to have better data with aliases of city names, I used it for a small tool named "geotag", see my post.




    Check also other posts with other tags.

     Topics
  • Kubuntu (4)
  • Lighttpd (3)
  • MacOSX (3)
  • UNIX (3)
  • AdBlock Plus (2)
  • Dell (2)
  • Firefox (2)
  • Geonames (2)
  • MaraDNS (2)
  • Perl (2)
  • Sqlite (2)
  • VirtualBox.org (2)
  • !Flickr (1)
  • !GoogleMaps (1)
  • !GPS (1)
  • Advertisement (1)
  • AFP (1)
  • Amrok (1)
  • Android (1)
  • Apache2 (1)
  • App Store (1)
  • Benchmark (1)
  • Beta Software (1)
  • Big Brother (1)
  • Bind9 (1)
  • Blackberry (1)
  • Bloatware (1)
  • Catch 22 (1)
  • Cellphone (1)
  • Cellphone Networks (1)
  • CGI (1)
  • Crap (1)
  • Cygwin (1)
  • DNS (1)
  • Donations (1)
  • DVD/HDD Recorder (1)
  • Ethics (1)
  • Eye Candy (1)
  • FastCGI (1)
  • GeoLite (1)
  • Geotagging (1)
  • GMail (1)
  • Google (1)
  • Grep (1)
  • Hassle (1)
  • Heuristics (1)
  • iDeneb (1)
  • IMAP (1)
  • Insanity (1)
  • Internet Explorer (1)
  • iPhone (1)
  • iTunes Alike (1)
  • JavaScript (1)
  • Keyboard Mapping (1)
  • Laptop (1)
  • Linux (1)
  • Location Lookup (1)
  • MacOS-X (1)
  • MacPorts.org (1)
  • Medion Life (1)
  • Memory Waste (1)
  • Micropayment (1)
  • Not Ready (1)
  • NSD (1)
  • NTFS (1)
  • Objective-C (1)
  • Old Laptop (1)
  • OpenStreetMap (1)
  • Opera (1)
  • Paypal (1)
  • PhoneGap (1)
  • QuickConnect (1)
  • Review (1)
  • Server Farm (1)
  • Servers (1)
  • Thieves (1)
  • Thunderbird (1)
  • Useability (1)
  • VMWare Fusion (1)
  • WebServer (1)
  • Windows XP (1)

  • .:.


    Welcome  ·  Travel  ·  Mobility  ·  Living  ·  Computer  ·  Art  ·  Spirituality



    Copyright 2007, 2008, 2009, 2010 © by René K. Müller <spiritdude@gmail.com>
    Illustrations and graphics made with Inkscape, GIMP and Tgif