cellhunter - openbmap - opencellid revisited... (was Re: CellHunter is moving to a new server)

[Alex (Maxious) Sadleir]

On Sun, Sep 6, 2009 at 7:55 PM, Risto H. Kurppa<risto at kurppa.fi> wrote:

>
> Ok, downloaded the cellhunter database, this is what it stores:
>
> provider        cell_mcc        cell_mnc        cell_la cell_id cell_arfcn      signal  gps_time        gps_lat gps_long        gps_alt gname   local_time      cell_type
> IL ORANGE       425     1       3AFC    A2F9    92      19      1252151461      32.3668973      34.8627705      17.44   Baruch  1252185642      old_oldgps_near
> IL ORANGE       425     1       3AFC    7D54    630     6       1252151461      32.3668973      34.8627705      17.44   Baruch  1252185370      old_oldgps_near
> IL ORANGE       425     1       1D60    79CD    107     17      1252151461      32.3668973      34.8627705      17.44   Baruch  1252185368      new_gps
>
> altitude is stored, good
> speed missing, bad
> hdop/vdop/pdop missing
> heading missing (ok, I can't right now see how to use it but why not
> to store it, it might become useful in the future..)
> cell arfcn, no idea what's that..
>
> So anyway also cellhunter could add some fields here and as it's on
> Freerunner only, it shouldn't be too hard to also add the support to
> the client.
>

I looked at "common api to submit cells" and found that the openmoko
OBM logger app was missing *one* field (arfcn) that cellhunter api
wanted. Both being opensource, here's my hackish solution to patch the
OBM logger to write that field and a python script to submit records
produced by that patched OBM to cellhunter. You have to run this after
collecting the logs but before you move them to the Processed Logs
folder in OBM. So I do it before I do Upload in the OBM logger app.
YMMV, hopefully the real logger app developers can work something out
between themselves.
-------------- next part --------------
from xml.dom import minidom
import os,dbus,urllib
"""
OBMhunter submitter 0.1.0  <maxious at lambdacomplex.org>
Submits appropriate openBMap xml logs to cellhunter DB.

Installation:
Patch the openBMap logger library:
patch < obm_hunter-logger.py.patch

Change the group name (gname), group password (gpass) and device id (a 
random number - check .cellhunter.conf if you want to be consistent) if
you want your results to count for a group's score. Otherwise leave 
defaults to remain anonymous.

Then just run the OBM logger as usual but you have to run this script after collecting the logs
but before you move them to the Processed Logs folder in OBM. So I do it before I do Upload in 
the OBM logger app.
"""
gname=""
gpass=""
device_id=0

bus = dbus.SystemBus()
ogsmd_obj = bus.get_object( "org.freesmartphone.ogsmd", "/org/freesmartphone/GSM/Device" )
ogsmd_network_iface = dbus.Interface( ogsmd_obj, "org.freesmartphone.GSM.Network" )
data = ogsmd_network_iface.GetStatus()
provider = urllib.quote(data['provider'])


path="/home/root/.openBmap/Logs/"
dirList=os.listdir(path)
for fname in dirList:
	print "Processing " + fname
	dom = minidom.parse(path + fname)
	for scannode in dom.getElementsByTagName("scan"):
		for gpsnode in scannode.getElementsByTagName("gps"):
			time = int(gpsnode.getAttribute("time"))
			lat  = float(gpsnode.getAttribute("lat"))
			long = float(gpsnode.getAttribute("lng"))
			alt  = float(gpsnode.getAttribute("alt"))
		for child in scannode.childNodes:
			if "gsm" in child.tagName:
				cell_mcc   = int(child.getAttribute("mcc"))
				cell_mnc   = int(child.getAttribute("mnc"))
				cell_la    = int(child.getAttribute("lac"))
				cell_id    = int(child.getAttribute("id"))
				if (child.getAttribute("rxlev") != "") & (child.getAttribute("arfcn") != ""):
					signal     = int(child.getAttribute("rxlev"))
					cell_arfcn = int(child.getAttribute("arfcn"))
					serving    = 1 if (child.tagName == "gsmserving") else 0
					URL = "http://ch.omoco.de/cellhunter/submit.php?provider=%s&cell_mcc=%d&cell_mnc=%d&cell_la=%x&cell_id=%x&signal=%d&time=%d&lat=%f&long=%f&alt=%f&gname=%s&gpass=%s&device_id=%d&cell_arfcn=%d&serving=%d " %(provider, cell_mcc,cell_mnc,cell_la,cell_id,signal,time,lat,long,alt,gname,gpass,device_id,cell_arfcn,serving)	
					os.system('wget --user-agent "OBMhunter 0.1.0 offline <maxious at lambdacomplex.org>" -q --output-document=- \"' + URL + '\"')
					print "\n"
	
-------------- next part --------------
A non-text attachment was scrubbed...
Name: obm_hunter-logger.py.patch
Type: application/octet-stream
Size: 6466 bytes
Desc: not available
Url : http://lists.openmoko.org/pipermail/community/attachments/20090906/0d8621e0/attachment.obj