Friday, April 17, 2015

My new ais stream processing code for libais

For a long time, I've had the not so great ais_normalize.py in noaadata to manage multi-line NMEA AIS VDM messages.  It worked for the old USCG format, but it was brittle and cruft code.  Egil added aisdecode to libais based on ais_normalize, but that was building on a terrible foundation.  Today, I pushed the final patch for my nmea_queue module that can handle text, bare NMEA, TAG Block (including Orbcomm extensions), and USCG old CSV metadata.  I still need to add a couple non-VDM messages to the system to work out how to parse those cleanly.  With the new gpsd_format library (led by SkyTruth), libais and gpsd are starting to really play well together.  The design goals of the two are extremely different, so think carefully which you want to use.  In my case, I use both.

Overall, I'm much happier with the state of libais and excited to talk to many folks in the AIS community at the AISSummit 2015 conference in Hamburg next month.  Paul Woods of SkyTruth and I will be giving a pair of talks that will go over how to use "Big Data" techniques and technologies to tackle larger AIS databases with ease.  We will talk about many of the specifics of what has gone into Global Fishing Watch.

I've also been putting work into gpsd, BitVector, and ais-areanotice with a little bit on noaadata (noadata is a mess).  More to come.