Friday, January 9, 2015

NOAA CSC and USCG do a lame job with Marine Cadastre AIS



This is a serious waste of our tax payer dollars.  Public data about vessel location is being removed from datasets.  This stuff is broadcast in the clear for anyone to receive.  And what do they mean by "encrypted"?  Did they just run a hash (e.g. md5) function on it?  If that's the case, if I we can figure the hash function, it's trivial to build a rainbow table style lookup.  Once that's done, there are lots of sources of MMSI to vessel name and call signs.  Oh, and by the way, it's 2015 and only 2009, 2010, and 2011 are available.  And then they publish it in the ESRI File Geo Database (FGDB) format that really isn't open.  And I don't really feel like getting a username and password again (or maybe my old one still works).  Sigh.

"Note: Ship name and call sign fields have been removed, and the MMSI (Maritime Mobile Service Identity) field has been encrypted for the 2010 and 2011 data at the request of the U.S. Coast Guard."





Saturday, January 3, 2015

Java and Eclipse on the mac

At one time, Apple seemed into Java and things were only slightly painful.  Then Java went out of favor with Apple and things went from bad to worse.  I downloaded the 64bit Mac build of Eclipse and got this after cd /Applications and tar xf ~/Downloads/eclipse-standard-luna-SR1-macosx-cocoa-x86_64.tar.gz and then double clicking the Eclipse icon.


I tried editing /Applications/eclipse/Eclipse.app/Contents/MacOS/eclipse.ini to look like this:


Looking at my system, I find this for java:

ls -l /Library/Java/JavaVirtualMachines
drwxr-xr-x  3 root  wheel  102 Apr  4  2014 jdk1.7.0_51.jdk
drwxr-xr-x  3 root  wheel  102 May  7  2014 jdk1.7.0_55.jdk
drwxr-xr-x  3 root  wheel  102 Jul  9 13:46 jdk1.7.0_60.jdk
drwxr-xr-x  3 root  wheel  102 Jul 30 13:27 jdk1.7.0_65.jdk
drwxr-xr-x  3 root  wheel  102 Sep 18 05:50 jdk1.7.0_67.jdk

drwxr-xr-x  3 root  wheel  102 Dec 10 08:45 jdk1.7.0_71.jdk

But no luck.  I finally found that this success:

cd /Applications/eclipse/Eclipse.app/Contents/MacOS
./eclipse -vm /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/bin/

And now I have eclipse running:


So I now have Eclipse Luna Service Release 1 (4.4.1) Build id 20140925-1800 on Mac OSX 10.9.5 with XCode 6.1.1.  It took me way too long to get this figured.

Wednesday, December 24, 2014

Merry NSA Christmas

Minox Cornflower Drill Wackenhut Air borne crypto anarchy import DATTA
dictionary pipeline CBM target ISA Emergency management Lebed North Korea
Sony explosion Maple Blowfish ASIO Epidemic strategic Wave ACC Bletchley
Park Hollyhock IRIDF MS13 Telecommunications Drug war benelux

Brought to you by M-x spook

Sunday, December 21, 2014

fink with 10.10 Yosemite

I'm finally trying out Mac OSX 10.10 (Yosemite)

system_profiler SPSoftwareDataType | grep 'System Version'

      System Version: OS X 10.10.1 (14B25)

uname -s -r

Darwin 14.0.0

xcodebuild -version
Xcode 6.1.1
Build version 6A2008a

I had to accept the xcode license:

sudo xcodebuild -license

My first serious PITA was Java.  I used the fink script for kickstarting and Java kept saying it needed to be installed.  I installed Java8 u25 and every time I tried to run java or javac, I got a GUI popup dialog asking me to install Java.  I installed Java7 and thing seem to work better now.

javac  -version
javac 1.7.0_71

Since I'm a fink developer, here is me setting up cvs as the update method for fink.

fink selfupdate # Select rsync
fink install cvs
fink selfupdate -m cvs

10.10 fink only has python 3.4 (no more 3.2 or 3.3), so I've got some cleanup to do.  Thanks to Hanspeter for pointing that out.

fink install wget
wget http://xquartz.macosforge.org/downloads/SL/XQuartz-2.7.7.dmg
fink install python3

Saturday, November 22, 2014

schwehr.org is temporarily offline

The undisclosed bunker location was recently sold.  The content at schwehr.org is not lost.  My main blog will be back when I get some time to try out a new solution.  I'm planning on going with AppEngine serving.  Just too many other important things taking precedence right now.
Wired UK and Windward is an entertaining article.  I have to say that it is great they are finally joining a conversation that has been going on for more than a decade.  And let me point out that I welcome working with Windward and others.  I started working on AIS troubles back in 2006 and there already had been really interesting work by folks for years before that.  I've been working hard to share ideas to help others get started.


Brian Calder and I wrote a paper that came out back in 2009, that talks about some of the issues with AIS.  I have written in my blog about the issues with security and integrity of AIS communications.  My blog is temporarily offline, but I've stashed the text here:  http://vislab-ccom.unh.edu/~schwehr/blog/archives-20141122.tar.xz And you can look through my papers here:  http://vislab-ccom.unh.edu/~schwehr/papers/  AIS really sucks for a lot of purposes and was intended just for safety of navigation when it was intitially put together in the 1990's.  Yet, it does show a huge fraction of the behavior of the world's vessels.  There are lots of ways to filter AIS data and conflating it with other data sources (of which there is a huge list).  And there are tons of different tools to help you sort out what's going on from the spectrum level to high level views like the Global Fishing Watch.

For example, this statement from the article:
"Until 2012, AIS data was super reliable because it wasn't commoditised. Nobody had it, so no one needed to clean the data or check it," Ami Daniel, a former naval officer and cofounder of Windward, tells WIRED.co.uk.
So not true.  The trend over 2000-2010 was for things with AIS getting way better, but it was still not great.  It took > 3 years for the USCG to notice my Class B receiver with the wrong country in the MMSI in it that was sitting right next to the USCG station in New Hampshire (like < 100m).  The number of vessels with bad MMSI's or "sailing to Casablanca" was huge.  Anyone not carefully inspecting AIS data now or then is crazy for certain applications.  And quotes like this:
Final ports of call were reported only 41 percent of the time
are meaningless.  Ships don't have to report their port of call for a lot of places.  It's a courtesy.  And even when mariners try to enter it correctly, the AIS required keyboard and typical interface is most kindly described as "painful."

Yes, anyone in the business knows that $100 of gear will let you spoof an invasion of the Spanish armada (yes, the Spanish admitted to practicing that), so the Wired article baffles me with all the grandstanding.  Sadly, the Wired/Windward article sounds more like an advertisement.  I hope they will come to the table with more constructive discussions in the future as it is going to take wide ranging effort from many parties (non-profits, companies, citizens and governments) to create a global economic environment that is effective and environmentally sustainable.  I've through contributions into the ring in my work with Google, the University of New Hampshire, Cornell, NOAA, USCG, IFAW, SkyTruth, EarthNC, Conserve.IO, and many others (e.g. WhaleAlert, Global Fishing Watch, libais, noaadata, my papers, my blog, my unpaid support of the Deepwater Horizon spill, going to AIS standards meetings, etc.).

So I look forward to seeing Windward and others join in the effort to make our planet a better place for the long hall.  Yes, economic growth, improved conditions for all people, and environmental protection can all happen together.  If you are a coder, go check out github and the standards.  If you are a policy person, attack the legal and political issues.  If you are a mariner, work with your colleagues to help ships get their data correct and that the laws and best practices actually make sense.  Biologists, oceanographers and other scientists, keep studying.  If you don't fit those categories, I'm you can find a way to help.... there are so many ways to help.  We are all on this planet together.


Monday, October 20, 2014

This is tomorrow...


USCG NATIONWIDE AUTOMATIC IDENTIFICATION SYSTEM (NAIS) NETWORK TESTING
On October 21, 2014, the U.S. Coast Guard (USCG) will conduct AIS test messaging from all
USCG NAIS base stations located across the United States; for up to 15 minutes on the hour,
from 1100-2215 UTC. These messages can be identified by their Maritime Mobile Service
Identity (MMSI) number: 003669139 or 00366613 and by the words ‘USCG TEST’ within their
contents. This is the first of many tests the USCG will conduct in the future as it seeks to provide
another timely delivery source of navigational information and services to promote safety,
efficiency, and integration of the Marine Transportation System (MTS).
 
Messages to be tested: AIS Application Specific (ASM) messages 8, 25, and 26, with DAC = 001,
367, FI’s = 0, 22, 29, 33, or 35; AIS Safety Related Text (SRM) messages 14, of various lengths,
using repetitions of the text ‘USCG TEST’. For further information on AIS messages, DACs, and
FI’s visit http://www.navcen.uscg.gov/?pageName=AISMessages.
 
Mariners capable of decoding these messages can ignore them and should contact the USCG
Navigation Center at 1-703-313-5900 or TIS-PF-NISWS@uscg.mil should they encounter any
negative impacts due to the testing.