Saturday, November 22, 2014
schwehr.org is temporarily offline
The undisclosed bunker location was recently sold. The content at schwehr.org is not lost. My main blog will be back when I get some time to try out a new solution. I'm planning on going with AppEngine serving. Just too many other important things taking precedence right now.
Wired UK and Windward is an entertaining article. I have to say that it is great they are finally joining a conversation that has been going on for more than a decade. And let me point out that I welcome working with Windward and others. I started working on AIS troubles back in 2006 and there already had been really interesting work by folks for years before that. I've been working hard to share ideas to help others get started.
Yes, anyone in the business knows that $100 of gear will let you spoof an invasion of the Spanish armada (yes, the Spanish admitted to practicing that), so the Wired article baffles me with all the grandstanding. Sadly, the Wired/Windward article sounds more like an advertisement. I hope they will come to the table with more constructive discussions in the future as it is going to take wide ranging effort from many parties (non-profits, companies, citizens and governments) to create a global economic environment that is effective and environmentally sustainable. I've through contributions into the ring in my work with Google, the University of New Hampshire, Cornell, NOAA, USCG, IFAW, SkyTruth, EarthNC, Conserve.IO, and many others (e.g. WhaleAlert, Global Fishing Watch, libais, noaadata, my papers, my blog, my unpaid support of the Deepwater Horizon spill, going to AIS standards meetings, etc.).
So I look forward to seeing Windward and others join in the effort to make our planet a better place for the long hall. Yes, economic growth, improved conditions for all people, and environmental protection can all happen together. If you are a coder, go check out github and the standards. If you are a policy person, attack the legal and political issues. If you are a mariner, work with your colleagues to help ships get their data correct and that the laws and best practices actually make sense. Biologists, oceanographers and other scientists, keep studying. If you don't fit those categories, I'm you can find a way to help.... there are so many ways to help. We are all on this planet together.
Brian Calder and I wrote a paper that came out back in 2009, that talks about some of the issues with AIS. I have written in my blog about the issues with security and integrity of AIS communications. My blog is temporarily offline, but I've stashed the text here: http://vislab-ccom.unh.edu/~schwehr/blog/archives-20141122.tar.xz And you can look through my papers here: http://vislab-ccom.unh.edu/~schwehr/papers/ AIS really sucks for a lot of purposes and was intended just for safety of navigation when it was intitially put together in the 1990's. Yet, it does show a huge fraction of the behavior of the world's vessels. There are lots of ways to filter AIS data and conflating it with other data sources (of which there is a huge list). And there are tons of different tools to help you sort out what's going on from the spectrum level to high level views like the Global Fishing Watch.
For example, this statement from the article:
"Until 2012, AIS data was super reliable because it wasn't commoditised. Nobody had it, so no one needed to clean the data or check it," Ami Daniel, a former naval officer and cofounder of Windward, tells WIRED.co.uk.So not true. The trend over 2000-2010 was for things with AIS getting way better, but it was still not great. It took > 3 years for the USCG to notice my Class B receiver with the wrong country in the MMSI in it that was sitting right next to the USCG station in New Hampshire (like < 100m). The number of vessels with bad MMSI's or "sailing to Casablanca" was huge. Anyone not carefully inspecting AIS data now or then is crazy for certain applications. And quotes like this:
Final ports of call were reported only 41 percent of the timeare meaningless. Ships don't have to report their port of call for a lot of places. It's a courtesy. And even when mariners try to enter it correctly, the AIS required keyboard and typical interface is most kindly described as "painful."
Yes, anyone in the business knows that $100 of gear will let you spoof an invasion of the Spanish armada (yes, the Spanish admitted to practicing that), so the Wired article baffles me with all the grandstanding. Sadly, the Wired/Windward article sounds more like an advertisement. I hope they will come to the table with more constructive discussions in the future as it is going to take wide ranging effort from many parties (non-profits, companies, citizens and governments) to create a global economic environment that is effective and environmentally sustainable. I've through contributions into the ring in my work with Google, the University of New Hampshire, Cornell, NOAA, USCG, IFAW, SkyTruth, EarthNC, Conserve.IO, and many others (e.g. WhaleAlert, Global Fishing Watch, libais, noaadata, my papers, my blog, my unpaid support of the Deepwater Horizon spill, going to AIS standards meetings, etc.).
So I look forward to seeing Windward and others join in the effort to make our planet a better place for the long hall. Yes, economic growth, improved conditions for all people, and environmental protection can all happen together. If you are a coder, go check out github and the standards. If you are a policy person, attack the legal and political issues. If you are a mariner, work with your colleagues to help ships get their data correct and that the laws and best practices actually make sense. Biologists, oceanographers and other scientists, keep studying. If you don't fit those categories, I'm you can find a way to help.... there are so many ways to help. We are all on this planet together.
Monday, October 20, 2014
This is tomorrow...
USCG NATIONWIDE AUTOMATIC IDENTIFICATION SYSTEM (NAIS) NETWORK TESTING
On October 21, 2014, the U.S. Coast Guard (USCG) will conduct AIS test messaging from all
USCG NAIS base stations located across the United States; for up to 15 minutes on the hour,
from 1100-2215 UTC. These messages can be identified by their Maritime Mobile Service
Identity (MMSI) number: 003669139 or 00366613 and by the words ‘USCG TEST’ within their
contents. This is the first of many tests the USCG will conduct in the future as it seeks to provide
another timely delivery source of navigational information and services to promote safety,
efficiency, and integration of the Marine Transportation System (MTS).
Messages to be tested: AIS Application Specific (ASM) messages 8, 25, and 26, with DAC = 001,
367, FI’s = 0, 22, 29, 33, or 35; AIS Safety Related Text (SRM) messages 14, of various lengths,
using repetitions of the text ‘USCG TEST’. For further information on AIS messages, DACs, and
FI’s visit http://www.navcen.uscg.gov/?pageName=AISMessages.
Mariners capable of decoding these messages can ignore them and should contact the USCG
Navigation Center at 1-703-313-5900 or TIS-PF-NISWS@uscg.mil should they encounter any
negative impacts due to the testing.
Wednesday, October 15, 2014
Encrypting non-military/law enforcement AIS is a bad idea
It was recently brought to my attention that some high level people are proposing that the IMO require a class of civilian (aka not military or law enforcement) vessels to use encrypted AIS to broadcast their position. This is an absolutely terrible idea.
Having ships broadcast their location encrypted defeats the primary safety aspect of AIS for those vessels, the encrypted messages do not expose their slot usage and thereby drastically increase the number of packet collisions in a local area (decreasing safety for normal AIS traffic), and encryption key management is not work well of large population of mariners. It would be a disaster in safety, technical and political terms.
The traditional "blue force" AIS units are just class A devices with slightly modified software (I own one). There really shouldn't be any cost difference. You could also change the programming for class B devices and have them transmit the encrypted position messages, but please don't do that.
Having ships broadcast their location encrypted defeats the primary safety aspect of AIS for those vessels, the encrypted messages do not expose their slot usage and thereby drastically increase the number of packet collisions in a local area (decreasing safety for normal AIS traffic), and encryption key management is not work well of large population of mariners. It would be a disaster in safety, technical and political terms.
The traditional "blue force" AIS units are just class A devices with slightly modified software (I own one). There really shouldn't be any cost difference. You could also change the programming for class B devices and have them transmit the encrypted position messages, but please don't do that.
Sunday, May 13, 2012
debian/fink and mbsystem
schwehr.org is currently in a "transitional stage"... it will be back soon. Too much going on with the group of use all hosting together for us to get this settled right away.
Jonny B asked a question this morning about tides and mbsystem. I didn't know the answer, but here is how I went about investigating. This is on a mac with 10.7 and fink. Hopefully this will give people some ideas about how to dig through what is around. "dpkg -L" lists the files in a package. As Jonny B said with a reference to the movie The Matrix, "Yowza, nice kung fu!". I didn't have a fish to give, so the best I could do was point to the tackle store and say you might be able to get what you need to fish in there.
# Which fink packages deal with sonar?
fink apropos sonar
# If you already know about mbsystem...
fink list mbsystem
fink install mbsystem
dpkg -L mbsystem | grep tide
# Nothing
cd ~/Desktop
ls /sw/src/mbsystem*
# See that the latest tar is version 5.3.1917
tar xf /sw/src/mbsystem-5.3.1917.tar.gz
cd mbsystem-5.3.1917
find . | grep -i tide
# Nothing
find . | xargs grep -l tide | wc -l
# 84
find . | xargs grep -l tide
# ./html/mbdatalist.html
# ./html/mbio.html
# ./html/mbnavadjust.html
# ./html/mbotps.html
# ./html/mbprocess.html
# ./html/mbset.html
# ./html/mbsystem.html
# ./html/mbsystem_how_to_get.html
# ./install_makefiles
# ./man/man1/mbdatalist.1
# ./man/man1/mbnavadjust.1
# ./man/man1/mbotps.1
# ./man/man1/mbprocess.1
# ./man/man1/mbset.1
# ./man/man1/mbsystem.1
# ...
# ./src/utilities/mb7kpreprocess.c
# ./src/utilities/mbcopy.c
# ./src/utilities/mbinfo.c
# ./src/utilities/mbprocess.c
# ./src/utilities/mbset.c
man mbset
# "Apply tides to bathymetry."
Jonny B asked a question this morning about tides and mbsystem. I didn't know the answer, but here is how I went about investigating. This is on a mac with 10.7 and fink. Hopefully this will give people some ideas about how to dig through what is around. "dpkg -L" lists the files in a package. As Jonny B said with a reference to the movie The Matrix, "Yowza, nice kung fu!". I didn't have a fish to give, so the best I could do was point to the tackle store and say you might be able to get what you need to fish in there.
# Which fink packages deal with sonar?
fink apropos sonar
# If you already know about mbsystem...
fink list mbsystem
fink install mbsystem
dpkg -L mbsystem | grep tide
# Nothing
cd ~/Desktop
ls /sw/src/mbsystem*
# See that the latest tar is version 5.3.1917
tar xf /sw/src/mbsystem-5.3.1917.tar.gz
cd mbsystem-5.3.1917
find . | grep -i tide
# Nothing
find . | xargs grep -l tide | wc -l
# 84
find . | xargs grep -l tide
# ./html/mbdatalist.html
# ./html/mbio.html
# ./html/mbnavadjust.html
# ./html/mbotps.html
# ./html/mbprocess.html
# ./html/mbset.html
# ./html/mbsystem.html
# ./html/mbsystem_how_to_get.html
# ./install_makefiles
# ./man/man1/mbdatalist.1
# ./man/man1/mbnavadjust.1
# ./man/man1/mbotps.1
# ./man/man1/mbprocess.1
# ./man/man1/mbset.1
# ./man/man1/mbsystem.1
# ...
# ./src/utilities/mb7kpreprocess.c
# ./src/utilities/mbcopy.c
# ./src/utilities/mbinfo.c
# ./src/utilities/mbprocess.c
# ./src/utilities/mbset.c
man mbset
# "Apply tides to bathymetry."
Friday, April 27, 2012
schwehr.org
Yes, http://schwehr.org is currently offline. InReach's colo facility is in a bad state. When I get a chance, I will find another place to get schwehr.org and my blog hosted.
4/24/2012 9:00am PST We continue to evaluate the existing HVAC system and initial indications for a repair are at least several weeks and possibly longer. We understand this is not an ideal situation so we would suggest to all of our customers that start moving as soon as possible to alternative locations in order to ensure servers and services remain up.Although we want to avoid it a full analysis may include a disruption of power. No definitive time, date or duration for this power disruption is scheduled but it could be within the next few days. We will update you all with the specifics once they become known.We are addressing security concerns as well and we have engaged the building management to help but their solution is to keep the door closed to the suite which will increase heat.
Using Google Cloud Storage - Part 1
This is my first attempt at a tutorial for Google Cloud Storage.
Using Google Cloud Storage… it's a service that let's you read/write
data objects (a blob of data of what ever type you want). Right now,
you get 5GB of storage for free and you can transfer 20 GB per month.
Beyond that, you'll have to check the pricing.
This is pretty much what is on https://developers.google.com/storage/docs/getting-started and https://developers.google.com/storage/docs/hellogooglestorage I've tweaked it a bit to fit my style. I'm doing this on a mac with fink and splitting between two acounts. I want to end up with a way to transfer files from one group to another. I started off by watching this 2011 Google IO video:
Google I/O 2011: Storing Your Application's Data in the Google Cloud
First setting up a virtual env in python to isolate us from messing up any existing python installs.
Let's test out setup.
This is pretty much what is on https://developers.google.com/storage/docs/getting-started and https://developers.google.com/storage/docs/hellogooglestorage I've tweaked it a bit to fit my style. I'm doing this on a mac with fink and splitting between two acounts. I want to end up with a way to transfer files from one group to another. I started off by watching this 2011 Google IO video:
Google I/O 2011: Storing Your Application's Data in the Google Cloud
First setting up a virtual env in python to isolate us from messing up any existing python installs.
cd ~/Desktop mkdir try-gs cd try-gs fink install virtualenv-py27 virtualenv ve source ve/bin/activate wget http://commondatastorage.googleapis.com/pub/gsutil.tar.gz tar xf gsutil.tar.gz pushd gsutil # gsutil does not have a proper setup.py that will work with virtualenv # so force it to play along perl -pi -e "s|/usr|`pwd`/../ve|g" setup.py python setup.py install popd
You should now have a working virtualenv with gsutil installed in it.
Your normal python will not be effective. You will have to rerun the
<b>activate</b> from any other shell that you want to have access to
gsutil. The virtualenv named "ve" will change your prompt:
$ pwd /Users/schwehr/Desktop/try-gs (ve) $
Let's test out setup.
type python # python is hashed (/Users/schwehr/Desktop/try-gs/ve/bin/python) type gsutil # gsutil is /Users/schwehr/Desktop/try-gs/ve/bin/gsutil gsutil --help # lots of help scrolls by # Try a world readable test data set gsutil ls gs://uspto-pair/applications/0800401* # gs://uspto-pair/applications/08004010.zip # gs://uspto-pair/applications/08004011.zip # gs://uspto-pair/applications/08004012.zip # gs://uspto-pair/applications/08004013.zip # gs://uspto-pair/applications/08004016.zip # gs://uspto-pair/applications/08004017.zip # gs://uspto-pair/applications/08004019.zip gsutil ls -l gs://uspto-pair/applications/08004010.zip # 3410 2010-09-29T20:14:15 gs://uspto-pair/applications/08004010.zip # TOTAL: 1 objects, 3410 bytes (3.33 KB) gsutil cp gs://uspto-pair/applications/08004010.zip . md5sum 08004010.zip # 04b4877377c16b1b49f19b0c7e161d04 08004010.zip
I have yet to write up using the oauth2 authentication. Hopefully, I'll get that done soon.
See Also:
- http://code.google.com/p/gsutil/
- Issue 92 covers using pypi / pip / distribute / setuptools
Subscribe to:
Posts (Atom)