Monday, September 25, 2017

GPS spoofing possibly seen in AIS data

Disclaimer:  The opinions here are my own and do not represent Google, SkyTruth, Global Fishing Watch, or the Univ. of New Hampshire.  I work for Google, co-founded Global Fishing Watch, and am an Affiliate Professor at the Univ. of New Hampshire.

Warning:  This was written off the top of my head in less than an hour with no review and not much fact checking.  I know I'm missing lots of interesting details.

Before I go an read the article that Bruce Schneier linked to, I'll try to think through what could explain these general sorts of issues.  I've heard rumbling before of this in AIS data, but it's time to take a few minutes.

After trawling through AIS data from recent years, evidence of spoofing becomes clear. Goward says GPS data has placed ships at three different airports and there have been other interesting anomalies. "We would find very large oil tankers who could travel at the maximum speed at 15 knots," says Goward, who was formerly director for Marine Transportation Systems at the US Coast Guard. "Their AIS, which is powered by GPS, would be saying they had sped up to 60 to 65 knots for an hour and then suddenly stopped. They had done that several times."
All of the evidence from the Black Sea points towards a co-ordinated attempt to disrupt GPS. A recently published report from NRK found that 24 vessels appeared at Gelendzhik airport around the same time as the Atria. When contacted, a US Coast Guard representative refused to comment on the incident, saying any GPS disruption that warranted further investigation would be passed onto the Department of Defence.
So what could cause deviations in speed?  This is without looking deeper at the data.   First, I need to think about what is being recorded here.  A ship has an AIS device that is trying to broadcast it's position.  It has an internal sense of location.  Where does that come from?  It usually has a GPS receiver, but that isn't always where the position comes from.  It could be from a different GNSS (aka satellite) nav system, Loran (Loran is not dead - only the US version), dead reckoning, or other ways of entering position.

If it was a GPS, there could be a couple things go on.  The GPS might be receiving differential corrections or ephemeris from someplace other than the satellites.  That could be a direct RF link or some internet link.  Those could be bogus either intensionally or unintentionally.  The GPS could be defective and I've seen that very clearly before.

But it might not be GPS.  These ships could be using pure GLONASSGalileo, or other system.  Or it could be a receiver that blends multiple constellations.  If one of these other systems has a problem, I'm not sure how that would impact the resulting position and velocity fixes.

The USCG and Europeans stopped using LORAN, but that doesn't mean there are no places where the same or similar local ground systems are in use for positioning.  The Russians have CHAYKA.  I've got used them and haven't thought through their failure modes or how to attack them.  They have a long and interesting history with things like Decca.

We also have the possibility the ship operator modifying the position.   Global Fishing Watch has seen clear evidence of ships monkeying with the position values on their AIS transceivers to thwart observers.  That could be from a modified AIS unit (e.g. customized embedded software), modified GPS, or a device between the GPS hardware and the AIS hardware.  A simple way to do this is to just use a computer with a software defined radio and send what ever you want.

There may also be Inertial Navigation System (INS) between any positioning system(s) and the AIS's embedded computer.  What are the ways an INS can have trouble?  There are plenty.

But, the ship's AIS might not have even been on a position system.  If the system's connection to a position system fails or it thinks it failed, the system can dead reckon.  Depending on the sensor inputs available, you can get all sorts of fun results.  And if the system later gets positioning input back, how does if get itself back in line?  Does it jump to the most recent position or does it try to gradually catch up.

There is probably plenty more to positioning, but it is time to move on to other topics like speed and time.

Next I wonder what is being used for speed.  AIS messages 1-3 have a speed (not velocity) field.  It's up to the AIS unit to fill that out.  How is it doing that?  It would be easy to alter or have that wrong.  I'm thinking of the NOAA Ship Thomas Jefferson that had bad bits in their AIS messages for months after a lightning strike to the ship.  Or are people with the messages using the times of the messages and their locations to externally calculate a speed for the ship?  That begs the question of what time to associate with each message and more specifically the position within the message.

Each AIS message 1,2,3 reports the seconds within the minute of the position report.  Only some reports have a more detailed time record with hour and minute (in the comm state block).  So if you are looking at a message, there are several questions.  Did the system know what time it was?  How far off could the time be?  Does the system think it doesn't have a decent sense of time from the GPS and internal clock(s)?  A substantial number of AIS transceivers report that they are getting time from the AIS transceivers around them.  What does that mean when most systems have a built in GPS that should provide an accurate sense of time?

Then we have the question of how long from the position/time tuple does it take the AIS to form the AIS message and get it out of the RF link?  It should be quick (less than 3-4 seconds), but who knows what a misbehaving transceiver might do.  And there is the added factor that AIS units will listen and act on remote commands without any authentication at all.  You can even tell an AIS unit to switch frequency.  That was accidentally demonstrated in the wild by the USCG of the US East coast many years ago.  Whoops!

So now we have an AIS message out on RF.  Where did it come from?  There is nothing in an AIS message that proves it is from that ship.  There is no cryptographic signature or other trick to make sure an external source isn't providing position.  You could jam the slots the ship transmits on and make fake messages for the ship in other slots and it would be very hard to spot from just typical NMEA logs of AIS.

There are plenty of cases with multiple ships having the same MMSI.  That's not a huge deal when they are in different oceans, but what happens when they are near each other?

Next, some AIS receivers do a careful job of recording metadata for a received AIS message, but most do not.  Some don't even record a received timestamp.  So you have to use the overall NMEA stream to keep a sense of time in the logs.  The USCG put atomic clocks in their receiving system (woo hoo, must be perfect time logs...), but then suffered Windows Time Protocol issues that had their logging systems recording times up to 24 hours different from the actual receive time.  Some people just use the loggers internal clock without NTP... that's full of all sorts of fun.  Sadly, oscillators in computers in are also temperature sensors.  They alter their frequency some with temperature.  I've seen that with a logger on the end of a peer in New Hampshire and it was especially bad when the 24 hour temperature swing was as large as near 0F to 60F.

There is also the case of where is the timestamp being recorded.  Back in 2010, I was getting Orbcomm data via the USCG.  About every 6 hours, I'd get a huge dump of AIS messages.  The timestamp metadata for all those messages would be the time of the dump.  So not helpful.

Then once a message is "received" there is method of getting the data to the decoder.  As seen by many, you might not be getting what you think.  If someone has access to the network, they could inject fake / altered messages and metadata.  If you can't trust the network or the people in the network, who knows what you will get.  And even in a trusted situation, you may not be receiving what the receiver saw.  The USCG provides NAIS feeds that decimate the data to things like only 1 copy of a message received at multiple points in the network and only 1 message per minute from a particular MMSI.  How do you check such decimated data?

Then finally, there is your decoding process.  Which software is converting AIS RF received bits into messages and finally NMEA and optional metadata (e.g. NMEA TAG BLOCKs)?  What library is decoding the NMEA and metadata?  What is being done to check timestamps find probable errors.  If your research is just on the decoding XML, json, or whatever decoded format, how can you verify that their decoding / interpretation of the original AIS is correct?  Perhaps they interpolate position reports (or did for just a bit of time) to make their system look more impressive?  Or their system rounds values or bugs in their processing snap positions to locations.  And so on and so on.

And there is also noise throughout the entire process.  The checksums in NMEA are pathetic.  And many encode and decode systems will just drop messages that don't meet their expectations.

Yes, you can spoof the GPS system, but the above shows that there are plenty of ways to have craziness in AIS data that are not caused by GPS spoofing.

Time to take a look at the article...  Skrevet av Henrik Lied 18. september 2017

While their timing with Putin's location is pretty convincing, I'm mostly left with questions.  Were these all Class A devices or where some Class B devices impacted too?  Were their any logs of positions from other devices?  e.g. a cell phone with GPS off, but able to see wifi can give you position estimates.  Were there any AIS satellite message (27?) position reports impacted?  What were the configurations and models of AIS and GPS for these ships?  Where there issues with other received AIS messages?  e.g. if you had any fixed base stations or ATONs reporting position, did they stay fixed and what positioning mode were they in (e.g. GNSS or surveyed?)  What happened with ships at dock or anchor in the region?  What was reported by different GPSes on the same ship?  Usually the AIS has a GPS receiver inside the device that is separate from the chart plotter.  Were any NMEA logs from the ship recorded?  Were their heading or commstate quirks in the AIS messages?

They quote from the master of the ship:
Thank you for your below answer, nevertheless I confirm my GPS equipment is fine.
We run self test few times and all is working good.
I confirm all ships in the area (more than 20 ships) have the same problem.
I personally contacted three of them via VHF, they confirmed the same.
Sometimes, position is correct, sometimes is not.
GPS sometimes looses position or displays inaccurate position (high HDOP).
For few days, GPS gave a position inland (near Gelendyhik aiport) but vessel was actually drifting more than 25 NM from it.
Important: at that time, GPS system considered the position as "Safe within 100m".
Really?  "my GPS equipment is fine"  How is a master supposed to really know that?  What do those self tests do?  What does their system do to declare that safety statement?  I'd love to see the raw log from even a regular fixed GPS for a time window that included the time of issues.  Even better if someone with a 100+ channel survey grade receiver had RINEX logs.  What does the phase tracking info say?


Really, these articles are just teases.  There is no really useful information in them.  The topic deserves a real research project with data published along with the results.  I encourage researchers out there to dig into these issues.

I also wonder about these hardware and software on the ship and receive stations.  I've never fuzz tested an AIS transceiver (RF input or ship comm channels), GPS, or ECDIS/ECS.  I've poked at fuzzing libais and it definitely needs work.  Also, some of these systems have OSes under the hood that could easily be attack via more traditional malware or vendor supplied issues.  e.g. If you have Transas ECDIS on your bridge that only runs on Windows?  Your machine could be pwned or Transas could have worked with the Russian government to have a way to allow the system to degrade positing when it wants... I variation on the US's ability to turn on Selective Availability (SA).

Note: I am NOT claiming that Transas has done this.  It's just they are a Russian company.  Correct?  I can't find any info to fact check my memory.

I am continually baffled by the maritime communities' believe that they are somehow special when is comes to computer security.  My message to all those working with ships:  Your IT security issues are not unique or special.   Your just worse at dealing with them than most and have your head in the sand even more than Equifax.

Saturday, September 2, 2017

AIS Integrity and Security - Part 0

Disclaimer:  The opinions here are my own and do not represent Google, SkyTruth, Global Fishing Watch, or the Univ. of New Hampshire.  I work for Google, co-founded Global Fishing Watch, and am an Affiliate Professor at the Univ. of New Hampshire.

With recent events with the US Navy alliding with commercial ships and people asking if the blue ship could have been hacked, it's time to dust off the old notes.  I've wanted to write a paper (or 10) about this topic since about 2005.  I might as well try in blog form to get out what I have.

First some old material material...

Back in 2012, I made this blog post:  AIS SECURITY AND INTEGRITY.   My whiteboard from that exercise:

And I started a doc for a paper and here is what I had as of 2010:

Integrity and Security of the global maritime Automatic Identification System (AIS)
Kurt Schwehr

In 2006, the several people in the US Government encouraged me not to write a paper on the security of AIS.  I have concluded that it is important to discuss these issues in the open.  Security through obscurity is not a good solution.  Many people have already thought through how AIS is vulnerable, but there has not been a source of information to the typical mariner or marine manage that lays out the issues.  AIS is an effective tool for maritime activities, but care needs to be exercised when using or trying to improve the system.  Changes to how AIS is used in real-time or post processed for analysis of water way activities can have unexpected consequences some of which can have negative environmental and economic impacts or put the mariner at greater risk of harm.


  • Bandwidth limitation
  • No official standard for logging with metadata (especially time) and transmitting over internet and other secondary channels
  • Limited reliable transmit capability
  • MKD is horrible and error prone.  Often not located in an easily visible portion of the bridge
  • AIS might not be integrated into ECS and/or ECDIS systems
  • No standards for autonomous vessels
  • Repeaters can quickly flood the AIS channel
  • Incorrect understanding by mariners and data analysis of what AIS means
  • Poor monitoring of AIS receiver networks
  • Larger messages have a higher probability of packet collision
  • Patent encumbered technology
  • Regulatory environment - not much room in the United States for experimentation and research
  • Can you issue penalties for behavior recorded only in AIS received reports?
  • Should fishermen install or block AIS on their vessels?  The proprietary knowledge of where they fish.
  • Attacks targeting vessels based on AIS.  Is this really any worse that without AIS?  e.g. Boston’s north LNG terminal.

Time and Positioning

  • What time does the position translated relate to?
  • Partial timestamps in packets and accuracy only to the second
  • RAIM seems like it is not really useful to indicate position accuracy
  • Offset issues between the ship reference point and the GPS reported by Pilots
  • Knowing where your receiver is located when getting data from networks such as NAIS
  • Timestamping of received messages that happens down stream may get very confusing.
  • GPS errror example from 2009 NOAA review in portsmouth NH

Radio noise and propagation issues

  • NOAA weather radio transmitters and other systems that put energy into the AIS spectrum
  • Inland use of the same spectrum
  • Noise on the GPS channel and solar issues
  • better propagation conditions that tunnel packets between cells will cause interference
  • currently not enough satellites and ground stations to give realtime global coverage

Receiver and Transmitter issues

  • Weak checksum scheme may create valid packets
  • Cheap receivers might decode noise as valid
  • Single channel receivers will miss half of the message traffic
  • VHF propagation issues prevent seeing distance ships most of the time
  • Transmit antenna height and power greatly matter for receive potential
  • For remote receivers, how do you know they are working if you aren’t sure there are transmitters in the area.  E.g. Antarctica (or little bay)

Class A or B units

  • - Turning off the unit (trivial to accomplish) or power failure
  • - Loss of GPS - antenna, urban canyon, multipath, kalman filter problems
  • - Unintentionally mis-configured devices.  Human error, system failure, sensor failure, antenna failure
  • - Intentionally incorrect devices.  How to confuse people.  What can you reprogram on a Class B?
  • Inability to match to an exact vessel in other registries
  • broadcasting bad time
  • Bad carrier sense in Class B
  • Long term scalability of the number of units that can be on the channels
  • How good is the self reported heading, ROT, speed?  Which sensor is being used?  Does the GNSS have any extra capabilities beyond just the traditional GPS?  WAAS, Diff, satellite transmitted corrections like CNAV, RTK?  

Transmissions by non-blue force (e.g. normal operations)

  • Not encrypted - Even addressed messages are received by all hardware
  • No cryptographic signature capability - impossible to assure identity
  • Specifications such as Circ 289 require people keep already public data protected
  • No way to TX position securely in areas with pirates
  • There are free and pay networks that will give realtime AIS to anyone.  Not everyone is willing to acknowledge this.

Blue force encrypted transmission

  • Lack of comm state in message 8 makes scheduling of slots difficult for other vessels
  • Direction finding may still identify the location of military or law enforcement vessels with COTS receivers
  • Is the encryption really secure?
  • It is unclear what the policy for switching between silent, secure, and normal modes
  • Is the encryption really secure?
  • The DAC/FI is not always the same and may infact be random

SAR and other aircraft

  • Broadcasting at altitude can span many cells and cause wide spread interference
  • May hear other cells and be less able to receive (solvable as evidenced by satellite)

Basestations and ATONs

  • reserving slots to consume bandwidth
  • telecommand to change reporting rates
  • changing the frequency of AIS units (see USCG announcement)
  • sensor failure for met hydro might go undetected or predicted not noticed as such

Software or hardware units that are not type approved

  • Accidental or intentional interference on slots
  • Ghost fleets of fake ships - Spanish?
  • Transmitting incorrect info for correct ships

Software and hardware QC/QA problems

  • Paywalled specifications
  • Closed testing
  • Connector design problems
  • Very few ports have knowledge, means of detecting, and technicians to correct bad or misconfigured AIS units.  SF is an exception
  • Complicated specifications lead to errors in implementation
  • Ambiguous message definitions because of the lack of use of a formal language to define messages.
  • Lack of evolution in some specifications have led vendors to innovate on their own with incompatible and/or undocumented changes  (e.g. Ohmex)
  • Units are inconsistent in between messages.  This should be uniform for the network and configurable at the presentation interface when displayed
  • No official catalog and registry system where countries can submit their regional messages

Possible improvements and solutions

All is not lost!  AIS is an effective tool for many maritime tasks and likely is making the job of safe navigation much easier.  The system as it stands is much better than in the 1990s.  Ships now know more of the names of ships around them so that when they call on VHF, they know who to address, they can get better directional information

  • Transmitting on other frequencies (and possibly coding schemes)
  • I-AIS standards
  • Best practices documentation for design of new messages
  • Open and free documentation and open certification processes
  • Define international standards for raw logging and transfer with JSON and XML messages between applications
  • Better suggestions for monitoring receiving integrity
  • Explicit documentation on how to calculate time to go with a position
  • Best practices for receive networks
  • Watch standers guides need to include sections on the configuration, maintenance, testing, and use of AIS data to make sure they are transmitting correctly and properly interpreting/using AIS data.  e.g. update rate is not on any AIS displays that I’ve seen

What does one component of AIS look like?

This is a sketch of what the whale alert system looks like at a high level back in 2012.

Thursday, April 20, 2017

AIS requirements for the Panama Canal has requirements for AIS on ships going through the Panama Canal.   I hear rumor that 16% of ships going through need to have a 2nd AIS added as their primary does not meet the requirements.  Some of these like the heading and pilot port are not too surprising.  But a lot of AIS units that I consider not okay would pass this.  All of this would be much easier if the IEC standards were not paywalled and there was open source software that could used to test AIS units.

And this cyber attack stuff is a load of crap.  Systems on ships are designed so stupidly, it hurts.  Why are people still using full windows machines for ship board operations?  Clickbait follows...

Monday, April 3, 2017

End of my USCG NAIS Feed

Last month, I finally called an end to my USCG NAIS feed.  The server running AISUser was running an ancient version of CentOS whose is end-of-life for security patches is this month. I had not been rebooted the machine since about 2011 (there is a large natural gas generator right outside its server room).  It feels like a weight has been lifted.

Friday, March 24, 2017

Danish AIS almost interesting

I just saw that the Danish have released AIS data publicly.  I was excited until I saw what one of the files looks like.  It's not RAW AIS NMEA data.  That means it's crap for anyone wanting to do use it for rigorous research into AIS.  You don't get to see the station info, the comm state, or any of the interesting channel messages.  You won't get to see each of the ship name messages.  Or the meteorology/hydrology messages.  You also won't be able to do anything if you find they have a processing bug or don't like the way they choose to conflate msgs 1-3 with 5.  Also, what exactly is the license / copyright status for the data? Big bummer.

That's in addition to the already annoying marine link story that doesn't actually link to the data.  MarineLink does an amazing job of annoying me.

And they named the files MMMYYYY so they don't sort nicely on their ftp server (wait, ftp?  what year is this?)  Extra bonus of using a zip so I can't zcat/bzcat/xzcat a partial download to see what I'm getting into.  At least the most recent data is available by day as an uncompressed file.

Thursday, January 26, 2017


Just to keep up with things, here is the latest.  I've definitely failed to blog enough this week.

Power GEODSS Sears Tower Social media Ti computer terrorism Task Force
nuclear Tehrik-i-Taliban Pakistan Gulf Cartel Listeria MARTA NATIA
JAVA Trump industrial intelligence 42

4c a7 9b 81 b9 3a ae 99 b1 40 a6 1f 08 c9 4e 43
41 2f 7a 4b ac 8a 5f 70 f9 60 a1 3e 79 dc da 1f
fb 97 19 72 6f 20 81 15 a5 04 c0 52 45 ae 86 7b
60 e6 71 6e ad 32 70 81 5b 30 8c 24 5f 83 d2 78
5b 32 f0 8f 1b 1a 16 3b f7 02 fc 87 df 02 0d ac
d0 32 c2 b0 d7 f2 08 1f 0d 87 06 9b 68 18 be e0
51 dd f3 38 a5 34 90 cd 64 8f ea f2 57 2b ba f1
ba 1e ff 22 9f 72 f0 3d b7 f2 82 52 f6 be 56 ea
70 24 62 b0 f4 bc b0 c0 89 34 62 2b 9d 12 44 09
f3 2b 5b 3c 04 e1 e8 bc 34 02 90 31 a4 92 89 38
2e 1d 82 c4 de fe 0c ca 81 b5 73 2c f7 70 db 35
96 c1 34 01 76 26 24 6b c6 df 64 29 74 90 ab bd
a1 5d 5a b6 5d 34 58 09 d2 3a 76 d5 93 d8 3c 67
a7 38 fd ef f0 f1 24 49 b7 76 94 9d fa 0e fe 75
02 f4 38 ef a1 36 0c 7e eb ac 7b a6 26 b6 24 11

88 24 2a c2 2c bc eb fb 1f eb 2e 06 99 28 7f 42

Sunday, January 8, 2017

Code reviews

I've sent out thousands of changes for code review and done reviews on many hundreds of changes.  It's kinda weird to read this RedHat post on code reviews.  I can't really imagine what the review process is like without things like the Google C++, Python and Java style guides (I haven't done Go, sorry Francesc).  Things like that and required automatic formatting rules take a lot of the pain out the review process.  But I have had changes that have taken more than a year to get into a code base.  That does get a wee bit tiring sometimes.  Pythonic code review (Red Hat Security Blog)

MIT 046 Algorithm course

I'm going through the MIT 046 algorithm course from 2015...

Sunday, January 1, 2017

Blogging again

I am going to see if I can get back into blogging again. Can I pull off a post a day for January?  What do I even want to write about these days?  A lot of my work is non-public Google infrastructure work.  I can take about some of the open source work and maybe I can talk about some of the public data I am around. I need to start digging into JTS, Geotools, and Java in general. I want to get more into algorithms and data structures.  These last couple of years have been paying down massive technical Dept, which isn't sexy. And buffet overruns and hard to debug code is no fun. Lately I have been playing with Flume and writing and reviewing some fun modern C++. But... Being a parent tends to dominate free time. If Linc could choose, I would probably blog about trains, King Bob, and wheels.  Or the things we see in the Santa Cruz area like this Red shouldered hawk we say today.

And can I figure out how to get the blogger app to work well on Android?  Once I had added a picture, everything went south.  I finally ended up copying the text from the app and taking it over to my laptop to use a real keyboard and the blogger web interface.