Sunday, November 15, 2026

Creating a synthetic test GeoTIFF image

 Recently, I was working on overview algorithms for Cloud Optimized GeoTIFFs (COGs). These extra smaller versions of a larger image are embedded in an image to help algorithms present a representation of the overall full resolution data much quicker. Downsampling data ahead of time can greatly optimize many processes that don’t need the full resolution data. The easiest task to think about that is benefited by overviews is visualization by humans. If you are zoomed out on a map, you would like to see an overview of what data is in that map. If, for example, you have 1 meter data, it’s not reasonable to fetch all of that 1 meter data to downsample the data into the few pixels that will be on your screen. Having a low resolution overview means that the computer only has to fetch a few pixels and it can get the user a nice view in a short time with little work. One of the key questions then is what algorithm should we use to produce these down-sampled extra images? GDAL’s COG driver lists the possible algorithms that it has available for overviews:


OVERVIEW_RESAMPLING=[NEAREST​/​AVERAGE​/​BILINEAR​/​CUBIC​/​CUBICSPLINE​/​LANCZOS​/​MODE​/​RMS]: (GDAL >= 3.2) Resampling method used for overview generation. For paletted images, NEAREST is used by default, otherwise it is CUBIC. This overrides, for overview generation, the value of RESAMPLING if it specified.


When should I use a particular resampling algorithm and why? Some of these are fairly obvious, but some I had no intuition for. If I have categorical data where an integer in each pixel has an assigned mean (e.g., 1 is grass land, 2 is forest, 3 is paved, 4 is bare ground, 5 is buildings, … 8 is water, etc.), then it makes no sense to use an algorithm that will interpolate neighboring pixels to produce a down sampled image. If we are trying to represent 9 pixels with 1 in a lower representation, we might have 6 pixels of grass (value 1) and 3 of water (value 8). If we use an average of the neighboring pixels, we have (1 * 6 + 8 * 3) / 9. = 3.33… Sticking with an integer representation, we might then round that to 3 and declare in the overview that the pixel is paved ground. That makes no sense. It might be better to use the nearest value or the most common value (mode). For elevation data, it might be best to use a cubic or cubic spline to interpolate the best representation. But maybe not. What properties are we trying to preserve in the overview and what algorithm will achieve that goal?


For each of the algorithms, it’s possible to apply it to a bunch of sample data and see what it does. That will give us concrete examples that we can look at, build intuition, and use algorithms to evaluate. So, what types of sample data should we use? And what if different people pick different data for the same evaluation? Two satellite images of even the same location on earth are likely to have different properties. The time of day may be different, the seasons change, land use may change, etc.  And those differences might be even more dramatic if we pick different parts of the planet. If one person picked mountain forest and the other picked flat desert, then the results are likely super different.


We can have a whole library of sample images. Picking the images that should be in a catalog of key test images is challenging, but we can do it. These might include: constant images, impulse points, phase sweeps, a Siemens star, a USAF-1951 bar target, steps, slants/edges, ISO 12233 camera calibration targets, nodata zones, thin features, classes/categorical data, ramps, gradients, terraces, dipoles, peaks/valleys/saddles, satellite images, images of animals, various types of noise, text, logos, and more.


Can we create a single sample image that contains a range of data types that will cover a wide range of situations and give us carefully crafted data that can help bring out various aspects of algorithms? Why yes of course! But what should that key single image look like? I think it works well if we designate sub regions of the images (a.k.a., chips) that each contain a specific data type. That way, we only have to track one image, right? Well, we might need to have cases for integer input and floating point input images. We might need different precision (int8, uint8, … int64, uint64 and float8…float64, and a range of complex values). What about algorithms that operate on color, multispectral, or hyperspectral images? If we keep things simple, we can start with an int16 single band image and a float32 single band image.


I’ve wanted this kind of thing for a long time, but never had the time to put it together. But once I got access to Opus 5.5, I figured I’d let an LLM agent give it a go. I wrote up a prompt and had the agent work in an explicit Specification Driven Development (SDD) mode. The results are here:


https://github.com/schwehr/gdal-autotest2/tree/master/generate/example_grid


Take a look at the Product Requirements Document in PRD.md, PLAN.md, and TASKS.md to see how it progressed. It created a 6x14 grid for a total of 84 of chips. Here are the results. I’ve placed them over a red background so you can see where there are nodata values.



This is already down sampled to be on this screen, so I encourage people to play with the original. Here is GDAL’s summary of the int16 version:


gdalinfo resample_grid_int16.tif

Driver: GTiff/GeoTIFF

Files: resample_grid_int16.tif

Size is 6848, 15840

Coordinate System is:

PROJCRS["WGS 84 / Pseudo-Mercator",


[SNIP]


   ID["EPSG",3857]]

Data axis to CRS axis mapping: 1,2

Origin = (0.000000000000000,0.000000000000000)

Pixel Size = (10.000000000000000,-10.000000000000000)

Metadata:

  GDAL_VERSION=3.12.0dev-0eb3a1a5fa

  RESAMPLE_GRID_DTYPE=int16

  RESAMPLE_GRID_LAYOUT={"width": 6848, "height": 15840, "raw_width": 6844, "raw_height": 15836, "panel_px": 1024, "guard_px": 100, "margin_px": 100, "columns": 6, "rows": 14, "overview_factors": [2, 4, 8, 16, 32]}

  RESAMPLE_GRID_SEED=559677208

  RESAMPLE_GRID_VERSION=0.1.0

  AREA_OR_POINT=Area

Image Structure Metadata:

  COMPRESSION=ZSTD

  INTERLEAVE=BAND

  PREDICTOR=2

Corner Coordinates:

Upper Left  (   0.0000000,   0.0000000) (  0d 0' 0.01"E,  0d 0' 0.01"N)

Lower Left  (       0.000, -158400.000) (  0d 0' 0.01"E,  1d25'22.03"S)

Upper Right (   68480.000,       0.000) (  0d36'54.60"E,  0d 0' 0.01"N)

Lower Right (   68480.000, -158400.000) (  0d36'54.60"E,  1d25'22.03"S)

Center      (   34240.000,  -79200.000) (  0d18'27.30"E,  0d42'41.21"S)

Band 1 Block=512x512 Type=Int16, ColorInterp=Gray

  NoData Value=-9999


It’s a GeoTIFF, so you can add it to a map in QGIS, Earth Engine, or other GIS tools. You can also use gdal_translate to place it anywhere you’d like on a globe. I used that idea combined with `gdal_translate` using the COG driver with the various overview algorithms to see how they work. The results have been super useful for me so far. Here is placing it on a map:



I encourage people to try out this image when learning new image processing / GIS algorithms. Or make your own calibration / test images that fit your needs. Everything in the sample image is LLM generated except that cat image (from SKLearn), the font for the text, and the (now old) Earth Engine logo. You are welcome to the machine generated images under the CC0 license.




 


Friday, November 6, 2026

LLM generated C++ test coverage of GDAL

 First off, even if GDAL were up for taking this LLM generated code by policy, the following is not ready. To start with, there are many quirks in there specific to the google3 monorepo.

For the local use of GDAL in google3, I've been in need of better C++ test coverage of GDAL to guard against trouble when updating from GDAL's master branch. I started gdal-autotest2 more than 10 years ago. It was slow going to get the 22K lines of C++ test code that I had in there. So, I've taken a go at creating a multiagent setup with one agent trying to write per GDAL C++ file test coverage that is > 90% test coverage and have multiple tools and LLM agents review the code. The result is 360K more lines of googletest based C++ test code. It's set up to not use files from disk. As such, the whole thing builds and runs much quicker than the python test code (which is still used). The tests have lots of redundancy and google3 only has a small subset of GDAL's drivers in it. A test suite covering all the GDAL drivers done this way would likely have more than double the lines of code. Be warned that these new tests are not human curated, so there are plenty of silly tests. But even with all those caveats, some folks may find the code useful. The repo is Apache 2.0 license, but I am fine with the license changing to MIT licensed for folks porting it to other repositories. 

https://github.com/schwehr/gdal-autotest2/tree/master/cpp

I'm planning to do the same in cpp/third_party for geos, hdf4, hdf5, libgeotiff, netcdf, proj, and maybe some others, but no promises. 

I see these tests as being in a similar category to fuzzers + a corpus of files that cover as many branches as possible. I used to maintain order 1M files for fuzzers to run though on each commit, but that had diminishing returns and was hard to maintain/update.

Tuesday, September 8, 2026

Surprised by the surprise about LLMs and agents

Two things recently surprised me in the world of LLMs and agent harnesses. My surprise is that these people are surprised by what they found.

  1. Steve Gibson and Leo Laporte were surprised that agents are just a stream of tokens with tool usage and other things are just tokens in the stream, which are just hints.
  2. OpenAI, Anthropic, and others are surprised that LLMs + harnesses under test escaped and cause trouble in multiple rounds.
First, but newer, SN 1093 (Security Now) has a section "Tokens in the Stream" talking about Prompt Injection as Role Confusion. I am surprised by Steve and Leos' surprise. Why? LLMs are just token streams and all this interaction is just convention. It was pretty obvious from the papers that Steve has said he has been reading, listening to Andrej Karpathy's lectures, and just observing these things how the model works. I have hated this phrase when I heard it in my graduate classes, but now I'm going to say it: "It obviously follows" that these things are a security catastrophe where we can do no better than train out known attacks and whack-a-mole on bad patterns in the harnesses. Think SQL injections, but worse as there very limited boundaries.

Second, people testing new models should always assume the worst and there are established ways to handle the worse. If you are testing a new model trained on the internet, it has all of the tricks and bad things on the internet for security holes, side channels, and the likes. Humans have been writing about spy-versus-spy and the like for thousands of years. Every silly hack that ever has been written about is in their training dataset. That includes side channels like blinking LEDs, pulsing power usage, vibration, skewing timings, writing data in odd places (e.g., DNS records, filename paths, metadata fields, steganography). Heck, I baked metadata into file name paths for Mars mission data files: the path to a file told you the spacecraft, the sensor, the sensor settings, the processing type, when it was collected, and more all just by parsing the full file path. These machines have IRC logs from hacking sessions. They have archives of CVEs, CWEs, and more. They probably have scrapings of the old rootshell dot org that I used back in the 90s. And they have trained on ridiculous amounts of source code.

If you want to test a new LLM, then you should really secure it. Hearing about the mistakes that the testers make is more surprising to me than anything. My not test in on a system that is physically disconnected the world during testing? Separate the network write lines and use a read only optical serial tap to follow the machines progress. When you are done with the test, power down the system, scrub the disks, BIOSes, and anything else the system could mutate. Treat all the read data as toxic and "red shell" it when inspecting it. It's not hard to create some really nasty stuff. For example, I have unintentionally created image files during fuzzing that took down the entire machine it was on if the desktop system ever touched the file (thank you OS for being helpful and scanning all new files so you are ready to help me search for them).

I guess what is really old is new. Sigh.

Written without aid of AI.

Monday, August 3, 2026

LLMs aren't so smart

 I'm working on updating my noaadata packages using spec-driven development (SDD). This stuff is definitely got a ways to go on smarts. It's doing plenty very well, but then it did this where it changed "Viet Nam" to "Viet Name" rather than "Vietnam". Sigh. I fixed it in 7233879.

--- a/data/mmsi_prefix.sql
+++ b/data/mmsi_prefix.sql
@@ -203,7 +203,7 @@ INSERT INTO "mmsi_prefix" VALUES(565,'Singapore (Republic of)','Singapore');
 INSERT INTO "mmsi_prefix" VALUES(567,'Thailand','Thailand');
 INSERT INTO "mmsi_prefix" VALUES(570,'Tonga (Kingdom of)','Tonga');
 INSERT INTO "mmsi_prefix" VALUES(572,'Tuvalu','Tuvalu');
-INSERT INTO "mmsi_prefix" VALUES(574,'Viet Nam (Socialist Republic of)','Viet Nam');
+INSERT INTO "mmsi_prefix" VALUES(574,'Viet Name (Socialist Republic of)','Viet Name');
 INSERT INTO "mmsi_prefix" VALUES(576,'Vanuatu (Republic of)','Vanuatu');
 INSERT INTO "mmsi_prefix" VALUES(578,'Wallis and Futuna Islands','Wallis and Futuna Islands');
 INSERT INTO "mmsi_prefix" VALUES(601,'South Africa (Republic of)','South Africa');

Tuesday, July 21, 2026

BitVector 0.0.4 - pylint focused

I'm learning lots. I'd love feedback and suggestions from the python community.

AI generated summary:

The v0.0.4 release of bitvector-modern primarily focuses on extensive code cleanup, linter adherence, and testing improvements. A major portion of the updates involves setting up pylint configurations (including the pylint-pytest plugin) and refactoring code across the repository to address linter rules—such as updating function/class docstrings, switching to f-strings and with statements, simplifying chained comparisons, and cleaning up break, return, and raise statements. Additionally, the release enhances overall quality by expanding unit test coverage for rotation logic, bit shifts, and BitVector methods such as reset(), __int__, __getitem__, and __setitem__, while resolving a bug related to slice assignment length calculations with negative start indices.

Sunday, July 19, 2026

bitvector-modern release 0.0.3

 I just did the first proper release of bitvector-modern with version 0.0.3. I hadn't setup 2 factor authentication (2FA) for my personal PyPI identity, but now I have so I can actually send updates out. I have been out of town with only mobile phone access to the internet, so I have been doing a bunch of updates to the package via Google Jules. That process went really well in general. I had some trouble getting it to do more ambitious changes line adding mmap and numpy based implementations, but you should see from release notes that there were plenty of successes.

Sadly, I did the 0.0.2 release just before this incorrectly, so that's where the changes are listed, but that triggered a 0.0.1 pypi release. I could yank it, but I don't think I need to as nothing is broken. The changes for 0.0.3 are in 0.0.1 on pypi.

I wouldn't recommend using bitvector-modern now for anything other than experimental work as I intend to rework the API in a few intrusive ways. Top of the list is simplifying __init__ and putting the extra functionality into class methods of the for from_. I have already removed the filename and filepointer args from __init__, but I have yet to add back file reading outside of __init__.

At this point, there are quite a few performance improvements over the old library. For starters, I switched the array.array storage from 16-bit to 64-bit blocks. This means that BitVectors smaller than 48-bits will take more storage, but that should not be a problem unless folks are working with large numbers of very small bitvectors that have to remain in memory. With the in-progress Protocol class work, that issue should be mitigated as the package should be able to let the user pick an implementation that works best with their use case.

I hope to start working on my old noaadata package soon and port it to bitvector-modern. I'm excited for the increase performance and improved usability.

The release:

The detailed release notes are here:

From the 0.0.2 release notes:




Friday, July 10, 2026

The future of Generic Sensor Format (GSF) for sonar data and processing

This is a response to this post: Lindsay Gee: GSF meeting at CCOM next week. I won’t be able to attend the meeting next week at CCOM even remotely, so I’ll try to capture my current take on GSF. I’ve not been shy in saying that the Generic Sensor Format (GSF) is not my vision of a generic sharing / processing format for multibeam sonar. However, it is critically important to the community. The key text from Lindsay:


The toolkit supports the GEBCO Technical Strategy by improving the sharing, archiving, and reuse of processed swath bathymetry data. It is designed to enhance interoperability across software applications, reduce barriers to working with GSF data, and support the generation of bathymetry and backscatter products without always needing to return to raw sonar data.


The first public release of the open-source toolkit, documentation, and code repository is available, and this session will provide an overview of the project, its current status, roadmap for future development, followed by an open discussion.


A key component of the project is a profile framework and JSON-based schema that captures differences between GSF library implementations. As this framework expands, community participation and feedback will be essential to ensure it meets the needs of the broader hydrographic and ocean mapping community.


The success of the toolkit depends on broad community engagement, and we welcome feedback from software developers, data producers, archives, and end users.


https://github.com/oceanmapping/mbtoolkit 


Trying to get my thoughts organized quickly is tough. Apologies as this is a bit disorganized.


I see there are at least 3 key packages that need to exist and be well maintained: The original GSF library from SAIC/Lidos originally written in C that should be converted to C++, the pure python package mbtoolkit, and a complete rewrite from scratch of a gsf library in Rust. Those packages should be on a public VC repo like GitHub, GitLab, etc and have permissive licenses allowing the most number of people to use and contribute to them. In the age of LLM coding agents, getting this work done will go faster than it ever could have in the past, but it’s not easy to do this correctly and make the results maintainable. But I have high hopes considering the positive messages from folks around the community recently.


0. Common to all three - best practices


For each package, they should be done to the best available modern software engineering practices. The surface area is pretty small, so achieving this isn’t that hard. What does it entail? (for Rust, a lot of this is already baked in)


  • Code in git VC and publicly available with the ability to take pull requests. I will assume GitHub for now.

  • Standard package management for the repo (a.k.a., CMake for C/C++, uv for Python, Cargo for Rust) 

  • Automatic code formatting so that code is always consistent no matter who contributes a change

  • Static analyzers automatically run (Coverity, cppcheck, clang static analyzer. ty/mypy/pyright/pyrefly, codespell, etc.)

  • 100% unittest coverage

  • A shared set of gsf files used for integration testing

  • Coverage based fuzzing to detect all sorts of bugs and provide artifacts that can be used to add testing

  • Performance benchmarks

  • GitHub actions and local automation to run all the checks (e.g. pre-commit)

  • The best security scanning available (e.g., zizmor for GitHub actions, bandit for python, CodeQL, etc.)

  • Great documentation built with each of the packages

  • A strong defensive AGENTS.md


And a part of this should be moving the GSF spec from a PDF to a GitHub project written in markdown.


1. The future of the C/C++ libgsf


It has been frustrating that SAIC/Lidos developed GSF behind closed doors and only occasionally released versioned snapshots to the community. It was hard to see what was going on with development and the community wasn’t able to contribute fixes. I tried to jump start this process with https://github.com/schwehr/generic-sensor-format/ back in 2015 (check out all the issues that I documented). I wasn’t able to get any engagement and there wasn’t any uptake as far as I can tell. This work should be started from the most recent release.


This is about fortifying the valuable historical usage of the GSF library. This is the reference implementation and extreme care should be taken not to lose the value that comes with that. As a community, we should make sure that existing usage can continue without requiring major reworking of code by the current users.


What should this entail? First off, I should be clear about switching to C++. I don’t mean converting the core to object oriented design. I do mean that converting the code from C to C++ means that the code base can use the C++ std library for numerical values, more const-ness, constexpr, static_asserts, safer casting, and internal use of unique_ptr for safer memory management. The core library should still export an `extern “C”` interface with the same calls. From there, it is reasonable to add an optional wrapper later on top that implements a more natural C++ interface that can be done as a class that knows how to manage memory and presents a more natural C++ error interface with std::optional, exceptions, and/or StatusOr<T>.


After that, it’s reasonable to provide a parallel all C++ from scratch rewrite of GSF that takes advantage of all the modern (and some not so modern ๐Ÿ™‚) capabilities like mmap, select, epoll, etc.


Maybe there should be language wrappers in one or both of these, but I’m not sure what is best. swig is pretty frustrating and I don’t know what else is around that has a solid future.


2. Pure Python


The community should get behind https://github.com/oceanmapping/mbtoolkit or something similar. Drive it to completely cover GSF with pure python. Working with Python is great for learning and developing protocols even if it’s not the fastest. I suggest aiming for python >= 3.14 and pushing for the strongest possible package. 


An example of some of what can be done for packages when starting from scratch can be seen in https://github.com/schwehr/bitvector-modern. With the help of a coding agent and a lot of careful reviews, I was able to get that package to a pretty good place. Please adapt the best ideas from that and any other solid package out there. Old school python packaging and setups was creaky and painful.


3. A Rust implementation


The Rust Cargo ecosystem is amazing. Unlike python and C++, there is one primary packaging tool: Cargo. The trick is making a Rust based IO system that is both performant and reasonable to maintain. The async code in Rust has a bad reputation for many people. I have little experience, but there is great potential. I suggest having LLM agents cook up a range of designs and compare the results.


Once that basic IO layer is done, that will unlock a huge potential. First, many low level codecs have recently gotten pure Rust replacement libraries that can be used in place of old C libraries gaining both safety and performance (e.g., zlib, png, jxl). Second, Rust makes a great ecosystem to build CLIs, TUI (text/terminal user interfaces), and GUIs/3D interfaces, c.f. ratatui and Bevy. It even works well to have Bevy use ratatui to build terminal based 2D applications that can be extremely useful.


Conclusion


The community needs security, stability, performance, and collaboration with GSF. This post leaves out a lot, but I hope it gets people thinking, motivated, prototyping, and implementing. The time to start is now!