This is a response to this post: Lindsay Gee: GSF meeting at CCOM next week. I won’t be able to attend the meeting next week at CCOM even remotely, so I’ll try to capture my current take on GSF. I’ve not been shy in saying that the Generic Sensor Format (GSF) is not my vision of a generic sharing / processing format for multibeam sonar. However, it is critically important to the community. The key text from Lindsay:
The toolkit supports the GEBCO Technical Strategy by improving the sharing, archiving, and reuse of processed swath bathymetry data. It is designed to enhance interoperability across software applications, reduce barriers to working with GSF data, and support the generation of bathymetry and backscatter products without always needing to return to raw sonar data.
The first public release of the open-source toolkit, documentation, and code repository is available, and this session will provide an overview of the project, its current status, roadmap for future development, followed by an open discussion.
A key component of the project is a profile framework and JSON-based schema that captures differences between GSF library implementations. As this framework expands, community participation and feedback will be essential to ensure it meets the needs of the broader hydrographic and ocean mapping community.
The success of the toolkit depends on broad community engagement, and we welcome feedback from software developers, data producers, archives, and end users.
https://github.com/oceanmapping/mbtoolkit
Trying to get my thoughts organized quickly is tough. Apologies as this is a bit disorganized.
I see there are at least 3 key packages that need to exist and be well maintained: The original GSF library from SAIC/Lidos originally written in C that should be converted to C++, the pure python package mbtoolkit, and a complete rewrite from scratch of a gsf library in Rust. Those packages should be on a public VC repo like GitHub, GitLab, etc and have permissive licenses allowing the most number of people to use and contribute to them. In the age of LLM coding agents, getting this work done will go faster than it ever could have in the past, but it’s not easy to do this correctly and make the results maintainable. But I have high hopes considering the positive messages from folks around the community recently.
0. Common to all three - best practices
For each package, they should be done to the best available modern software engineering practices. The surface area is pretty small, so achieving this isn’t that hard. What does it entail? (for Rust, a lot of this is already baked in)
Code in git VC and publicly available with the ability to take pull requests. I will assume GitHub for now.
Standard package management for the repo (a.k.a., CMake for C/C++, uv for Python, Cargo for Rust)
Automatic code formatting so that code is always consistent no matter who contributes a change
Static analyzers automatically run (Coverity, cppcheck, clang static analyzer. ty/mypy/pyright/pyrefly, codespell, etc.)
100% unittest coverage
A shared set of gsf files used for integration testing
Coverage based fuzzing to detect all sorts of bugs and provide artifacts that can be used to add testing
Performance benchmarks
GitHub actions and local automation to run all the checks (e.g. pre-commit)
The best security scanning available (e.g., zizmor for GitHub actions, bandit for python, CodeQL, etc.)
Great documentation built with each of the packages
A strong defensive AGENTS.md
And a part of this should be moving the GSF spec from a PDF to a GitHub project written in markdown.
1. The future of the C/C++ libgsf
It has been frustrating that SAIC/Lidos developed GSF behind closed doors and only occasionally released versioned snapshots to the community. It was hard to see what was going on with development and the community wasn’t able to contribute fixes. I tried to jump start this process with https://github.com/schwehr/generic-sensor-format/ back in 2015 (check out all the issues that I documented). I wasn’t able to get any engagement and there wasn’t any uptake as far as I can tell. This work should be started from the most recent release.
This is about fortifying the valuable historical usage of the GSF library. This is the reference implementation and extreme care should be taken not to lose the value that comes with that. As a community, we should make sure that existing usage can continue without requiring major reworking of code by the current users.
What should this entail? First off, I should be clear about switching to C++. I don’t mean converting the core to object oriented design. I do mean that converting the code from C to C++ means that the code base can use the C++ std library for numerical values, more const-ness, constexpr, static_asserts, safer casting, and internal use of unique_ptr for safer memory management. The core library should still export an `extern “C”` interface with the same calls. From there, it is reasonable to add an optional wrapper later on top that implements a more natural C++ interface that can be done as a class that knows how to manage memory and presents a more natural C++ error interface with std::optional, exceptions, and/or StatusOr<T>.
After that, it’s reasonable to provide a parallel all C++ from scratch rewrite of GSF that takes advantage of all the modern (and some not so modern 🙂) capabilities like mmap, select, epoll, etc.
Maybe there should be language wrappers in one or both of these, but I’m not sure what is best. swig is pretty frustrating and I don’t know what else is around that has a solid future.
2. Pure Python
The community should get behind https://github.com/oceanmapping/mbtoolkit or something similar. Drive it to completely cover GSF with pure python. Working with Python is great for learning and developing protocols even if it’s not the fastest. I suggest aiming for python >= 3.14 and pushing for the strongest possible package.
An example of some of what can be done for packages when starting from scratch can be seen in https://github.com/schwehr/bitvector-modern. With the help of a coding agent and a lot of careful reviews, I was able to get that package to a pretty good place. Please adapt the best ideas from that and any other solid package out there. Old school python packaging and setups was creaky and painful.
3. A Rust implementation
The Rust Cargo ecosystem is amazing. Unlike python and C++, there is one primary packaging tool: Cargo. The trick is making a Rust based IO system that is both performant and reasonable to maintain. The async code in Rust has a bad reputation for many people. I have little experience, but there is great potential. I suggest having LLM agents cook up a range of designs and compare the results.
Once that basic IO layer is done, that will unlock a huge potential. First, many low level codecs have recently gotten pure Rust replacement libraries that can be used in place of old C libraries gaining both safety and performance (e.g., zlib, png, jxl). Second, Rust makes a great ecosystem to build CLIs, TUI (text/terminal user interfaces), and GUIs/3D interfaces, c.f. ratatui and Bevy. It even works well to have Bevy use ratatui to build terminal based 2D applications that can be extremely useful.
Conclusion
The community needs security, stability, performance, and collaboration with GSF. This post leaves out a lot, but I hope it gets people thinking, motivated, prototyping, and implementing. The time to start is now!
