Back a few years ago, I wrote slogcxx as an experiment in creating and easy to use logging system. It spent a little under a year as the logging system for GeoZui. I haven't touched the code in a long time and I am sure it can be done better. So, I have two questions that were triggered by a discussion with BRC. What packages would you suggest for C++ logging and unit testing. I did not including anything real in libais and would really like to work towards the best possible infrastructure to make this a long lived package. Some basic thoughts on requirements:
- Must not be GPL, but LGPL is okay. It has to play nice
- Be open source
- Must be thread safe - a unit test may fire off a bunch of threads for a test
- The logger must be able to handle multiple producers and consumers that are both in the same process and external
- Likely to stick around for a few years
- Relatively easy to use
- Cross platform to Windows, Mac, and Linux
- If not already available in fink, rpm, and deb form, then easily packaged
- Have a safe for work project name... ahem
Just looking on Freshmeat, there are a lot of projects out there and slogcxx is in the pile still: C++ logging and C++ unit testing. The thought is often to look at Boost (e.g. Boost Logging Lib v2 and Boost test) because of the incredible peer review, but it is a pretty heavy dependency.
There are just so many options. Building a large system on the wrong one will lead to serious pain, but using a good one will encourage better software.
Dreaming of being able to subscribe to an AMQP feed for a cluster and user interface tearing through massive data crunching and knowing that the system passed a slew of unit tests that give us confidence that new code is less likely to pull the system down...
Another source of info: Wikipedia List of unit testing frameworks. There doesn't seem to be an equivalent page for loggers, but there is a little in the Java lib4j ports section.
P.S. I'm really starting to think that GNU autoconf sucks. It may be the standard, but a configure system is supposed to help, not break my brain. It shouldn't require massive training and experimentation to get it into place. I really should go look at cmake and scons again.
Dan commented offline. Here is what he said:
ReplyDeleteYou might want to put together a desired feature list: timestamps (and precision), formatting options, simultaneous screen+file, logging levels vs selectors, ... See More extendibility, parsing support, log rotation, etc.
One thing that can be nice is to be able to log in a highly compact format, like 'protobufs' or 'thrift'. XML can really bloat logs (although compression can help a lot). An added benefit is that binary logs don't lose precision for numbers and can be parsed much faster. You can guess where I was worried about massive log volume.
Maybe you should research "standard" log formats and parsing tools, and then work backward from there. Just a thought.
As for the build system, I switched mstone to cmake and never looked back. scons replaces 'make', but I don't think it really handles the platform identification issues that autoconf or cmake address (but then I haven't really used scons).
Thrift is interesting. It's similar to Google protobufs, but the code is considerably more complete than what Google has released (so far).
I'm not sure of your goals for the logs. I ran across a nice comparison of (gui) system log viewers. It might be worth looking at the complexity of the file formats they parse and how the highlighting/triggering works. It may be too simple for what you want, but it's a data point.
ReplyDeletehttp://www.linuxlinks.com/article/20100603155736857/LogfileViewers.html
-Dan
Ran into another serialization system, msgpack. http://msgpack.org/
ReplyDeleteLooks nice, but I haven't used it.
-Dan
And yet another one: http://avro.apache.org/docs/current/
ReplyDeleteAnd a log file processor (that's probably too big for what you're doing).
http://github.com/cloudera/flume
-Dan