Does anybody have suggestions or pointers for a demo Fortran 2008 file or files? I'd like to find a couple that do a good job of simply demonstrating excellent Fortran 2008 coding style. I'm looking for things like properly defining functions outside of modules, simple modules, pointers, and handling command line arguments, and parsing data files. What should people do if they want a linked list, hash/dictionary, deque (deck), or set? Are there implementations of these in the Fortran standard library?
I gave a somewhat pathetic attempt at doing this...
First impressions of Fortran 2008
Friday, November 12, 2010
Wednesday, September 1, 2010
Open Source and GSF - the "Generic Sensor Format" for multibeam sonars
I am repeating myself... see Generic Sensor Format (GSF) Meeting (Sept 2008).
What makes open source software successful is a community that contributes back to the code base to make it better. Val is making a huge step towards that for the multibeam Generic Sensor Format by working on a sidescan addition to the format and posting about how to use GSF: A GSF Primer. Val even called for a code review. Yesterday, 5 of us sat down with Val and the code to give it a look. Many eyes for review is a great thing (unlike design by committee, which typically makes everyone equally unhappy).
That said, I worry about people using GSF as an archive or interchange format for multibeam sonar data right now. Here are some of the issues, some of which can be fixed and others that are intrinsic to the design. There needs to be open discussion and I argue that the original data (pre GSF) and the code that generated that data need to be archived.
First, the name implies that it is a "generic" format, but if you look into the code. One look at the gsfSensorSpecific struct should put this question to rest. There is a huge amount of information in a GSF file that is not generic. For some vendors, there are even multiple model specific structures (I count 7 for Simrad/EM/Kongsberg). This comes from the rapid evolution of sonars since GSF was first started in the early 1990's (I see a first date of 1994). If we really do want to have a generic sonar format, I think we need to design a series of new messages that cover the basic multibeam and back scatter data returned such that we don't need these vendor specific payloads. How does the MGD77 format need a special case??? This is just x, y, z, depth, gravity, and magnetic field. The 77 means 1977. This format has been around for a long time.
The next thing that is needed is a major code over hall. This would entail distributing a better build system (maybe CMake) that builds proper libraries for all the major architectures. As a part of this, GSF needs a series of unit tests that take a very wide range of sample multibeam files and convert them to a GSF file. Then read these back in and verify that the resulting GSF file makes sense. Even simpler yet, we need code that exercises all of the GSF code base that doesn't need large input files to test it. This unit test suit needs to be public and the non-input-file based code should be a part of the standard build process - aka unit testing. These unit tests also serve a second purpose of providing documentation for how the library should be used. To go along with this code base update, the library should be put into the major linux distributions as a standard package. This will mean that the library can't be called "libgsf" as that conflicts with the libgsf that is the GNOME Structured File library. Gnome is going to trump the sonar library for Ubuntu, Debian, RedHat, etc.
The next code update would be to have functions that can do very basic validation of every structure that is passed around the GSF library. Client code can then call these to verify that they are, at least at a basic level, passing in data that makes sense. There is still tons of room for errors, but if roll is +/- 180 degrees, we should not pass in a roll of 720 degrees. NOTE: roll right now is +/- 90 degrees, which will cause trouble for vehicles under overhangs (e.g. under ice, under ships, or in caves). The no data value for roll is 99.0. That is going to be a problem. I guess we can have rolls that go from -270 to +90 to get around this.
We also need to look at the performance of GSF. A mmap implementation of GSF would likely be much faster. What else can be done to speed up the code? We should discuss the idea of a SQLite standard 2nd file to go along with GSF and other multibeam log files that is similar to what MBSystem does. If is contains the basic metadata and possibly a prebuild index of packets, anything beyond the first pass of a GSF file will go much faster. An example would be pulling the navigation and sound velocity profiles (SVP) from the tail of the prior log file, would be faster if the file was already indexed in a standard way.
A final step of the code update would be to turn on all the major compiler warnings and fix them. At least -Wall for gcc should return no warnings. There appear to be headers that should be included and lots of point magic that works right now, but should be better documented so compilers can check any code changes. Also, the readers and writers should probably be switched to use a set of inline functions that do type checking that wrap the byte swapping and memcpy packing. Is the code totally 32 and 64 bit safe for all execution paths???
A very useful addition would be to package native reader/writer interfaces the common languages used by people who process this kind of data. This means having perl, python, and matlab interfaces. This should be a part of GSF and distributed along side. I know many people who have written their own interfaces to GSF and, while that is instructional to create one, at least one for each major language should be included in the distribution.
Finally, the documentation that goes with GSF needs to be updated. I have heard from several people who have written GSF code that the documentation is not enough to write a working encoder/decoder. Missing from the specification document are a lot of the motivations behind these packets. SAIC has put 16 years of hard work into GSF and learned a lot of lessons that can benefit the whole sonar community. We need to capture this.
It is super important to note that SAIC is only able to work on GSF based on its contracts with the US Navy. Either someone needs to pay SAIC to do some of this work or we, as a community, need to get cracking on this if GSF is going to have staying power. The same goes for MBSystem and other critical software. The lead authors are up to their eyeballs in work. This is a plea for the community to jump in. I try to contribute back as much as possible, but am maxed out. Find any nitch and pitch in. It doesn't matter if your contribution is large or small. Code, documentation, translating documentation to other languages, quality bug reports, building test cases, even just learning the tools and how they work... it's all important.
What makes open source software successful is a community that contributes back to the code base to make it better. Val is making a huge step towards that for the multibeam Generic Sensor Format by working on a sidescan addition to the format and posting about how to use GSF: A GSF Primer. Val even called for a code review. Yesterday, 5 of us sat down with Val and the code to give it a look. Many eyes for review is a great thing (unlike design by committee, which typically makes everyone equally unhappy).
That said, I worry about people using GSF as an archive or interchange format for multibeam sonar data right now. Here are some of the issues, some of which can be fixed and others that are intrinsic to the design. There needs to be open discussion and I argue that the original data (pre GSF) and the code that generated that data need to be archived.
First, the name implies that it is a "generic" format, but if you look into the code. One look at the gsfSensorSpecific struct should put this question to rest. There is a huge amount of information in a GSF file that is not generic. For some vendors, there are even multiple model specific structures (I count 7 for Simrad/EM/Kongsberg). This comes from the rapid evolution of sonars since GSF was first started in the early 1990's (I see a first date of 1994). If we really do want to have a generic sonar format, I think we need to design a series of new messages that cover the basic multibeam and back scatter data returned such that we don't need these vendor specific payloads. How does the MGD77 format need a special case??? This is just x, y, z, depth, gravity, and magnetic field. The 77 means 1977. This format has been around for a long time.
The next thing that is needed is a major code over hall. This would entail distributing a better build system (maybe CMake) that builds proper libraries for all the major architectures. As a part of this, GSF needs a series of unit tests that take a very wide range of sample multibeam files and convert them to a GSF file. Then read these back in and verify that the resulting GSF file makes sense. Even simpler yet, we need code that exercises all of the GSF code base that doesn't need large input files to test it. This unit test suit needs to be public and the non-input-file based code should be a part of the standard build process - aka unit testing. These unit tests also serve a second purpose of providing documentation for how the library should be used. To go along with this code base update, the library should be put into the major linux distributions as a standard package. This will mean that the library can't be called "libgsf" as that conflicts with the libgsf that is the GNOME Structured File library. Gnome is going to trump the sonar library for Ubuntu, Debian, RedHat, etc.
The next code update would be to have functions that can do very basic validation of every structure that is passed around the GSF library. Client code can then call these to verify that they are, at least at a basic level, passing in data that makes sense. There is still tons of room for errors, but if roll is +/- 180 degrees, we should not pass in a roll of 720 degrees. NOTE: roll right now is +/- 90 degrees, which will cause trouble for vehicles under overhangs (e.g. under ice, under ships, or in caves). The no data value for roll is 99.0. That is going to be a problem. I guess we can have rolls that go from -270 to +90 to get around this.
We also need to look at the performance of GSF. A mmap implementation of GSF would likely be much faster. What else can be done to speed up the code? We should discuss the idea of a SQLite standard 2nd file to go along with GSF and other multibeam log files that is similar to what MBSystem does. If is contains the basic metadata and possibly a prebuild index of packets, anything beyond the first pass of a GSF file will go much faster. An example would be pulling the navigation and sound velocity profiles (SVP) from the tail of the prior log file, would be faster if the file was already indexed in a standard way.
A final step of the code update would be to turn on all the major compiler warnings and fix them. At least -Wall for gcc should return no warnings. There appear to be headers that should be included and lots of point magic that works right now, but should be better documented so compilers can check any code changes. Also, the readers and writers should probably be switched to use a set of inline functions that do type checking that wrap the byte swapping and memcpy packing. Is the code totally 32 and 64 bit safe for all execution paths???
A very useful addition would be to package native reader/writer interfaces the common languages used by people who process this kind of data. This means having perl, python, and matlab interfaces. This should be a part of GSF and distributed along side. I know many people who have written their own interfaces to GSF and, while that is instructional to create one, at least one for each major language should be included in the distribution.
Finally, the documentation that goes with GSF needs to be updated. I have heard from several people who have written GSF code that the documentation is not enough to write a working encoder/decoder. Missing from the specification document are a lot of the motivations behind these packets. SAIC has put 16 years of hard work into GSF and learned a lot of lessons that can benefit the whole sonar community. We need to capture this.
It is super important to note that SAIC is only able to work on GSF based on its contracts with the US Navy. Either someone needs to pay SAIC to do some of this work or we, as a community, need to get cracking on this if GSF is going to have staying power. The same goes for MBSystem and other critical software. The lead authors are up to their eyeballs in work. This is a plea for the community to jump in. I try to contribute back as much as possible, but am maxed out. Find any nitch and pitch in. It doesn't matter if your contribution is large or small. Code, documentation, translating documentation to other languages, quality bug reports, building test cases, even just learning the tools and how they work... it's all important.
Sunday, August 29, 2010
Which clock should I get?
A seriously geeky clock:
We need to get a new clock for the table across the room in the
bedroom. It would be best if it was readable at night without my
glasses (my vision isn't that bad, but a lighted display is essential)
and if it reset itself using radio broadcasts (refered to as a
clock) as we have so many power outages. More features would be
nice (weather, AM/FM/Scanner, etc), but I don't want a cluttered
display. What to people recommend?
NIST has a list of manufactures of time and frequency receivers
I'd like to spend less than $100.
We need to get a new clock for the table across the room in the
bedroom. It would be best if it was readable at night without my
glasses (my vision isn't that bad, but a lighted display is essential)
and if it reset itself using radio broadcasts (refered to as a
clock) as we have so many power outages. More features would be
nice (weather, AM/FM/Scanner, etc), but I don't want a cluttered
display. What to people recommend?
NIST has a list of manufactures of time and frequency receivers
I'd like to spend less than $100.
Friday, August 27, 2010
Thursday, August 12, 2010
NSFastEnumeration success?
I am definitely a beginner when it comes to Objective-C! Any help would be hugely appreciated.
This code works for me, but I really feel like it will certainly blow up on me in the future. For example, what if someone calls autorelease drain in the middle of a for loop. Also, what the heck is the difference between itemPtr and stackbuf? The documentation for NSFastEnumeration is really weak and my code is not behaving as described:
That's not very helpful. I'm using only itemsPtr and it works. What exactly am I supposed to do with stackbuf and how should I handle memory allocation/deallocation of stackbuf and itemsPtr??? I read this discussion on Implementing NSFastEnumeration and feel even less confident that I have any idea what is going on.
The code is online here: simple-fast-enum2.m. I know this is a lot of code, but hang in there with me.
Here is the output of the program that says that, at least in this case, it works.
This shows that code runs all the way through and deallocs all of the datagrams when the pool was drained. I'm leaking my itemsPtr array as "Finished loop" never appears in the output. I need to test for the loops to build itemsPtr does not add any items and free itemPtr then.
So... help! Is this right? How do I make it better? What should I be doing with stackBuf? How do I not get in trouble with a break?
This code works for me, but I really feel like it will certainly blow up on me in the future. For example, what if someone calls autorelease drain in the middle of a for loop. Also, what the heck is the difference between itemPtr and stackbuf? The documentation for NSFastEnumeration is really weak and my code is not behaving as described:
stackbuf
A C array of objects over which the sender is to iterate.
itemsPtr
A C array of objects
That's not very helpful. I'm using only itemsPtr and it works. What exactly am I supposed to do with stackbuf and how should I handle memory allocation/deallocation of stackbuf and itemsPtr??? I read this discussion on Implementing NSFastEnumeration and feel even less confident that I have any idea what is going on.
The code is online here: simple-fast-enum2.m. I know this is a lot of code, but hang in there with me.
001: #import <Foundation/Foundation.h>
002: #include <assert.h>
003:
004: @interface Datagram : NSObject
005: {
006: int dgId;
007: }
008: -(id)initWithDatagramType:(int)datagramType;
009: -(void)dealloc;
010: -(NSString *)description;
011: @property (readonly) int dgId;
012: @end
013:
014: @implementation Datagram
015: @synthesize dgId;
016: - (NSString *)description {
017: return [NSString stringWithFormat: @"Datagram: dgId:", dgId];
018: }
019:
020: -(id)initWithDatagramType:(int)datagramType {
021: self = [super init];
022: if (!self) return self;
023: dgId = datagramType;
024: return self;
025: }
026:
027: -(void)dealloc {
028: NSLog(@"dealloc datagram: %d",dgId);
029: [super dealloc];
030: }
031: @end
032:
033: // Pretend sequence of packet ID's coming from a sonar
034: int testSeq[] = {
035: 3, 12, 4, 19, 8,
036: 2, 2, 2, 2, 2, 2, 2, 2, 2, 9,
037: 2, 2, 2, 2, 9,
038: 2,2,2,2,9,
039: 1,2,3,4,5,6,7,8,9,
040: 11,12,13,14,15,16,17,18,19,
041: 3,
042: 0 // End of sequence / array sentinal
043: };
044:
045: @interface DatagramFile : NSObject <NSFastEnumeration>
046: {
047: // No ivars
048: }
049: -(id)init;
050: @end
051:
052: @implementation DatagramFile
053: -(id)init {
054: self = [super init];
055: if (!self) return self;
056: // NOP
057: return self;
058: }
059:
060: - (NSUInteger)countByEnumeratingWithState:(NSFastEnumerationState *)state objects:(id *)stackbuf count:(NSUInteger)len
061: {
062: NSLog(@"In countByEnumeratingWithState: stackbuf: %p, count: %d", stackbuf, len);
063: NSLog(@"\t state struct: state=%d %p %p", state->state, state->itemsPtr, state->mutationsPtr);
064: if (stackbuf) {
065: NSLog(@"***INSPECTING STACKBUF\n");
066: for(int i=0;i<1000 && stackbuf[i]!=0;i++) {
067: NSLog(@"Stackbuf %d: %p",i,stackbuf[i]); // What should I do with stackbuf[i]?
068: }
069: }
070: if (0 == state->state) {
071: NSLog(@"Initializing loop");
072: assert(0==state->itemsPtr);
073: state->itemsPtr = malloc(sizeof(id)*16);
074: memset(state->itemsPtr, 0, sizeof(id)*16);
075: } else if (0==len) {
076: // Will this get called if the call uses break inside the for loop?
077: NSLog(@"Finished loop. cleanup");
078: free(state->itemsPtr);
079: state->itemsPtr = 0;
080: return 0;
081: }
082: state->mutationsPtr = (unsigned long *)self; // Tell the caller that the file has not changed
083:
084: NSUInteger count=0;
085: for (; count < len && testSeq[state->state]!=0; count++, state->state++) {
086: NSLog(@"Creating datagram of type %d state: %d count %d",testSeq[state->state], state->state, count);
087: Datagram *dg = [[Datagram alloc] initWithDatagramType:testSeq[state->state]];
088: state->itemsPtr[count] = dg;
089: [dg autorelease];
090: }
091: NSLog(@"countByEnumeratingWithState read %d datagrams. state->state: %d",count, state->state);
092: return count;
093: }
094: @end // implementation DatagramFile
095:
096: int main (int argc, const char * argv[]) {
097: NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
098:
099: DatagramFile *df = [[DatagramFile alloc] init];
100: for (Datagram *dg in df) {
101: NSLog(@"About to read datagram in for");
102: NSLog(@" Datagram type: %d", [dg dgId]);
103: }
104:
105: NSLog(@"about to drain pool");
106: [pool drain];
107: NSLog(@"pool drained. ready for winter");
108: return 0;
109: }
Here is the output of the program that says that, at least in this case, it works.
2010-08-12 07:56:25.213 simple-fast-enum2[22266:a0f] In countByEnumeratingWithState: stackbuf: 0x7fff5fbfd930, count: 16
2010-08-12 07:56:25.215 simple-fast-enum2[22266:a0f] state struct: state=0 0x0 0x0
2010-08-12 07:56:25.216 simple-fast-enum2[22266:a0f] ***INSPECTING STACKBUF
2010-08-12 07:56:25.216 simple-fast-enum2[22266:a0f] Stackbuf 0: 0x7fff5fc005f8
2010-08-12 07:56:25.217 simple-fast-enum2[22266:a0f] Stackbuf 1: 0x7
2010-08-12 07:56:25.217 simple-fast-enum2[22266:a0f] Stackbuf 2: 0x7fff5fc3c028
2010-08-12 07:56:25.218 simple-fast-enum2[22266:a0f] Stackbuf 3: 0x7fff5fbfda40
2010-08-12 07:56:25.218 simple-fast-enum2[22266:a0f] Stackbuf 4: 0x7fff5fbfda00
2010-08-12 07:56:25.219 simple-fast-enum2[22266:a0f] Stackbuf 5: 0x7fff5fc016d2
2010-08-12 07:56:25.219 simple-fast-enum2[22266:a0f] Initializing loop
2010-08-12 07:56:25.220 simple-fast-enum2[22266:a0f] Creating datagram of type 3 state: 0 count 0
2010-08-12 07:56:25.220 simple-fast-enum2[22266:a0f] Creating datagram of type 12 state: 1 count 1
[snip - parsing data and adding it to itemPtr]
2010-08-12 07:56:25.227 simple-fast-enum2[22266:a0f] Creating datagram of type 9 state: 14 count 14
2010-08-12 07:56:25.227 simple-fast-enum2[22266:a0f] Creating datagram of type 2 state: 15 count 15
2010-08-12 07:56:25.228 simple-fast-enum2[22266:a0f] countByEnumeratingWithState read 16 datagrams. state->state: 16
2010-08-12 07:56:25.228 simple-fast-enum2[22266:a0f] About to read datagram in for
2010-08-12 07:56:25.229 simple-fast-enum2[22266:a0f] Datagram type: 3
[snip - inside for loop using datagrams]
2010-08-12 07:56:25.243 simple-fast-enum2[22266:a0f] About to read datagram in for
2010-08-12 07:56:25.244 simple-fast-enum2[22266:a0f] Datagram type: 2
[ snip - 16 blocks handled]
[ snip - 12 blocks handled ... finished the list of datagrams]
Last block...
2010-08-12 07:56:25.314 simple-fast-enum2[22266:a0f] In countByEnumeratingWithState: stackbuf: 0x7fff5fbfd930, count: 16
2010-08-12 07:56:25.314 simple-fast-enum2[22266:a0f] state struct: state=44 0x1001100a0 0x10010c760
2010-08-12 07:56:25.315 simple-fast-enum2[22266:a0f] ***INSPECTING STACKBUF
2010-08-12 07:56:25.315 simple-fast-enum2[22266:a0f] Stackbuf 0: 0x7fff5fc005f8
2010-08-12 07:56:25.316 simple-fast-enum2[22266:a0f] Stackbuf 1: 0x7
2010-08-12 07:56:25.316 simple-fast-enum2[22266:a0f] Stackbuf 2: 0x7fff5fc3c028
2010-08-12 07:56:25.317 simple-fast-enum2[22266:a0f] Stackbuf 3: 0x7fff5fbfda40
2010-08-12 07:56:25.317 simple-fast-enum2[22266:a0f] Stackbuf 4: 0x7fff5fbfda00
2010-08-12 07:56:25.318 simple-fast-enum2[22266:a0f] Stackbuf 5: 0x7fff5fc016d2
2010-08-12 07:56:25.318 simple-fast-enum2[22266:a0f] countByEnumeratingWithState read 0 datagrams. state->state: 44
2010-08-12 07:56:25.319 simple-fast-enum2[22266:a0f] about to drain pool
2010-08-12 07:56:25.319 simple-fast-enum2[22266:a0f] dealloc datagram: 3
2010-08-12 07:56:25.320 simple-fast-enum2[22266:a0f] dealloc datagram: 19
2010-08-12 07:56:25.320 simple-fast-enum2[22266:a0f] dealloc datagram: 18
2010-08-12 07:56:25.321 simple-fast-enum2[22266:a0f] dealloc datagram: 17
[ snip ]
2010-08-12 07:56:25.340 simple-fast-enum2[22266:a0f] dealloc datagram: 12
2010-08-12 07:56:25.341 simple-fast-enum2[22266:a0f] dealloc datagram: 3
2010-08-12 07:56:25.341 simple-fast-enum2[22266:a0f] pool drained. ready for winter
This shows that code runs all the way through and deallocs all of the datagrams when the pool was drained. I'm leaking my itemsPtr array as "Finished loop" never appears in the output. I need to test for the loops to build itemsPtr does not add any items and free itemPtr then.
So... help! Is this right? How do I make it better? What should I be doing with stackBuf? How do I not get in trouble with a break?
Wednesday, August 4, 2010
A cleaner way to have an NSMutableArray of counters?
How do I make this clean. I know how to make this really simple with a C array or a C++ vector, but I'm trying to learn ObjC! Any help greatly appreciated.
NSMutableArray *count_id = [NSMutableArray new];
// Can this be done better for an initialization?
for (int i; i < 256; i++) {
[count_id addObject:[NSNumber numberWithInt:0]];
}
NSData *data = [NSData dataWithContentsOfFile:fileName];
for (size_t dgStart = 0; dgStart < data.length; ) {
unsigned char id;
unsigned int size;
[data getBytes:&size range:NSMakeRange(dgStart, 4)];
[data getBytes:&id range:NSMakeRange(dgStart+5, 1)];
//
// Yuck! I just want to increment the number by 1. How do I improve this?
//
[count_id replaceObjectAtIndex:id withObject: [NSNumber numberWithInt:[[count_id objectAtIndex:id] integerValue]+1 ] ];
// Jump to the next packet
dgStart += size+4;
}
Tech and science podcasts?
I have been listening to a lot of FLOSS
podcasts lately. What suggestions do you all have for great
podcasts for Mac, Linux, spatial stuff, and science pod casts?
podcasts lately. What suggestions do you all have for great
podcasts for Mac, Linux, spatial stuff, and science pod casts?
Subscribe to:
Posts (Atom)