Thursday, December 1, 2011
Wednesday, November 30, 2011
RT 25 - R for Statistics
Rob Braswell was our guest lecturer covering R for statistical analysis.
mp3, pdf of screenshots, 25-R-lab1-Intro.pdf, 25-r-statistics.org (on bitbucket via hg).
mp3, pdf of screenshots, 25-R-lab1-Intro.pdf, 25-r-statistics.org (on bitbucket via hg).
Wednesday, November 23, 2011
RT 24 - Part 4: Parsing binary SBET files with python's struct
Topics include "What is GIS," using glob.glob to expand file names with "*", using sys.argv directly to list input files and using argparse to properly handle command line arguments.
html, pdf, mp3 and org (in BitBucket hg).
html, pdf, mp3 and org (in BitBucket hg).
Saturday, November 19, 2011
RT 23 - Part 3: Parsing binary SBET files with python's struct
In this class, we do our first mercurial pull of changes. We then add helper functions to our sbet.py module to give us the number of datagrams in an sbet file, tell us at what offset any particular datagram is located and add a generator function allowing cleaner for loops over sbet files.
23-python-binary-files-part-3.html, mp3 and pdf
Remember that at this point, you should be getting the org mode formatted class notes via mercurial (hg). To get set up:
And every time you start working on the class, do a pull and update to get the latest versions.
23-python-binary-files-part-3.html, mp3 and pdf
Remember that at this point, you should be getting the org mode formatted class notes via mercurial (hg). To get set up:
mkdir ~/projects
cd ~/projects
sudo apt-get install mercurial # Install hg on ubuntu & debian linux
hg clone https://bitbucket.org/schwehr/researchtools
And every time you start working on the class, do a pull and update to get the latest versions.
cd ~/projects/researchtools
hg pull # Bring the changes down to the local "repo"
hg update # Change the working files to have the latest changes
Wednesday, November 16, 2011
RT 22 - Part 2: Parsing binary SBET files with python's struct
Rob Braswell has tentatively agreed to give us a lecture on Data Analysis with R on Nov. 29th. Rob taught EOS 864 while he was a full-time Research Professor at UNH. I haven't done any work with R myself, but I've seen some really great work done with it. I'm super excited to sit in on his class.
Notes for the class should now be retrieved via mercurial/hg from https://bitbucket.org/schwehr/researchtools. The rest of the material is in the usual locations in the class directory: mp3, pdf, html
Notes for the class should now be retrieved via mercurial/hg from https://bitbucket.org/schwehr/researchtools. The rest of the material is in the usual locations in the class directory: mp3, pdf, html
Friday, November 11, 2011
RT 21 - Parsing binary SBET files with python's struct
Using struct to decode Applanix POSPAC SBET navigation files:
21-python-binary-files.org,
html, mp3 and pdf
21-python-binary-files.org,
html, mp3 and pdf
Sunday, November 6, 2011
RT 20 - BAGs 3
Thursday, November 3, 2011
RT 19 - BAGs 2, XML Metadata
Wednesday, November 2, 2011
Friday, October 21, 2011
Friday, October 14, 2011
Research Tools Lecture 13 - if, while, functions, classes
13-python-if-while-def-class.mp3 and org, html, and pdf
I go over boolean expressions (1==1 gives True and 1==2 gives False),
while loops, functions, classes, and start into collecting data with
socat from a weather station on the roof of our building.
I go over boolean expressions (1==1 gives True and 1==2 gives False),
while loops, functions, classes, and start into collecting data with
socat from a weather station on the roof of our building.
Saturday, October 8, 2011
Wednesday, October 5, 2011
Research Tools Lecture 9 - Babel, bash, Google Earth
Research Tools Lecture 1 - Introduction
1-introduction.org and 1-introduction.mp3
This is a place where people can discuss lecture 1 of research tools. I'm interested in questions, clarifications, corrections, suggestions, etc for all the lectures. I'll do a post per lecture and video. However, the videos are on youtube, so commenting there is best.
The course material is all in emacs org-mode and in later lectures, I will use org-babel to execute code from inside the document to follow the concept of reproducible-research and "tangling" documentation and code. The Research Tools class website has HTML versions of the notes too if you are not interested in the emacs aspect of the course.
I am creating the lectures on the fly (despite trying to get ahead over the last year), so any feedback I get will help me improve the lectures for the rest of the semester.
Please note that I edit the audio files down from the full hour and 20 minutes to about 50-60 minutes. I clean up some of my "um"'s and "ahh"s along with white space, repeats and other non-useful bits. I walk around the class and there I often explain the same or similar concept to multiple people. I try to reduce the repetition. It's helpful in class, but with the audio, you can just rewind it.
The youtube videos are unedited, so they will show more of my typos.
Thanks!
-kurt
This is a place where people can discuss lecture 1 of research tools. I'm interested in questions, clarifications, corrections, suggestions, etc for all the lectures. I'll do a post per lecture and video. However, the videos are on youtube, so commenting there is best.
The course material is all in emacs org-mode and in later lectures, I will use org-babel to execute code from inside the document to follow the concept of reproducible-research and "tangling" documentation and code. The Research Tools class website has HTML versions of the notes too if you are not interested in the emacs aspect of the course.
I am creating the lectures on the fly (despite trying to get ahead over the last year), so any feedback I get will help me improve the lectures for the rest of the semester.
Please note that I edit the audio files down from the full hour and 20 minutes to about 50-60 minutes. I clean up some of my "um"'s and "ahh"s along with white space, repeats and other non-useful bits. I walk around the class and there I often explain the same or similar concept to multiple people. I try to reduce the repetition. It's helpful in class, but with the audio, you can just rewind it.
The youtube videos are unedited, so they will show more of my typos.
Thanks!
-kurt
Thursday, July 28, 2011
Turned off adds
I was playing with adds just to see what it was like. I reached $13 since I started adds and have not cached that out. I'm visiting Google, it does not seem right to have adds on my blog (unless there is a way to point the money at a charity of my choosing), and they were a bit overwhelming the way I had them setup.
Geotagging in Blogger
Draft Blogger now has geotagging! Where am I? I'm a Visiting Faculty at Google at the moment!
Tuesday, June 14, 2011
Wednesday, June 8, 2011
Examples of great metadata?
Today, I am sitting in a metadata workshop at UNH run by Northeast Coastal and Ocean Data Partnership (formerly: Gulf of Maine Ocean Data Partnership) NeCODP... it's also their annual meeting. Back in January, I attended a Metavist metadata workshop put on at the Alaska Science Symposium by NBII's metadata folks.
I have been struggling for a long time with metadata. I have felt strongly that something is fundamentally wrong. I'm starting to feel that this is that we are missing examples that are held up to the community as best practices and examples of how this metadata is used. As Google has popularized, we need to dog food the metadata. And it has to be understandable by the 1st year graduate student, not just metadata experts!
So, what are example of best practices in ISO 19115 metadata (I consider FGDC metadata as dead and pointless for a global community). And it really sucks how closed the ISO standards are. So totally frustrating. Am I going to try to teach UML to new grad students? We need to be able to kickoff with metadata and get going in 15-30 minutes and UML is counter to this understanding in short time frames.
The keynote speaker is Ted Habermann who gave a fantastic talk. However, strongly disagree with Ted on a couple points. First is with the pay for standards model. We are not making light bulbs and the standards bodies are not paying us for our time writing these standards. Hiding software and data standards only degrades the ability of individuals to be empowered. Large companies like ESRI could care less one way or another about the costs, but it is up to us, the data producers and consumers to be empowered. It's getting way way too complicated! Ted's Metadata Standards page doesn't even cover a small fraction of the standards he talked about during his keynote. e.g. what is EML? He also mentioned UDDC - Unidata Data Discovery Conventions, NetCDF Markup Language (NcML), etc.
There has got to be something simpler, because with complexity comes increased error rates and all sorts of training problems. Can we use simpler strategies? e.g. Geodata Discovery and geo micro format
And to be different than the speaker, when it comes to researchers and especially graduate students, the do go to wikipedia to read multiple papes and you can make books out of groups of pages. Wikipedia:Books
I have been struggling for a long time with metadata. I have felt strongly that something is fundamentally wrong. I'm starting to feel that this is that we are missing examples that are held up to the community as best practices and examples of how this metadata is used. As Google has popularized, we need to dog food the metadata. And it has to be understandable by the 1st year graduate student, not just metadata experts!
So, what are example of best practices in ISO 19115 metadata (I consider FGDC metadata as dead and pointless for a global community). And it really sucks how closed the ISO standards are. So totally frustrating. Am I going to try to teach UML to new grad students? We need to be able to kickoff with metadata and get going in 15-30 minutes and UML is counter to this understanding in short time frames.
The keynote speaker is Ted Habermann who gave a fantastic talk. However, strongly disagree with Ted on a couple points. First is with the pay for standards model. We are not making light bulbs and the standards bodies are not paying us for our time writing these standards. Hiding software and data standards only degrades the ability of individuals to be empowered. Large companies like ESRI could care less one way or another about the costs, but it is up to us, the data producers and consumers to be empowered. It's getting way way too complicated! Ted's Metadata Standards page doesn't even cover a small fraction of the standards he talked about during his keynote. e.g. what is EML? He also mentioned UDDC - Unidata Data Discovery Conventions, NetCDF Markup Language (NcML), etc.
There has got to be something simpler, because with complexity comes increased error rates and all sorts of training problems. Can we use simpler strategies? e.g. Geodata Discovery and geo micro format
And to be different than the speaker, when it comes to researchers and especially graduate students, the do go to wikipedia to read multiple papes and you can make books out of groups of pages. Wikipedia:Books
Monday, May 16, 2011
Examples of good cruise reports and descriptive reports (DR)?
I am on the hunt for great cruise reports and hydrographic survey Descriptive Reports (DR). I'm asking a few people directly for their opinions, but here I'm asking anyone who is willing to share their take on examples of what makes a great report.
For example, Jim Gardner shared his CRUISE KM1009 May 17, to June 16, 2010.
For example, Jim Gardner shared his CRUISE KM1009 May 17, to June 16, 2010.
Tuesday, May 3, 2011
Tuesday, March 29, 2011
C++ testing 2011
I started really digging in to C++ testing with a post listing lots of options. If you have comments on any of these systems and especially if you can link to people reviewing their experiences with the testing systems (preferably in the last couple years), it would be a huge help.
My new post: unittesting c++ in 2011 - part 1. My old post last year: Which c++ logging and unit test framework(s)?
My new post: unittesting c++ in 2011 - part 1. My old post last year: Which c++ logging and unit test framework(s)?
Thursday, January 20, 2011
Wednesday, January 19, 2011
Deepwater Horizon Lessons learned
If you are in Anchorage, AK this evening, come by the Captain Cook Hotel and join us for Lessons Learned from the Gulf of Mexico oil spill from 5:45-8:00PM at the Alaska Marine Science Symposium. Here are some lessons learned in my oppinion to try to help get the discussion going:
Some are being worked on now, some are not. What would you add?
- Ships of all sizes should have AIS and there must be a way to track which ships are a part of a response
- Have portable AIS systems ready to put on ships (suitcase version)
- Know the bathymetry and be willing to collect your own if necessary. Have charts, the Coast Pilot and local names
- Have a manual that you get to all of the team. People coming to the area don’t know the local names, physical layout, and available data / resources
- Start using mobile devices to collect data – position and information. Avoid handwritten and voice communication to prevent confusion
- Train with actual ships being virtually moved around
- Get necessary data integrated into AOOS, ERMA and other systems before you are in a crisis. Unlike Deepwater Horizon, initial response time is usually critical
- Train people to work with the data types before a crisis
- PDFs and paper are not good primary delivery devices
- Communication across all the teams makes or breaks the response
- We should have AIS from a couple response vessels forwarded through satellite to supplement normal stations
Some are being worked on now, some are not. What would you add?
Sunday, January 2, 2011
How to learn emacs lisp?
During 2011, I would really like to start writing org-spatial for
emacs. I spend most of my life in emacs and it is crazy that org-mode
and emacs do not know about location and projections. If that is going
to change, I'm going to have to step up and get it started. However, I
suck at reading lisp. What is a great source for getting going? ShowMeDo
only has one video tagged with emacs.
I just found rpdillon on youtube. I just wish he had more videos.
How would you recommend to a total LISP beginner that they get into org-mode programming?
emacs. I spend most of my life in emacs and it is crazy that org-mode
and emacs do not know about location and projections. If that is going
to change, I'm going to have to step up and get it started. However, I
suck at reading lisp. What is a great source for getting going? ShowMeDo
only has one video tagged with emacs.
I just found rpdillon on youtube. I just wish he had more videos.
How would you recommend to a total LISP beginner that they get into org-mode programming?
Subscribe to:
Posts (Atom)