debian/fink and mbsystem

schwehr.org is currently in a "transitional stage"... it will be back soon.  Too much going on with the group of use all hosting together for us to get this settled right away.

Jonny B asked a question this morning about tides and mbsystem.  I didn't know the answer, but here is how I went about investigating.  This is on a mac with 10.7 and fink.  Hopefully this will give people some ideas about how to dig through what is around.  "dpkg -L" lists the files in a package.  As Jonny B said with a reference to the movie The Matrix, "Yowza, nice kung fu!".  I didn't have a fish to give, so the best I could do was point to the tackle store and say you might be able to get what you need to fish in there.


# Which fink packages deal with sonar?
fink apropos sonar
# If you already know about mbsystem...
fink list mbsystem
fink install mbsystem
dpkg -L mbsystem | grep tide
# Nothing
cd ~/Desktop
ls /sw/src/mbsystem*
# See that the latest tar is version 5.3.1917
tar xf /sw/src/mbsystem-5.3.1917.tar.gz
cd mbsystem-5.3.1917
find . | grep -i tide
# Nothing


find . | xargs grep -l tide | wc -l
# 84
find . | xargs grep -l tide
# ./html/mbdatalist.html
# ./html/mbio.html
# ./html/mbnavadjust.html
# ./html/mbotps.html
# ./html/mbprocess.html
# ./html/mbset.html
# ./html/mbsystem.html
# ./html/mbsystem_how_to_get.html
# ./install_makefiles
# ./man/man1/mbdatalist.1
# ./man/man1/mbnavadjust.1
# ./man/man1/mbotps.1
# ./man/man1/mbprocess.1
# ./man/man1/mbset.1
# ./man/man1/mbsystem.1
# ...
# ./src/utilities/mb7kpreprocess.c
# ./src/utilities/mbcopy.c
# ./src/utilities/mbinfo.c
# ./src/utilities/mbprocess.c
# ./src/utilities/mbset.c


man mbset
# "Apply tides to bathymetry."

schwehr.org

Yes, http://schwehr.org is currently offline.  InReach's colo facility is in a bad state.  When I get a chance, I will find another place to get schwehr.org and my blog hosted.

4/24/2012 9:00am PST We continue to evaluate the existing HVAC system and initial indications for a repair are at least several weeks and possibly longer. We understand this is not an ideal situation so we would suggest to all of our customers that start moving as soon as possible to alternative locations in order to ensure servers and services remain up.

Although we want to avoid it a full analysis may include a disruption of power. No definitive time, date or duration for this power disruption is scheduled but it could be within the next few days. We will update you all with the specifics once they become known.

We are addressing security concerns as well and we have engaged the building management to help but their solution is to keep the door closed to the suite which will increase heat.

Using Google Cloud Storage - Part 1

This is my first attempt at a tutorial for Google Cloud Storage.

Using Google Cloud Storage… it's a service that let's you read/write data objects (a blob of data of what ever type you want).  Right now, you get 5GB of storage for free and you can transfer 20 GB per month. Beyond that, you'll have to check the pricing.

This is pretty much what is on https://developers.google.com/storage/docs/getting-started and https://developers.google.com/storage/docs/hellogooglestorage I've tweaked it a bit to fit my style. I'm doing this on a mac with fink and splitting between two acounts. I want to end up with a way to transfer files from one group to another. I started off by watching this 2011 Google IO video:

Google I/O 2011: Storing Your Application's Data in the Google Cloud

First setting up a virtual env in python to isolate us from messing up any existing python installs.

cd ~/Desktop
mkdir try-gs
cd try-gs
fink install virtualenv-py27
virtualenv ve
source ve/bin/activate
wget http://commondatastorage.googleapis.com/pub/gsutil.tar.gz
tar xf gsutil.tar.gz
pushd gsutil

# gsutil does not have a proper setup.py that will work with virtualenv
# so force it to play along
perl -pi -e "s|/usr|`pwd`/../ve|g" setup.py

python setup.py install
popd

You should now have a working virtualenv with gsutil installed in it. Your normal python will not be effective. You will have to rerun the <b>activate</b> from any other shell that you want to have access to gsutil. The virtualenv named "ve" will change your prompt:

$ pwd
/Users/schwehr/Desktop/try-gs
(ve)
$ 

Let's test out setup.

type python
# python is hashed (/Users/schwehr/Desktop/try-gs/ve/bin/python)
type gsutil
# gsutil is /Users/schwehr/Desktop/try-gs/ve/bin/gsutil
gsutil --help 
# lots of help scrolls by

# Try a world readable test data set
gsutil ls gs://uspto-pair/applications/0800401*
# gs://uspto-pair/applications/08004010.zip
# gs://uspto-pair/applications/08004011.zip
# gs://uspto-pair/applications/08004012.zip
# gs://uspto-pair/applications/08004013.zip
# gs://uspto-pair/applications/08004016.zip
# gs://uspto-pair/applications/08004017.zip
# gs://uspto-pair/applications/08004019.zip
gsutil ls -l gs://uspto-pair/applications/08004010.zip 
#       3410  2010-09-29T20:14:15  gs://uspto-pair/applications/08004010.zip
# TOTAL: 1 objects, 3410 bytes (3.33 KB)
gsutil cp gs://uspto-pair/applications/08004010.zip .
md5sum 08004010.zip 
# 04b4877377c16b1b49f19b0c7e161d04  08004010.zip

I have yet to write up using the oauth2 authentication.  Hopefully, I'll get that done soon.

See Also:

• http://code.google.com/p/gsutil/
• Issue 92 covers using pypi / pip / distribute / setuptools

RT 26 - Part 6: Parsing binary SBET files with python's struct

html, mp3, pdf and 26-python-binary-files-part-5.org

RT 25 - R for Statistics

Rob Braswell was our guest lecturer covering R for statistical analysis.

mp3, pdf of screenshots, 25-R-lab1-Intro.pdf, 25-r-statistics.org (on bitbucket via hg).

RT 24 - Part 4: Parsing binary SBET files with python's struct

Topics include "What is GIS," using glob.glob to expand file names with "*", using sys.argv directly to list input files and using argparse to properly handle command line arguments.

html, pdf, mp3 and org (in BitBucket hg).

RT 23 - Part 3: Parsing binary SBET files with python's struct

In this class, we do our first mercurial pull of changes. We then add helper functions to our sbet.py module to give us the number of datagrams in an sbet file, tell us at what offset any particular datagram is located and add a generator function allowing cleaner for loops over sbet files.

23-python-binary-files-part-3.html, mp3 and pdf

Remember that at this point, you should be getting the org mode formatted class notes via mercurial (hg). To get set up:

mkdir ~/projects
cd ~/projects
sudo apt-get install mercurial # Install hg on ubuntu & debian linux
hg clone https://bitbucket.org/schwehr/researchtools

And every time you start working on the class, do a pull and update to get the latest versions.

cd ~/projects/researchtools
hg pull   # Bring the changes down to the local "repo"
hg update # Change the working files to have the latest changes