the Daily Burrito

Introducing numpyIO.py

I implemented numpyIO.py because I often use scipy.io.numpyio.fread() and scipy.io.numpyio.fwrite(), but I don’t want to depend on scipy. It uses numpy’s tofile/fromfile functions instead.

Why might you want this instead of SciPy’s numpyio? Well, I can think of a few reasons:

it imports faster because it only depends on numpy, not scipy
easier to package with py2exe or py2app
it is slightly faster
it lets you avoid rewriting legacy code that makes extensive use of scipy’s deprecated fread/fwrite

Grab the numpyIO project from bitbucket or clone the repo with:

hg clone http://bitbucket.org/lannybroo/numpyio

Updating to Python 2.6 on Mac OS X Leopard 10.5.6

Install Python from source.

cd ~/tmp
wget http://www.python.org/ftp/python/2.6.1/Python-2.6.1.tgz
untar Python-2.6.1.tgz
cd Python-2.6.1
mate Mac/README
./configure --enable-framework
make
make install
cd ..

Update setuptools so that easy_install works.

wget http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c9-py2.6.egg
sh setuptools-0.6c9-py2.6.egg

Add the following to my .profile so that the binaries installed by easy_install take precedence over my old Python 2.5 stuff.

export PATH=/Library/Frameworks/Python.framework/Versions/2.6/bin:$PATH

Easy install various packages I use frequently.

easy_install IPython
easy_install nose
easy_install pexpect
easy_install cheetah
easy_install clonedigger
easy_install pyserial
easy_install markdown2
easy_install euclid
easy_install psyco

Do the slightly tougher installs: NumPy (svn rev 6271), SciPy (svn rev 5300), matplotlib, and euclid. See http://www.scipy.org/Installing_SciPy/Mac_OS_X for details. Before starting, I also updated to the latest apple developer tools, although this isn’t strictly necessary.

wget http://r.research.att.com/gfortran-4.2.3.dmg
open gfortran-4.2.3.dmg # install using GUI

wget http://www.fftw.org/fftw-3.2.tar.gz
untar fftw-3.2.tar.gz
cd fftw-3.2
./configure
make
sudo make install
cd ..

svn co http://svn.scipy.org/svn/numpy/trunk numpy
cd numpy
python setup.py build
python setup.py install
cd ..

svn co http://svn.scipy.org/svn/scipy/trunk scipy
cd scipy
python setup.py build
python setup.py install
cd ..

python
>>> import numpy
>>> numpy.test('1','10') # 6 problems in 1897 tests, but usable
>>> import scipy
>>> scipy.test('1','10') # 168 errors of 3990 tests, but usable

sudo port install libpng
sudo port install freetype
easy_install matplotlib

ipython -pylab
>>> plot([1,2,3])

svn checkout http://pyeuclid.googlecode.com/svn/trunk/ pyeuclid
cd pyeuclid
python setup.py install
cd ..

Install wxPython and PythonCard for GUI development.

wget http://downloads.sourceforge.net/wxpython/wxPython2.8-osx-unicode-2.8.9.1-universal-py2.6.dmg
wget http://downloads.sourceforge.net/wxpython/wxPython2.8-osx-docs-demos-2.8.9.1-universal-py2.6.dmg
open wxPython2.8-osx-unicode-2.8.9.1-universal-py2.6.dmg
open wxPython2.8-osx-docs-demos-2.8.9.1-universal-py2.6.dmg

wget http://prdownloads.sourceforge.net/pythoncard/PythonCard-0.8.2.tar.gz
untar PythonCard-0.8.2.tar.gz
cd PythonCard-0.8.2
python setup.py install
open /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/PythonCard/samples/minimal/

Then, Ctrl-click and open minimal.py with Python Launcher to test.

Hi-Def Video Playback on Windows

Jeff Atwood’s great tips on Hi-Def video playback for Windows.

Speedy Python

A great reference: Tools for Accelerating Python. I learned about quite a few new ways of getting Python code to run faster …

Git Link Roundup

I’ve been working more with git lately and have found a few more interesting sites I’d like to share.

The Git Community Book is a great quick reference for finding out how to use a command if you can’t recall from the man page.

If you want to really understand the gory details of branching and merging with git (and I do highly recommend understanding them … easy branching is the best part of git coming from a svn background), then read this long article on LWN.net.

Also, R. Tyler Ballance has a series of blog entries on moving his team from subversion to git. I especially liked Team Development with Git which details the bad habits of developers that are just becoming familiar with distributed source control.

The Battery Powered blog tells us how to deploy a Git Repository Server in Ubuntu, which may prove useful soon. Currently, I’m just using Git for my local work and then pushing to a SVN repo, but I’d like to move others to Git soon.

scie.nti.st also explains how to host a Git repo. Both this and the Battery Powered link above use gitosis to make setup easier.

Amanda’s Marathon Training Video

Amanda and I are running the Chicago Marathon this year. We’re hoping our friends will want to come watch us run on Sunday, October 12!

We are running as part of a fundraising group that supports the Chicago Children’s Memorial Hospital. So, Amanda made a fun video showing her unique Rocky Balboa training methods.

Anyway, hope you enjoy the video. If you like it and want to sponsor us, head to Amanda’s donation site and make a pledge to support her. Or, you can pledge to support me here.

Watch below, or check it out in HD on YouTube.

Git-SVN Cheat Sheet

Set good “don’t mess with my line endings” option before first checkout:

git config --global core.autocrlf false

Do this one time to suck in the repo w/history from SVN:

git svn clone svn://myrepo.com/project/

Do this loop over and over until your code ready to push back to SVN. The cool thing about this is that the adds and commits can be done off the network! Don’t forget to merge and re-test your changes after the rebase:

git add .
git commit -m "Added some awesome stuff"
git svn rebase

Push all local changes back to the SVN repo, which will now have all the local commits you did when you were on the airplane:

git svn dcommit

One other key tip is what when you want to decommit or rebase but you are in the middle of some changes, don’t worry. git-stash to the rescue:

git stash
git svn rebase
git stash apply

Also, here are a few links that inspired this post:

Using Git for Local Version Control
git awesome-ness
Learning git-svn in 5min
Everyday GIT With 20 Commands Or So
git-config(1)
Git - SVN Crash Course
Steve’s ramblings slowly convincing me
Git Magic book (a great quick-reference for git commands w/examples)

Announcing Haiku Of the Day (HOD)

I helped put together a simple website and RSS feed of Haiku published by Bronze Man Books. Read the announcement or check out today’s haiku of the day!

Run Your Software on Your GPU

Check out GPULib from Tech-X Corp. It gives you the ability to run mathematical functions on your GPU card (certain NVIDIA models only, as the moment). Includes a blog and bindings for Python (pystream) and MATLAB.

Idiomatic Python, the Cheese Shop, PyEuclid

Today is becoming python link day, so here are a few more:

Idiomatic Python has some great tips on how to use python as it was intended.
The Python Package Index is a good place to look for code someone has already written that might solve the problem you’re thinking about, and probably in a much cleaner way than you were thinking. This used to be known as the Cheese Shop.
pyeuclid is a nice 2D/3D vector, matrix, and quaternion math library (that needs better docstrings and a ray-sphereoid intersection).

Numpy for MATLAB Users

Great reference for those of us recovering from a MATLAB addiction, like this guy, who recently moved his scientific workflow to Python.

Python-by-example

Straight-forward Python reference the explains the standard library rather completely and cleanly just by showing a simple example for each module and function. Very nice for a quick lookup of something you don’t quite remember.

Universal Sole 10

Amanda, Shayna, Mark, and I ran the 10-mile Universal Sole race in Lincoln Park yesterday. Here’s an excerpt with our times (full results):

O'All No. Name   Div/Tot  Rank 5 Mile  Pace  Finish    Pace 
===== === ====== ======== ==== ======= ===== ========= =====
  788 369 MARK   103/120   960   49:11  9:51 1:33:00.4  9:19
  883 155 ALAN    57/66    961   49:12  9:51 1:35:53.2  9:36
 1148 370 SHAYNA 119/130  1181 1:00:52 12:11 1:56:42.8 11:41
 1149 156 AMANDA 120/130  1182 1:00:53 12:11 1:56:44.4 11:41

Installing IPython, SciPy, and Matplotlib into Apple’s Python on upgraded Leopard

Start by reading this reference and some tips at MacResearch.

Because it obscures the nice Leopard python that includes Numpy and Dtrace, remove your old Python completely.

sudo rm -rf /Library/Frameworks/Python.framework/

Check that NumPy is already installed and working.

python
import numpy
numpy.test(1,10)

Install latest IPython, version 0.8.2. Note that I always alais untar='tar xvzf' in my bash setup.

wget http://ipython.scipy.org/dist/ipython-0.8.2.tar.gz
untar ipython-0.8.2.tar.gz
cd ipython-0.8.2
sudo python setup.py install

Install a fortran compiler binary because it is needed to compile/install Scipy. I get the PPC version of gfortran. You can also find the Intel version at HPC Mac OS X.

wget http://internap.dl.sourceforge.net/sourceforge/hpc/\
gfortran-ppc-leopard-bin.tar.gz
sudo tar -xvf gfortran-ppc-leopard-bin.tar.gz -C /

Edit a disttools file in Apple’s NumPy distribution so it is more lenient in allowing the use of the above gfortran compiler when building SciPy. Inspired by this note.

cd /System/Library/Frameworks/Python.framework/Versions/2.5/\
Extras/lib/python/numpy/distutils/fcompiler
sudo cp -p gnu.py gnu.py.bak

And edit the following lines as follows (I use mate gnu.py to edit in TextMate).

- version_match = simple_version_match(start=r'GNU Fortran (?!95)')
+ version_match = simple_version_match(start=r'GNU Fortran')
- version_match = simple_version_match(start='GNU Fortran 95')
+ version_match = simple_version_match(start='GNU Fortran')

Build and install SciPy 0.6.0. Source available here.

wget http://internap.dl.sourceforge.net/sourceforge/scipy/\
scipy-0.6.0.tar.gz
untar scipy-0.6.0.tar.gz
cd scipy-0.6.0
sudo python setup.py install

This installs SciPy here: /Library/Python/2.5/site-packages/scipy, which isn’t seen by Python at first. I added the following to my .profile.

export PYTHONPATH="/Library/Python/2.5/site-packages"

Test it after closing your terminal and re-opening. I actually have 2 failures and 3 errors, but they are things I don’t care about at the moment.

python
import scipy
scipy.test(1,10)

Install matplotlib binary.

wget http://internap.dl.sourceforge.net/sourceforge/matplotlib/\
matplotlib-0.91.2-py2.5-macosx-10.3-fat.egg
sudo easy_install -N matplotlib-0.91.2-py2.5-macosx-10.3-fat.egg

Test it out and be happy that you’re done!

ipython -pylab
plot([1,2,3])

Rocky Fetching, Fighting the Roomba

Our boy continues to be quite the fun addition to the family. He can run around playing fetch all day. He also likes to fight with Little Buddy (a.k.a. our Roomba). I think we should teach him to ride it around.

Markdown in Python

A clean, fast Markdown implementation in python.

Meet Rocky Brooks

We love our little 4-month-old Chihuahua we adopted on 20-Oct-2007. He also loves to play with his inverted self in the mirror …

Clean up your Contextual Menus

Thanks to this TidBITS article, I was tipped off to find this Mac OS X Hint about how you can rebuild the Launch Services database that determines which applications show up in your “Open With” contextual menu. My Open With had a huge amount of cruft accumulated over time with Classic apps, apps on my backup hard drive, and an annoying number of image droplets in the Photoshop/Samples/Droplets folder.

So I rebuilt my Launch Services database to make my Open With much cleaner and faster. Here’s how I did it.

Eject my backup hard drive so that the database doesn’t include duplicate copies of all my applications.
Archive the Photoshop/Samples/Droplets/ folder (right click, Create Archive, then delete the folder).

Add the following line to my ~\.profile so typing the lsregister command is easier. (alias ... should all be on one line)

# make launch services register easy to find
alias lsregister='/System/Library/Frameworks/
    ApplicationServices.framework/Frameworks/
    LaunchServices.framework/Support/lsregister'

Save a description of the old database (out of curiosity, not necessary).
```
lsregister -dump | less > lsregDumpOld.txt
```

Rebuild the database. (all on one line)

lsregister -kill -r -f -domain system -domain system
    -domain user "/Applications"

Save a description of the new database (again just curious).
```
lsregister -dump | less > lsregDumpNew2.txt
```

The new database is now about 1 MB, down from the original 6 MB. There sure was a lot of junk in there!

Steve vs. Alan on Hearing High Stuff

I can hear up to 17.1kHz. Steve can only hear up to 14.7kHz. So I win this round. Although maybe that means he can save more disc space and compress his MP3s a little more?

If you want to perform your own test, you can use Python on Windows as follows:

from winsound import Beep
Beep(14000, 1000)

14000 is the frequency 14kHz and 1000 is beep duration 1000ms (1 second). Crank up the frequency until you can’t hear it.

Update: alternate MATLAB one-liner for doing the same:

f=15000;fs=44100;sound(sin(f*2*pi*[0:fs-1]/fs),fs)

Update 2: Paul says he can hear up to 16.9kHz well but also claims to faintly hear 18-20kHz. I think that means he wins, because I cannot hear above 17.1kHz at all. He said he used headphones whereas I used crappy laptop speakers, so I consider that cheating a little.

Update 3: So apparently, the results are very dependent on the speakers being used. I tested Amanda at home using the Bose speakers and she can hear up to 17.5kHz. Funny thing: with these speakers, I can hear up to about 18.4kHz. Not sure what to make of that except that I’m winning again.

Punched before Eating

Maybe the funniest SNL Digital Short yet.

Hidden Dictionary on The New York Times

Woah, I just randomly double-clicked a word while reading an article nytimes.com and it popped up a definition/theauraus with information on that word. Seems to work while reading articles, but not on the front page.

It is actually more than just a dictionary. For example, go read about Led Zeppelin finally doing digital downloads and when you click Zeppelin, it gives a short bio on the band. Says it’s powered by Answers.com.