Class 12: python and control structures

Introduction

I am trying out adding in the time offsets in the lecture audio as comments. It will be a minute:seconds offset into the audio.

Log files that you are keeping for the class homework

It is okay to keep one long log file for the course log
There must be one entry for each day of class
The entry must start with a single “*” header
The entry must be tagged with “day”. More is okay
The entry line must contain the date
If you miss a class, I still expect an entry

Google Group

I have set up a google group for the class. Feel free to discuss anything related to the class there. It is open to the public, so there might end up being more than just the people you see in the classroom.

http://groups.google.com/group/researchtools2011

Comments per class

I will try to make a post per class on my blog as a place to comment on a specific class. I leave it up to you to decide if you want to decide to comment on the Google Group or as a comment to a blog post on blogspot/blogger (Google's blog service)

Slashdot and Steve Jobs apple stevejobs

If you haven't heard, Steve Jobs, co-founder of Apple Computers, passed away yesterday.

http://apple.slashdot.org/story/11/10/06/000211/steve-jobs-dead-at-56

Setup

Before you start class, make sure you have your environment set up. Here is what I suggest

mkdir -p ~/class/12
cd ~/class/12
wget http://vislab-ccom.unh.edu/~schwehr/Classes/2011/esci895-researchtools/src/12-python.org

Setting your editor emacs editor bashvariable

We can set the bash shell variable EDITOR to emacs, but then every time we want to edit a file, ipython is going to wait for us to "finish" editing and exit emacs. We will loose our place each time. There is a special way to setup emacs as a "server" that can be told to open a file from somewhere else. emacs will stay running and can get multiple requests. Here is how to make it work!

Start emacs. Applications -> Programming -> GNU Emacs 23.

In emacs, we need to start the server that will listen for requests to edit a file.

M-x server-start

Now, open a terminal. Applications -> Accessories -> Terminal Once we have a terminal, we can set the EDITOR variable to use the program called emacsclient. Remember that you can read more about the program with man emacsclient.

export EDITOR=emacsclient

Now start ipython. As ipython to edit a python script file:

edit helloworld.py

Now you can finish editing the file with C-x #. Unfortunately, a couple things are not yet correct. First, emacs will close that file so we can't keep editing. Second, this setup is not permanent. It only exists as long as this copy of emacs and this terminal are running. We need to fix both at the same time by editing two configuration files in our account.

First, let us edit our .emacs file and add two lines plus some comments. In emacs lisp, comments start with the ";" character. Please do not worry about trying to understand the lisp programming language. That is outside of the scope of this class. If you are interested, please talk to me and I can get you started.

;;; Emacs server

; Do not close the file that was being edited when C-x # is typed
(setq server-kill-new-buffers nil)

; Start the emacs server for emacsclient
(server-start)

Now, add this line to the bottom of your .bashrc:

export EDITOR=emacsclient

Next time you log in to your virtual machine, everything should be setup for you!

NOTE: remember to start emacs before using the edit command! Also, only start 1 emacs. The way it is setup here, we can only have one emacs. Any addition emacs instances will complain when they get to the server-start command and find there is already a server running.

Now in ipython, editing a file should look like this. When you use C-x # in emacs to let ipython know that you are done editing, ipython will try to run your code.

In [1]: edit "helloworld.py"
Editing...Waiting for Emacs...

In emacs, make the file look like this:

print 'hello world'

Now press C-x # in emacs.

 done. Executing edited code...
hello world

Creating a python script and running it from python compile script python ipython

Start ipython:

cd ~/class/12
ipython --pylab

Check that your EDITOR variable is set correctly.

import os
os.environ['EDITOR']

It should respond with:

'emacsclient'

Running the script using ipython

Ask ipython to start editing a file. It will look at your EDITOR variable and call emacsclient:

edit first.py
Editing...Waiting for Emacs...

You can now start editing the file. Put this in your first.py:

#!/usr/bin/env python

print 'hello world'

Now let ipython know you are done editing with C-x #

You can keep editing first.py. If you save in emacs and want to try the program again from ipython, you can "run" the code from ipython:

run first.py

Reading a file with python

Copy data.csv from ~/class/11 or just run C-c C-c in this source code block. The weird "<< EOF" syntax says to read in everything until it gets to a line with the string EOF on it. That could be any string, but EOF is a convention meaning "End Of File".

This is your shell trick of the day. Weird, but useful.

cat << EOF > data.csv
1,2
4,5
9,-1
EOF

Last class, we left off with reading a file. We read the file like this:

datafile = open('data.csv')
type( datafile )
datafile.readline()
datafile.readline()
datafile.readline()
datafile.readline()

We can read an entire text file with one readlines (not the plural 's') command. This is easy and works for small files, but it will get really slow for large files.

datafile = open('data.csv')
lines = datafile.readlines()
print lines
print len(lines)

line = lines[0]
print 'line: ', line
line = line.strip() # remove blank space on the left or right of the string
print 'line with strip:', line
fields = line.split(',')
print 'first:', fields

print lines[0]
print lines[0].strip()
print lines[0].strip().split(',')  # yikes!  you can chain things together
print 'second:', lines[0].strip().split(',')

Run that with F1. You will get a lot more back than I show here:

./first.py
hello world
my mistake
first: ['1', '2']
second ['1', '2']

A for loop for

It is better (faster) to loop over each line with a for loop when it comes to huge files. Let's try some for loops.

Try these in ipython

for item in [ 1, 3, 6, 'nine' ]:
    print item

Here is where I stop and talk about indentation controlling the beginning and ending of "blocks"!!!

A file also works as a sequence.

myfile = open('data.csv')
for line in myfile:
    print line

Gives:

1,2

4,5

9,-1

Hmmm… why to the blank line in between? We get a new line from print and one from the line. We need to strip that line.

myfile = open('data.csv')
for line in myfile:
    print line.strip()