Class 11: ipython and python data types

• http://vislab-ccom.unh.edu/~schwehr/Classes/2011/esci895-researchtools/hw/hw-3-work-log.html
• I will be periodically be sending you org text with all of your assignment grades

There are 3 more videos online since the last class. Make sure you watch them before the next class. Runtime is 71 minutes for all 3.

Playlist of extra class videos on YouTube

SCHEDULED: 2011-10-10 Mon 14:00

I forgot to announce a matlab session last week. Next Monday, you may want to check out:

• "Mathematica in Education and Research"
• 2-3pm, including Q&A
• Kingsbury Hall room N101

My talk is given 100% in Mathematica, and a big part of what I
want to discuss is the exciting new free-form input in
Mathematica 8. 

Here's a quick video preview: http://www.wolfram.com/common/includes/m8videos/quicktour.html

UNH has an Information Technology (IT) newsletter called Signals.

Vantagepoint: Inside UNH IT with Chief Information Officer Joanna Young

You can meet our CIO. If you read her answers to questions, you will see that I think she missed something huge with passwords: all your passwords should be different. I worry less about people changing their passwords and more about each password being unrelated to compartmentalize any damage. If someone grabbed your password for a site requiring your password, there is a non-zero chance that if you change your password that they will get that new password the same way Unless it is you that gave your password to a bad guy you thought was a good guy… and hopefully you learned never to give your password to anyone Do not have that stop you from changing your passwords, but it is important to understand the overall situation.

Never give your password to anyone. Period. Full stop.

And do not get me started on "security questions".

Did you try it with an echo command?

echo ~/hw/03/log-$USER-$(date +%Y%m%d).org

On my Mac laptop, I get:

/Users/schwehr/hw/03/log-schwehr-20111001.org

I talk through the details of that command in Video 6, Bash part 2: shell variables

In this class we are using Python 2.7. You will want to avoid reference material for python 3. While python 3 is even better than python 2.x, there is still work to be done to get all of the add ons ready for python 3 and you will have trouble with getting examples to work. To reduce confusion, just avoid python 3 for now. If you learn python 2.7, the switch to python 3 will be very easy and there is even a program to automatically make the number of small changes required for code to work with python 3.

If you like the concept of a 1 double sided reference card, here some for python and ipython:

• http://asd.gsfc.nasa.gov/Rodrigo.Nemmen/ipython_quickref.pdf

There are a number of very good free books to get you started. I've sorted them in the order that I think you might want to approach them.

• Dive Into Python by Mark Pilgram
• Python Quick Reference by Richard Gruet
• http://en.wikibooks.org/wiki/Python_Programming
• Think Python by Allen Downey
• Byte of Python by Swaroop C H. Make sure to get the 2.x version. Not 3!
• Data Structures and Algorithms with Object-Oriented Design Patterns in Python by Bruno R. Preiss
• The Programming Historian by William J. Turkel and Alan MacEachern (for a research area focused take on python)

Books in Safari:

FIX

Inside of ipython, we can ask to edit a file. The default editor to use is called vim (or often refered to as just vi). We just spent a number of lectures learning GNU Emacs and we would rather take advantage of that. Without setting anything up, here is vi as the editor:

ipython
edit helloworld.py
:q!
Exit()

That ":q!" is the vi command to "quit without saving".

We can set the bash shell variable EDITOR to emacs, but then every time we want to edit a file, ipython is going to wait for us to "finish" editing and exit emacs. We will loose our place each time. There is a special way to setup emacs as a "server" that can be told to open a file from somewhere else. emacs will stay running and can get multiple requests. Here is how to make it work!

Start emacs. Applications -> Programming -> GNU Emacs 23.

In emacs, we need to start the server that will listen for requests to edit a file.

M-x server-start

Now, open a terminal. Applications -> Accessories -> Terminal Once we have a terminal, we can set the EDITOR variable to use the program called emacsclient. Remember that you can read more about the program with man emacsclient.

export EDITOR=emacsclient

Now start ipython. As ipython to edit a python script file:

edit helloworld.py

Now you can finish editing the file with C-x #. Unfortunately, a couple things are not yet correct. First, emacs will close that file so we can't keep editing. Second, this setup is not permanent. It only exists as long as this copy of emacs and this terminal are running. We need to fix both at the same time by editing two configuration files in our account.

First, let us edit our .emacs file and add two lines plus some comments. In emacs lisp, comments start with the ";" character. Please do not worry about trying to understand the lisp programming language. That is outside of the scope of this class. If you are interested, please talk to me and I can get you started.

;;; Emacs server

; Do not close the file that was being edited when C-x # is typed
(setq server-kill-new-buffers nil)

; Start the emacs server for emacsclient
(server-start)

Now, add this line to the bottom of your .bashrc:

export EDITOR=emacsclient

Next time you log in to your virtual machine, everything should be setup for you!

NOTE: remember to start emacs before using the edit command! Also, only start 1 emacs. The way it is setup here, we can only have one emacs. Any addition emacs instances will complain when they get to the server-start command and find there is already a server running.

Now in ipython, editing a file should look like this. When you use C-x # in emacs to let ipython know that you are done editing, ipython will try to run your code.

In [1]: edit "helloworld.py"
Editing...Waiting for Emacs...

In emacs, make the file look like this:

print "hello world"

Now press C-x # in emacs.

 done. Executing edited code...
hello world

The main web page for python documentation is: http://docs.python.org/

Inside of python, there are a number of ways to get help.

First, you can directly ask for help. Here we are asking for help on the open "function":

help open

You can also put a "?" after a bunch of text and it will try to tell you what it can about that string. You can put the "?" before or after then word.

open?

And to answer the question from class last time about the difference between exit() and Exit(). We just have to ask!

?exit
?Exit

The key is to read through all that and ignore most of it. The last line of ?Exit tells us the key detail: "Exit IPython without confirmation." That can also be said: you will not be asked yes/no when you quit ipython with Exit().

Later on, we will see more about functions or "methods" on variables that are accessed with a ".". Here I will create a string variable and ask it what I can do with a string by pressing <TAB>.

In [1]: mystring = "hello world"

In [2]: mystring.
mystring.__add__                      mystring.decode
mystring.__class__                    mystring.encode
mystring.__contains__                 mystring.endswith
mystring.__delattr__                  mystring.expandtabs
mystring.__doc__                      mystring.find
mystring.__eq__                       mystring.format
mystring.__format__                   mystring.index
mystring.__ge__                       mystring.isalnum
mystring.__getattribute__             mystring.isalpha
...

There is a lot of "noise" in that output, but you will learn to read that and often be able to recognize what you want to do with a string.

Here we are faced with a little problem before we go on. I would like the examples to be runnable both in org-mode with C-c C-c and as something you can paste into ipython without modification. However, that is not possible. The setup for python in org-babel is that it ignores what we will print. So if I try a print statement in python and run it with org-babel:

print 1

The results above are "None". Say what?!?! It turns out that we have to "return" what we want org-babel to print.

return 1

That is more like what we wanted. If you just paste the text without the return, all will be well. So, in ipython, it will look like this:

ipython
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) 
Type "copyright", "credits" or "license" for more information.

IPython 0.10.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: 1
Out[1]: 1

Onwards to working with some strings! The python documentation is here:

http://docs.python.org/library/string.html

Strings can be 'in single quotes' or "in double quotes". I will stick with singles quotes.

First just a basic string:

return 'this is a string'

We can ask python to manipulate a string a little bit:

return 'this is a string'.capitalize()

Or we can get fancier. The title method for a string makes it have each word capitialized.

return 'this is a string'.title()

We can add strings together.

return 'this ' + 'is ' + 'a string'

We can ask python the type of a variable.

return type('my string')

shipname='Coastal Surveyor'
len(shipname)
shipname[0] # Count from zero
shipname[5:8]
shipname.find('S') # returns 8
shipname.find('x') # returns -1 ... not found
shipname[8:] # from position 8 to the end
shipname[-4:] # last 4 characters

As sequence of characters:

shipname='Coastal Surveyor'
len(shipname)
shipname[0] # Count from zero
shipname[5:8]
shipname.find('S') # returns 8
shipname.find('x') # returns -1 ... not found
shipname[8:] # from position 8 to the end
shipname[-4:] # last 4 characters

1
type(1)
1.1
type(1.1)
str(1.1)
float('3.1415')
import math
math.pi
math.sin(math.pi/2)
math.radians(180)
math.degrees(2*math.pi)
math. # then press the <TAB> key to get a list

complex(1,4j)
4j * (2 + 9j)

range(4)
range(3,7)
range(3,28,5)

ships = [ 'tug','row boat', 303902000, 123456789 ]
type(ships)
ships.append(369970120)
ships.sort()
ships[0]
ships[-1]
ships.remove('row boat')
ships. # press <TAB>

shipname='Gulf Challenger, R/V'
shipname.split()
fields = shipname.split(',')
len(fields)
name = fields[0]
name * 4
' -- '.join(fields)