14. Internet Scripting

Topics

♦ Sockets in Python

♦ The FTP module (client)

♦ Email processing (client)

♦ CGI scripts (server)

♦ Jython: Python for Java systems

♦ Active Scripting and COM

♦ Other tools: urllib, web frameworks, XML (see Text), and more

Using sockets in Python

♦ Basic network and local client/server communication

♦ At the heart of Internet communications

♦ A standard C extension type: BSD socket wrapper

♦ C functions become socket object methods

Socket module details

♦ Supports all types: TCP/IP, UDP datagram, Unix domain

♦ Also supports timeouts, non-blocking, SSL

♦ Encrypted sockets supported if SSL library enabled

♦ Urllib modules do https:// if secure sockets enabled

♦ Email also uses secure sockets if enabled

♦ Python also supports “select” multiplex processing

Basic client/server example

● Server: echoes all data that it receives back

● Client: sends data to the server

● Client calls: “socket”, “connect”

● Server calls: “socket”, “bind”, “listen”, “accept”

file: echoserver.py

#########################################################

# Server side: open a socket on a port, listen for

# a message from a client, and send an echo reply;

# this is a simple one-shot listen/reply per client,

# but it goes into an infinite loop to listen for

# more clients as long as this server script runs;

#########################################################

from socket import * # get socket constructor and constants

myHost = '' # server machine, '' means local host

myPort = 50007 # listen on a non-reserved port number

sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object

sockobj.bind((myHost, myPort)) # bind it to server port number

sockobj.listen(5) # listen, allow 5 pending connects

while 1: # listen until process killed

connection, address = sockobj.accept() # wait for next client connect

print 'Server connected by', address # connection is a new socket

while 1:

data = connection.recv(1024) # read next line on client socket

if not data: break # send a reply line to the client

connection.send('Echo=>' + data) # until eof when socket closed

connection.close()

file: echoclient.py

#############################################################

# Client side: use sockets to send data to the server, and

# print server's reply to each message line; 'localhost'

# means that the server is running on the same machine as

# the client, which lets us test client and server on one

# machine; to test over the net, run server on a remote

# machine, set serverHost to machine's domain name or IP addr;

#############################################################

import sys

from socket import * # portable socket interface plus constants

serverHost = 'localhost' # server name, or: 'starship.python.net'

serverPort = 50007 # non-reserved port used by the server

message = ['Hello network world'] # text to send to server

sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP/IP socket object

sockobj.connect((serverHost, serverPort)) # connect to serve and port

for line in message:

sockobj.send(line) # send line to server over socket

data = sockobj.recv(1024) # receive from server: up to 1k

print 'Client received:', `data`

sockobj.close() # close to send eof to server

Forking socket servers

import os, time, sys, signal, signal

from socket import * # get socket constructor and constants

myHost = '' # server machine, '' means local host

myPort = 50007 # listen on a non-reserved port number

sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP socket object

sockobj.bind((myHost, myPort)) # bind to server port number

sockobj.listen(5) # up to 5 pending connects

signal.signal(signal.SIGCHLD, signal.SIG_IGN) # avoid child zombie processes

def now(): # time on server machine

return time.ctime(time.time())

def handleClient(connection): # child process replies, exits

time.sleep(5) # simulate a blocking activity

while 1: # read, write a client socket

data = connection.recv(1024)

if not data: break

connection.send('Echo=>%s at %s' % (data, now()))

connection.close()

os._exit(0)

def dispatcher(): # listen until process killed

while 1: # wait for next connection,

connection, address = sockobj.accept() # pass to process for service

print 'Server connected by', address,

print 'at', now()

childPid = os.fork() # copy this process

if childPid == 0: # if in child process: handle

handleClient(connection) # else: go accept next connect

dispatcher()

Threading socket servers

import thread, time

from socket import * # get socket constructor and constants

myHost = '' # server machine, '' means local host

myPort = 50007 # listen on a non-reserved port number

sockobj = socket(AF_INET, SOCK_STREAM) # make a TCP/IP socket object

sockobj.bind((myHost, myPort)) # bind to server port number

sockobj.listen(5) # upto 5 pending connects

def now():

return time.ctime(time.time()) # current time on the server

def handleClient(connection): # in spawned thread: reply

time.sleep(5) # simulate a blocking activity

while 1: # read, write a client socket

data = connection.recv(1024)

if not data: break

connection.send('Echo=>%s at %s' % (data, now()))

connection.close()

def dispatcher(): # listen until process killd

while 1: # wait for next connection,

connection, address = sockobj.accept() # pass to thread for service

print 'Server connected by', address,

print 'at', now()

thread.start_new(handleClient, (connection,))

dispatcher()

Select socket servers

# event loop: listen and multiplex until server process killed

from select import select

print 'select-server loop starting'

while 1:

readables, writeables, exceptions = select(readsocks, writesocks, [])

for sockobj in readables:

if sockobj in mainsocks: # for ready input sockets

# port socket: accept new client

newsock, address = sockobj.accept() # accept should not block

print 'Connect:', address, id(newsock) # newsock is a new socket

readsocks.append(newsock) # add to select list, wait

else:

# client socket: read next line

data = sockobj.recv(1024) # recv should not block

print '\tgot', data, 'on', id(sockobj)

if not data: # if closed by the clients

sockobj.close() # close here and remv from

readsocks.remove(sockobj) # del list else reselected

else:

# this may block: should really select for writes too

sockobj.send('Echo=>%s at %s' % (data, now()))

Standard library server types

● Threading/Forking TCP/UDP server classes

● Asyncore select-based server classes

● See also: HTTP/CGI webserver.py in CGI section below

● See also: Twisted 3^rd party system

● See also: asyncio module in 3.4+, async/await syntax in 3.5+

import SocketServer, time # get socket server, handler objects

myHost = '' # server machine, '' means local host

myPort = 50007 # listen on a non-reserved port number

def now():

return time.ctime(time.time())

class MyClientHandler(SocketServer.BaseRequestHandler):

def handle(self): # on each client connect

print self.client_address, now() # show this client's addr

time.sleep(5) # simulate blocking activity

while 1: # self.request is client socket

data = self.request.recv(1024) # read, write a client socket

if not data: break

self.request.send('Echo=>%s at %s' % (data, now()))

self.request.close()

# make a threaded server, listen/handle clients forever

myaddr = (myHost, myPort)

server = SocketServer.ThreadingTCPServer(myaddr, MyClientHandler)

server.serve_forever()

The FTP module

♦ ftplib library module uses sockets to transfer files

♦ Supports both binary and text retrieve/store operations

♦ Handles all handshaking with the remote site

♦ See Python library manual for more details

Example: fetch file

file: getone.py

import os, sys

from getpass import getpass

nonpassive = False

filename = 'lawnlake2-jan-03.jpg' # file to download

dirname = '.' # remote directory

sitename = 'ftp.rmi.net' # ftp site to contact

userinfo = ('lutz', getpass('Pswd?')) # use () for anonymous

if len(sys.argv) > 1: filename = sys.argv[1] # file on command-line?

print 'Connecting...'

from ftplib import FTP # socket-based ftp tools

localfile = open(filename, 'wb') # local file to store to

connection = FTP(sitename) # connect to ftp site

connection.login(*userinfo) # default anonymous login

connection.cwd(dirname) # xfer 1k at a time

if nonpassive: # if server requires

connection.set_pasv(False)

# thread me in a GUI

print 'Downloading...'

connection.retrbinary('RETR ' + filename, localfile.write, 1024)

connection.quit()

localfile.close()

Example: simple FTP site mirror (PP book)

#!/bin/env python

########################################################################

# use ftp to copy all files from a remote site/directory to a local dir;

# e.g., run me periodically from a unix cron job to mirror an ftp site;

# script assumes this is a flat directory--see the mirror program in

# Python's Tools directory for a version that handles subdirectories;

########################################################################

import os, sys, ftplib

from getpass import getpass

remotesite = 'home.rmi.net'

remotedir = 'public_html'

remoteuser = 'lutz'

remotepass = getpass('Please enter password for %s: ' % remotesite)

localdir = (len(sys.argv) > 1 and sys.argv[1]) or '.'

cleanall = raw_input('Clean local directory first? ')[:1] in ['y', 'Y']

print 'connecting...'

connection = ftplib.FTP(remotesite) # connect to ftp site

connection.login(remoteuser, remotepass) # login as user/password

connection.cwd(remotedir) # cd to directory to copy

if cleanall:

for localname in os.listdir(localdir): # try to delete all locals

try: # to remove old files

print 'deleting local', localname

os.remove(os.path.join(localdir, localname))

except:

print 'cannot delete local', localname

count = 0 # download remote files

remotefiles = connection.nlst() # nlst() gives files list

# dir() gives all details

for remotename in remotefiles:

localname = os.path.join(localdir, remotename)

print 'copying', remotename, 'to', localname

if remotename[-4:] == 'html' or remotename[-3:] == 'txt':

# use ascii mode xfer

localfile = open(localname, 'w')

callback = lambda line, file=localfile: file.write(line + '\n')

connection.retrlines('RETR ' + remotename, callback)

else:

# use binary mode xfer

localfile = open(localname, 'wb')

connection.retrbinary('RETR ' + remotename, localfile.write)

localfile.close()

count = count+1

connection.quit()

print 'Done:', count, 'files downloaded.'

Example: upload site by FTP (PP book)

#!/bin/env python

########################################################################

# use ftp to upload all files from a local dir to a remote site/directory;

# e.g., run me to copy an ftp site's files from your machine to your ISP,

# especially handy if you only have ftp access to your website, not a

# telnet/shell account access (else you could tar up all files and

# transfer in a single step to the remote machine and untar there);

# to upload subdirectories too, use os.path.isdir(path), FTP().mkd(path),

# and recursion--see uploadall.py for a version that supports subdirs.

##########################################################################

import os, sys, ftplib, getpass

remotesite = 'starship.python.net' # upload to starship site

remotedir = 'public_html/home' # from win laptop or other

remoteuser = 'lutz'

remotepass = getpass.getpass('Please enter password for %s: ' % remotesite)

localdir = (len(sys.argv) > 1 and sys.argv[1]) or '.'

cleanall = raw_input('Clean remote directory first? ')[:1] in ['y', 'Y']

print 'connecting...'

connection = ftplib.FTP(remotesite) # connect to ftp site

connection.login(remoteuser, remotepass) # login as user/password

connection.cwd(remotedir) # cd to directory to copy

if cleanall:

for remotename in connection.nlst(): # try to delete remotes

try: # to remove old files

print 'deleting remote', remotename

connection.delete(remotename)

except:

print 'cannot delete remote', remotename

count = 0

localfiles = os.listdir(localdir) # upload all local files

# listdir() strips dirpath

for localname in localfiles:

localpath = os.path.join(localdir, localname)

print 'uploading', localpath, 'to', localname

if localname[-4:] == 'html' or localname[-3:] == 'txt':

# use ascii mode xfer

localfile = open(localpath, 'r')

connection.storlines('STOR ' + localname, localfile)

else:

# use binary mode xfer

localfile = open(localpath, 'rb')

connection.storbinary('STOR ' + localname, localfile, 1024)

localfile.close()

count = count+1

connection.quit()

print 'Done:', count, 'files uploaded.'

Email processing

♦ POP, IMAP (retrieve); SMTP (send); on top of sockets

♦ Newer email.* module package: parse+compose, attachments, MME en/decoding …

Reading a POP email account

#!/usr/local/bin/python

######################################################

# use the Python POP3 mail interface module to view

# your pop email account messages;

######################################################

import poplib, getpass, sys, mailconfig

mailserver = mailconfig.popservername # e.g., 'pop.rmi.net'

mailuser = mailconfig.popusername # e.g., 'lutz'

mailpasswd = getpass.getpass('Password for %s?' % mailserver)

print 'Connecting...'

server = poplib.POP3(mailserver)

server.user(mailuser) # connect,login to server

server.pass_(mailpasswd) # pass is a reserved word

try:

print server.getwelcome() # print greeting message

msgCount, msgBytes = server.stat()

print 'There are', msgCount, 'mail messages in', msgBytes, 'bytes'

print '-'*80

raw_input('[Press Enter key]')

for i in range(msgCount):

hdr, message, octets = server.retr(i+1)

for line in message: print line # retrieve, print all

print '-'*80 # mbox locked till quit

if i < msgCount - 1:

raw_input('[Press Enter key]')

finally:

server.quit() # make sure to unlock

print 'Bye.'

Sending email via a SMTP server

#!/usr/local/bin/python

########################################################

# use the Python SMTP mail interface module to send mail

########################################################

import smtplib, sys, time, mailconfig # oct15: dropped string

mailserver = mailconfig.smtpservername # e.g., starship.python.net

From = raw_input('From? ').strip() # e.g, lutz@rmi.net

To = raw_input('To? ').strip() # e.g., spam@python.org

To = To.split(',') # allow a list of tos

Subj = raw_input('Subj? ').strip()

# prepend standard headers (this may be nonstandard)

date = time.ctime(time.time())

text = ('From: %s\nTo: %s\nDate: %s\nSubject: %s\n'

% (From, string.join(To, ','), date, Subj))

text = text + '\n' # blank line between hdrs,body

print 'Type message text, end with line=(ctrl + D or Z)'

while True:

line = sys.stdin.readline()

if not line:

break # exit on ctrl-d

text += line # servers do this auto

if sys.platform[:3] == 'win': print

print 'Connecting...'

server = smtplib.SMTP(mailserver) # connect, no login step

errors = server.sendmail(From, To, text)

server.quit()

if errors: # smtplib also raises excepts

print 'Errors:', errors

else:

print 'No errors.'

print 'Bye.'

The email package

● Parses mail text fetched with poplib

● Generates mail text to send with smtplib

# Composing a simple message

>>> from email.Message import Message

>>> m = Message()

>>> m['from'] = 'Sue Jones <sue@jones.com>'

>>> m['to'] = 'pp3e@earthlink.net'

>>> m.set_payload('The owls are not what they seem...')

>>> s = str(m)

>>> print s

From nobody Sun Jan 22 21:26:53 2006

from: Sue Jones <sue@jones.com>

to: pp3e@earthlink.net

The owls are not what they seem...

# Parsing a simple message

>>> from email.Parser import Parser

>>> x = Parser().parsestr(s)

>>> x

<email.Message.Message instance at 0x00A7DA30>

>>> x['From']

'Sue Jones <sue@jones.com>'

>>> x.get_payload()

'The owls are not what they seem...'

>>> x.items()

[('from', 'Sue Jones <sue@jones.com>'), ('to', 'pp3e@earthlink.net')]

>>> for part in x.walk():

... print x.get_content_type()

... print x.get_payload()

...

text/plain

The owls are not what they seem...

# Composing a multi-part message (attachments)

>>> from email.MIMEMultipart import MIMEMultipart

>>> from email.MIMEText import MIMEText

>>>

>>> top = MIMEMultipart()

>>> top['from'] = 'Art <arthur@camelot.org>'

>>> top['to'] = 'pp3e@earthlink.net'

>>>

>>> sub1 = MIMEText('nice red uniforms...\n')

>>> sub2 = MIMEText(open('data.txt').read())

>>> sub2.add_header('Content-Disposition', 'attachment',

filename='data.txt')

>>> top.attach(sub1)

>>> top.attach(sub2)

>>> text = top.as_string() # same as str() or print

>>> print text

Content-Type: multipart/mixed; boundary="===============0257358049=="

MIME-Version: 1.0

from: Art <arthur@camelot.org>

to: pp3e@earthlink.net

--===============0257358049==

Content-Type: text/plain; charset="us-ascii"

MIME-Version: 1.0

Content-Transfer-Encoding: 7bit

nice red uniforms...

--===============0257358049==

Content-Type: text/plain; charset="us-ascii"

MIME-Version: 1.0

Content-Transfer-Encoding: 7bit

Content-Disposition: attachment; filename="data.txt"

line1

line2

line3

--===============0257358049==--

# Parsing a multi-part message

>>> from email.Parser import Parser

>>> msg = Parser().parsestr(text)

>>> msg['from']

'Art <arthur@camelot.org>'

>>> for part in msg.walk():

... print part.get_content_type()

... print part.get_payload()

... print

...

multipart/mixed

[<email.Message.Message instance at 0x00A82058>,

<email.Message.Message instance at 0x00A82260>]

text/plain

nice red uniforms...

text/plain

line1

line2

line3

Running the email examples

● See also: pymail.py and PyMailGui.py in examples directory

● Also in Extras, PP4E (book examples)

● PyMailGui is a Python/Tk mail client (screen shots in GUI unit)

C:\examples\Part3\Internet\Email>python smtpmail.py

From? lutz@rmi.net

To? lutz@rmi.net

Subj? testing 1 2 3

Type message text, end with line=(ctrl + D or Z)

words

Connecting...

No errors.

Bye.

C:\examples\Part3\Internet\Email>python popmail.py

Password for pop.rmi.net?

Connecting...

+OK Cubic Circle's v1.31 1998/05/13 POP3 ready

There are 1 mail messages in 780 bytes

-----------------------------------------------------------------------

[Press Enter key]

Received: by chevalier (mbox lutz)

(with Cubic Circle's cucipop (v1.31 1998/05/13) Sun Feb 13 13:56:44 2000)

X-From_: lutz@rmi.net Sun Feb 13 13:43:01 2000

Return-Path: <lutz@chevalier.rmi.net>

Received: from server.python.net (server.python.net [209.50.192.113])

by chevalier.rmi.net (8.9.3/8.9.3) with SMTP id NAA16859

for <lutz@rmi.net>; Sun, 13 Feb 2000 13:43:00 -0700 (MST)

Message-Id: <200002132043.NAA16859@chevalier.rmi.net>

Received: (qmail 31944 invoked from network); 13 Feb 2000 20:43:22 -0000

Received: from dial-67.73.denco.rmi.net (166.93.67.73)

by server.python.net with SMTP; 13 Feb 2000 20:43:22 -0000

From: lutz@rmi.net

To: lutz@rmi.net

Date: Sun Feb 13 13:44:16 2000

Subject: testing 1 2 3

words

------------------------------------------------------------------------

Bye.

Other client-side tools

■ urllib: fetching web pages

■ nntplib: reading and posting usenet news

■ httplib: lower-level web conversations

■ telnet, gopher, imap,…

■ htmllib, xml package: parsing fetched web pages and data (soap)

■ Active Scripting: embedded Python in HTML

■ Jython: client-side applets in Python

■ RIAs: pyjamas, Silverlight, … (see GUIs section)

Building web sites with Python

♦ CGI scripting

● Simpler techniques for simpler sites

● Augment with state retention: hidden form fields, query parameters, cookies, server-side databases

● Embeds HTML in Python (opposite of PSP, etc.)

● Cgi module parses inputs, escapes outputs

● See also HTMLgen and similar for reply generation

● Require web server that can run Python scripts

♦ Web frameworks and tools

● Zope (OO meets websites: DTML, TAL, ZMI, ZODB, …)

● Plone (CMS built on top of Zope, workflow)

● TurboGears (multi-tool package: AJAX, MVC, SQLObject, CherryPy,…)

● Django (model view controller, implicit)

● CherryPy, WebWare, Quixote, Pylons, web2py (embarrassment of riches!)

● PSP (Webware and mod_python, like PHP/ASP)

● mod_python for Apache (speed, sessions, PSP,…)

● Fast CGI (persistent processes for state retention)

● Google AppEngine, Amazon AWS, Microsoft Azure

● Twisted (network server framework)

Writing server-side CGI scripts

♦ Server-side scripts referenced from HTML pages

♦ Run on server, connected to client/browser via sockets

♦ cgi module handles input parsing, output formatting

Typical operation

♦ Inputs: fetch info typed into forms (cgi.FieldStorage)

● cgi scripts get input from stdin plus environment info

♦ cgi library module

● provides dictionary interface to parsed form data

♦ Outputs: sesults in browser (cgi.escape, urllib.quote)

● cgi scripts write HTML to stdout to display results

Basic CGI interfaces

♦ FieldStorage class

● Reads the form contents from standard input or the environment

print "Content-type: text/html" # HTML text follows

print # blank line: end headers

print "<TITLE>CGI script output</TITLE>"

form = cgi.FieldStorage()

form_ok = 0

if form.has_key("name") and form.has_key("addr"):

if (form["name"].value != "" and

form["addr"].value != ""):

form_ok = 1

if not form_ok:

print "<H1>Error</H1>"

print "Please fill in the name and addr fields."

return

...more form processing here...

♦ FormContent class

● A (now) outdated interface: same functionality, but FieldStorage above is preferred scheme

import cgi # see library manual

. . .

form = cgi.FormContent() # parse input stream

if form.has_key("fieldname"):

data = form["fieldname"][0]

Basic CGI example

# running a local webserver: Extras\Code\Internet\webserver.py

#!/usr/bin/python3

"""

===========================================================================

Implement an HTTP web server in Python that knows how to run server-side

CGI scripts coded in Python; serves files and scripts from current working

dir; Python scripts must be stored in webdir\cgi-bin or webdir\htbin;

===========================================================================

"""

import os, sys

from http.server import HTTPServer, CGIHTTPRequestHandler

webdir = '.' # where your html files and cgi-bin script directory live

port = 80 # default http://localhost/, else use http://localhost:xxxx/

os.chdir(webdir) # run in HTML root dir

srvraddr = ("", port) # my hostname, portnumber

srvrobj = HTTPServer(srvraddr, CGIHTTPRequestHandler)

srvrobj.serve_forever() # run as perpetual daemon

# input form: cgi101.html (use GET instead of POST for some server/browser combos)

<html>

<title>Interactive Page</title>

<body>

<form method=POST action="cgi-bin/cgi101.py">

<P><B>Enter your name:</B>

<P><input type=text name=user>

<P><input type=submit>

</form>

</body></html>

# reply script: cgi-bin/cgi101.py

#!/usr/bin/python3

import cgi

form = cgi.FieldStorage() # parse form data

print('Content-type: text/html\n') # hdr plus blank line

print('<title>Reply Page</title>') # html reply page

if not 'user' in form:

print('<h1>Who are you?</h1>')

else:

print('<h1>Hello <i>%s</i>!</h1>' % cgi.escape(form['user'].value))

# testing with query strings and urllib (see also: htmllib parser)

>>> from urllib.request import urlopen # 2.X: from urllib import…

>>> conn = urlopen('http://localhost/cgi-bin/cgi101.py?user=Sue+Smith')

>>> reply = conn.read()

>>> reply

b'<title>Reply Page</title>\r\n<h1>Hello <i>Sue Smith</i>!</h1>\r\n'

>>> urlopen('http://localhost/cgi-bin/cgi101.py').read()

b'<title>Reply Page</title>\r\n<h1>Who are you?</h1>\r\n'

>>> urlopen('http://localhost/cgi-bin/cgi101.py?user=Bob').read()

b'<title>Reply Page</title>\r\n<h1>Hello <i>Bob</i>!</h1>\r\n'

CGI controls: test5.html/.cgi (PP book)

test5b.html

<HTML><BODY>

<TITLE>CGI 101</TITLE>

<H1>Common input devices: alternative layout</H1>

<P>Use the same test5.cgi server side script, but change the

layout of the form itself. Notice the separation of user interface

and processing logic here; the CGI script is independent of the

HTML used to interact with the user/client.</P><HR>

<FORM method=POST action="http://starship.python.net/~lutz/test5.cgi">

<H3>Please complete the following form and click Submit</H3>

<P><TABLE border cellpadding=3>

<TR>

<TH align=right>Name:

<TD><input type=text name=name>

<TR>

<TH align=right>Shoe size:

<TD><input type=radio name=shoesize value=small>Small

<input type=radio name=shoesize value=medium>Medium

<input type=radio name=shoesize value=large>Large

<TR>

<TH align=right>Occupation:

<TD><select name=job>

<option>Developer

<option>Manager

<option>Student

<option>Evangelist

<option>Other

</select>

<TR>

<TH align=right>Political affiliations:

<TD><P><input type=checkbox name=language value=Python>Pythonista

<P><input type=checkbox name=language value=Perl>Perlmonger

<P><input type=checkbox name=language value=Tcl>Tcler

<TR>

<TH align=right>Comments:

<TD><textarea name=comment cols=30 rows=2>Enter spam here</textarea>

<TR>

<TD colspan=2 align=center>

<input type=submit value="Submit">

<input type=reset value="Reset">

</TABLE>

</FORM>

</BODY></HTML>

test5.cgi

#!/usr/local/bin/python

# runs on the server, reads form input, prints html;

# executable privileges, stored in ~/public_html,

import cgi, sys, string

form = cgi.FieldStorage() # parse form data

print "Content-type: text/html" # plus blank line

html = """

<TITLE>test4.cgi</TITLE>

<H1>Greetings</H1>

<HR>

<H4>Your name is %(name)s</H4>

<H4>You wear rather %(shoesize)s shoes</H4>

<H4>Your current job: %(job)s</H4>

<H4>You program in %(language)s</H4>

<H4>You also said:</H4>

<P>%(comment)s</P>

<HR>"""

data = {}

for field in ['name', 'shoesize', 'job', 'language', 'comment']:

if not form.has_key(field):

data[field] = '(unknown)'

else:

if type(form[field]) != type([]):

data[field] = form[field].value

else:

values = map(lambda x: x.value, form[field])

data[field] = string.join(values, ' and ')

print html % data

In typical interactions, a user directs a browser to a HTML file, which includes a form action that automatically invokes the server-side CGI script on submit button press. It’s also possible to invoke a CGI script directly, provided that input fields are given values within the referencing URL address itself:

test5c.html

<HTML><BODY>

<TITLE>CGI 101</TITLE>

<H1>Common input devices: URL parameters</H1>

<P>

This demo invokes the test5.cgi server-side script again,

but hardcodes input data to the end of the script's URL,

within a simple hyperlink (instead of packaging up a form's

inputs). Click your browser's "show page source" button

to view the links associated with each list item below.

<P>This is really more about CGI than Python, but notice that

Python's cgi module handles both this form of input (which is

also produced by GET form actions), as well as POST-ed forms;

they look the same to the Python CGI script. In other words,

cgi module users are independent of the method used to submit

data.

<P>Also notice that URLs with appended input values like this

can be generated as part of the page output by another CGI script,

to direct a next user click to the right place and context; together

with type 'hidden' input fields, they provide one way to

save state between clicks.

</P><HR>

<UL>

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?name=Bob&shoesize=small">

Send Bob, small</A>

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?name=Tom&language=Python">

Send Tom, Python</A>

<LI><A href=

"http://starship.python.net/~lutz/test5.cgi?job=Evangelist&comment=spam">

Send Evangelist, spam</A>

</UL>

</UL>

<HR>

</BODY></HTML>

The Grail web browser (educational only today)

● Dated but educational: a contemporary of Netscape!

● Portable: written in Python, uses Tkinter as browser GUI

● Browser downloads/runs applets referenced in HTML

● Applets more powerful than HTML+CGI: a full GUI API

● Python also supports ActiveX, Netscape plug-ins,…

HTML file

<HEAD>

<TITLE>Grail Applet Test Page</TITLE>

</HEAD>

<BODY>

<H1>Test an Applet Here!</H1>

Click this button!

<APP CLASS=Question>

</BODY>

file: Question.py

# Python applet file: Question.py

# in the same location (URL) as the html file

# that references it; adds widgets to browser;

from Tkinter import *

class Question: # run by grail?

def __init__(self, parent): # parent=browser

self.button = Button(parent,

bitmap='question',

command=self.action)

self.button.pack()

def action(self):

if self.button['bitmap'] == 'question':

self.button.config(bitmap='questhead')

else:

self.button.config(bitmap='question')

if __name__ == '__main__':

root = Tk() # run stand-alone?

button = Question(root) # parent=Tk: top-level

root.mainloop()

Grail sample screen: a Python/Tk GUI

Caveat: this is a very old example: [tT[kinter looks a lot better today!

Jython: Python for Java systems

♦ An independent implementation of Python

♦ Python for Java-based applications

♦ Python scripting complements Java systems language

♦ Renamed from “JPython” to “Jython” (2000: copyright)

♦ IronPython is very similar, but for .Net/C#

Structure

● A collection of Java classes that compile and run Python code

● “jython” program: equivalent to “python”, interactive, scripts

● “jythonc” compiler/packager program: makes .jar, .class files

● Embedding: “PythonInterpreter” class runs Python from Java

● Extending: automatic, Python coded imports, uses Java classes

Advantages

♦ Compiles Python code to JVM byte-code

● “100% Pure Java” (written in Java, generates JVM)

● More seamless than Tcl and Perl approaches

● Can run as client-side applets in Java-aware browsers

♦ Includes Python/Java integration support

● Python can access Java classes

● Java can embed/call Python scripts

● Access to AWT and Swing from Python for GUIs

Examples

Hello World

from java.applet import Applet

class HelloWorld(Applet):

def paint(self, gc):

gc.drawString("Hello world", 20, 30)

A (very!) simplistic calculator (see PP’s PyCalc)

from java import awt

from pawt import swing

labels = ['7', '8', '9', '+',

'4', '5', '6', '-',

'1', '2', '3', '*',

'0', '.', '=', '/' ]

keys = swing.JPanel(awt.GridLayout(4, 4))

display = swing.JTextField()

def push(event): # Callback for regular keys

display.replaceSelection(event.actionCommand)

def enter(event): # Callback for '=' key

display.text = str(eval(display.text))

display.selectAll()

for label in labels:

key = swing.JButton(label)

if label == '=':

key.actionPerformed = enter

else:

key.actionPerformed = push

keys.add(key)

panel = swing.JPanel(awt.BorderLayout())

panel.add("North", display)

panel.add("Center", keys)

swing.test(panel)

Jython downsides

♦ Not yet fully compatible with standard Python

♦ Only applicable where a JVM is installed or shipped

♦ Requires users to learn Java too (environment, libs)

♦ Doesn’t support standard C/C++ extension modules

♦ Slower than C Python (1.7x, 10x, 100X), for now?

Active Scripting and COM

● Active Scripting: Python embedded in HTML, like JavaScript or VBScript

● Extracted and run on client (browser object module) or server (form and reply object models)

● Windows only, IE and IIS only today, requires Python + PyWin32 (formerly win32all) package installs

● More useful on server side: only one machine requires Python + PyWin32 installs, any browser will work

● COM and DCOM: Microsoft component object models; DCOM allows client and server to be remote

● Host uses Active Scripting to export COM object interfaces for use in Python code

● See also Python Soap, XML-RPC support for web services

● More recent: IronPython Python port to C#/.Net framework, xlwings Excel plugin

Active Scripting example

→ Embedded Python in HTML was disabled on client due to rexec security issue, though Javascript can still call registered Python COM objects there; see also Pyjs, IronPython, ASP, PSP, Zope,…

<HTML>

<BODY>

<H1>Embedded code demo: Python</H1>

<SCRIPT Language=Python>

# embedded python code shows three alert boxes

# as page is loaded; any Python code works here,

# and uses auto-imported global funcs and objects

def message(i):

if i == 2:

alert("Finished!")

else:

alert("A Python-generated alert => %d" % i)

for count in range(3): message(count)

</SCRIPT>

</BODY></HTML>

A Python COM client

from sys import argv

docdir = 'C:\\temp\\'

if len(argv) == 2: docdir = argv[1] # ex: comclient.py a:\

from win32com.client import Dispatch # early or late binding

word = Dispatch('Word.Application') # connect/start word

word.Visible = 1 # else word runs hidden

# create and save new doc file

newdoc = word.Documents.Add() # call word methods

spot = newdoc.Range(0,0)

spot.InsertBefore('Hello COM client world!') # insert some text

newdoc.SaveAs(docdir + 'pycom.doc') # save in doc file

newdoc.SaveAs(docdir + 'copy.doc')

newdoc.Close()

A Python COM server

import sys

from win32com.server.exception import COMException # what to raise

import win32com.server.util # server tools

globhellos = 0

class MyServer:

# com info settings

_reg_clsid_ = '{1BA63CC0-7CF8-11D4-98D8-BB74DD3DDE3C}'

_reg_desc_ = 'Example Python Server'

_reg_progid_ = 'PythonServers.MyServer' # external name

_reg_class_spec_ = 'comserver.MyServer' # internal name

_public_methods_ = ['Hello', 'Square']

_public_attrs_ = ['version']

# python methods

def __init__(self):

self.version = 1.0

self.hellos = 0

def Square(self, arg): # exported methods

return arg ** 2

def Hello(self): # global variables

global globhellos # retain state, but

globhellos = globhellos + 1 # self vars don't

self.hellos = self.hellos + 1

return 'Hello COM server world [%d, %d]' % (globhellos, self.hellos)

# registration functions

def Register(pyclass=MyServer):

from win32com.server.register import UseCommandLine

UseCommandLine(pyclass)

def Unregister(classid=MyServer._reg_clsid_):

from win32com.server.register import UnregisterServer

UnregisterServer(classid)

if __name__ == '__main__': # register server if file run or clicked

Register() # unregisters if --unregister cmd-line arg

Other Internet-related tools

See Extras\Code\Internet directory on the class CD for more examples

♦ HTMLgen

● generates HTML from class-based page descriptions (see CD)

♦ HTML, SGML parsers

● useful for extracting information from web pages (standard lib, BeautifulSoup)

♦ rfc822 module, newer email package

● parses standard message headers (and much more)

♦ ILU, fnorb, and OmniORB packages

● can be used for CORBA networking (if anyone still does)

♦ urllib module

● opens URL, returns file-like object: read, readline (see CD)

● Parse fetched page with htmllib, string.find, xml package

♦ Restricted execution mode (security issue)

● See Extras\Code\Internet directory on CD

● for running code fetched from untrusted sources

● modules “rexec” and “bastion”

● → Withdrawn in Python 2.4: security concerns

♦ Other Internet protocol support

● web server classes (see Extras\Code\Internet on CD)

● telnetlib, nntplib (news), SSL sockets,…

♦ Zope & Plone (www.zope.org)

● An open-source web application framework

● Written in and customized with Python

● Plone runs on Zope: workflow-based content management

● “http://server/module/func?arg1=val1&arg2=val2” URL becomes

“module.func(arg1=val1, arg2=val2)” call on server-side Python

● [Defunct] Supplemental Zope/Plone workbook available

♦ Django: Python’s Rails?

Lab Session 10

Click here to go to lab exercises

Click here to go to exercise solutions

Click here to go to solution source files

Click here to go to lecture example files