![[Home page]](PythonPowered.gif "Home page"){kind=small}
| Books | Code | Blog | Python | Author | Train | Find | ©M.Lutz |
This page provides updates on points that arose after publication, and serves as a virtual appendix to the book. It contains a list of Python and book updates which I've found important enough to post, but have not had time to organize more coherently yet (or post on PyErrata). More recent notes appear first in these lists. Some of these represent recent Python developments. Other entries are book clarifications or corrections. Although this text went through rigorous testing and review, a few bugs and typos are statistically inevitable in a 1250 page book and 42K lines of example source code. Moreover, future Python changes and coding techniques may impact parts of the book over time. Contents:
See also: lists below of updates applied thus far in the book reprint runs that I have seen or been told about, after the initial March 2001 release. You can tell which printing you have by looking near the bottom of the publishing details page (near the front). If there are no details here for a printing listed below, it is because I was not consulted for edits before that print run; see O'Reilly's errata list for any changes made. I also stopped keeping track of printings after 2001, so printing numbers after that time are unknown.
I mentioned the new nested scopes lookup on this page previously. I should also point out a common mistake that can arise if you generate callback handlers within a loop, and try to use enclosing scope references to refer to the loop variable within the generated functions. Because enclosing scope references are evaluated when the function is called (not created), they all wind up referring to the same setting of that variable--the one made by the last loop iteration. Even though nested scope lookup does simplify much callback code, you must still use defaults to rembember the value in this case, because defaults are evaluated at function creation time instead.
For details, see the "Nested scopes gotcha" write-up in the Learning Python 2E updates page, near the end of its "More Python Gotchas" section.
A reader wrote to ask about displaying GUI pop-up windows from a non-GUI program. This gives me as good an excuse as any to say a few more words about the Tkinter mainloop call, event-driven programs, and socket connections.
The best answer to this query is probably to take the plunge and restructure the program as a real GUI: initialize widgets on start-up, call mainloop once to start event processing and display the main window, and move all program logic into callback functions triggered in response to user actions. For instance, the original non-GUI code can be started by widgets on the main GUI window. That's the way that GUI programs are typically structured, and makes for a more coherent user experience than pop-up GUI windows appearing at seemingly random times, from an otherwise non-GUI script.
Alternatively, the GUI could be spawned by the non-GUI script as a separate program; user interaction results can be communicated from the spawned GUI to the script using sockets, files, or other IPC mechanisms. The advantage to this approach is that it provides a separation of GUI and non-GUI code; the non-GUI script would only have to be modified to spawn and wait for user results to appear from the separate GUI program, but could otherwise be used as is. Moreover, the non-GUI script would not be blocked while an in-process mainloop call runs (only the GUI process would run a mainloop), and the GUI program could persist after the point at which user inputs are required by the script, leading to fewer pop-up windows.
However, if you want to add a simple GUI user interaction to an existing non-GUI script (e.g., to select files to open or save), it is possible to do so by configuring widgets and calling mainloop from the non-GUI main program when you need to interact with the user. This essentially makes the program GUI-capable, but without a persistent main window. The trick is that mainloop doesn't return until the GUI main window is closed by the user (or quit method calls), so you cannot retrieve user inputs from widgets after mainloop returns. To work around this, all you have to do is be sure to save user inputs in a Python object: the object lives on, after the GUI is destroyed. For an example of this technique, see the off-page demo program:
This demo makes a simple 2-button main window, which launches file selection dialogs. Its output, printed as the main windows are closed, looks like this:popup1... C:/Python23/python.exe D:/temp/new.txt popup2... C:/Python23/dir1/__init__.py D:/temp/public_html/calendar.html ending...
Notice that this program calls mainloop twice, to implement two modal user interactions from an otherwise non-GUI script. It's okay to call mainloop more than once, but this script takes care to recreate the GUI's widgets before each call, because they are destroyed when the previous mainloop call exits (widgets are destroyed internally inside Tk, even though the corresponding Python dialog object still exists). Again, this can make for an odd user experience compared to a traditional GUI program structure -- windows seem to pop up from nowhere -- but it's a quick way to put a GUI face on a script without reworking its code.
Also notice that this is different from using recursive/nested mainloop calls to implement modal dialogs, as shown in the book on page 317. In that mode, the nested mainloop call returns when the dialog's quit method is called, but we return to the enclosing mainloop layer, and remain in the realm of event-driven programming. The mainloopdemo.py example instead runs mainloop two different times, stepping into and out of the event-driven model twice.
Finally, note that this scheme only works if you don't have to run any non-GUI code while the GUI is open, because your script is inactive and blocked while mainloop runs. You cannot, for example, apply this technique to use utilities like the guiStreams.py example on page 457-459 to route user interaction from non-GUI code to GUI windows. The GuiInput and GuiOutput classes in that example assume that there is a mainloop call running somewhere (they're GUI, after all). But once you call mainloop to popup these windows, you can't return to your non-GUI code to interact with the user until the GUI is closed, and the mainloop call returns. The net effect is that these classes can only be used in the context of a fully GUI program.
Really, the mainloopdemo.py demo above works only because the GUI can interact with the user independently, while the mainloop call runs; the script is able to surrender control to the Tkinter mainloop call and wait for results. That scheme won't work if you must run any non-GUI code while the GUI is open. Because of such constraints, you will generally need a main-window-plus-callbacks model in most GUI programs; callback code runs in response to interaction, but while the GUI remains open. Simply make a main GUI window with widgets that start the original non-GUI script's code in a callback handler, and post new dialog windows from callback handler code as needed. That way, you can run code while GUI windows are active. See the way that the non-GUI pack and unpack scripts are run from a GUI so that their results appear in a GUI on page 460 for an example.
Spawning the GUI as a separate program might help with the guiStreams example too, as long as inputs and outputs are communicated to the GUI over IPC mechanisms, and the widget after() method (or similar) is used by the GUI program to detect incoming output to be displayed. The non-GUI script would not be blocked by a mainloop call. For a simplistic example of this technique, see the following off-page scripts:
Start the nongui script--it spawns the gui script, which displays a popup window that shows the text printed in the nongui script (the date and time, sent once every 2 seconds from nongui to gui over a socket). The nongui script can keep running code to produce data, because only gui runs a mainloop call. Of course, if you have to work that hard anyhow, you might as well just make your script a traditional GUI program with a main window.Beginning with Python release 2.3, you can no longer use the "col" keyword argument as a short synonym for "column", when using the "grid" widget method to arrange widgets with gridding. This appears to have been broken between Python 2.2 and 2.3, though at first glance it seems to reflect a change in the underlying Tk toolkit used by 2.3, rather than Python itself (a TclError is issued when "col" is used in 2.3). It's difficult to understand the rationale for such a trivial and non-backward-compatible change, but "col" is clearly no longer allowed today.
So far, two book examples have been identified which use the "col" synonym, both of which appear near page 409 in Chapter 8, "A Tkinter Tour, Part 2" (other gridding examples use the longer "column"). The impacted examples are grid5b.py, and grid5c.py, both in directory Examples\PP2E\Gui\Tour\Grid on the CD. To fix, simply change "col" to "column" in the "grid()" calls; for example, in grid5b.py, change line 19:
from:
l.grid(row=numrow, col=i, sticky=NSEW)
to:
l.grid(row=numrow, column=i, sticky=NSEW)
I've been testing some of the book's examples on my new Zaurus Linux-based PDA lately, and thought I'd post three of the first scripts I coded and ran on that device, as supplemental examples. I worked on two of these on a plane, while simultaneously listening to MP3s on the same PDA; it doesn't get much better than that...).
You can find more about this PDA on this page, including a couple mid-page screen shots showing Python running in console mode; you can also run Python programs as Qtopia apps, with a bit of configuration (see also www.zaurususergroup.com).
As you might expect, Python works very well on this machine, because it's Linux. It is a complete, up-to-date Python, which comes with threads, sockets for Internet scripting, a full os module including popen and fork, PyQt for GUIs, and so on. You can also get interfaces for MySql, SQLite, numeric tools, etc. You can even use gcc to compile C extensions to .so files right on the PDA, and import them into your scripts.
One usage note: some standard library modules are missing in the Riverbank package; as a work-around, I simply copied the 2.3 standard library directory (from the source package) onto a CF card, and put that directory on my PYTHONPATH, to get some extra library modules (e.g., glob). A few library modules coded in C may require a compile with gcc or similar (I haven't needed to yet).
[Update, 12/03] It appears that it is possible to run the Tk library, and hence Python's Tkinter GUI toolkit, on this device. Details about the Tk configuration are available on this page. In short, X Windows and Tk need to be enabled in one of a variety of ways, and the Python Tk interface module must be compiled as well. This is not yet a simple process, but is an alternative to using Qt, anygui, or Jython for GUIs on the Zaurus, especially if you have existing Tkinter programs.
[Update 9/04: Per a hint from Fredrik Lundh, change the code below to use state='readonly' for the entry field to get a disabled but not greyed-out display in the calculator. This is not exactly like disabled in 2.2 (the field background is a bit shaded), but is better than either the disabled or normal states in 2.3.]
The Tkinter in the latest Python release, 2.3, now greys-out (stipples) text in Entry widgets that are marked as disabled. This is a change from all earlier Python releases, and likely reflects a change in the underlying Tk library. In 2.2 and earlier, disabled Entry widgets showed their text normally, but did not allow edits, and did not participate in keyboard tab focus traversals. Unfortunately, the numeric display in the book's final calculator GUI uses a disabled Entry field, to disallow user changes. Under 2.3, this field is now difficult to read because of the new grey-out effect. To fix, comment out the disable call in file PP2E\Lang\Calculator\calculator.py:
class CalcGui(GuiMixin, Frame):
...
def __init__(self, parent=None, fg=Fg, bg=Bg, font=Font):
...
if not parent or not isinstance(parent, Frame):
...
# 2.3: no way to disable but show normal?
# self.entry.config(state='disabled')
else:
...
Note that the grey-out effect in 2.3 has only been observed on Windows; this may or may not be an issue on other platforms. Also note that leaving the Entry field enabled means you can change its text (but probably shouldn't; a better solution may be to use a Label for the display). 2.3 also renders some buttons a bit smaller on Windows, but this is less intrusive.
A number of people have emailed me to say that they broke or lost their CDs, and need a copy of the book's examples. I've heard of a few CDs being broken on receipt as well. Luckily, O'Reilly maintains the book CD's Examples directory online, at this site: http://examples.oreilly.com/python2.
Get the Examples.zip file there; it contains all the book's examples, and is a simple zip file which you can download and unpack. Note that this is not the entire CD, but you probably will only care about getting this part of it. The CD also contains various open source packages (e.g., Python), but you can find more up-to-date versions of all the CD's packages elsewhere on the web. If you are unable to access the site above, please contact O'Reilly. I don't own the examples, and do not keep them online anywhere myself.
In this earlier note, I suggested that it may now be possible to change the red Tk icon in Python Tkinter GUI programs. I've now verified this: as of Python 2.3, you can change the icon to one of your own, with code lack that in this example script: testicon.py, which makes a window that looks like this.
Note that this changes the icon both for the open window itself, as well as for the program's minimized entry in the launcher bar. This is new as of Python 2.3; the example does not change the icon in Python 2.2 (yet another reason to upgrade).
Are you as tired of waiting for your email client to download huge spam messages over 50k dial-up lines as I am? A recent rash of such spam, about 150k each, prompted me to tweak one of the book's POP email examples, to only fetch message headers, and allow an email to be deleted from the server without actually downloading the entire message. You can find the script here.
It's a simple script, and true spam filters such as spambayes are much more powerful solutions; but running this script before my email client by clicking a desktop shortcut is usually all I need. Also, many spam filters must download full messages (not workable over a modem when there are 10M of them), this script is immune to differences in the filtering tools of email clients, and coding this was much quicker than becoming an expert in such things. It's an example of the sort of tactical things you can do when you know a scripting language like Python.
A few notes on this approach:
Someone wrote recently to express concern about buying this 2nd edition, given that it is "already two years old." For the most part, this edition is still valid and perfectly compatible with the current version of Python (2.3), and its examples still work as described. Python does evolve, but not so fast that two years is the software eternity that some might imagine.
However, in the last two and a half years, Python did indeed sprout a handful of new features that do not invalid this edition's examples, but do mean that some of them can be coded slightly more simply than shown. All of these features will be covered in detail in the upcoming book Learning Python 2nd Edition, and some are described elsewhere on this updates page. But as a service to readers concerned about life on the bleeding edge, and in lieu of a hypothetical 3rd Edition, here is a quick summary of the most prominent Python changes that impact this edition, in roughly decreasing relevance order:
Python recently started showing many more decimal digits for the repr() string of certain floating point numbers. That is, for repr(N), (and backquotes `N`, which is the same as repr), Python today shows more digits after the decimal point than it used to. Type "2 / 5.0" at the Python interactive prompt, and you'll see what I mean: today you get 0.40000000000000002, not the 0.4 of prior releases. The format of such numbers returned by the str(N) call (and by proxy, the print statement) is as before, without the full numeric precision: str(2 / 5.0) yields '0.4'.
This language change happened well after the book, and very recently (in release 2.2?). Unfortunately, because the calculator GUIs that show up in the book's Text and Language chapter convert numeric results for display with backquotes (`N`), you may now sometimes see more decimal digits than before. Because this can be annoying when doing simple calculations, you might want to patch the calculator.py file to use str(N) instead of `N`, as follows:
def runstring(self, code):
try:
###############################################################################
## 2/03: else new full precision oddness
## result = `eval(code, self.names, self.names)` # try expr: string
###############################################################################
result = str(eval(code, self.names, self.names)) # try expr: string
self.hist.append(code + ' => ' + result) # add to hist log
except:
exec code in self.names, self.names # try stmt: None
self.hist.append(code)
result = None
return result
As one reader wrote recently, some of the readme files on the book CD might be a bit misleading about the source-code tree's structure and installation. This may partly be because some of these files are legacy code meant only as extra examples, and have comments that reflect a prior tree structure different than the one shipped (e.g., some directory names don't match reality anymore). It may also be due to misconceptions of package imports in general. Although this is outside this book's scope, here are a few more words of clarification.
In Python-speak, a directory of modules and subdirectories is called a package. At the top-level of the book CD, the "Examples\PP2E" directory contains all the examples' source-code. Really, the "PP2E" subdirectory is the example package root, and directory "Examples" represents an arbitrary container for the package root. In book example code, all cross-directory imports are always relative to the PP2E root directory (e.g., import PP2E.Gui.xxx); imports of files that live in the top-level script's home directory are not (e.g., import xxx) because the home directory is searched first. This scheme avoids potential name collisions.
To setup your module search path for the examples source tree, you will add the directory containing PP2E on your machine. That is, PP2E's container must be the home directory, or listed in PYTHONPATH or other settings. On the CD, the top-level Examples directory is PP2E's container, and so is the one to add to the module search path (unless you use the self-configuring launchers). To copy the examples to your hard-drive, copy the PP2E directory to some other container, and then add that container to your module search path. This is all documented in the CD's Examples\PP2E\README-PP2E.txt file, which is the official information source for the tree.
The __init__.py files in package directories seem to be most prone to confuse. These files are present to identify a directory as an importable module package, and initialize the directory's namespace. Each directory listed in a script's import statement must have an __init__.py file. For example, given a directory structure: Root\A\B\C.py, and an import statement of the form: import A.B.C:
For the book's package, this means that if you copy the PP2E package directory from CD to a directory on your hard-drive, make sure to add the directory containing the PP2E directory to your module search path. The directory containing PP2E does not need an __init__.py file (though it is simply ignored if present). PP2E itself, and its package subdirectories, already have __init__.py files. Typically, __init__.py files are not meant to be useful if executed directly; they are run automatically during imports, the first time Python goes through a directory.
Part of the confusion expressed by some readers may revolve around the Examples\__init__.py example file on the CD. The Examples CD directory in general is only meant to stand for any PP2E container, and does not require an __init__.py file in practice. In fact, Examples\__init__.py is ignored and never run by imports--it if was, book examples would generate error messages and abort (Examples\__init__.py contains imports of old directory names which no longer exist in the examples tree!). Because of that, you should take the Examples directory's __init__.py file as nothing more than an example, not a required component, and view the CD's Examples root directory as simply a representative stub.
Finally, remember that import basics like these are not really part of this book's scope; it assumes you already understand things like __init__.py files. The package import mechanism is covered in Learning Python--the book that you should consult first for such core language topics. Package imports are introduced in that book's 1st Edition, and covered in more depth in that book's upcoming 2nd Edition. As stated in its preface, Programming Python 2nd Edition is oriented more towards application-level programming concepts, and so does not itself pretend to teach package import details in any sort of depth. At 1250 pages, we had to draw a few lines.
I'd like to insert footnotes about five unrelated issues that came up recently, by email or otherwise:
Due to a recent change in the Python C API, the ppembed-callables.c embedding example source file needs to be changed for use with recent Python releases. When compiling under 2.2, the following error is reported:
ppembed-callables.c: In function `PP_Debug_Function': ppembed-callables.c:51: too many arguments to function `_PyTuple_Resize'
Basically, API function _PyTuple_Resize has been changed in Python's API to accept just 2 arguments now, as documented in 2.2 Python release notes:
tupleobject.h: extern DL_IMPORT(int) _PyTuple_Resize(PyObject **, int);
To make the ppembed example compile, change this line:
oops = _PyTuple_Resize(&args, (1 + PyTuple_Size(args)), 1);to this:
oops = _PyTuple_Resize(&args, (1 + PyTuple_Size(args)));
Note that this fix will make ppembed compile, but may not by itself make debugged function calls work as before, though non-debugged calls should work fine. (To be resolved; the resize copying order may differ now, and so may require additional code to move tuple components after each resize).
The 2.2 release notes say of the original 3rd argument removal: "If this affects you, you were cheating." That seems a bit harsh, given that this example was originally present in the 1st edition, worked fine when the 2nd edition was published, and had been compiling and working correctly since 1994--for some 8 years. Regrettably, the Python C API is more prone to change than the core language.
I've recently received a handful of email queries about copyright issues associated with reusing the book's example source code in other systems. Here is my opinion, and O'Reilly's.
As far as I'm concerned, you may freely use the book examples in your own software systems, in any fashion you like. As an author, I consider this a form of compliment, and I'm always glad to hear that the examples are finding practical application. In fact, I encourage code reuse whenever possible--Python code wants to be both reused, and free.
Technically, though, my publisher (O'Reilly) owns the copyright on the book and its examples, so you'll want to consult their policy if you need a legally binding answer. The good news is that O'Reilly's policy is very liberal: in short, if you reuse example code directly, they only require some sort of attribution that names the original book/author. Naturally, there are additional regulations regarding book citations in other publications, which any good editor should be able to recite from memory.
If you want to read about O'Reilly's example code reuse policy in full, please see their web pages:
The following notes pertain to the auto-launcher scripts presented near the end of chapters 4 and 8. They seem to be standing up to time well, but a few tweaks are worth mentioning:
Just for fun, I recently coded four simple scripts that count the number of lines of source code in the examples shipped with the book. They reuse the finder and visitor examples to do directory traversals. To take a look, or to use these scripts for metrics of your own, click here. The verdict: the book examples tree shipped on the CD includes roughly 900 source files, and 42,000 source lines. Given that most of this line count represents Python code, multiply by 3 or 4 to derive the SLOC equivalent for C, C++, or Java.
Python 2.2 was released near the end of 2001, roughly a year after this book's 2nd Edition was finished. With respect to the book's examples, perhaps the most significant change in 2.2 is the introduction of arbitrarily nested static (a.k.a. lexical) scopes. In brief, this new nested scopes rule allows code within both nested def statements and lambda expressions to automatically have access to names defined in the scope of all enclosing function defs.
This is a fairly major language change. In all prior releases, code that referenced a variable name only had access to names defined in the local scope (def or lambda body), the enclosing global scope (module), and the built-in scope--a 3-scope search known as the LGB rule. In 2.2, the names in the local scopes of any and all enclosing function defs will also be visible to the nested function. This new variable lookup layer has been added between the L and the G in the LGB rule: on variable references, Python now searches the local scope, then all enclosing defs (if any) from inner to outer, then the module, and finally the built-in scope. This scope rule change was available in 2.1 as an optional extension, but had to be explicitly enabled with a "from __future__" import statement.
This Python change does not break examples in the book (they will still work with 2.2 and later), but it does mean that callbacks in some of the examples can be coded in somewhat simpler terms after the 2.2 release. Because enclosing scope names are now visible, it is usually no longer required to save names from an enclosing scope by passing them into nested defs and lambdas with default arguments. One noteable exception: defaults are still sometimes required in loops (see this note).
This is an especially common case in lambda expressions used to generate callback handler functions in Tkinter code (lambda introduces a new local scope, just like a nested def statement). For instance, code of the following form:
class MyGui:
def action(self, tree):
print tree
def makeWidgets(self):
tree = 'larch'
Button(text='Spam',
command=(lambda: s=self, t=tree: s.action(t))).pack()
can now be coded in the simpler way:
class MyGui:
def action(self, tree):
print tree
def makeWidgets(self):
tree = 'larch'
Button(text='Spam',
command=(lambda: self.action(tree))).pack()
simply because the variables "self" and "tree" from
the enclosing def scope (the method) are automatically retained
for later use by code in the body of the lambda function.
The lambda is still needed to defer execution of the action
method call (until the invocation of the generated anonymous
function at event time), but there is no need to also pass
in data from the enclosing scope. The impacted parts of the
book include pages 271-274 and 310-311, and various examples.
We still occasionally find typos in the narrative that were missed during the copyedit (it happens, no matter how many people proof a book). None of these impact program examples, or the book's technical accuracy. But I will use this entry to note any odd or missing word reports. Items found recently:
A few people have asked about ways to change the #! line at the top of script files (this line gives the path to the Python interpreter on UNIX-like machines). It's easy to do this with the visitor_replace.py script in the book's PyTools directory, and described on page 216. For example, say something like this to replace all #!/usr/bin/python lines with #!\Python21\python:
C:\...\PP2E>python PyTools\visitor_replace.py
#!/usr/bin/python #!\Python21\python
Lots of status messages scroll by unless redirected to a file.
visitor_replace does a simple global search-and-replace operation
on all non-binary files in an entire directory tree (it recursively
visits all subdirectories).
It's also a bit naive: it won't change other #! line patterns that
mention python (e.g., you'll have to run it again to change
#!/usr/local/bin/python), and might change occurrences besides
those on a first line. That probably won't matter, but if it does,
it's easy to write your own Visitor subclass to be more accurate.
Or just use one I coded: click here to fetch script visitor_poundbang.py. Copy this file into your PP2E\PyTools directory. When run, this script converts all #! lines in all script files in the entire book examples tree. It changes every first line that starts with #! and name "python", to a line you pass in on the command line or assign in the script:
C:\...\PP2E>python PyTools\visitor_poundbang.py #!\MyPython21\python Are you sure?y . ... 1 => .\__init__.py 2 => .\PyDemos2.pyw 3 => .\towriteable.py ... 1474 => .\Integrate\Mixed\Exports\ClassAndMod\output.prog1 1475 => .\Integrate\Mixed\Exports\ClassAndMod\setup-class.csh Visited 1475 files and 133 dirs, changed 190 files .\towriteable.py .\Launch_PyGadgets.py .\Launch_PyDemos.pyw ... C:\...\PP2E>type .\Launch_PyGadgets.py #!\MyPython21\python ############################################### # PyGadgets + environment search/config first ...This script caught and changed 190 files (more than visitor_replace), so there must be other #! line patterns lurking in the examples tree besides #!/usr/bin/python. The Visitor class framework essentially wraps the os.path.walk call (see chapter 5).
As implied by the socket chapter, it is possible to test server-side CGI scripts without connecting to the Internet. Simply run a CGI-capable web server as a background process on the local machine, and use a URL of the form "http://localhost/..." in your web browser to contact the local server. This should work on any machine with true multi-tasking and socket support, and can be both a testing technique, and a way to leverage your web browser as a GUI interface (CGI scripts replace in-process GUI callbacks, albeit with the tradeoffs explored on page 811). This was not demonstrated directly in the book due to problems with Python's HTTP server support on Windows in 1.5.2 and 2.0 described below; in other contexts, this technique works fine.
Some people run a full-blown server like Apache locally in such a role. As of Python 2.0. the 7 line webserver.py script on page 940 can also, with a few customizations, be used to run a CGI-capable web server on your local Windows (or other) machine. That is, it can be used to serve up local HTML files and CGI scripts with just a standard Python install on Windows, and without connecting to the Internet at all. In this scheme, the Python-coded web server and your web browser run on the same machine. As such, this is another way to test and run CGI scripts without having an account on a web server machine that supports Python.
Simply start webserver.py on your local machine, and use server name "localhost" in your URLs (e.g., type "http://localhost/cgi-bin/test0.cgi" at the top of your browser to contact your locally running webserver). This can work portably, because the Python 2.0 CGIHTTPServer module now tries os.popen2 to spawn the CGI script if no os.fork exists (and falls back on execfile() if neither exists). Prior to 2.0, that module only tried os.fork, and so was a UNIX-only tool; in 2.0 and beyond, os.popen2 is used to start CGI scripts on Windows. To see why the server name "localhost" works such magic, see the client-side socket calls overview on page 530 and beyond (as noted in the book, your web browser is a socket client). Such URLs should also allow Python scripts to contact a locally running server with the Python urllib module.
You'll probably need to subclass CGIHTTPRequestHandler to tailor directory and file extension names--it only recognizes .py and .pyc files by default, (not .cgi), and wants to find CGI scripts segregated in a subdirectory like cgi-bin. A more Windows-friendly webserver.py may be posted here later, if I get time to code one myself; till then, consider it a suggested exercise, and a possible future Python extension. There are a few additional UNIX-only calls in some of the book's CGI scripts (e.g., os.path.samefile in getfile.cgi) that may require more customization for a server running on Windows, and you'll want to run the fixsitename.py script to replace any lurking "starship.python.net/~lutz" references in HTML files to "localhost" (see the Server Side Scripting chapter for details). Naturally, you should be able to run webserver.py locally under Linux and UNIX as well.
Of course, whether or not running a server locally like this is easier than using a server machine, depends on your constraints and resources. For many (and perhaps most), configuring and starting a local web server may actually be more work than simply telneting into their server. If you don't have access to a remote server, though, this can be convenient (for example, I started using the webserver.py script in 2003 to test CGI-based programs on a Linux PDA, without a network or modem card).
5/28 Note: When running such a Python-coded web server locally on Windows and contacting it with an Internet Explorer client on the same machine, there appears to be an issue with POST style requests to scripts. POSTed forms fail, but GET requests to the exact same script (a form with method=GET, or a URL with an appended query string) work fine. Scripts generate identical output for both POST and GET, so this is not a bug in the examples, but seems to be related to the Windows socket model used by os.popen2. In fact, tracing shows that scripts receive correct inputs and generate correct outputs for POST too, but the output appears to be lost or munged on the way back to the web browser. Oddly, this issue appears to only exist on Windows for the combination of Python's CGIHTTPServer server module, and the Internet Explorer client. For instance, a locally running Apache server will handle POST correctly, and Netscape 4.7 on NT reportedy handles POST requests properly as well. More details will be posted here as they emerge (no pun intended). This issue was reported on the Python bug list at sourceforge on 5/29; you can probably read the report's status here.
Python now has a sys.executable builtin that gives the full directory path name of the Python interpreter program that is running the calling script:
>>> import sys >>> sys.executable 'C:\\PROGRAM FILES\\PYTHON\\PYTHON.EXE'In some cases, and on some platforms, this call might be able to replace some of the Python interpreter searches that are performed in a few examples early in the book (see the launcher utilities for more details). This is not a replacement for those directory search tools in general, though--they are also used to search for other files in various examples. It also may not be as general or portable a solution; for instance, the searchers fall back on brute-force disk searches if needed, and the interpreter needs to do some fairly complex and platform-specific work to set sys.executable including a similar PATH search (see Python source file Modules/getpath.c). This call is a nice option for tools like os.spawnv, though, which need a Python interpreter path passed in.
Update 5/22/01: ActiveState will include a fix for this bug in an ActivePython2.1 patch release, scheduled to be out within a month; it was apparently a Windows resource build option bug. So, these examples should work fine under all ActivePython distributions, except for the original 2.1 release (and you may simply fetch the patch release if you have the original 2.1 release). For more details, please consult the ActivePython bug reports pages at activestate.com. This patch release, ActivePython 2.1.0 Build 211 (June 15 2001), apparently also fixes a Tkinter radiobutton bug.
There is a bug in ActivePython2.1's implementation of Tkinter that makes a few of the book examples fail on Windows. At the least, any example that uses the guimaker.py tool will not work as is under ActivePython2.1. This includes PyEdit, PyView, PyMail (its mail composition windows), and probably big_gui and shellgui. However, all of these examples still work under the standard Python2.1 release for Windows that is available from www.python.org (as well as all older versions available on the book's CD).
The reason is that ActivePython2.1's Tkinter for Windows does not recognize the cursor='hand2' and cursor='gumby' configuration options used by the guimaker.py module for toolbars and help buttons. Note that these are standard Tk cursor settings, and should work in all Tkinters. Much worse, ActivePython's Tkinter not only does not implement these options, it raises an exception for them, which makes programs that use these (and likely other) cursors fail.
This is a bug in ActivePython2.1, and has been reported to ActiveState. The settings do work as advertised in the standard Python 2.1 release. But if you have installed ActivePython 2.1 and need a workaround for the problem, you have three options:
Again, this is a bug in ActivePython2.1, not in the examples or the standard Python release. Because of that, I won't post a patch to fix this in the examples themselves; it doesn't make sense to cripple code for a particular distribution's bug. Also note that although I am happy to investigate such issues, their resolution is beyond my control; please report such issues to ActiveState directly (this specific bug has already been reported).
Be sure to click/run the Launch_PyDemos.pyw auto-configuring script in the Examples\PP2E directory for a quick look, not file PyDemos.pyw itself. The latter of these assumes that your Python module search path (PYTHONPATH) is set to include the book's source-code package root. The Launch_PyDemos script accomodates this by configuring the search path automatically, and then spawning PyDemos.pyw. This works, because PyDemos, and programs it spawns, inherit the module search path setting made by Launch_PyDemos. If your PYTHONPATH is not set (either manually, or by the top-level Launch_PyDemos script), PyDemos pograms that depend on the module search path setting will not work.
I thought that this launcher structure was explained clearly enough in the book's Preface and CD README files, but a few people have already been burned by this (and to be fair, there is a less clear descripton of the demo launchers structure in the Pragmatics appendix). While I'm on the subject: please note that the launchers do work with the standard Python installs available on the book's CD and at www.python.org; if you do something custom, you may need to patch the launcher scripts' code. The book's demo launchers do work with the ActivePython install too, but that install includes an unrelated Tkinter bug in release 2.1 that makes some demos fail (see the prior note).
And finally, if you set up your module search path manually, be sure to add the directory that contains the PP2E root to your PYTHONPATH setting. This is all described in the CD README files and book, and is required because cross-directory imports in the book examples are always relative to the PP2E module package root; for example:
from PP2E.Gui.Tools import guimakerIn particular, be careful to not add the PP2E directory itself to the module path, but instead add its parent. For instance, if you copy the CD's Examples\PP2E directory to "C:\PP2E", then you must add "C:" to PYTHONPATH (not "C:\PP2E"). This is the way that module package imports work in general. (Update 01/03: see the newer and related note).
The last code listing on page 66 almost looks like a typo, but it's not. If you look closely, you'll notice that the first output line shows up second in the output file. That is, the text written first with the file object write method appears after the text written second with the os module write call. This looked like a bug to me too when I reread it, but it does work exactly as shown. Here's why: In this case, the file object's output written first is buffered by the stdio file system, and is not actually transferred to the file until the object's close method is called at the end of the interaction. In contrast, the os module's write call does not do buffering by default, so its text winds up first in the file, even though it is written second. Strange, but true. See the close and buffering descriptions on pages 60 and 63 for more details. And speaking of files: be sure to see the standard struct module if you're looking for tools that unpack and pack strings transferred from/to binary mode files; this seems to be a commonly asked question.
In the code listing on page 837, the generateIndexed function adds "#" characters to both href and name attributes for HTML section (fragment) links, generated for the with-index display mode of PyErrata browse requests. The "#" character is generally only required at the href attribute (of course), but is completely harmless if used in the name attribute as well, under Internet Explorer. That is, the code works as is and as shown so this is not quite a bug; but the # at fragment name attributes is unnecessary, and is usually omitted.
There has been a report that file names on the book CD get mangled to 8.3 DOS format when the High Sierra file system is mounted on the CD under Solaris (Rock Ridge extensions aren't supported). Subsequent reports have suggested that this also may occur on some other flavors of UNIX. Unfortunately, import statements and file searches in the examples really need to see the original, non-8.3, unmangled names. On at least some UNIX systems, this may mean that although the examples on the CD and in the book are indeed coded to be platform neutral, the initial version of the CD itself is not. However, this is only known to be an issue on a handful of platforms; the CD should work fine on recent Windows, Linux, and Mac machines, and on UNIX platforms with proper support.
We're exploring a fix, but if you're having trouble seeing the original file names on the CD, don't have access to a supported machine, and can't find another way to read the CD on your machine, for now please send an email (see the end of this page) for a link to an alternative examples package on the web (a tar/gzip file). Naturally, if you have a Windows or Linux box on your network, you may also be able to simply copy the CD contents to one of those machines, and ftp or otherwise transfer it over to your UNIX box. Also note that on recent versions of Linux, the CD works fine, but be careful to read it in such a way as to preserve the original unmangled file names; see your mount and fstab man pages if you need help (e.g., you may need to specify iso9660 and/or joliet keywords in your /etc/fstab file on some installations). In the worst case, most Linux users may also copy the CD to a hard disk under Windows, and then copy it over to your Linux partition.
Update #1, 5/27: The gory details. It looks as if the issue is that the book CD was formatted with the Joliet standard used by most commercial CD burners available for PCs. Joliet is an extension to the ISO-9660 CD-R standard that is used to support long filenames, and overcome the 8.3 file naming limit of straight ISO. Long file names on Joilet CDs can be read correctly today on Windows, Linux, and Macs, but not on some UNIX systems that lack Joliet support. UNIX systems that don't support Joliet format generally want to find the Rock Ridge long filenames extensions on the CD instead; you can still read the book CD on such UNIX systems, but you'll get the straight ISO 8.3 mangled filenames, not the Joliet long names. Unfortunately, Joliet won't work on some UNIX systems, and Rock Ridge apparently won't work on Windows. Unless you provide both formats on the same CD, one or the other gets 8.3 ISO names. And just for fun, Macs define yet another incompatible format.
For more background details, see the CD-R FAQ entries here and here. A workaround (e.g., a 8.3-to-long file copy and rename script) is still being researched, but please contact me as requested above for the time being if this is effecting you. If you feel ambitious and get to this before I do, you might also be able to code a copy-and-rename script yourself (see this gzipped text file for the output of a "dir /S /B" CD listing run on DOS, sorted by full pathname). Again, this only impacts users on UNIX systems for which a Joliet-format CD reader does not exist, and who do not have access to other kinds of machines; the CD should work fine today on recent versions of Windows, Linux, Macs, and other UNIX boxes. If you do have trouble seeing long filenames on Linux and Mac machines see this page; the standard Linux kernal started supporting Joliet as of release 2.0.34, and some Macs may require an extension download.
In the first server-side scripting chapter, I gave a handful of hints for locating a web server with Python (you generally need one in order to install and experiment with the server-side CGI scripts in the book). Another option not mentioned in this section: it's also possible to install the Apache open source web server on your own machine, and run the book's server scripts from there. See Apache's site for details; this is probably not for the novice, but isn't too difficult in general. Alternatively, you could try coding and running a web server in Python with the standard CGIHTTPServer module (see the example near the end of the Advanced Internet Topics chapter), or consult a list of Python-friendly web providers (one is maintained here, but you should also check the archives at python.org; most ISPs likely have or will install Python these days).
Update: See also the note above about running webserver.py locally. With a few customizations, that script can serve up HTML files and CGI scripts on the local machine, and without having to connect to the net; simply use a URL of the form "http://localhost/..." in your browser to contact the local server. It can be run locally on a Linux or UNIX machine almost out of the box, and should work on Windows with a few caveats mentioned in the prior note (there was a bug with POST requests and Internet Explorer as I wrote this, which may be fixed by the time you read this).
I found two typos that will be repaired in the book's next printing. On page 1041, in the first indented paragraph, "awg" should be "awk" (of course!). And on page 1040, in the first line of the last interaction listing, it should read "@= ..\bases\zoo.kb" (a weird upside-down apostrophe showed up where the first backslash should be). Like I said, typos happen.
Update: these typos will be fixed in the third printing of the book, scheduled to happen in early June 2001. For a complete list of typo fixes I requested for the third printing, click here.
[Update, 09/03]: Someone reported that the Tk icon issue may have been fixed in Tk 8.4, which is part of Python 2.3. Calling the window's iconbitmap method with the name of a properly formatted icon file as the argument apparently does change the icon today (I haven't had a chance to verify this yet). Update: now verified: see this note.
[Update, 08/01]: The py2exe packaging/distribution tool now also has a way to change the icon: simply pass a .ico file name to the script's --icon command-line argument. py2exe is similar to the installer tool, and builds upon the distutils extension. It converts python scripts into standalone executable Windows programs, that can be run without requiring a Python installation (which is quite amazing, even apart from its ability to change the Tk icon). Get py2exe here.
[Update, 07/01]: Matt Butler recently announced a small extension which lets you easily change the window icon of toplevel windows in Tk on Windows (you pass it a .ico icon file name to replace the default red icon). You can grab it from the following URLs, or at the Vaults site-- home: http://hackicon.sourceforge.net, download: http://prdownloads.sourceforge.net/hackicon/hackicon00003.zip .
This isn't discussed in the book at all, but I've gotten a few questions about this in classes, and it has come up on the Python newsgroup as well. Apparently, there is no way in a Python script to replace the default red "Tk" icon that shows up in the upper left hand corner of all toplevel Tkinter windows. Top-level window object protocols that deal with icons only deal with the icon shown when a window is iconified (minimized), and don't work on Windows in any event.
There might be a way to replace the window border icon with a bitmap of your own in the Tk C library: the C function Tk_SetWindowBorderPixmap(tkwin, pixmap) looks like a good bet, but it can only be used from C, not Python. You can always build a small C extension module to expose this C function to Python, but that requires a C compiler. (If you want to try, the Tk binary library file installs with Python, and Tk source code is already available on the book CD). If I find a better solution, I'll post it here. At the least, I hope this functionality is exposed to Python in a future Tkinter release.
Speaking of Tkinter: I also recommend the Manning book "Python and Tkinter Programming" as a follow-up to the material in the GUI part of this 2nd edition; in particular, its coverage of the supplemental widgets in the PMW extension may prove valuable if you need to do any industrial strength GUI development. As mentioned in the book, PMW adds additional widgets to the roughly 20 widgets native to Tkinter. When you install PMW, you also get a tabbed notebook, paned windows, spin (combo) boxes, and so on. The combination of Tkinter and PMW is at least as rich as wxPython, and does not suffer from wxPython's initial framework complexity and documentation shortage (he says, trying very hard to be objective :-). You may also find the PMW package and documentation on the book's CD.
PyMailGui, the Tkinter client-side email tool in Part III, seems to run into "Bad File Descriptor" type errors after running awhile on Windows, especially after downloading lots of email. When it does, it produces an error popup, and needs to reload all email from scratch. I have no idea why this occurs, and it seems to have gotten worse in Python 2.0. It is not a bug in PyMailGui itself, but appears to be in the Python libraries or lower. Since poplib is careful to close and delete the socket used to fetch email, this is probably related to the underlying socket library on Windows. If anyone has a better clue, I'd love to hear it; for now, you should simply ignore this socket error message pop-up if you ever see it and press "Load" again, unless and until a workaround surfaces.
Subtle thing: When sending email with the smtplib module, you should generally insert a blank line between the mail header lines and the text of the message, to be robust. The book doesn't described this format, and you don't actually need the blank line in most cases. In fact, a few email examples in Part III of the book omit the blank line (smtpmail, PyMailGui, and PyMailCGI), yet all the email examples in the book work correctly as shown. However, some servers and systems detect the end of the headers by simply noticing a blank line, or a line that does not contain a colon (":"). They do not perform a deeper structural analysis to detect valid header lines (Python doesn't either--see method readheaders in the source code of Python's rfc822 module for an example). Therefore, without the blank line separator, if the first line of the message text happens to contain a ":", it might be classified as a header, and dropped from the message body.
I've never been burned by this in the two years that I've been using these examples, so it's a very obscure thing, and not quite a bug. But to be general, either concatenate an extra "\n" after the header lines, or be sure to type a blank first line in the message if the first real line contains a ":". For more details on mail message formats, please see http://www.faqs.org/rfcs/rfc822.html. And as it states repeatedly, please also keep in mind that this book is about Python coding; it is not intended as a formal guide to all the jots and tittles of Internet protocol standards (many of which are not enforced by commonly used Internet systems). You should consult other sources for protocol details.
Update: these examples will be made more robust as described above, in the second printing of the book. The second printing is scheduled to happen in April 2001 (about a month after the first printing), so this issue is unlikely to exist in most copies of the book. But for details on the changes made, click here. (Update 6/02: see also the new "email" standard library package in Python2.2, for other ways to format and parse email messages.)
You can fetch a slightly enhanced version of the book's PyDemos top-level demo launcher script, by clicking here. Use your browser's save-as option to store locally, if needed, and place this file in your PP2E root directory. This version's new main window looks like this. It adds buttons on the right that automatically pop-up each demo's source code files in PyEdit windows. This is a minor mutation, but makes it easy to consult example code quickly--something I've done often enough to warrant the change. This version also demonstrates PyEdit being used in pop-up (Toplevel) mode, rather than as an attached component or spawned stand-alone program.
I want to clarify two points related to the Preface, about which I've already gotten email. First of all, if you have an existing Python 1.5.2 installation, you do not need to install Python 2.0 to run most examples in this book; they should work fine under 1.5.2, even though the Preface states that this edition covers 2.0. This backwards compatibility was one reason that examples use things like string module calls, not 2.0 string methods (string methods are discussed in the text itself). In fact, I still have Python 1.5.2 installed on one of the development machines used to write this book--for which, of course, I am deeply ashamed :-).
Secondly, I want to reiterate that this edition has been redesigned to be an advanced follow-up to Learning Python. It is now mostly about applications of Python, not core language fundamentals. That is, it focuses on ways to use Python syntax for real tasks, rather than the syntax itself. If you are hunting for a detailed description of the class statement, for example, you won't find it here--core language details have now been delegated to the book Learning Python. This is a break from both the 1st Edition and O'Reilly Perl books, but was made deliberately for at least three reasons. First of all, using the language for real tasks represents the second, and much larger half of the Python learning experience. Secondly, many people learn best by example, especially after mastering language basics. And thirdly, the scope of this new edition's examples makes it possible to illustrate larger design concepts such as OOP and component reuse in a more meaningful and satisfying way. If this design shift is still not clear, please see the Preface for more about this edition's structure.
See also: New recent Python changes summary earlier on this page.
Python 2.2 was released late in December 2001. It includes new language features such as:
The latest and greatest Python release, version 2.1, may now be fetched from python.org. There are not as many features in this release as there were in 2.0, but it's worth a peek. In short, 2.1 introduces rich comparison overloading methods, a warning framework, "weak" references, function attributes, the PyDoc and PyUnit tools, a new xreadlines method for files, a new C build system, and a bit more. As mentioned elsewhere on this page, arbitrarily nested scopes were retracted from 2.1, but can be turned on as an option, and will likely appear in 2.2 late this year. You can play with nested scopes in 2.1 by saying "from __future__ import nested_scopes".
Two new packages were recently announced that merit a blurb here: check out the PyGame 1.0 package, a set of extension modules for writing games with Python and the cross-platform SDL DirectMedia library. Also see the SOAP.py release for a Python interface to the XML-based SOAP protocol.
A beta release of Pippy--a port of Python to the Palm platform--is now available. Once installed, Pippy actually provides a Python command-line on the Palm; you can import modules installed, and type and run Python code interactively on a Palm device. For instance, with Pippy, you can use the Python socket and Internet support modules to implement things like web servers and mail readers on your Palm (no, really!). More generally, Pippy promises to bring Python's productivity and maintainability benefits to the embedded systems domain. See the Pippy home page or python.org for downloads and more details; to go straight to some Pippy screen-shots, click here.
Python 2.1 will include the PyDoc system--a tool that extracts documentation strings from Python programs, and nicely formats them into a HTML (and other) formats for viewing. There is also a dedicated website, www.pydoc.org, which includes PyDoc pages for the Python standard library. Release 2.1 will also include the PyUnit automated testing system--see 2.1 release notes for more details. Speaking of tools, also see PyChecker--a new program that does simple lint-like program analysis, and pyprof--a simplified interface to the Python profiler. You can find all these on python.org, or the vaults of parnassus.
Fredrik Lundh has implemented some nice tools for doing XML-RPC from Python--a protocol that lets scripts invoke methods on a remote machine, and passes data between caller and callee as XML-formatted text streams over sockets. Consult python.org, or read about xmlrpclib on its page PythonWare. Fredrik also has a soaplib module in the works to support the somewhat similar SOAP protocol.
ActiveState's Komodo system is a Python development IDE that is free, portable, and based on the Mozilla application framework. Among other things, Komodo allows GUIs to be specified in XUL--an XML based description language. See ActiveState for more details. Technically speaking, Komodo takes something of an agnostic approach to language choice: it will serve as an IDE for a variety of programming languages. ActiveState will also sell VisualPython, a Microsoft-based IDE that lets you develop scripts under Visual Studio (which must be purchased separately).
A newly emerged IDE for Python development, demoed at the IPC9 conference. See the Wing IDE home page, or python.org for links.
The PythonWorks IDE has sprouted a very nice refactoring feature, which automatically changes and rearranges code for clarity and maintainability. See PythonWare for details.
PythonWare is also at work on an optimized replacement for the current Tkinter system; it will be a drop-in, compatible replacement for the current Tkinter, and will ship with future Python releases. PythonWare is also polishing uiToolkit--a framework built on top of Tkinter, which allows widgets to be easily constructed in efficient Python code.
Python 2.1 alpha introduced, and then retracted, a change which will allow scopes to nest arbitrarily, when scope constructs (class, def, and lambda) are nested syntactically. Because this feature breaks some existing code, it is disabled by default in the upcoming 2.1 release, but may be enabled as a standard feature in 2.2. When it is turned on, this feature eliminates the need to use default arguments in lambdas and nested defs to save values from the enclosing scope. Such code will still work under the new scheme, but names from an enclosing scope will be naturally visible without the use of default arguments. This change will also obviate the familiar LGB scope rule (local-global-builtin 3-scope name lookup); or if you prefer, it generalizes "L" to be an arbitrary number of scopes. This change also breaks uses of both "from *", and "eval" and "exec" without explicit namespace dictionaries, whenever they appear within a function (the compiler cannot resolve names it doesn't know about). In other words, this is a use-at-your-own-risk feature, and is thus not enabled in the upcoming 2.1 release. See more about the 2.2 release earlier on this page, and see the book note on this page that describes this language change in more detail.
Check out the new online Python Journal. It has been resurrected, and is being refashioned into an interactive site. (Update, 6/02: see also the new "Py" print magazine; search python.org for more details.)
Back to the PP2E home page