[PP4E cover]

Running PP4E Examples on Python 3.3 Through 3.5


The book Programming Python, 4th Edition (a.k.a. PP4E) was published when Python 3.2 was current. This page summarizes the 4 minor steps required to run this book's major examples with Python versions 3.3, 3.4, and 3.5, and documents the corresponding changes applied to the current release 1.4 of the book's examples package, created October 15, 2013. In brief:

  1. [Windows only] Set PY_PYTHON=3 for new Windows launcher
  2. [Windows only] Fix #! lines unrecognized by new Windows launcher
  3. Install 2-file PyMailGUI patch required for 3.3 email package change
  4. Install the Pillow drop-in replacement for the PIL image library

The first 2 of these steps are needed for Windows only, and the first 3 are incorporated into release 1.4 of the book's examples package automatically. For instance, with release 1.4 the PyGadgets and PyDemos auto-launchers, as well as the larger stand-alone programs such as PyMailGUI, PyEdit, and PyCalc will work on 3.3 through 3.5 out of the box. Still, you'll need to manually install Pillow instead of PIL for image processing examples such as PyPhoto, and may need to set your PY_PYTHON on Windows for general 3.X use.

Note that all these changes are backward-compatible with earlier 3.X releases, so release 1.4 of the book's examples package works with Pythons 3.0 through 3.5, and perhaps later. Later Pythons may or may not require additional patches; watch for updates to be posted here as warranted. Also note that most book examples will run unchanged under 3.3 through 3.5; the steps outlined here are mainly required for specific programs only, and mostly for larger graphical programs. Some minor examples may require other changes not given here, but change is a constant in software development.

Updates (most recent first)


[Windows only] Set PY_PYTHON=3 for new Windows launcher

As noted here, the new Windows launcher automatically installed with Python 3.3 runs scripts and commands with an installed 2.X by default, unless they give a more specific release number in #! patterns or command argument options. This can effectively break programs meant to be used with 3.X if they use generic Python commands or have top-level script files with missing or release-ambiguous #! lines. As it's common practice on Windows to omit a #! line formerly useful only in some Unix usage modes, this can have broad impacts. Even on Unix, #! lines with a generic "python" resolved by file links is a norm, but will now fail if run by Python 3.3 on Windows.

To work around this, you can either:

  1. Add a Unix #! line to the top of every Python 3.X script you run on Windows—a bizarre expectation that needlessly adds extra work, and seems just plain rude to people who use Python on Windows.

  2. Set your PY_PYTHON environment variable to "3" to force the 3.3 launcher to default to the highest-numbered 3.X globally for all programs that don't give a version explicitly.
While a manual PY_PYTHON setting applies to all scripts run on your machine, it doesn't help much for scripts meant to run immediately with no user configuration steps—like some of those in the book. In lieu of PY_PYTHON settings, the changes required in auto-launcher scripts are subtle, and depend on how the launched code is run: Manually setting your PY_PYTHON makes both of these changes unnecessary and is recommended in general, but that doesn't address auto-launchers' goals.

In the book examples package

To work even in the absence of PY_PYTHON settings, release 1.4 of the examples package applies the required changes to the book's four auto-launcher scripts, plus one utility module that they leverage; the launchers see regular action as drug-out shortcuts on my Windows desktops:

     
PyGadgets
PP4E\Launch_PyGadgets_bar.pyw
Added #!/usr/bin/python3 to force 3.X execution of itself; this script runs Launcher.py tools in-process via imports.
PyDemos
PP4E\Launch_PyDemos.pyw
Added #!/usr/bin/python3 to force 3.X execution of itself; this script runs Launcher.py tools in-process via imports.
PyEdit
PP4E\Gui\TextEditor\pyedit.pyw
Added #!/usr/bin/python3 line to force 3.X execution of itself; this script runs the main PyEdit script in-process with exec().
PyMailGUI
PP4E\Internet\Email\PyMailGui\altconfigs\launch_PyMailGui.py
Added os.environ['PY_PYTHON']='3' to force 3.X execution of the PyMailGUI subprocess spawned with os.system(); a #! line for this script's own execution is irrelevant, as its code runs on both 2.X and 3.X.
Utility
PP4E\Launcher.py
Added both #!/usr/bin/python3 to force 3.X execution of itself (when run standalone), and os.environ['PY_PYTHON']='3' to force 3.X execution of any subprocess it spawns (see below).

Note that the last entry in this table, Launcher.py, sets PY_PYTHON=3 in os.environ, to be inherited by spawned programs. This may be moot for its immediate subprocesses, because this module runs explicit "python ..." command lines with os.spawnv() after setting Python and system paths automatically, thereby cutting out the 3.3 "py" launcher in the Windows registry; deeper program descendants, however, may still require this setting. Complex to be sure, but avoidable if you're willing to require an extra environment setting for using existing 3.X code under 3.3 on Windows, a manual task PP4E's top-level launchers were designed to obviate.

Also keep in mind that this patch applies to the selected auto-launcher scripts only. When using 3.3 and later, you'll still probably want to manually set your PY_PYTHON environment variable eventually in order to run other version-agnostic 3.X scripts on Windows with 3.X instead of 2.X. This includes any book examples not updated by this patch; see the Advanced system settings in your Control Panel's System entry to set the version default globally.

Three caveats on this process

Although this patch suffices in the normal case, there are three issues that may impact your Windows launcher experience worth noting here:

For more on the 3.3 Windows launcher in general, see the new Appendix B in Learning Python, 5th Edition, or its early draft here. The often prescribed alternative to environment settings mentioned earlier—adding a Unix #! line at the top of every script on Windows—seems absurd and extreme, especially given other launcher issues like those of the former and following sections.


[Windows only] Fix #! lines unrecognized by new Windows launcher

As described here, the new Windows launcher installed with Python 3.3 fails when a script begins with a Unix #! line that it cannot recognize. Some book example scripts used a "#!/bin/env python" pattern that was ignored on Windows by every Python through 3.2, but is now treated as an error by 3.3. Such #! forms are valid and even required on some Unix systems, but must be changed to a launcher-accepted pattern such as "#!/usr/bin/python" or "#!/usr/bin/env python" in order to run under 3.3 on Windows—a platform where such lines are otherwise irrelevant.

Note that this is true even if you use "python3" in the #! line or set your PY_PYTHON=3 per the prior section; an unrecognized #! pattern fails in 3.3 even if it's not version-ambiguous, and even if it's valid on a Unix system.

In the book examples package

In release 1.4, this fix was applied to the #! lines at the top of the following 10 files, changing their "/bin/env" to "/usr/bin/env"; no other code or environment changes are required:

     
PP4E\Launch_PyDemos.pyw (also changed for prior section)
PP4E\Launch_PyGadgets_bar.pyw (also changed for prior section)
PP4E\LaunchBrowser.pyw
PP4E\Internet\Ftp\Mirror\cleanall.py
PP4E\Internet\Ftp\Mirror\downloadflat.py
PP4E\Internet\Ftp\Mirror\downloadflat_modular.py
PP4E\Internet\Ftp\Mirror\ftptools.py
PP4E\Internet\Ftp\Mirror\uploadall.py
PP4E\Internet\Ftp\Mirror\uploadflat.py
PP4E\Internet\Ftp\Mirror\uploadflat_modular.py

You can locate and edit these files yourself in older example packages by using the Search/Grep tool in the book's PyEdit GUI example on the package's PP4E root directory, or by running one of the directory search and edit/replace utilities presented in the System Programming part of the book (e.g., see search_all.py and Visitor subclasses later in Chapter 6). Fixing any additional unrecognized #! patterns lurking in the examples package is an officially suggested exercise.

For more on the 3.3 Windows launcher in general, see the new Appendix B in Learning Python, 5th Edition, or its early draft here.


Install 2-file PyMailGUI patch required for 3.3 email package change

As covered here, Python 3.3 changed a standard library email utility that the book's largest example, PyMailGUI, depends on to format non-ASCII email address names for display. The patch is simple, and spans just two files:

This is a small bit of "monkey-patching"—replacing part of a module at runtime—which should be avoided in general, but in this case suffices to make the existing code run as is. The changes required to fully support Python's email API changes since the book's publication would be much more involved.

In the book examples package

The new and modified file of the patch are automatically included in PyMailGUI's source code directory of the new release 1.4 of the examples package; no other code or environment changes are required.


Install the Pillow drop-in replacement for the PIL image library

As mentioned here, the third-party PIL image-processing library used by examples in the book has been subsumed by the Pillow fork—an open source drop-in replacement for PIL which is being actively supported for use in newer Python releases. You'll want to fetch and install Pillow to use most of the book's image examples, including the PyPhoto viewer, which employs image display, thumbnail generation, and resize operations. I've also used Pillow successfully for EXIF photo metadata tag processing.

Pillow is currently available at this site. For example, all image-based book examples run fine under on Python 3.3 after installing the following on my Windows 32- and 64-bit systems, respectively:

See PIL's site for other Pythons and platforms; at this writing there are also Pillow installers for Python 3.4 and 3.5 (see Pillow 3.x).

Also note that Pillow is not required of all image display-only programs: as of Python 3.4 tkinter can display PNG images (thanks to Tk 8.6), and all Python 3.X's can display GIF and PPM/PPG images. Pillow is still required for other image type and Python combinations, as well as for other tasks beyond display—including the thumbnail and resizing operations implemented by PyPhoto.

In the book examples package

No fix was applied, because the example package doesn't ship 3rd-party systems as bundled items as a rule; you'll want to fetch and install Pillow for your Python and platform, from its official site above.



[Python Logo] Home Books Programs Blog Python Author Training Email Search ©M.Lutz