Using Python

In USENIX ;login:, January 1999

Mark Lutz is a software developer, and a Python writer and trainer. He has worked on compilers, programming tools, scripting languages, and assorted client/server systems.

Note: Adapted from Mark Lutz, "Python: An Object-Oriented Scripting Language" in Handbook of Programming Languages, vol. 3 "Tools and Little Languages" (P.H. Salus, ed.). Indianapolis: Macmillan, 1998. Used with the permission of the publisher.

If you're like most developers, one of the first questions you may ask when you hear about a new programming language or tool is: what can I use it for? After all, programming languages are tools; although they can be interesting to study in isolation, their real utility lies in the context of applications.

Python Defined

In terms of acronyms, Python can be classified as both a VHLL (very high level language) and an OODL (object-oriented dynamic language). Although this puts Python in the same general category as languages such as Smalltalk and Eiffel, Python has a flavor all its own. For instance, Python has a very practical orientation: unlike some OO (object-oriented) languages, Python was created by and for engineers interested in solving day-to-day programming tasks.

Because Python is a general-purpose language, arriving at a single all-encompassing definition isn't an easy task. What Python is depends much on how it is used. To some, Python serves as programmable front-end to libraries coded in a compiled language like C or C++. For others, it takes the form of an embedded scripting language for customizing larger systems. And many programmers use Python as a standalone language, leveraging its library of precoded system interfaces. But in terms of basic functionality, Python can be described in a variety of ways.

Object-oriented Scripting Language

Python is a fundamentally object-oriented language. Its "class" system supports modern OO development paradigms and makes Python ideal for use in OO systems. Perhaps more important for a scripting language, Python's object model is surprisingly easy to use, yet it supports advanced OO concepts such as operator overloading, multiple inheritance, and more.

Standalone Rapid Development Tool

Python provides both a conceptually simple language and rapid development-cycle turnaround. It is specifically optimized to support speed of development. In fact, many programmers find Python development to be orders of magnitude faster than other approaches. By combining development speed with integration tools, Python fosters radically faster development modes.

Next-generation Extension Language

Like many scripting languages, Python programs can be both extended with modules coded in a compiled language like C and embedded in a C program. But unlike many scripting tools, Python provides both integration with external components and a powerful OO language. Because of this combination of utility, Python can address a wide range of problem domains.

Freely Available, Interpreted Language

By most definitions, Python is an interpreted language. More accurately, Python code is compiled to portable byte code that is run by a virtual machine. This scheme provides both fast turnaround after program changes and reasonable execution speed. Finally, Python is true freeware: its source code is freely available, and it may be embedded in products without copyright constraints or fees.

Some Quotable Quotes

But don't take this writer's word for it. Some of the comments that have been made by Python programmers on the Internet speak volumes about the language's design. Although personal reactions vary, the following quotes are representative of the level of enthusiasm Python typically generates in newcomers:

"Python looks like it was designed, not accumulated." Python sports a refreshingly coherent design. Python's creator carefully balanced the need to support practical programming needs with the desire to avoid feature glut and complexity. The result is a simple language that scales well to support larger systems. Moreover, Python has a remarkably clear syntax: for many, its inherent readability makes it ideal for programs that may be reused or maintained by others.

"Python bridges the gap between scripting languages and C." Python's design and implementation place it somewhere between compiled languages such as C and traditional scripting languages like csh, Awk, and Perl. Python's syntax resembles languages like C (e.g., there are no "$" variable prefixes), but its high-level programming tools and lack of compile and link steps are familiar to users of other scripting languages. In some sense, Python provides the best of both worlds: it supports scripting and embedding, but doesn't ask programmers to abandon normal standards of development quality.

"Python is the BASIC of the 90s." Although Python is certainly being applied in real companies and products, many current Python users fall into the "hobbyist" category: they use Python because they want to, not because they have to. In fact, Python's popularity is largely due to a strong grassroots following among engineers, similar in spirit to the support BASIC enjoyed in the early days of the PC revolution. Naturally, Python isn't a new and improved BASIC (it incorporates advanced programming tools such as exceptions, modules, and classes), but it generates similar excitement.

"Python is as easy or as powerful as you want it to be." Python programs can range in complexity from simple five-line scripts, to complex object-oriented frameworks. Program complexity can be scaled for both the programmer's skill level and the problem at hand. For instance, Python supports OOP, but does not impose it: classes are an optional language tool. Programmers (not language designers) decide whether OOP, and other advanced Python tools, are warranted for each task.

"Python: less filling, tastes great." This quote will probably be more meaningful if you're familiar with American beer advertisements, but it underscores one of the central concepts in Python. If you've used other compiled, strongly typed languages like C or C++, you'll find that Python eliminates much of the complexity inherent in traditional tools. In fact, some have called Python "executable pseudocode": because of their simplicity, Python programs can more closely reflect the problems they are designed to solve.

Some of these quotes will make more sense after we examine the language in more detail. For instance, some of Python's simplicity stems from its high-level built-in object types (lists, dictionaries, etc.) and the absence of type and size declarations in Python (objects are created dynamically and reclaimed automatically when no longer used).

A Python History Lesson

Python was created by Guido van Rossum in Amsterdam around 1990. Before Python, Guido worked on the ABC programming language and the Amoeba distributed operating-system and both of these influenced Python's design. For instance, Python was originally conceived as a scripting language for the Amoeba system, and it inherits ABC's usability study-inspired syntax features. But Python adds practical features to ABC, such as a C-like nature and integration with external C components. Python borrows features from a variety of languages, including Modula, C, Icon, and C++.

Despite the proliferation of snake icons and book covers, Python is named after the BBC comedy series "Monty Python's Flying Circus." According to Python folklore, Guido viewed various Monty Python videos at the time he was busy inventing a new object-oriented language; the association naturally arose. This legacy tends to foster a generally humorous (and occasionally irreverent) flavor to the Python community. For instance, the standard labels "foo" and "bar" become "spam" and "eggs" in Python examples, and quotes from Monty Python skits pervade the Python culture (and documents like this).

Python was first offered to the public domain in 1991, and a USENET newsgroup devoted to Python first appeared in 1994 (<comp.lang.python>). Python now enjoys a growing international user community, with a strong following in the U.S., Europe, Japan, and Australia. A nonprofit support organization, the Python Software Activity (PSA), has been established to help manage some of its growth. The PSA maintains Python's Internet sites and helps organize Python conferences. At this writing, the PSA is hosted by the Corporation for National Research Initiatives (CNRI), which also pays Guido's salary.

Uses for Python

Python's general-purpose nature makes it applicable to more domains than I can list here. But as a sampling, here's a quick look at some of the most common things people are using Python for today.

System Utilities (Shell Tools)

Python's existing interfaces to operating system tools make it ideal for writing portable system utilities. Among other things, Python supports UNIX tools such as sockets, regular expressions, and POSIX bindings. On MS-Windows, Python also interfaces with COM, OLE, and more.

Graphical User Interfaces

Python's high-level nature and rapid turnaround are well suited to GUI development. It comes with an OO interface to the Tk GUI library, MFC, X11, and more. The Tk interface runs on X Windows, MS-Windows, and the Macintosh, making it a portable GUI solution.

Internet Scripting

Python also comes with a full suite of Internet tools. It has support for CGI scripts, Web browser applets, Web agents and crawlers, HTML generation, and more. Precoded library modules support most Internet protocols: FTP, URLs, HTTP, email, etc. On MS-Windows systems, Python can also be in conjunction with ActiveX scripting, and the new JPython system supports Python/Java integration.

Database Programming

Object persistence is a standard part of Python: programmers can save entire Python objects to a persistent store and retrieve them later. Moreover, Python offers interfaces to more traditional database systems, such as Oracle, Informix, Sybase, mSql, PostGres, and ODBC.

Component Integration

Because Python can be integrated with components written in compiled languages like C and C++, many use Python as a glue language, to control or extend application components. Wrapping libraries in a high-level language like Python, makes them easier to use. And embedding Python in a C/C++ program, opens up the system to customization without recompilation.

Rapid Application Development

To Python programs, components written in Python and C look the same. This allows developers to implement in Python first and later move selected components to a compiled language like C. This development mode is often called prototype-and-migrate: when used well, it can leverage both the development speed of Python and the performance advantages of C.

More Specific Domains

Python is also being used in more specific application areas, such as numeric programming, artificial intelligence, image processing, and distributed object systems (e.g., CORBA). Although these are interesting applications, many are really instances of the component integration potential of Python. For example, Python numeric programming is implemented as compiled-language libraries, integrated into Python for ease of use.

What Python Can't Be Used For

Naturally, no language can be optimized for every development need. Like most interpreted languages, Python sacrifices some performance in order to maximize development speed. Because of that, Python is not the ideal tool for implementation of time-critical components. For instance, compute-intensive image processing libraries are probably best coded in a compiled language such as C, C++, or FORTRAN.

But even in performance-critical domains, we can make use of Python's development speed. By implementing compute-intensive components in a compiled language and exporting them to Python, we get a system that's both easy to use and efficient. The numeric programming example mentioned previously is a perfect example of this approach to development.

Python is optimized for speed of development and designed for multiple-language systems: by adding it to the mix, we can make use of a rapid development language for our system interface and infrastructure while simultaneously achieving performance goals. Moreover, Python can be used to prototype demanding systems before investing the extra time needed to code critical components in C.

Compulsory Features List

Another way to understand Python is by listing its major features. Table 1 summarizes some of the things people seem to like best about the language. It's by no means complete; the point to notice is that Python combines a wide variety of features in a single, relatively simple language. For many, the end result is a remarkably expressive and responsive language, which can actually be fun to use.

Table 1: Major Python Features

Note that most of Python's features aren't new by themselves: it's really how Python combines features that make it what it is. For many, it seems to be the right mix of programming language concepts.

Python in the "Real World"

Python is more than a collection of features. It has become a tool of choice for real programmers in real companies around the world. To get a feel for how companies apply Python for real development tasks, here's a sampling of some of the larger organizations in Python's current user base. This list is necessarily incomplete; check Python's Web site <http://www.python.org> for up-to-date information about other companies applying Python.

Surfing Made Easy (Infoseek, Four11)

Many major Internet-related companies use Python in a variety of roles. For instance, Infoseek was an early adopter of Python in its Web-search systems.

Cockroaches and Pepsi Jingles (Blue Sky Animation)

If you've seen the movie Joe's Apartment, you've seen Python at work. Python has been successfully applied as a scripting language for creating commercial-grade animation.

Blowing Up the Energizer Bunny (the Alice VR System)

The University of Virginia uses Python as the scripting interface for its virtual reality system, Alice. One Alice demo features the Energizer Bunny (along with methods to make it explode).

Rocket Science (Lawrence Livermore Labs, NASA)

Lawrence Livermore Labs use Python in numerical programming applications and sponsored a recent Python conference. Python is also being applied in a number of ways at NASA.

Linux Install Tools (Red Hat)

Any time you install the Red Hat Linux system, you're using Python: portions of Red Hat's installation system are written in Python.

Distributed Object Systems (ILU, Hector)

ILU and Hector both make use of Python in distributed object programming systems. ILU, from Xerox, is a CORBA-based system that provides bindings to Python's object model.

Finding the Grail, Intelligent Agents (CNRI)

Besides Python, Guido van Rossum also created an extensible Internet/Web browser called (appropriately enough) Grail. It's written in Python, and allows Python programs to define applets executed in the client (browser), much like Java. CNRI also uses Python in research on intelligent agents for the Internet.

Apples and Oranges and Bananas and Coconuts

Python is often compared to other free (and not so free) languages, especially on Internet forums. Although language choice is often a matter of style, a brief comparison to similar tools can help point out both some of Python's primary distinctions and some of the main reasons people choose to use it. Table 2 summarizes some of the most common arguments of Python proponents.

Naturally, your mileage may vary. Some of the distinctions in Table 2 reflect different design goals. For instance, Perl is optimized for text processing, C++ and Java are generally considered to be systems languages (not scripting tools), and Tcl is targeted more toward simple glue tasks than larger systems development.

Programmers matter, too: although Python is deliberately designed to prevent unreadable code, there's nothing stopping us from writing it if we try. Moreover, having many languages to choose from can only be a good thing. In fact, support for multiple-language systems is one of the central concepts in Python: no language can satisfy all our needs at once.

Programmers must be empowered to pick and choose the best language for each task at hand. Python is a great language to have in your development toolbox, but it should not be the only one.

Table 2: Python versus similar tools

For More Information

To learn more about Python, visit Python's Web site <http://www.python.org>, the author's Web site <http://learning-python.com>, or the Python USENET newsgroup <comp.lang.python>.

Besides a vigorous online community, you'll find tutorials, reference manuals, and Python books. For instance, at this writing, O'Reilly <http://www.oreilly.com> publishes two Python books, with a third on the way.