6. Modules

 

 

 

 

 

Why use modules?

 

 

¨  Code reuse

¨  System name-space partitioning

¨  Implementing shared services or data

 

 

 

 

Module topics

 

¨  The basics

¨  Import variations

¨  Reloading modules

¨  Design concepts

¨  Modules are objects

¨  Package imports

¨  Odds and ends

¨  Module gotchas

 

 


 

Module basics

 

 

¨   Creating modules:        Python files, C extensions; Java classes (Jython)

¨   Using modules:             import, from, reload()

¨   Module search path:    $PYTHONPATH

 

 

 

 

file: module1.py

 

def printer(x):           # module attribute

    print x

 

 

 

Module usage

 

% python

>>> import module1                    # get module

>>> module1.printer('Hello world!')

Hello world!

 

>>> from module1 import printer       # get an export

>>> printer('Hello world!')

 

>>> from module1 import *             # get all exports

>>> printer('Hello world!')

 

 

 

 

 


 

 

“from *” can obscure variables’ meaning

 

>>> from module1 import *     # may overwrite my names

>>> from module2 import *     # no way to tell what we get

>>> from module3 import *

>>> . . .

>>> func()     # ç ?!

 

 

 

 

“from” does not play well with “reload”

 

>>> from module1 import func       # copy variable out

>>> func()                         # test it

 

…change module1.py…

 

>>> reload(module1)                # ç fails: bad name!

 

>>> import module1                 # must bind name here

>>> reload(module1)                # ok: loads new code

>>> func()                         # ç fails: old object!

 

>>> module1.func()                 # this works now

 

>>> from module1 import func       # so does this

>>> func()


 

 

Module files are a namespace

 

 

¨   A single scope: local==global

¨   Module statements run on first import

¨   Top-level assignments create module attributes

¨   Module namespace: attribute ‘__dict__’, or dir()

 

 

 

file: module2.py

print 'starting to load...'

 

import sys

name = 42

 

def func(): pass

 

class klass: pass

 

print 'done loading.'

 

 

Usage

>>> import module2

starting to load...

done loading.

 

>>> module2.sys

<module 'sys'>

>>> module2.name

42

>>> module2.func, module2.klass

(<function func at 765f20>, <class klass at 76df60>)

 

>>> module2.__dict__.keys()

['__file__', 'name', '__name__', 'sys', '__doc__', '__builtins__', 'klass', 'func']


 

 

 

Name qualification

 

 

¨   Simple variables

·        ‘X’ searches for name ‘X’ in current scopes

¨   Qualification

·        ‘X.Y’ searches for attribute ‘Y’ in object ‘X’

¨   Paths

·        ‘X.Y.Z’ gives a path of objects to be searched

¨   Generality

·        Qualification works on all objects with attributes: modules, classes, C types, etc.

 

 

 

 

 

“sys.stdin.readlines”

 
 

 

 

 


                       

                             1) find “sys” in the current name-space (scope)

 

                             2) find “stdin” in the “sys” object’s name-space

 

                             3) find “readlines” in the “sys.stdin” object

 

 

 

 

 


 

Import variants

 

 

 

¨  Module import model

·        module loaded and run on first import or from

·        running a module’s code creates its top-level names

·        later import/from fetches already-loaded module

 

 

¨  import and from are assignments

·        import assigns an entire module object to a name

·        from assigns selected module attributes to names

 

 

 

 

Operation

Interpretation

import mod

fetch a module as a whole

from mod import name

fetch a specific name from a module

from mod import *

fetch all top-level names from a module

reload(mod)

force a reload of module’s code

 

 

 

 

 

=

 

 from M import X

 

 import M

 X = M.X

 del M

 
          

 

 

 

 

 

 

 

 

 

 

 

 

 

Relative import syntax (2.5+)

 

 

from __future__ import absolute_import    # till 2.7?, only from

 

import string    # always finds the standard library's version

 

from .string import name1, name2     # import names from pkg.string

from . import string                 # import pkg.string

 

 

 

 

 

 


 

Reloading modules

 

 

¨   Imports only load/run module code first time

¨   Later imports use already-loaded module

¨   reload function forces module code reload/rerun

¨   Allows programs to be changed without stopping

 

 

General form

 

import module                # initial import

[use module.attributes]

...                          # change module file

...

reload(module)               # get updated exports

[use module.attributes]

 

 

 

Usage details

 

¨   A function, not a statement

¨   Requires a module object, not a name

¨   Changes a module object in-place:

·        runs module file’s new code in current namespace

·        assignments replace top-level names with new values

·        impacts all clients that use ‘import’ to fetch module

·        impacts future ‘from’ clients

 

 

>>> from M import func

>>> func()                 # edit func’s source

>>> reload(M)              è FAILS: no M in this scope

 

>>> import M

>>> reload(M)

>>> func()                 è FAILS: old object!

 

>>> M.func()               # okay: refetch from M now

 

>>> from M import func

>>> func()                 # okay: refetched name

Reload example

 

¨   Changes and reload file without stopping Python

¨   Other common uses: GUI callbacks, embedded code, etc.

 

 

 

% cat changer.py

message = "First version"

 

def printer():

    print message

 

 

% python

>>> import changer

>>> changer.printer()

First version

>>> 

 

 

[modify changer.py without stopping python]

 

% vi changer.py

% cat changer.py

message = "After editing"

 

def printer():

    print 'reloaded:', message

 

 

[back to the python interpreter/program]

 

>>> import changer

>>> changer.printer()      # no effect: uses loaded module

First version

 

>>> reload(changer)        # forces new code to load/run

<module 'changer'>

>>> changer.printer()

reloaded: After editing

 


 

 

Package imports

 

Module package (directory) imports

·        module name == “dir.dir.dir” in import statements and reloads

·        “import dir1.dir2.module”    =>   load dir1\dir2\mod.py

·        “from dir1.dir2.mod import name”

·        dir1 must be contained by a directory on PYTHONPATH

·        each dir must have a  “__init__.py”  file, possibly empty

·        __init__.py gives directory’s namespace, __all__ for “from*”

·        simplifies path, disambiguates same-named modules files

 

Example

 

For:

dir0\dir1\dir2\mod.py

And:

import dir1.dir2.mod

 

·        dir0 (container) must be listed on the module search path

·        dir1 and dir2 both must contain an __init__.py file

·        dir0 does not require an __init__.py

 

dir0\

    dir1\

        __init__.py

        dir2\

            __init__.py

            mod.py

 

 

Another example tree

 

root\

    sys1\

        __init__.py       (__init__ needed if dir in import)

        util.py

        main.py           (“import util” finds here)

        other.py

    sys2\

        __init__.py

        util.py

        main.py

        other.py

    sys3\                 (here or elsewhere)

        __init__.py       (your new code here)

        myfile.py         (“import util” depends on path)

                          (“import sys1.util” doesn’t)

 

Odds and ends

 

¨   Python 2.0: “import module as name”

·        Like “import module” + “name = module”

·        Also good for packages: “from sys1.util import util”, versus “import sys1.util as util”

 

¨   Loading modules by name string

·        exec ‘import ‘ + name

·        __import__(name)

 

¨   Modules are compiled to byte code on first import

·        ‘.pyc’ files serve as recompile dependency

·        compilation is automatic and hidden

 

¨   Data hiding is a convention

·        Exports all names defined at the top-level of a module

·        Special case: __all__ list gives names exported by “from *”

·        Special case: “_X” names aren’t imported by a “from*”

 

¨   __name__ and ‘__main__’

·        name set to ‘__main__’ when run as a script

·        allows modules to be imported and/or run

·        Simplest unit test protocol, dual usage modes for code

 

file: runme.py

 

def tester():

    print "It's Christmas in Heaven"

 

if __name__ == '__main__':       # only when run

    tester()                     # not when imported

 

Usage modes

 

% python

>>> import runme

>>> runme.tester()

It's Christmas in Heaven

 

% python runme.py

It's Christmas in Heaven


 

 

 

Module design concepts

 

 

¨   Always in a module: interactive = module __main__

¨   Minimize module coupling: global variables

¨   Maximize module cohesion: unified purpose

¨   Modules should rarely change other module’s variables

 

mod.py

X = 99                  # reader sees only this

 

import mod

print mod.X ** 2        # always okay to use

 

import mod

mod.X = 88              # almost always a Bad Idea!

 

import mod

result = mod.func(88)   # better: isolates coupling

 

 

 

 

 

 

 

 

 


 

Modules are objects: metaprograms

 

 

¨   A module which lists namespaces of other modules

¨   Special attributes: module.__name__, __file__, __dict__

¨   getattr(object, name) fetches attributes by string name

¨   Add to $PYTHONSTARTUP to preload automatically

 

 

 

File mydir.py

verbose = 1

 

def listing(module):

    if verbose:

        print "-"*30

        print ("name: %s file: %s" %

                 (module.__name__, module.__file__))

        print "-"*30

 

    count = 0

    for attr in module.__dict__.keys():      # scan names

        print "%02d) %s" % (count, attr),

        if attr[0:2] == "__":

            print "<built-in name>"          # skip specials

        else:

            print getattr(module, attr)      #__dict__[attr]

        count = count+1

 

    if verbose:

        print "-"*30

        print module.__name__, "has %d names" % count

        print "-"*30

 

if __name__ == "__main__":

    import mydir

    listing(mydir)      # self-test code: list myself

 


 

 

¨  Running the module on itself

 

 

 

C:\python> python mydir.py

------------------------------

name: mydir file: mydir.py

------------------------------

00) __file__ <built-in name>

01) __name__ <built-in name>

02) listing <function listing at 885450>

03) __doc__ <built-in name>

04) __builtins__ <built-in name>

05) verbose 1

------------------------------

mydir has 6 names

------------------------------

 

 

 


 

Another program about programs

 

 

¨   ‘exec’ runs strings of Python code

¨   ‘os.system’ runs a system shell command

¨   __dict__ attribute is module namespace dictionary

¨   ‘sys.modules’ is the loaded-module dictionary

 

 

 

file: fixer.py

editor = 'vi'  # your editor's name

 

def python(cmd):

    import __main__

    namespace = __main__.__dict__

    exec cmd in namespace, namespace

 

def edit(filename):

    import os

    os.system(editor + ' ' + filename)

 

def fix(modname):

    import sys                      # edit,(re)load

    edit(modname + '.py')

    if modname in sys.modules.keys():

        python('reload(' + modname + ')')

    else:

        python('import ' + modname)

 

 

 

% python

>>> from fixer import fix

>>> fix("spam")             # browse/edit, import by name

 

>>> spam.function()         # spam was imported in __main__

>>> fix("spam")             # edit and reload() by name

 

>>> spam.function()         # test new version of function


 

 

 

Module gotchas

 

 

 

“from” copies names but doesn’t link

 

nested1.py

X = 99

def printer(): print X

 

 

nested2.py

from nested1 import X, printer    # copy names out

X = 88                            # changes my "X" only!

printer()

 

 

nested3.py

import nested1             # get module as a whole

nested1.X = 88             # change nested1's X

nested1.printer()

 

 

% python nested2.py

99

% python nested3.py

88

 

 

 

Statement order matters at top-level

 

file: order1.py

func1()               # error: "func1" not yet assigned

 

def func1():

    print func2()     # okay:  "func2" looked up later

 

func1()               # error: "func2" not yet assihned

 

def func2():

    return "Hello"

 

func1()               # okay:  "func1" and "func2" assigned


 

Recursive “from” import gotchas

 

¨   Solution: use “import”, or “from” inside functions

file: recur1.py

X = 1

import recur2           # run recur2 now if doesn't exist

Y = 2

 

file recur2.py

from recur1 import X    # okay: "X" already assigned

from recur1 import Y    # error: "Y" not yet assigned

 

 

>>> import recur1

Traceback (innermost last):

  File "<stdin>", line 1, in ?

  File "recur1.py", line 2, in ?

    import recur2

  File "recur2.py", line 2, in ?

    from recur1 import Y    # error: "Y" not yet assigned

ImportError: cannot import name Y

 

 

 

 

“reload” may not impact “from” imports

 

¨   “reload” overwrites existing module object

¨   But “from” names have no link back to module

¨   Use “import” to make reloads more effective

import module        Þ module.X reflects module reloads

from module import X Þ X may not reflect module reloads!

 

 

 

“reload” isn’t applied transitively

 

¨   Use multiple “reloads” to update subcomponents

¨   Or use recursion to traverse import dependencies

 

See CD’s Extras\Misc\reloadall.py


 

 

 

Optional reading: a shared stack module

 

 

¨   Manages a local stack, initialized on first import

¨   All importers share the same stack: single instance

¨   Stack accessed through exported functions

¨   Stack can hold any kind of object (heterogeneous)

 

 

file: stack1.py

 

stack = []                           # on first import

error = 'stack1.error'               # local exceptions

 

def push(obj):

    global stack                     # 'global' to change

    stack = [obj] + stack            # add item to front

 

def pop():   

    global stack

    if not stack:

        raise error, 'stack underflow'  # raise local error

    top, stack = stack[0], stack[1:]    # remove front item

    return top

 

def top():

    if not stack:                       # raise local error

        raise error, 'stack underflow'  # or let IndexError

    return stack[0]

 

def empty():      return not stack      # is the stack []?

def member(obj):  return obj in stack   # item in stack?

def item(offset): return stack[offset]  # index the stack

def length():     return len(stack)     # number entries

def dump():       print '<Stack:%s>' % stack

 

 

 


 

Using the stack module

 

% python

>>> import stack1

>>> for i in range(5): stack1.push(i)

...

>>> stack1.dump()

<Stack:[4, 3, 2, 1, 0]>

 

 

Sequence-like tools

>>> stack1.item(0), stack1.item(-1), stack1.length()

(4, 0, 5)

>>> stack1.pop(), stack1.top()

(4, 3)

>>> stack1.member(4), stack1.member(3)

(0, 1)

>>> for i in range(stack1.length()): print stack1.item(i),

...

3 2 1 0

 

 

Exceptions

>>> while not stack1.empty(): x = stack1.pop(),

...

>>> try:

...     stack1.pop()

... except stack1.error, message:

...     print message

...

stack underflow

 

 

Module clients

file: client1.py

from stack1 import *

push(123)                  # module-name not needed

result = pop()

 

file: client2.py

import stack1

if not stack1.empty():     # qualify by module name

    x = stack1.pop()

stack1.push(1.23)          # both clients share same stack


 

 

 

Lab Session 5

 

Click here to go to lab exercises

Click here to go to exercise solutions

Click here to go to solution source files

Click here to go to lecture example files