Why use functions?
♦ Code reuse
♦ Procedural decomposition
♦ Alternative to cut-and-paste: redundancy
Function topics
♦ The basics
♦ Scope rules
♦ Argument matching modes
♦ Odds and ends
♦ Generator expressions and functions
♦ Design concepts
♦ Functions are objects
♦ Function gotchas
Function
basics
♦ def is an executable statement; usually run during import
♦ def creates a function object and assigns to a name
♦ return sends a result object back to the caller
♦ Arguments are passed by object reference (assignment)
♦ Arguments, return types, and variables are not declared
♦ Polymorphism: code to object interfaces, not datatypes
General form
def <name>(arg1, arg2,… argN):
<statements>
return <value>
Definition
>>> def times(x, y): # create and assign function
... return x * y # body executed when called
...
Calls
>>> times(2, 4) # arguments in parenthesis
8
>>> times('Ni', 4) # functions are 'typeless'
'NiNiNiNi'
→ “Polymorphism”
The
meaning of an operation depends on its subject.
By not
caring about types, code becomes more flexible.
Any object
with a compatible interface will work.
Most errors are best caught by Python, not your code.
Example: intersecting sequences
● Definition
def intersect(seq1, seq2):
res = [] # start empty
for x in seq1: # scan seq1
if x in seq2: # common item?
res.append(x) # add to end
return res
● Calls
>>> s1 = "SPAM"
>>> s2 = "SCAM"
>>> intersect(s1, s2) # strings
['S', 'A', 'M']
>>> intersect([1, 2, 3], (1, 4)) # mixed types
[1]
Scope
rules in functions
♦ Enclosing module is a ‘global’ scope
♦ Each call to a function is a new ‘local’ scope
♦ Assigned names are local, unless declared “global”
♦ All other names are global or builtin
♦ Added in 2.2: ‘nonlocal’ enclosing function locals (if any) searched before global
♦ Added in 3.X: ‘nonlocal’ variables can be changed if declared, just like ‘globals’
Name resolution: the “LEGB” rule
● References search up to 4 scopes:
1. Local (function)
2. Enclosing functions (if any)
3. Global (module)
4. Builtin (__builtin__ (2.X), builtins (3.X))
● Assignments create or change local names by default
● “global” declarations map assigned names to module
Example
♦ Global names: ‘X’, ‘func’
♦ Local names: ‘Y’, ‘Z’
♦ Interactive prompt: module ‘__main__’
X = 99 # X and func assigned in module
def func(Y): # Y and Z assigned in function
Z = X + Y # X not assigned: global
return Z
func(1) # func in module: result=100
Enclosing Function Scopes (2.2+)
def f1():
x = 88
def f2():
print x # 2.2: x found in enclosing function
f2()
f1() # prints 88
def f1():
x = 88
def f2(x=x): # before 2.2: pass in values with defaults (ahead)
print x
f2()
f1()
# More useful with
lambda (ahead)
def func( ):
x = 42
action = (lambda n: x ** n) # 2.2
def func( ):
x = 42
action = (lambda n, x=x: x ** n) # before 2.2
# Most useful for closures: state retention
(non-OOP)
>>> def
maker(N):
... def action(X): # make, don’t call
... return X ** N
... return action
# return new func
...
>>> f = maker(3) # “remembers” 3 (N)
>>> f(2),
f(3) # arg to X, not N
(8, 27)
>>> g = maker(4) # “remembers” 4 (N)
>>> g(2), g(3)
(16, 81)
>>> f(2),
f(3) # f still has 3
(8, 27)
More on
“global”, and 3.X “nonlocal”
♦ ‘global’ means assigned at top-level of a module file
♦ Global names must be declared only if assigned
♦ Global names may be referenced without being declared
♦ 3.X: “nonlocal X” means X in enclosing def changeable
All Pythons
y, z = 1, 2 # global variables in module
def all_global():
global x # declare globals assigned
x = y + z # no need to declare y,z: 3-scope rule
Python 3.X Only
# enclosing function vars are changeable (state retention)
>>> def
outer():
... x =
1
... def inner():
... nonlocal x
...
x += 1
... print(x)
... return inner
...
>>> f = outer() # f is really an inner
>>> f()
2
>>> f() # each outer() would have own “x”
3 # see “closures” above
More on
“return”
♦ Return sends back an object as value of call
♦ Can return multiple arguments in a tuple
♦ Can return modified argument name values
>>> def multiple(x, y):
... x = 2
... y = [3, 4]
... return x, y
...
>>> X = 1
>>> L = [1, 2]
>>> X, L = multiple(X, L)
>>> X, L
(2, [3, 4])
More on argument
passing
♦ Pass by object reference: assign shared object to local name
♦ In def, assigning to argument name doesn’t effect caller
♦ In def, changing mutable object argument may impact caller
♦ Not pass ‘by reference’ (C++), but:
■ Immutables act like ‘by value’ (C)
■ Mutables act like ‘by pointer’ (C)
>>> def changer(a, b):
... a = 2 # changes local name's value only
... b[0] = 'spam' # changes shared object in-place
...
>>> X = 1
>>> L = [1, 2]
>>> changer(X, L)
>>> X, L
(1, ['spam', 2])
Equivalent to these assignments:
>>> X = 1
>>> a = X # they share the same object
>>> a = 2 # resets a only, X is still 1
>>> L = [1, 2]
>>> b = L # they share the same object
>>> b[0] = 'spam' # in-place change: L sees the change too
Special
argument matching modes
♦ Positional matched left-to-right in header (normal)
♦ Keywords matched by name in header
♦ Varargs catch unmatched positional or keyword args
♦ Defaults header can provide default argument values
♦ 3.X: Keyword Only, after “*” in def, must pass by name
|
Operation |
Location |
Interpretation |
|
func(value) |
caller |
normal
argument: matched by position |
|
func(name=value) |
caller |
keyword
argument: matched by name |
|
def func(name) |
function |
normal
argument: matches any by name or position |
|
def func(name=value) |
function |
default
argument value, if not passed in call |
|
def func(*name) |
function |
matches
remaining positional args (tuple) |
|
def func(**name) |
function |
matches
remaining keyword args (dictionary) |
|
func(*args, **kargs) |
caller |
subsumes
old apply(): unpack tuple/dict of args |
|
def func(a, *b, c) def func(a, *, c) |
function |
3.X keyword-only (c must be passed by
name only) |
Examples
Positionals and keywords
>>> def f(a, b, c): print a, b, c
>>> f(1, 2, 3)
1 2 3
>>> f(c=3, b=2, a=1)
1 2 3
>>> f(1, c=3, b=2)
1 2 3
Defaults
>>> def f(a, b=2, c=3): print a, b, c
>>> f(1)
1 2 3
>>> f(1, 4, 5)
1 4 5
>>> f(1, c=6)
1 2 6
Arbitrary positionals
>>> def f(*args): print args
>>> f(1)
(1,)
>>> f(1,2,3,4)
(1, 2, 3, 4)
Arbitrary keywords
>>> def f(**args): print args
>>> f()
{}
>>> f(a=1, b=2)
{'a': 1, 'b': 2}
>>> def f(a, *pargs, **kargs): print a, pargs, kargs
>>> f(1, 2, 3, x=1, y=2)
1 (2, 3) {'y': 2, 'x': 1}
Example: min value functions
Example
● Only deals with matching: still passed by assignment
● Defaults retain an object: may change if mutable
def func(spam, eggs, toast=0, ham=0): # first 2 required
print (spam, eggs, toast, ham)
func(1, 2) # output: (1, 2, 0, 0)
func(1, ham=1, eggs=0) # output: (1, 0, 0, 1)
func(spam=1, eggs=0) # output: (1, 0, 0, 0)
func(toast=1, eggs=2, spam=3) # output: (3, 2, 1, 0)
func(1, 2, 3, 4) # output: (1, 2, 3, 4)
Ordering rules
♦ Call: keyword arguments after non-keyword arguments
♦ Header: normals, then defaults, then *name, then **name
Matching algorithm (see exercise)
1. Assign non-keyword arguments by position
2. Assign keyword arguments by matching names
3. Assign extra non-keyword arguments to *name tuple
4. Assign extra keyword arguments to **name dictionary
5. Unassigned arguments in header assigned default values
Odds and
ends
♦ lambda expression creates anonymous functions
♦ list comprehensions, map, filter apply expressions to sequences (see also prior unit)
♦ Generator expressions (2.4+)
♦ Generator functions and yield (new in 2.2, 2.3)
♦ apply function calls functions with arguments tuple
♦ Functions return ‘None’ if they don’t use a real ‘return’
♦ Python 3.X function annotations and keyword arguments
■ Lambda expressions
>>> def func(x, y, z): return x + y + z
...
>>> func(2, 3, 4)
9
>>> f = lambda x, y, z: x + y + z
>>> f(2, 3, 4)
9
hint: embedding logic
in a lambda body
(A and B) or C
((A and [B]) or [C])[0] # or newer: B if A else C
hint: embedding loops
in a lambda body...next topic
■ List comprehensions (added in 2.0)
(See also prior unit)
>>> ord('s')
115
>>> res = []
>>> for x in 'spam': res.append(ord(x))
...
>>> res
[115, 112, 97, 109]
>>> map(ord, 'spam') # apply func to sequence
[115, 112, 97, 109]
>>> [ord(x) for x in 'spam'] # apply expr to sequence
[115, 112, 97, 109]
# adding arbitrary expressions
>>> [x ** 2 for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> map((lambda x: x**2), range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> lines = [line[:-1] for line in open('README.txt')]
>>> lines[:2]
['This is Python version 2.4 alpha 3', '==================================']
# adding if tests
>>> [x for x in range(10) if x % 2 == 0]
[0, 2, 4, 6, 8]
>>> filter((lambda x: x % 2 == 0), range(10))
[0, 2, 4, 6, 8]
# advanced usage
>>> [x**2 for x in range(10) if x % 2 == 0]
[0, 4, 16, 36, 64]
>>> [x+y for x in 'abc' for y in 'lmn']
['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']
>>> res = []
>>> for x in 'abc':
... for y in 'lmn':
... res.append(x+y)
...
>>> res
['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']
# nice for matrixes
>>> M = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]
>>> M[1]
[4, 5, 6]
>>> col2 = [row[1] for row in M]
>>> col2
[2, 5, 8]
>>> quad = [M[i][j] for i in (0,1) for j in (0, 1)]
>>> quad
[1, 2, 4, 5]
List comprehensions can become
incomprehensible when nested, but map and list comprehensions may be faster
than simple for loops
(In 2.4, comprehensions are twice as
fast as for loops, and for loops are now quicker than map: see middle of this page, and
CD’s Extras\Code\Misc\timerseqs.py)
■ Generator expressions (2.4+)
# list comprehensions generate entire list in memory
>>> squares = [x**2 for x in range(5)]
>>> squares
[0, 1, 4, 9, 16]
# generator expressions yield 1 result at a time: saves memory, distributes work
>>> squares = (x**2 for x in range(5))
>>> squares
<generator object at 0x00B2EC88>
>>> next(squares)
0
>>> next(squares)
1
>>> next(squares)
4
>>> list(squares)
[9, 16]
# iteration contexts automatically call next()
>>> for x in (x**2 for x in range(5)):
print x,
0 1 4 9 16
>>> sum(x**2 for x in range(5))
30
■ Generator functions and yield
Generators
implement iteration protocol: .next() (3.X .__next__())
Retains
local scope when suspended
Distributes
work over time, may save memory (see also: threads)
Related:
generator expressions, enumerate function, file iterators
# functions compiled specially when contain yield
>>> def gensquares(N):
... for i in range(N): # suspends and resumes itself
... yield i ** 2 # <- return value and resume here later
# generator objects support iteration protocol: next()
>>> x = gensquares(10)
>>> x # also retain all local variables between calls
<generator object at 0x0086C378> # classes define __iter__ to return iter object
>>> next(x)
0
>>> next(x)
1
>>> next(x)
4
…
>>> next(x)
…StopIteration exception raised at end…
# for loops (and others) automatically call next()
>>> for i in gensquares(5): # resume the function each time
... print i, ':', # print last yielded value
...
0 : 1 : 4 : 9 : 16 :
# (advanced) application to coroutines via task switchers
►See Extras\Code\Misc\asynch1.py
# (advanced) generators, through the years…
►See Extras\Code\Misc\gendemo.py
Original
def with yield
statement supports iteration protocol, returning yielded value for next()
In 2.5+
caller can resume
generator and pass it value via G.send(X): result of “yield”
expression
In
3.3+
“return” value
allowed in generators: attached to exception, result of “yield from” expression
In
3.5+
“async”
and “await” distinguish some coroutine roles, similar
to “yield from”
■ Apply built-in (2.X only) and syntax (all Pys)
>>> def func(a, b, c):
return a + b + c
>>> apply(func, (2, 3, 4)) # 2.X only
9
>>> apply(func, (2, 3), {'c': 4})
9
Versus
Python 2.0+ apply-like call syntax
func(*a) …like… apply(func, a)
func(*a, **b) …like… apply(func, a, b)
>>> func(*(2, 3, 4))
9
>>> func(*(2, 3), **{'c': 4}) # 2.X and 3.X
9
# call syntax is more flexible:
>>> def func(a,
b, c, d): return a + b + c + d
>>>
args1 = (1, 2)
>>>
args2 = {'c': 3, 'd': 4}
>>> func(*args1,
**args2)
10
>>> func(1,
*(2,), **args2)
10
See
also: “About the stars” sidebar above
■ Default return values
>>> def proc(x):
... print x
...
>>> x = proc('testing 123...')
testing 123...
>>> print x
None
■ Python 3.X function annotations
>>> def func(a: int, b: 'spam', c: 88 = 99) -> float:
... print(a, b, c)
...
>>> func(1, 2)
1 2 99
>>> func.__annotations__
{'b': 'spam', 'return': <class 'float'>, 'a': <class 'int'>, 'c': 88}
■ Python 3.X keyword-only arguments
>>> def f(a, b, *, c=3, d): print(a, b, c, d)
...
>>> f(1, 2)
TypeError: f() missing 1 required keyword-only argument: 'd'
>>> f(1, 2, 3)
TypeError: f() takes 2 positional arguments but 3 were given
>>> f(1, 2, d=4)
1 2 3 4
>>> f(1, 2, c=3, d=4)
1 2 3 4
Function
design concepts
♦ Use global variables only when absolutely necessary
♦ Use arguments for input, ‘return’ for outputs
♦ Don’t change mutable arguments unless expected
♦ But globals are only state-retention tool without classes
♦ But classes depend on mutable arguments (‘self’)
Functions
are objects: indirect calls
♦ Function objects can be assigned, passed, etc.
♦ Can call objects generically: function, bound method, ...
def echo(message): print message
x = echo
x('Hello world!')
def indirect(func, arg):
func(arg)
indirect(echo, 'Hello world!')
schedule = [ (echo, 'Hello!'), (echo, 'Ni!') ]
for (func, arg) in schedule:
func(arg)
File scanners
file: scanfile.py
def scanner(name, function):
file = open(name, 'r') # create file
for line in file.readlines():
function(line) # call function
file.close()
file: commands.py
import string
from scanfile import scanner
def processLine(line):
print string.upper(line)
scanner("data.txt", processLine) # start scanner
Function
gotchas
Local names are detected statically
# cant use both local+global version of name, unless reference through
module
>>> X = 99
>>> def selector(): # X used but not assigned
... print X # X found in global scope
...
>>> selector()
99
>>> def selector():
... print X # does not yet exist!
... X = 88 # X classified as a local name
...
>>> selector()
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in selector
NameError: X
>>> def selector():
... global X # force X to be global
... print X
... X = 88
...
>>> selector()
99
Mutable defaults created just once
# to avoid: check for None inside def
and set to [] if so
>>> def grow(A, B=[]):
B.append(A)
return B
>>> grow(1)
[1]
>>> grow(1)
[1, 1]
>>> grow(1)
[1, 1, 1]
Nested functions weren’t nested scopes
# this works as of 2.2 – enclosing function
scopes!
>>> def outer(x):
... def inner(i): # assign in outer’s local
... print i, # i is in inner’s local
... if i: inner(i-1) # not in my local or global!
... inner(x)
...
>>> outer(3)
3
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "<stdin>", line 6, in outer
File "<stdin>", line 5, in inner
NameError: inner
>>> def outer(x):
... global inner
... def inner(i): # assign in enclosing module
... print i,
... if i: inner(i-1) # found in my global scope
... inner(x)
...
>>> outer(3)
3 2 1 0
Use defaults to save references
# normally no longer necessary as of 2.2:
enclosing function scopes
# BUT still required to retain current value
of loop variables!
>>> def outer(x, y):
def inner():
return x ** y
return inner
>>> x = outer(2, 4)
>>> x()
16
# code before 2.2
>>> def outer(x, y):
... def inner(a=x, b=y): # save x,y bindings/objects
... return a**b # from the enclosing scope
... return inner
...
>>> x = outer(2, 4)
>>> x()
16
>>> def outer(x, y):
... return lambda a=x, b=y: a**b
...
>>> y = outer(2, 5)
>>> y()
32
for I in someiterable:
actions.append(lambda I=I: …) # retain current I, not last!
Optional
reading: set functions
♦ Functions process passed-in sequence objects
♦ Work on any type of sequence objects
♦ Supports mixed types: list and tuple, etc.
file: inter.py
def intersect(seq1, seq2):
res = [] # start with an empty list
for x in seq1: # scan the first sequence
if x in seq2:
res.append(x) # add common items to end
return res
def union(seq1, seq2):
res = map(None, seq1) # copy of seq1 (see ch13)
for x in seq2: # add new items in seq2
if not x in res:
res.append(x)
return res
% python
>>> from inter import intersect, union
>>> s1 = "SPAM"
>>> s2 = "SCAM"
>>> intersect(s1, s2), union(s1, s2) # strings
(['S', 'A', 'M'], ['S', 'P', 'A', 'M', 'C'])
>>> intersect([1,2,3], (1,4)) # mixed types
[1]
>>> union([1,2,3], (1,4))
[1, 2, 3, 4]
Supporting multiple operands: *varargs
file: inter2.py
def intersect(*args):
res = []
for x in args[0]: # scan first sequence
for other in args[1:]: # for all other args
if x not in other: break # this in each one?
else:
res.append(x) # add items to end
return res
def union(*args):
res = []
for seq in args: # for all args
for x in seq: # for all nodes
if not x in res:
res.append(x) # add items to result
return res
% python
>>> from inter2 import intersect, union
>>> s1, s2, s3 = "SPAM", "SCAM", "SLAM"
>>> intersect(s1, s2), union(s1, s2) # 2 operands
(['S', 'A', 'M'], ['S', 'P', 'A', 'M', 'C'])
>>> intersect([1,2,3], (1,4))
[1]
>>> intersect(s1, s2, s3) # 3 operands
['S', 'A', 'M']
>>> union(s1, s2, s3)
['S', 'P', 'A', 'M', 'C', 'L']
Lab Session 4
Click here to go to
lab exercises
Click here to go to exercise
solutions
Click here to go to
solution source files
Click here to go to
lecture example files