![[LP5E]](lp5e-large.jpg)
| This material became part of a new decorators chapter in the 4th Edition of Learning Python, and was revised and expanded in the 5th Edition. Watch for related-reading links at the end of this page. This page has a toggled Contents display and floating Top button if JavaScript is enabled in your browser. |
|---|
Spring 2009 (updated June-July 2018)
As a final example of the utility of decorators, this section develops a function decorator that automatically tests whether arguments passed to a function or method are within a valid numeric range. It's designed to be used either during development or production, and can be used as a template for similar tasks (e.g., argument type testing, if you must). As we'll see, supporting arbitrary argument passing modes leads us down a variety of Python introspection and tool-building paths, and underscores subtleties in Python's argument matching modes.
Let's start with a basic range test implementation. In the chapter "A More Realistic Example", we wrote a class that gave a raise to objects based upon a passed-in percentage [on this website, this example is available on this page]:
class Person:
...
def giveRaise(self, percent):
self.pay = int(self.pay * (1 + percent))
There, we noted that we should probably check the percentage to make sure
it's not too large or small, if we want the code to be robust. Although
we could do this with an if statement in the method, it might be more
useful and interesting to develop a more general tool that can perform
range tests for us automatically, for the arguments of any function or
method we might code now or in the future.
The following function decorator implements this idea. To keep this simple at
first, it works only for positional arguments, and assumes they always appear
at the same position in every call; they cannot be passed by keyword name,
and we don't support additional **args keywords in calls because this can
invalidate the positions declared in the decorator:
def rangetest(*argchecks): # validate positional arg ranges
def onDecorator(func):
if not __debug__: # True if "python -O main.py args.."
return func # no-op: call original directly
else: # else wrapper while debugging
def onCall(*args):
for (ix, low, high) in argchecks:
if args[ix] < low or args[ix] > high:
errmsg = 'Argument %s not in %s..%s' % (ix, low, high)
raise TypeError(errmsg)
return func(*args)
return onCall
return onDecorator
As is, this code is mostly a rehash of the coding patterns we explored for privacy decorators in the prior section [in the book], so I won't describe it in detail here. A few quick highlights:
rangetest function returns the actual decorator
to be applied to the subject function for the @ syntax.
__debug__ here, to see if we're running in
development mode. Python sets this variable to True only if has not been run
with a -O command line argument. If False, we return the onCall wrapper to
validate arguments. If True, we simply return the original function, and
skip the validation wrapper altogether, thereby avoiding the performance cost of
the onCall wrapper. The __debug__ name is True if you launch the script
with a system command-line like python -O main.py args...; doing so effectively
turns off range tests, and optimizes your function call's speed.
self
(first) argument is passed automatically, so argument specification numbers for methods must
begin with 1, not 0. When used on a class method, the method name is rebound to the
onCall function in the decorator, which remembers both the original method function
and the argchecks tuple in enclosing scopes. When the original class method name is
later called, onCall receives the instance in argument 1 as usual in *args,
and passes this along to self in the original method function.
To test this decorator, the following code applies it to functions, as well
as a class method—the one from the prior chapter's example. Notice that we can
pass argument specifications in any sequence object (e.g., tuple or list); onCall's
*args bundles them in a tuple, and the inner for loop's header line extracts their
three parts with sequence assignment. Trace through the decorated Person.giveRaise
method call in this to see how the self argument is propagated to the original
method naturally:
#if __name__ == '__main__': # to make this a library
print(__debug__) # True if no -O python cmdline arg
# False if "python -O main.py args"
@rangetest((1, 0, 120)) # person = rangetest(..)(person)
def person(name, age):
print('%s is %s years old' % (name, age))
@rangetest([0, 1, 12], [1, 1, 31], [2, 0, 2009])
def birthday(M, D, Y):
print('birthday = {0}/{1}/{2}'.format(M, D, Y))
class Person:
"""
revisit the Person class to validate argument
"""
def __init__(self, name, job, pay):
self.job = job
self.pay = pay
# arg 0 is the self instance here
@rangetest([1, 0.0, 1.0]) # giveRaise = rangetest(..)(giveRaise)
def giveRaise(self, percent):
self.pay = int(self.pay * (1 + percent))
# comment lines raise TypeError unless "python -O" on shell command line
person('Bob Smith', 45) # really runs onCall(..) with state
#person('Bob Smith', 200) # or person() if -O cmd line argument
birthday(5, 31, 1963)
#birthday(5, 32, 1963)
sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10) # really runs onCall(self, .10)
print(sue.pay) # or giveRaise(self, .10) if -O
#sue.giveRaise(1.10)
#print(sue.pay)
When run, valid calls in this code produce the following output (all the code
in this section works the same under Python 2.X and 3.X, because function
decorators are supported in both, and we use 3.X-style print call syntax):
C:\misc> C:\Python30\python rangetest.py True Bob Smith is 45 years old birthday = 5/31/1963 110000
Uncommenting any of the invalid calls cause a TypeError to be raised by
the decorator; here's the result when the last two lines are allowed to
run (I've omitted some of the error message text here to save space):
C:\misc> C:\Python30\python rangetest.py True Bob Smith is 45 years old birthday = 5/31/1963 110000 ... TypeError: Argument 1 not in 0.0..1.0
Running Python with its -O flag at a system command-line will disable
range-testing, but also avoid the performance overhead of the wrapping
layer—we wind up calling the original undecorated function directly.
Assuming this is a debugging tool only, you can use this flag to optimize
your program for production use:
C:\misc> C:\Python30\python -O rangetest.py False Bob Smith is 45 years old birthday = 5/31/1963 110000 231000
The prior version works and illustrates the basics we need to
employ, but it's fairly limited—it supports validating
arguments passed by position only, and assumes that no keyword arguments
are passed in such a way that makes validated argument position
numbers incorrect (they won't be in *args if they were passed by name,
and will generate an index error in the wrapping layer's code).
That's fine if all your arguments are passed by position, but a
less-than-ideal constraint in a general tool. Python supports more
flexible argument-passing modes, which we're not yet addressing.
Our next version does better. It supports range validations for
arguments passed by either position or keyword name. Arguments
specified by position are assumed to be passed by position, and
arguments specified by keyword are assumed always passed by name.
When the original function name is called, we simply step through
both the *args tuple and **kargs dictionaries, running
the specified positional and keyword argument validations, respectively.
def rangetest(*argchecks, **kargchecks): # validate keyword arg ranges too
def onDecorator(func):
if not __debug__: # True if "python -O main.py args.."
return func # no-op: call original directly
else: # else wrapper while debugging
def onCall(*args, **kargs):
for (ix, low, high) in argchecks:
if args[ix] < low or args[ix] > high:
errmor = 'Argument #%s not in %s..%s' % (ix, low, high)
raise TypeError(error)
for (key, (low, high)) in kargchecks.items():
if kargs[key] < low or kargs[key] > high:
error = 'Argument "%s" not in %s..%s' % (key, low, high)
raise TypeError(error)
return func(*args, **kargs)
return onCall
return onDecorator
Here this version's self-test code; at the @ decorator, arguments expected to be
passed by position are specified by number as before, and expected keyword arguments
are given by name now. Run this on your own to experiment; the commented-out lines
here are invalid, and raise exceptions in the decorator as expected:
@rangetest((1, 0, 120)) # person = rangetest(..)(person)
def person(name, age):
print('%s is %s years old' % (name, age))
@rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009))
def birthday(M, D, Y):
print('birthday = {0}/{1}/{2}'.format(M, D, Y))
@rangetest((0, 0, 999999999), height=(0.0, 7.0)) # ssn always by position
def record(ssn, category, height): # height always by keyword
print('record: %s, %s' % (ssn, category)) # category not checked
class Person:
"""
revisit the Person class to validate argument
"""
def __init__(self, name, job, pay):
self.job = job
self.pay = pay
# giveRaise = rangetest(..)(giveRaise)
@rangetest((1, 0.0, 1.0)) # assume percent passed by position
def giveRaise(self, percent):
self.pay = int(self.pay * (1 + percent))
@rangetest(percent=(0.0, 1.0)) # same, but assume passed by keyword
def giveRaise2(self, percent, bonus=.10):
self.pay = int(self.pay * (1 + percent + bonus))
# comment lines raise TypError unless "python -O" on shell command line
person('Bob Smith', 45) # really runs onCall(..) with state
#person('Bob Smith', 200) # or person() if -O cmd line argument
birthday(M=5, D=31, Y=1963)
birthday(Y=1963, M=5, D=31)
#birthday(M=5, D=32, Y=1963)
record(123456789, 'spam', height=5.5)
record(123456789, height=5.5, category='spam')
#record(1234567899, height=5.5, category='spam')
#record(123456789, height=99, category='spam')
sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10) # really runs onCall(self, .10)
print(sue.pay) # or giveRaise(self, .10) if -O
sue.giveRaise2(percent=.10)
print(sue.pay)
#sue.giveRaise2(percent=1.10)
#print(sue.pay)
Here's an example run, with the last two lines uncommented to trigger an error:
C:\misc> C:\Python30\python rangetest.py True Bob Smith is 45 years old birthday = 5/31/1963 birthday = 5/31/1963 record: 123456789, spam record: 123456789, spam 110000 132000 ... TypeError: Argument "percent" not in 0.0..1.0
Although both positional and keyword arguments can be validated now, this version does not support an argument being passed in either positional or keyword modes at different calls. Once decorated, the argument to test must always appear at the same position in every call, or always be passed by keyword in every call. This limits the flexibility of argument passing in calls, and other arguments cannot be passed in a way that changes expected positions. This may be a reasonable constraint during development, since expected call patterns can often be predicted; it's still not ideal in a general tool, though.
The root of inflexibility in this version is a bit subtle, and stems from Python's argument matching algorithm; but argument position numbers may not be reliable if arguments may be passed arbitrarily, by name or position. Given just an argument's name, we don't have enough information to know which position it may show up in if it is passed by position instead. Similarly, a given argument's expected position may not be valid if other arguments are passed in arbitrary ways. To address this uncertainty, this version requires that checked arguments must always be passed by either name or a fixed position.
To be fully general, we might be able to do better by requiring information at decoration time for each argument, validated or not; this seems a less than user-friendly policy. Would could also try to mimic Python's argument matching logic in its entirety to see which names have been passed in which modes, but that seems like far too much complexity for our tool—and is prone to change arbitrarily over time.
It might be better if we could somehow match arguments passed by name against the set of all expected arguments' names, in order to determine in which position arguments actually appear during a given call. To this end, we may be able to leverage the introspection API available on function objects and their associated code objects (which gives access to expected arguments' names). The next section shows one way to code this plan.
As described in the prior section, in order to allow validated arguments to be
passed by either position or name, we need to be able to match against the set
of all expected argument names, in order to determine which argument was passed
to which position. It turns out that Python provides exactly what we need, as
part of the introspection tools available on function objects, and their
associated code objects. The set of expected argument names is the first N
variables names attached to a function's code object:
# In Python 2.6 and earlier:
>>> def func(a, b, c=3, d=4):
x = 1
y = 2
>>> dir(func)
['__call__', '__class__', '__closure__', '__code__', '__defaults__',
...more omitted...
'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals',
'func_name']
>>> dir(func.func_code)
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__',
...more omitted...
'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename',
'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name',
'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
>>> func.func_code.co_varnames
('a', 'b', 'c', 'd', 'x', 'y')
>>> func.func_code.co_nlocals
6
>>> func.func_code.co_argcount
4
>>> code = func.func_code # same as func.__code__ in 2.6 and 3.0
>>> code.co_varnames[:code.co_argcount] # argument names at the front
('a', 'b', 'c', 'd')
Although the API has changed slightly in 3.0, it's still straightforward to
access—we use func.__code__ instead of func.func_code, and can use the
Python version number in the sys module to choose one or the other (the new
__code__ attribute is also available in 2.6 redundantly for portability,
but func_code must be used in prior 2.X Pythons):
# In Python 3.0 (and 2.6 for compatibility):
>>> sys.version_info # [0] is major release number
(3, 0, 0, 'final', 0)
>>> def func(a, b, c, d):
... x = 1
... y = 2
...
>>> code = func.__code__ # different attribute name
>>> code.co_varnames[:code.co_argcount]
('a', 'b', 'c', 'd')
Now, we can put these to use to validate ranges for arguments passed in either mode, by matching against all expected arguments. The algorithm is as follows:
**kargs dictionary.
**kargs are assumed to have been passed
by position, and must be mapped to their actual position in the call,
as follows:
**kargs, from the function's list of expected arguments.
*pargs tuple.
**kargs by name,
and then in *pargs by their relative position in the remaining expected
arguments list.
Under this scheme, both passing modes are supported for arguments to be checked:
arguments expected by position but passed by name will show up in **kargs and be
processed by name; conversely, arguments expected by name but passed by position
will be mapped to their positions automatically in the remaining expected arguments
list. Here's how this translates to code:
def rangetest(**argchecks): # validate ranges for both arg types
def onDecorator(func): # onCall remembers func and argchecks
if not __debug__: # True if "python -O main.py args.."
return func # wrap if debugging else use original
else:
import sys
code = func.__code__ if sys.version_info[0] == 3 else func.func_code
allargs = code.co_varnames[:code.co_argcount]
def onCall(*pargs, **kargs):
positionals = list(allargs)
for argname in kargs:
# for all passed by name, remove from expected
positionals.remove(argname)
for (argname, (low, high)) in argchecks.items():
# for all args to be checked
if argname in kargs:
# was passed by name
if kargs[argname] < low or kargs[argname] > high:
errmsg = 'Argument "{0}" not in {1}..{2}'
errmsg = errmsg.format(argname, low, high)
raise TypeError(errmsg)
else:
# was passed by position
position = positionals.index(argname)
if pargs[position] < low or pargs[position] > high:
errmsg = 'Argument "{0}" not in {1}..{2}'
errmsg = errmsg.format(argname, low, high)
raise TypeError(errmsg)
return func(*pargs, **kargs) # okay: run original call
return onCall
return onDecorator
To test, the following makes sure that the prior calls still work in this scheme, but add an additional function to stress the code further. Notice how arguments to test are given only by name now, since the decorator automatically maps them to their expected positions if they are not passed by keyword in the actual call:
@rangetest(age=(0, 120)) # person = rangetest(..)(person)
def person(name, age):
print('%s is %s years old' % (name, age))
@rangetest(M=(1, 12), D=(1, 31), Y=(0, 2009))
def birthday(M, D, Y):
print('birthday = {0}/{1}/{2}'.format(M, D, Y))
@rangetest(ssn=(0, 999999999), height=(0.0, 7.0))
def record(ssn, category, height): # args by name or position
print('record: %s, %s' % (ssn, category)) # category not checked
class Person:
"""
revisit the Person class to validate argument
"""
def __init__(self, name, job, pay):
self.job = job
self.pay = pay
# giveRaise = rangetest(..)(giveRaise)
@rangetest(percent=(0.0, 1.0)) # percent passed by name or position
def giveRaise(self, percent):
self.pay = int(self.pay * (1 + percent))
# comment lines raise TypError unless "python -O" on shell command line
person('Bob Smith', 45) # really runs onCall(..) with state
person(age=45, name='Bob Smith')
#person('Bob Smith', 200) # or person() if -O cmd line argument
#person('Bob Smith', age=200)
#person(age=200, name='Bob Smith')
birthday(M=5, D=31, Y=1963)
birthday(Y=1963, M=5, D=31)
birthday(5, 31, Y=1963)
birthday(5, D=31, Y=1963)
birthday(5, Y=1963, D=31)
#birthday(5, 32, 1963)
#birthday(5, 32, Y=1963)
#birthday(5, D=32, Y=1963)
#birthday(5, Y=1963, D=32)
#birthday(M=5, D=32, Y=1963)
record(123456789, 'spam', height=5.5)
record(123456789, height=5.5, category='spam')
#record(1234567899, height=5.5, category='spam')
#record(123456789, height=99, category='spam')
sue = Person('Sue Jones', 'dev', 100000)
sue.giveRaise(.10) # really runs onCall(self, .10)
print(sue.pay) # or giveRaise(self, .10) if -O
sue.giveRaise(percent=.10)
print(sue.pay)
#sue.giveRaise(1.10)
#sue.giveRaise(percent=1.10)
#print(sue.pay)
One final twist: the preceding version works as intended, allowing checked arguments to be passed either by name or position. Unfortunately, it still fails to address default arguments that are omitted in a call. As a last step, let's generalize the decorator to skip range tests for arguments that are defaulted and omitted.
The reason the prior version does not support defaults lies in its assumption
that checked arguments not in the **kargs dictionary are assumed to have been
passed by position. That's not true if an argument was defaulted—it won't
be in **kargs, and may not be in the *pargs positionals list either, even if
it's part of the remaining expected arguments list. Because the decorator does
not skip testing for arguments that are allowed to default in the call, it can
still fail by indexing off the end of the actual arguments list.
With the prior version, for instance, the valid call in the second last line of the
following triggers an index error in the decorator's code, because argument d is not
in the keyword **kargs, but is also not present in the positional *pargs since it was defaulted;
indexing the actual arguments list on the relative position of d is out of bounds. The
last line similarly fails, because c is in the expected list, but not in the actual *pargs:
@rangetest(a=(1,10), b=(1,10), c=(1, 10), d=(1, 10))
def omitargs(a, b=7, c=8, d=9):
print(a, b, c, d)
omitargs(1, 2, 3, 4) # works
omitargs(1, 2, 3) # FAILS! - index out of range, pargs[position]
omitargs(1, 2, d=4) # FAILS! - same reason: d in **, but c fails
On first glance, accounting for this is a potentially difficult problem. When the wrapper is invoked for an actual call, all we have access to are the sets of positional and keyword arguments passed; argument names in the decorated function have not been assigned yet, and indeed, we can't even be sure if the call is legal. Although we could try to emulate Python's argument matching mechanism at this point, we can get the same mileage by simply modifying the algorithm used by the prior version.
The simpler solution relies on two constraints on argument passing order imposed by Python (these still hold true both in 2.6 and 3.0):
def, all non-default arguments appear before all default arguments
That is, a non-keyword argument cannot follow a keyword argument at a call,
and a non-default argument cannot follow a default argument at a definition.
All name=value must appear after any simple name in both places.
To simplify our task, we can also make the assumption that a call is valid in general, such that all arguments will receive values (by name or position), or were omitted to pick up defaults. This isn't quite true, because the function hasn't yet been actually called when the wrapper logic runs to test arguments—it may still fail later when invoked by the wrapper layer, due to incorrect argument passing. As long as that doesn't cause the wrapper to fail any more badly, though, we can finesse the validity of the call (and if the call will fail anyhow, we don't care much what it does to our decorator). This helps, because validating calls before the call is actually made would require us to emulate the argument-matching algorithm of Python exactly—a much too complex procedure for the purposes of our development tool.
Now, given these assumptions, we can allow for omitted default arguments in the call with a slightly different algorithm:
N passed positional
arguments match the first N expected arguments. This is true per Python's call
ordering rules given above, since all positionals precede all keywords.
*pargs positionals tuple.
N actual positionals, then, were either passed
by keyword, or were defaulted by omission at the call.
**kargs it was passed by
name; if it is instead in the first N expected arguments, it was passed by
position; otherwise, we can assume it was omitted in the call and defaulted,
and need not be checked.
In other words, we can skip tests for arguments that were omitted in a call,
by assuming that the first N actually passed positional arguments in *pargs
must match the first N argument names in the list of all expected arguments.
Any others must have been passed by keyword and thus be in **kargs, or have
been defaulted. Under this scheme, if an argument to be checked was omitted
between the rightmost positional argument and the leftmost keyword argument,
between keyword arguments, or after the rightmost positional in general,
it will simply be skipped by the decorator, and not trigger an index error.
The following final form captures this approach:
trace = True
def rangetest(**argchecks): # validate ranges for both+defaults
def onDecorator(func): # onCall remembers func and argchecks
if not __debug__: # True if "python -O main.py args.."
return func # wrap if debugging else use original
else:
import sys
code = func.__code__ if sys.version_info[0] == 3 else func.func_code
allargs = code.co_varnames[:code.co_argcount]
funcname = func.__name__
def onCall(*pargs, **kargs):
# all pargs match first N args by position
# the rest must be in kargs or omitted defaults
positionals = list(allargs)
positionals = positionals[:len(pargs)]
for (argname, (low, high)) in argchecks.items():
# for all args to be checked
if argname in kargs:
# was passed by name
if kargs[argname] < low or kargs[argname] > high:
errmsg = '{0} argument "{1}" not in {2}..{3}'
errmsg = errmsg.format(funcname, argname, low, high)
raise TypeError(errmsg)
elif argname in positionals:
# was passed by position
position = positionals.index(argname)
if pargs[position] < low or pargs[position] > high:
errmsg = '{0} argument "{1}" not in {2}..{3}'
errmsg = errmsg.format(funcname, argname, low, high)
raise TypeError(errmsg)
else:
# assume not passed: default
if trace:
print('Argument "{0}" defaulted'.format(argname))
return func(*pargs, **kargs) # okay: run original call
return onCall
return onDecorator
To test, we'll repeat prior cases again to verify no new breakage, and add a new function with defaults to verify our final mutation (I've omitted repeated test cases' code here for space; see the preceding version's self-test code, or the source file link at the end of this section):
...repeated tests omitted are coded and run the same...
@rangetest(a=(1, 10), b=(1, 10), c=(1, 10), d=(1, 10))
def omitargs(a, b=7, c=8, d=9):
print(a, b, c, d) # skip omitted defaults
# comment lines raise TypeError unless "python -O" on shell command line
...repeated tests omitted are coded and run the same...
omitargs(1, 2, 3, 4)
omitargs(1, 2, 3)
omitargs(1, 2)
omitargs(1)
omitargs(1, 2, 3, d=4)
omitargs(1, 2, d=4)
omitargs(1, d=4)
omitargs(a=1, d=4)
omitargs(d=4, a=1)
omitargs(1, b=2)
omitargs(1, b=2, d=4)
omitargs(a=1)
omitargs(d=8, c=7, b=6, a=1)
omitargs(d=8, c=7, a=1)
#omitargs(1, 2, 3, 11)
#omitargs(1, 2, 11)
#omitargs(1, 11, 3)
#omitargs(11, 2, 3)
#omitargs(1, 11)
#omitargs(11)
#omitargs(1, 2, 3, d=11)
#omitargs(1, 2, 11, d=4)
#omitargs(1, 2, d=11)
#omitargs(1, 11, d=4)
#omitargs(1, d=11)
#omitargs(11, d=4)
#omitargs(d=4, a=11)
#omitargs(1, b=11, d=4)
#omitargs(1, b=2, d=11)
#omitargs(1, b=2, c=11)
#omitargs(1, b=11)
#omitargs(d=8, c=11, b=6, a=1)
#omitargs(d=8, c=7, a=11)
Two final caveats: First, calls to the original function that are not valid still fail in our final decorator. The following both trigger exceptions, for example:
omitargs() omitargs(d=8, c=7, b=6)
These only fail, though, where we try to invoke the original, with func(*pargs, **kargs).
While we could try to imitate Python's argument matching to avoid this, there's
not much reason to do so—since the call would fail at this point anyhow, we
might as well let Python's own argument matching logic detect the problem for us.
Lastly, although our final version handles positional arguments, keyword arguments,
and omitted defaults, it still doesn't do anything explicit about *args and **args that
may be used in a decorated function that accepts arbitrarily many arguments. We probably
don't need to care for our purposes, though:
**kargs and can be tested
normally by name if mentioned to the decorator.
**kargs or the sliced
expected positionals list, and will not be checked—it is treated as though it were defaulted,
even though it is really an optional extra argument.
In other words, as is the code supports testing arbitrary keyword arguments by name, but not arbitrary positionals that are unnamed and hence have no set position in the function's argument signature.
For instance, consider the following simplified version of our decorator's code:
def func(a, b, *pargs, **kargs):
print('func:', pargs, kargs) # jun-2018: 3.X compatible prints
code = func.__code__
print('expected:', code.co_varnames[:code.co_argcount])
def wrap(func):
def onCall(*pargs, **kargs):
print('onCall:', pargs, kargs)
func(*pargs, **kargs)
return onCall
func = wrap(func)
func(1, 2, 3, 4, x=4, y=5)
When run, only a and b show up as expected argument names on the
function object; kargs and pargs do not. Within the wrapped function
itself, arguments that don't match a and b wind up in *pargs or **kargs.
In the wrapping layer, however, all actual arguments show up in *pargs or **kargs:
expected: ('a', 'b')
onCall: (1, 2, 3, 4) {'y': 5, 'x': 4}
func: (3, 4) {'y': 5, 'x': 4}
Slicing the expected arguments list on the length of *pargs will leave it unchanged
in this case, since there are more actual than expected. Extra keyword arguments
can be range-tested as is, because their names will either appear in **kargs, or
be unnamed in the sliced expected arguments list and thus skipped. Extra
positionals will simply be skipped, because their names do not appear on
the sliced expected arguments list. If we omit a default argument, we get
the same net result—the expected arguments list is reduced to just a,
so checks on name b will be skipped, though any keyword arguments can still
be tested out of **kargs:
def func(a, b=8, *pargs, **kargs):
...
func(1, x=4, y=5)
...
expected: ('a', 'b')
onCall: (1,) {'y': 5, 'x': 4}
func: () {'y': 5, 'x': 4}
In principle, we could extend the decorator's interface to support *pargs in the
decorated function too, for the very rare cases where this might be useful (e.g.,
a special argument name with a test to apply to all arguments in the wrapper's
*pargs beyond the length of the expected arguments list). Since we've already
exhausted the space allocation for this example, though, if you care about such
improvements, you've officially crossed over into the realm of suggested exercise.
Now that we've worked so hard to carefully craft
a general and flexible decorator solution, I should mention that we
can often achieve the same effect by adding manual tests or using
assert statements inside the function itself. For example, the
following range testing using our final decorator code:
class Person:
@rangetest(percent=(0.0, 1.0)) # use decorator to validate
def giveRaise(self, percent):
self.pay = int(self.pay * (1 + percent))
could be implemented by simply testing ranges manually inside the function:
class Person:
def giveRaise(self, percent): # validate with in-line code
if percent < 0.0 or percent > 1.0:
raise TypeError, 'percent invalid'
self.pay = int(self.pay * (1 + percent))
or by using the Python assert statement, which is designed for
this sort of development-time testing:
class Person: # validate with asserts
def giveRaise(self, percent):
assert percent >= 0.0 and percent <= 1.0, 'percent invalid'
self.pay = int(self.pay * (1 + percent))
As we'll see in the next part of the book, the assert statement is like
a conditional raise: it raises an exception if a test is not true.
Like our decorator, assert statements also check the __debug__ flag and
do nothing if the Python -O flag is used to run the script (e.g., python -O m.py);
in fact assert goes a minor step further, and generates no code at all
when -O is used.
So why bother with the decorator? In the simple case of testing one
argument's range, assert may be just as good—it's one line of code
versus one line of code. In less straightforward cases, though, asking
programmers to in-line more complex logic in their functions is much
less friendly than coding it in a reusable decorator (imagine in-lining the
equivalent of the [book's] last section's private and public declarations, for example).
For more complex requirements than those we've seen here, the decorator coding
pattern may provide better support than in-line tests or assertions.
Moreover, if decoration logic must ever change, there is only one copy to
update in a decorator; without one, there may be arbitrarily many in-lined
copies to find and change. Although the @ syntax is not strictly required
to leverage this benefit (we could rebind the function name with a manual
assignment, and still avoid code redundancy with encapsulation), the special
@ line can make our intent more obvious.
In addition, once you see the coding pattern we've arrived at for proceeding arguments in decorators, it can be applied in other contexts. Checking argument data types at development time, for example, is a straightforward extension:
def typetest(**argchecks):
def onDecorator(func):
....
def onCall(*pargs, **kargs):
positionals = list(allargs)[:len(pargs)]
for (argname, type) in argchecks.items():
if argname in kargs:
if not isinstance(kargs[argname], type):
...
raise TypeError(errmsg)
elif argname in positionals:
position = positionals.index(argname)
if not isinstance(pargs[position], type):
...
raise TypeError(errmsg)
else:
# assume not passed: default
return func(*pargs, **kargs)
return onCall
return onDecorator
@typetest(a=int, c=float)
def func(a, b, c, d): # like func = typetest(...)(func)
...
func(1, 2, 3.0, 4) # okay
func('spam', 2, 99, 4) # triggers exception
As you should have learned in the book, though, this particular role is a generally bad idea in working code, and not at all "Pythonic"—type testing restricts your function to work on specific types only, instead of allowing it to operate on any types with compatible interfaces. In effect, it limits your code, and breaks its flexibility. On the other hand, every rule has exceptions; type checking may come in handy in isolated cases while debugging, and when interfacing with code written in more restrictive languages such as C++. This general pattern of argument processing might also be applicable in a variety of less controversial roles.
In the end, an author's role is to raise questions about the merits of alternative techniques, unless a best practice has already emerged. As usual, your role should always be to judge such tradeoffs for yourself.
For additional reading, try these other articles popular at learning-python.com:
These and more are available on the blog page.
![[Home page]](PythonPowered.gif "Home page"){kind=small}