Function Arguments/Return Type-Testing Decorator

[Jan-2015] As part of a recent project, I extended an arguments type-checking function+method decorator that appears in Chapter 39 of Learning Python, 5th Edition, to also perform return type testing, and use 3.X function attributes instead of decorator arguments. That makes it 3.X only, but it's simple to translate this code to use 2.X-compatible decorator arguments instead, as shown in the book.

Fetch the decorator module here:

⇨ debugtypes.py 

Usage

See the decorator module's self-test code for usage examples, and the book for the logic underlying the decorator itself (its origins go back to this post from 2008/2009).

As another use case—and an example of bytes and bit-level processing—I used this decorator to verify that type annotations coded in simple file obfuscation functions were being respected. Here's a partial excerpt from this (not yet published) client:

"""
[Scheme 2 of 4]
>>> data = do_encode2(b'spam', ord('x'))
120 b'spam'
bytearray(b'\x00\xeb\x00\xe8\x00\xd9\x00\xe5')
 
>>> data = do_decode2(bytes(data), ord('x'))         # need bytes() for decorator only!
120 bytearray(b'\x00\xeb\x00\xe8\x00\xd9\x00\xe5')
bytearray(b'spam')
"""

from debugtypes import debugtypes

@debugtypes
def do_encode2(data: bytes, adder: int) -> bytearray:
    trace(adder, data[:4])
    newdata = bytearray()
    for byte in data:                         # bytes => int (or via bytes[i])
        word  = byte + adder                  # add scrambler
        byte1 = (word & 0xFF00) >> 8          # upper byte 
        byte2 = (word & 0X00FF)               # lower byte
        newdata.extend([byte1, byte2])        # 2 bytes for 1, to binary file
    trace(newdata[:8])                        # int => bytes (or bytes([int]))
    return newdata

@debugtypes
def do_decode2(data: bytes, adder: int) -> bytearray:
    trace(adder, data[:8])
    newdata = bytearray()
    ix = 0
    while ix < len(data):
        byte1, byte2 = data[ix], data[ix+1]   # bytes => int
        word = (byte1 << 8) + byte2           # 1 word to 2 bytes
        word -= adder                         # remove scrambler
        newdata.append(word)                  # retain low byte
        ix += 2
    trace(newdata[:4])
    return newdata

...
from encoder import *
encode, decode = do_encode2, do_decode2       # choose your weapon
data = open(filename, 'rb').read()            # load from original name
data = encode(data, adder)                    # scramble byte data
newname = filename + encext                   # write to new enc name
with open(newname, 'wb') as newfile:          # guarantee closes
    newfile.write(data)

Caveats

This decorator might be useful during development to ensure coding-time expectations, but has some major downsides:

• As the book notes, manual type testing in general can limit code flexibility in most contexts. In this specific case, the decorator precludes processing other unrelated objects with compatible interfaces. In the example client below, for instance, a bytearray won't pass a bytes type test when the decorator is deployed (and requires extra manual conversion), but works fine if the decorator is removed because its interface is compatible with the code. The type test fails because bytes objects are not instances of bytearray, and vice-versa. Adding extra generic base types can address some such issues, but would add new complexity all their own for an artificial and arguably dubious cause. Python is about object interfaces, not type constraints.
• Besides limiting code scope, type constraints are also often fully pointless in Python—the run-time error checking already performed by the language will generally catch type mismatches automatically during testing. That is, it's better to write code expecting a compatible object interface, and let Python's own error checking detect cases of interface mismatch. See the self-test code in the decorator module listed above for a prime example; most type errors caught by the decorator's manual type testing generate normal Python errors if the decorator's type testing is disabled (via the "-O" command-line switch). The decorator's coding pattern may have valid use cases, but type checking may not be one of them.

Update Apr-2015: A similar model is being proposed as standard type declarations for Python 3.5; it threatens to escalate the same caveats to best practice. For an arguably better use case for decorators, see the Private/Public attributes class decorator example from LP5E.

Books

Code

Blog

Python

Author

Train

Find

©M.Lutz