17. Advanced Topics

 

 

 

 

 

Topics

 

 

     Unicode text and binary data

     Managed Attributes

     Decorators

     Metaclasses

     Context Managers

     Python 3.X Changes

 

 

 

 

 

 

 

 

Unicode Text and Binary Data

 

 

Python 3.Xs dichotomy

      Unicode text strings for all possibly-internationalized text use cases

      Binary data byte strings for C integration, devices, and networks

How Unicode works

      Unicode encodings specify how characters are stored as raw bytes

      Unicode translations largely occur at IO boundaries: decode in input, encode on output

      In memory, text strings are sequences of decoded integer code points identifying characters per Unicode

For simple text

      ASCII is a simple kind of Unicode: strings and files work same in 2.X and 3.X for ASCII-only text

      This is why some programmers may not need to care

      But web, email, JSON, XML, GUIs, sockets, pipes, databases, may require Unicode knowledge

 

 

 

 

 

Python 2.X string model

      str for both 8-bit text and binary data

      unicode for unicode text

      open() returns 8-bit str, codecs.open() is unicode

      str.decode()/encode(), unicode.encode() to convert

      codecs.open() supports encoding names too

      unicode has same interface as and may mix with str

 

 

Python 3.X string model

      str for all text: Unicode and 8-bit (ASCII is Unicode)

      bytes for 8-bit binary data, bytearray mutable variant

      open() uses str in text mode, bytes in binary mode

      str.encode(), bytes.decode() to convert

      open() supports encoding names too

      bytes has similar interfaces but doesnt mix with str

 

 

 

 

 

Coding Unicode Text (Python 3.X)

 

 

Coding ASCII characters works as it always did, and the same in 2.X and 3.X, and most encodings are no-ops:

 

 

C:\misc> py -3

>>> ord('X') # 88 in default encoding

88

>>> chr(88) # 88 stands for char 'X'

'X'

 

>>> S = 'XYZ'

>>> S

'XYZ'

>>> len(S) # 3 characters long

3

 

# Values 0..127 in 1 byte (7 bits) each

 

>>> S.encode('ascii')

b'XYZ'

 

# Values 0..255 in 1 byte (8 bits) each

 

>>> S.encode('latin-1')

b'XYZ'

 

# Values 0..127 in 1 byte, 128..2047: 2, others: 3 or 4

 

>>> S.encode('utf-8')

b'XYZ'

 

 

 

Coding non-ASCII characters: via hex or Unicode escapes in your strings; hex escapes are limited to a single bytes value, but Unicode escapes can name characters with values two and four bytes wide. The hex values 0xC4 and 0xE8, for instance, are codes for two special characters outside the 7-bit range of ASCII, but we can embed them in 3.X str objects, because str supports Unicode today (use u'' in 2.X):

 

 

>>> chr(0xc4) # 0xC4, 0xE8: outside ASCII range

''

>>> chr(0xe8)

''

 

>>> S = '\xc4\xe8' # Single byte 8-bit hex escapes

>>> S

''

 

>>> S = '\u00c4\u00e8' # 16-bit Unicode escapes

>>> S

''

>>> len # 2 characters long (not num of bytes!)

2

 

 

 

Manually encoding a non-ASCII string into raw bytes using as ASCII generates an error. Encoding as Latin-1 works, though, and allocates one byte per character; encoding as UTF-8 allocates 2 bytes per character instead. If this string is written to a file, the raw bytes shown here is what is actually stored on the file for the encoding types given (see ahead):

 

 

>>> S = '\u00c4\u00e8'

>>> S

''

>>> len(S) # for str, number characters (code points), not bytes!

2

 

>>> S.encode('ascii')

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)

 

>>> S.encode('latin-1') # One byte per character

b'\xc4\xe8'

 

>>> S.encode('utf-8') # Two bytes per character

b'\xc3\x84\xc3\xa8'

 

>>> len(S.encode('latin-1')) # 2 bytes in latin-1

2

>>> len(S.encode('utf-8')) # 4 in utf-8

4

 

 

 

Manual decoding from raw bytes back to a Unicode string works too. You could read raw bytes from a file and decode manually this way, but the encoding mode you give to the open() call causes this decoding to be done for you automatically on input (and avoids issues that may arise from reading partial character sequences when reading by blocks of bytes):

 

 

>>> B = b'\xc4\xe8'

>>> B

b'\xc4\xe8'

>>> len(B) # 2 raw bytes, 2 chars

2

>>> B.decode('latin-1') # Decode to latin-1 text

''

 

>>> B = b'\xc3\x84\xc3\xa8'

>>> len(B) # 4 raw bytes

4

>>> B.decode('utf-8')

''

>>> len(B.decode('utf-8')) # 2 unicode characters

2

 

 

 

Unicode escapes: both 16- and 32-bit Unicode values for characters may be specified in strings; "\u..." with 4 hex digits for the former, and "\U...." with 8 hex digits for the latter:

 

 

>>> S = 'A\u00c4B\U000000e8C'

>>> S # A,B,C + 2 nonASCII

'ABC'

>>> len(S) # 5 characters long

5

 

>>> S.encode('latin-1')

b'A\xc4B\xe8C'

>>> len(S.encode('latin-1')) # 5 bytes in latin-1

5

 

>>> S.encode('utf-8')

b'A\xc3\x84B\xc3\xa8C'

>>> len(S.encode('utf-8')) # 7 bytes in utf-8

7

 

>>> S.encode('cp500') # Two unusual encodings

b'\xc1c\xc2T\xc3'

>>> S.encode('cp850') # 5 bytes each

b'A\x8eB\x8aC'

 

 

 

Piecemeal coding: non-ASCII strings can also be built-up piecemeal using chr(), but it becomes tedious for large strings:

 

 

>>> S = 'A' + chr(0xC4) + 'B' + chr(0xE8) + 'C'

>>> S

'ABC'



 

ASCII versus Unicode: ASCII is a type of Unicode that encodes the same way in most encodings:

 

 

>>> S = 'spam' # ASCII is same in most

>>> S.encode('latin-1')

b'spam'

>>> S.encode('utf-8')

b'spam'

>>> S.encode('cp500') # But not in IBM EBCDIC!

b'\xa2\x97\x81\x94'

>>> S.encode('cp850')

b'spam'

 

 

 

Escapes summary: Python 3.X allows special characters to be coded with both hex and Unicode escapes in str strings, but only hex escapes in bytes:

 

 

>>> S = 'A\xC4B\xE8C' # str: hex and unicode

>>> S

'ABC'

 

>>> S = 'A\u00C4B\U000000E8C'

>>> S

'ABC'

 

>>> B = b'A\xC4B\xE8C' # bytes: hex, not unicode

>>> B

b'A\xc4B\xe8C'

 

 

 

 

Text and Binary Files

 

   Text mode files interpret file contents according to an encodingeither the default for your platform, or one whose name you pass in. Text mode also translated line-ends to \n by default.

   Binary mode files instead return file content to you raw, as a sequence of integers representing byte values, with no encoding or decoding, and no line-end translations.

 

 

 

C:\misc> py -2

 

>>> open('temp', 'w').write('abd\n') # text mode

>>> open('temp', 'r').read()

'abd\n'

>>> open('temp', 'rb').read() # binary mode

'abd\r\n'

 

>>> open('temp', 'wb').write('abc\n') # binary mode

>>> open('temp', 'r').read() # \n verbatim

'abc\n'

>>> open('temp', 'rb').read()

'abc\n'

 

 

C:\misc> py -3

 

# Write and read a text file (default encoding)

 

>>> open('temp', 'w').write('abc\n')

4

>>> open('temp', 'r').read() # str for text mode

'abc\n'

>>> open('temp', 'rb').read() # bytes for binary mode

b'abc\r\n'

 

 

# Write and read a binary file

 

>>> open('temp', 'wb').write(b'abc\n')

4

>>> open('temp', 'r').read()

'abc\n'

>>> open('temp', 'rb').read()

b'abc\n'

 

 

# Write and read truly binary data

 

>>> open('temp', 'wb').write(b'a\x00c')

3

>>> open('temp', 'r').read()

'a\x00c'

>>> open('temp', 'rb').read()

b'a\x00c'

 

 

 

 

Reading and writing Unicode data

 

C:\misc> py -3

 

>>> S = 'A\xc4B\xe8C' # 5 chars, nonASCII

>>> S

'ABC'

>>> len(S)

5

 

 

# Manual encoding

 

>>> L = S.encode('latin-1') # 5 bytes as Latin-1

>>> L

b'A\xc4B\xe8C'

>>> len(L)

5

 

>>> U = S.encode('utf-8') # 7 bytes as UTF-8

>>> U

b'A\xc3\x84B\xc3\xa8C'

>>> len(U)

7

 

 

# File output encoding

 

>>> S

'ABC'

 

>>> open('latindata', 'w', encoding='latin-1').write(S)

5

>>> open('utf8data', 'w', encoding='utf-8').write(S)

5

 

>>> open('latindata', 'rb').read()

b'A\xc4B\xe8C'

>>> open('utf8data', 'rb').read()

b'A\xc3\x84B\xc3\xa8C'

 

 

# File input encoding

 

>>> open('latindata', 'r', encoding='latin-1').read()

'ABC'

>>> open('utf8data', 'r', encoding='utf-8').read()

'ABC'

 

>>> X = open('latindata', 'rb').read()

>>> X.decode('latin-1')

'ABC'

>>> X = open('utf8data', 'rb').read()

>>> X.decode()

'ABC'

 

 

 

More details: see Learning Python 4th and 5th Editions

 

 

 

 

 

 

 

 

Managed Attributes

 

 

 

Attribute access control techniques:

   Properties: class attrs that run code on attr access

   Descriptors: class attrs with __get__/__set__ methods

   __getattr__ method: run for undefined attrs

   __getattribute__ method: run for all attrs

 

      Accessors are most useful when computation requirements unknown or prone to change, or require special processing (e.g., validation, database mapping).

      __getattr__ and __getattribute__ are more general and support delegation, but may incur extra calls, and are not run in 3.X for implicit __X__ attr fetches made by built-in operations (proxy classes requires special code for operator overloading: see LP5E).

 

Properties

 

class C:

classAttr = property(fget, fset, fdel, doc)

 

 

Example

 

class PropSquare:

def __init__(self, start):

self.value = start

def getX(self): # On attr fetch

return self.value ** 2

def setX(self, value): # On attr assign

self.value = value

X = property(getX, setX) # No delete, docs

 

P = PropSquare(3) # Instances of class with prop

Q = PropSquare(32) # Each has different state

 

print(P.X) # 3 ** 2

P.X = 4

print(P.X) # 4 ** 2

print(Q.X) # 32 ** 2

 

 

 

 

Descriptors

 

class Descriptor:

"docstring goes here"

def __get__(self, instance, owner): ...

def __set__(self, instance, value): ...

def __delete__(self, instance): ...

 

 

Example

 

class DescSquare:

def __init__(self, start): # State info

self.value = start

def __get__(self, instance, owner): # On get

return self.value ** 2

def __set__(self, instance, value): # On set

self.value = value

 

class Client1:

X = DescSquare(3) # Instance is class attr

 

class Client2:

X = DescSquare(32) # Again in another class

c1 = Client1()

c2 = Client2()

 

print(c1.X) # 3 ** 2

c1.X = 4

print(c1.X) # 4 ** 2

print(c2.X) # 32 ** 2

 

 

 

 

__getattr__

 

def __getattr__(self, name): # obj.undef

def __getattribute__(self, name): # obj.all

def __setattr__(self, name, value): # obj.all=val

def __delattr__(self, name): # del obj.all

 

 

Example

 

class AttrSquare:

def __init__(self, start):

self.value = start # setattr!

def __getattr__(self, attr): # Undef gets

if attr == 'X':

return self.value ** 2

else:

raise AttributeError(attr)

 

def __setattr__(self, attr, value): # All sets

if attr == 'X':

attr = 'value'

self.__dict__[attr] = value

 

A = AttrSquare(3)

B = AttrSquare(32)

 

print(A.X) # 3 ** 2

A.X = 4

print(A.X) # 4 ** 2

print(B.X) # 32 ** 2

 

 

 

 

__getattribute__

 

 

Example

 

class AttrSquare:

def __init__(self, start):

self.value = start # setattr!

def __getattribute__(self, attr): # All gets

if attr == 'X':

return self.value ** 2 # recurs!

else:

return object.__getattribute__(self, attr)

 

def __setattr__(self, attr, value): # All sets

if attr == 'X':

attr = 'value'

object.__setattr__(self, attr, value)

 

A = AttrSquare(3)

B = AttrSquare(32)

 

print(A.X) # 3 ** 2

A.X = 4

print(A.X) # 4 ** 2

print(B.X) # 32 ** 2

 

 

 

 

Coding properties with decorators (2.6+)

   See next section for more details

 

class Person:

@property

def name(self): ...

 

# Same as

 

class Person:

def name(self): ...

name = property(name)

 

 

 

 

Example

 

class Person:

def __init__(self, name):

self._name = name

@property

def name(self): # name=property(name)

"name property docs"

print('fetch...')

return self._name

 

@name.setter

def name(self, value): # name=name.setter(name)

print('change...')

self._name = value

 

@name.deleter

def name(self): # name= name.deleter(name)

print('remove...')

del self._name

 

bob = Person('Bob Smith')

print(bob.name) # Runs name getter

bob.name = 'Robert Smith' # Runs name setter

print(bob.name)

del bob.name # Runs name deleter

 

 

 

Other use cases: see Learning Python 4th and 5th Editions

 

 

 

 

 

 

 

 

 

Decorators

 

 

 

   Function decorators: run at end of def statement to manage function itself, or later call to it with proxy objects (2.5 and later)

   Class decorators: run at end of class statement to manage class itself, or later instance-creation calls to it with proxy objects (2.6+ and 3.X)

   See also the complete function decorator and class decorator examples in the top-level Extras\Code\OOP

 

 

Function Decorators

 

# This syntax:

 

@decorator # Decorate function

def F(arg):

...

 

F(99) # Call function

 

 

# Is translated to:

 

def F(arg):

...

F = decorator(F) # Rebind function name

 

F(99) # Calls decorator(F)(99)

 

 

# Managing functions:

 

def decorator(F):

# process function F

return F

 

@decorator

def func(): ... # func = decorator(func)

 

 

# Inserting wrappers/proxies:

 

def decorator(F):

# save or use function F

# return a different callable that invokes F:

# nested def, class with __call__, etc.

 

@decorator

def func(): ... # func = decorator(func)

 

 

 

 

Class Decorators

 

# This syntax:

 

@decorator # Decorate class

class C:

...

 

x = C(99) # Make an instance

 

 

# Is translated to:

 

class C:

...

C = decorator(C) # Rebind class name

 

x = C(99) # Calls decorator(C)(99)

 

 

# Managing classes:

 

def decorator(C):

# process class C

return C

 

@decorator

class C: ... # C = decorator(C)

 

 

# Inserting wrappers/proxies:

 

def decorator(C):

# save or use class C

# return a different callable that invokes C:

# nested def, class with __call__, etc.

 

@decorator

class C: ... # C = decorator(C)

 

 

 

 

Decorator arguments

      Actual decorator returned by decorator named

      Decorator named uses or retains arguments

 

 

@decorator(A, B)

def F(arg):

...

 

F(99)

 

 

# Is translated to:

 

def F(arg):

...

F = decorator(A, B)(F) # Rebind F to return val

 

F(99) # decorator(A, B)(F)(99)

 

 

# Implementation:

 

def decorator(A, B):

# save or use A, B

def actualDecorator(F):

# save or use function F

# return F or another callable:

# nested def, class with __call__, etc.

# A, B, and F may all be accessible here

return callable

return actualDecorator

 

 

 

 

Decorator nesting

      Each layer can manage or wrap prior

      Works for both function and class decorators

 

def d1(F): return lambda: 'X' + F()

def d2(F): return lambda: 'Y' + F()

def d3(F): return lambda: 'Z' + F()

 

@d1

@d2

@d3

def func(): # func = d1(d2(d3(func)))

return 'spam'

 

print(func()) # Prints "XYZspam"

 

 

 

 

Example: __call__ wrappers

 

class tracer:

def __init__(self, func): # On @

self.calls = 0

self.func = func

def __call__(self, *args): # On later call

self.calls += 1

print('call %s to %s' %

(self.calls, self.func.__name__))

self.func(*args)

 

@tracer

def spam(a, b, c): # spam = tracer(spam)

print(a + b + c) # Wraps spam in obj

 

 

 

 

Example: nested scopes

def tracer(func): # State via enclosing scope

calls = 0

def wrapper(*args, **kwargs):

nonlocal calls

calls += 1

print('call %s to %s' %

(calls, func.__name__))

return func(*args, **kwargs)

return wrapper

 

@tracer

def spam(a, b, c):

print(a + b + c)

 

@tracer

def eggs(x, y):

print(x ** y)

 

 

 

 

Example: tracing object interfaces

      Does not catch operator-overloading methods in 3.X

      See the book LP5E for alternative ways to address this

 

def Tracer(aClass):

class Wrapper:

def __init__(self, *args, **kargs):

self.fetches = 0

self.wrapped = aClass(*args, **kargs)

def __getattr__(self, attrname):

print('Trace: ' + attrname)

self.fetches += 1

return getattr(self.wrapped, attrname)

return Wrapper

 

@Tracer

class Spam: # Spam = Tracer(Spam)

def display(self): # Spam is Wrapper

print('Spam!' * 8)

 

@Tracer

class Person: # Person = Tracer(Person)

def __init__(self, name, hours, rate):

self.name = name

self.hours = hours

self.rate = rate

def pay(self):

return self.hours * self.rate

 

food = Spam() # Triggers Wrapper()

food.display() # Triggers __getattr__

print([food.fetches])

 

bob = Person('Bob', 40, 50) # bob is really a Wrapper

print(bob.name) # Wrapper embeds a Person

print(bob.pay())

 

print('')

sue = Person('Sue', rate=100, hours=60)

print(sue.name)

print(sue.pay())

 

print(bob.name)

print(bob.pay())

print([bob.fetches, sue.fetches])

 

 

 

 

Example: registering objects to an API

 

registry = {}

def register(obj): # class+func decor

registry[obj.__name__] = obj # Add to registry

return obj # Return obj itself

 

@register

def spam(x):

return(x ** 2) # spam = register(spam)

 

@register

def ham(x):

return(x ** 3)

 

@register

class Eggs: # Eggs = register(Eggs)

def __init__(self, x):

self.data = x ** 4

def __str__(self):

return str(self.data)

 

print('Registry:')

for name in registry:

print(name, '=>',

registry[name], type(registry[name]))

 

print('\nManual calls:')

print(spam(2))

print(ham(2))

X = Eggs(2)

print(X)

 

print('\nRegistry calls:')

for name in registry:

print(name, '=>', registry[name](3))

 

 

 

 

Other use cases: private attribute management for classes, function argument range and type testing, etc. see Learning Python 4th and 5th Editions, and the complete examples referenced at the start of this section

 

 

 

 

 

 

 

 

Metaclasses

 

 

 

 

   Subclasses of type (usually), run at end of class statement

   Manages class or later instance-creation calls to it

   Overlaps with newer class decorator in most roles

 

 

 

# Metaclass declaration:

 

class Spam(metaclass=Meta): # 3.0 and later

 

class spam(object): # 2.6 version (only)

__metaclass__ = Meta

 

 

 

# At end of class statement, Python runs:

 

class = Meta(classname, superclasses, attrdict)

 

 

 

# type.__call__ runs Metas __new__ and __init__:

 

Meta.__new__(Meta, classname, superclasses, attrdict)

Meta.__init__(class, classname, superclasses, attrdict)

 

 

 

# Meta redefines for its own purposes:

 

class Meta(type):

def __new__(meta, clsname, supers, clsdict):

# run by inherited type.__call__

return type.__new__(meta, clsname, supers, clsdict)

 

 

 

# Mapping at end of class statement:

 

class Spam(Eggs, metaclass=Meta):

data = 1

def meth(self, arg):

pass

 

 

Spam = Meta(

'Spam',

(Eggs,),

{'data': 1, 'meth': meth, '__module__','__main__'})



 

 

Example: tracing the protocol

 

class MetaOne(type):

def __new__(meta, classname, supers, classdict):

print('In MetaOne.new: ', classname, supers, classdict, sep='\n...')

return type.__new__(meta, classname, supers, classdict)

def __init__(Class, classname, supers, classdict):

print('In MetaOne init:', classname, supers, classdict, sep='\n...')

print('...init class object:', list(Class.__dict__.keys()))

class Eggs:

pass

 

print('making class')

class Spam(Eggs, metaclass=MetaOne):

data = 1

def meth(self, arg):

pass

 

print('making instance')

X = Spam()

print('data:', X.data)

 

 

# Output

 

making class

In MetaOne.new:

...Spam

...(<class '__main__.Eggs'>,)

...{'__module__': '__main__', 'data': 1, 'meth': <function meth at 0x02AAB810>}

In MetaOne init:

...Spam

...(<class '__main__.Eggs'>,)

...{'__module__': '__main__', 'data': 1, 'meth': <function meth at 0x02AAB810>}

...init class object: ['__module__', 'data', 'meth', '__doc__']

making instance

data: 1

 

 

 

 

Example: augmenting classes - metaclasses

 

# Extend with a metaclass - supports future changes better

 

def eggsfunc(obj):

return obj.value * 4

 

def hamfunc(obj, value):

return value + 'ham'

 

class Extender(type):

def __new__(meta, classname, supers, classdict):

classdict['eggs'] = eggsfunc

classdict['ham'] = hamfunc

return type.__new__(meta,

classname, supers, classdict)

 

class Client1(metaclass=Extender):

def __init__(self, value):

self.value = value

def spam(self):

return self.value * 2

 

class Client2(metaclass=Extender):

value = 'ni?'

 

X = Client1('Ni!')

print(X.spam())

print(X.eggs())

print(X.ham('bacon'))

 

Y = Client2()

print(Y.eggs())

print(Y.ham('bacon'))

 

 

 

 

Example: augmenting classes - decorators

 

# Extend with a decorator: same as __init__ in a metaclass

 

def eggsfunc(obj):

return obj.value * 4

 

def hamfunc(obj, value):

return value + 'ham'

 

def Extender(aClass):

aClass.eggs = eggsfunc

aClass.ham = hamfunc

return aClass

 

@Extender

class Client1: # Client1=Extender(Client1)

def __init__(self, value):

self.value = value

def spam(self):

return self.value * 2

 

@Extender

class Client2:

value = 'ni?'

 

X = Client1('Ni!') # X is a Client1 instance

print(X.spam())

print(X.eggs())

print(X.ham('bacon'))

 

Y = Client2()

print(Y.eggs())

print(Y.ham('bacon'))

 

 

 

Other use cases: applying a function decorator to all methods of a class, etc. see Learning Python 4th and 5th Editions

See also: this summary PDF, giving metaclasss role in new style inheritance

 

 

 

 

 

 

 

 

 

Context Managers

 

 

 

 

   Alternative to try/finally termination action exception handler

   Also supports entry actions

   Makes for less code in clients, for supported objects

   Some built-in objects have them (files, thread locks)

   Available in 2.6+ and 3.X (and 2.5 with from __future__)

 

 

 

 

# try/finally

 

myfile = open(r'C:\misc\data')

try:

for line in myfile:

print(line)

...more code here...

finally:

myfile.close()

 

 

 

# context managers: auto-closed

 

with open(r'C:\misc\data') as myfile:

for line in myfile:

print(line)

...more code here...

 

 

 

# thread locks too

 

lock = threading.Lock()

with lock:

# critical section of code

...access shared resources...

 

 

 

 

# User-defined context managers

 

class TraceBlock:

def message(self, arg):

print('running', arg)

def __enter__(self):

print('starting with block')

return self

def __exit__(self, exc_type, exc_value, exc_tb):

if exc_type is None:

print('exited normally\n')

else:

print('raise an exception!', exc_type)

return False # propagate

 

with TraceBlock() as action:

action.message('test 1')

print('reached')

 

with TraceBlock() as action:

action.message('test 2')

raise TypeError

print('not reached')

 

 

 

% python withas.py

starting with block

running test 1

reached

exited normally

 

starting with block

running test 2

raise an exception! <class 'TypeError'>

Traceback (most recent call last):

File "withas.py", line 20, in <module>

raise TypeError

TypeError

 

 

 

 

 

 

 

 

Python 3.X Changes

 

 

 

For recent 3.X changes, see LP5Es Appendix C, and this online review

 

 

   Below: 3.0 changes impacting the book Learning Python 4th and 5th Editions

   There are more!see Whats new in Python 3.0 in standard manuals

   Released Dec 2008 but you may not need to care until 2010 or 2011

   Update: 2.X still in wide use as of 2015, 7 years later

   Many third-party libraries and much current code is still 2.X

   3.1 fixes radical I/O slowdown, adds ,d for numbers in str.format()

 

 

 

Extension

The print() function in 3.0

The nonlocal x,y statement in 3.0

The str.format() method in 2.6 and 3.0

String types in 3.0: str for Unicode text, bytes for binary data

Text and binary file distinctions in 3.0

Class decorators in 2.6 and 3.0: @private('age')

New iterators in 3.0: range(), map(), zip()

Dictionary views in 3.0: keys(), values(), items()

Division operators in 3.0: remainders, / and //

Set literals in 3.0: {a, b, c}

Set comprehensions in 3.0: {x**2 for x in seq}

Dictionary comprehensions in 3.0: {x: x**2 for x in seq}

Binary digit-string support in 2.6 and 3.0: 0b0101, bin(I)

The fraction number type in 2.6 and 3.0

Function annotations in 3.0: def f(a: 99, b: str)->int

Keyword-only arguments in 3.0: def f(a, *b, c)

Extended sequence unpacking in 3.0: a, *b = seq

Relative import syntax for packages enabled in 3.0

with/as context managers enabled in 2.6 and 3.0

Exception syntax changes in 3.0: raise, except, superclass

Exception chaining in 3.0: raise e2 from e1

Reserved word changes in 3.0 and 2.6

New-style class cutover in 3.0

Property decorators in 2.6 and 3.0

Descriptor use in 2.6 and 3.0

Metaclass use in 2.6 and 3.0

Abstract base classes support in 2.6 and 3.0

 

 

 

 

 

Removed

Replacement

reload(M)

imp.reload(M) (or exec())

apply(f, ps, ks)

f(*ps, **ks)

`X`

repr(X)

X <> Y

X != Y

long

Int

9999L

9999

D.has_key(K)

K in D (or D.get(key) != None)

raw_input()

input()

old input()

eval(input())

xrange()

range()

file()

open() (and io module classes)

X.next()

X.__next__(), called by next(X)

X.__getslice__()

X.__getitem__() passed a slice() object

X.__setslice__()

X.__setitem__() passed a slice() object

reduce()

functools.reduce() (or loop code)

execfile(filename)

exec(open(filename).read())

exec open(filename)

exec(open(filename).read())

0777

0o777

print x, y

print(x, y)

print >> F, x, y

print(x, y, file=F)

print x, y,

print(x, y, end=' ')

u'ccc'

'ccc'

'bbb' for byte strings

b'bbb'

raise E, V

raise E(V)

except E, X:

except E as X:

def f((a, b)):

def f(x): (a, b) = x

file.xreadlines()

for line in file: (or X=iter(file))

D.keys(), etc. as lists

list(D.keys()) (dictionary views)

map(), range(), etc. as lists

list(map()), list(range()) (built-ins)

map(None, ...)

zip() (or manual code to pad results)

X=D.keys(); X.sort()

sorted(D) (or list(D.keys()))

cmp(x, y)

(x > y) - (x < y)

X.__cmp__(y)

__lt__, __gt__, __eq__, etc.

Sort comparison functions

Use key=transform or reverse=True

Dictionary <, >, <=, >=

Compare sorted(D.items()) (or loop code)

types.ListType

list (types is non-built-in names only)

__metaclass__ = M

class C(metaclass=M):

__builtin__

Builtins

Tkinter

Tkinter

sys.exc_type, exc_value

sys.exc_info()[0], [1]

function.func_code

function.__code__

__getattr__ run by built-ins

Redefine __X__ methods in wrapper classes

-t, tt command-line switches

Inconsistent tabs/spaces use is always an error

from *, within a function

May only appear at the top-level of a file

import mod, in same package

from . import mod, package-relative form

class MyException:

class MyException(Exception):

exceptions module

Built-in scope, library manual

os.popen2/3/4()

subprocess.Popen()

String-based exceptions

Class-based exceptions (also required in 2.6)

String module functions

String object methods

Unbound methods

Functions (staticmethod to call via instance)

Mixed type comparisons, sorts

Non-numeric mixed type comparisons are errors

 

 

 

 

Lab Session 13

 

Click here to go to lab exercises

Click here to go to solution source files