Coroutines 101: portable Python 3.X and 2.X flavor. (Copyright M. Lutz, 2015)
Demonstrate nonpreemptive, asynchronous multitasking, with a simple event loop
that switches between generator functions.

This model is cooperative: tasks must voluntarily and manually yield control 
to an event loop, and be short enough so as to not monopolize the CPU.  Hence, 
this requires a rigid and perhaps unnatural code structure, and is better used
for programs that can be designed as a set of tasks that run in quick bursts.

By contrast, threads and processes offer much more generalized preemptive 
multitasking, which shares the CPU among normally-structured tasks without
requiring code to manually yield control.  But they may also require use of 
synchronization tools such as thread queues, sockets, or pipes, as the tasks
may overlap arbitrarily (see Programming Python).  Both models can be used to 
interleave work over time, and avoid blocking while a program waits for IO or
other non-CPU operations, even when they don't boost overall program speed.

The simple generators here are adequate for task switching, but reflect only 
the first level of this volatile and obscure corner of the Python language:

-In 2.3+: original model - yield (plus expressions in 2.4)
-In 2.5+: sends (plus throws, closes)
-In 3.3+: returns and subgenerators
-In 3.5+: asynch/await (plus asyncio in 3.4)
For examples of most of the above:

from __future__ import print_function  # 2.X
import time, sys
def itime(): return '@%d' % round(time.clock())

# Event loop

def switcher(*tasks):
    taskqueue = [task() for task in tasks]    # Start all generators
    while taskqueue:                          # While any running tasks remain
        front = taskqueue.pop(0)              # Fetch next task at front
            result = next(front)              # Resume task, run to its next yield
        except StopIteration:
            pass                              # Returned: task finished
            name = front.__name__             # Yielded: reschedule at end of queue
            print(name, result, itime())

# Tasks

def task1():
    for i in range(6):       # Resumed here by next() in switcher (or for)
         time.sleep(1)       # Simulate a nonpreemptable action here
         yield i             # Suspend and yield control back to switcher (or for)
                             # Removed from queue on return (or exit for)
def task2():
    for i in range(3):
         yield i

def task3():
    for i in range(2):
         yield i

# Task launcher

if __name__ == '__main__':

    if len(sys.argv) > 1:
        # SERIAL: with command-line arg, run the functions by themselves.
        # Each task checks in at regular intervals and runs atomically.
        start = time.clock()
        for i in task1(): print('task1', i, itime()) 
        for i in task2(): print('task2', i, itime())
        for i in task3(): print('task3', i, itime())
        print('all tasks finished, total time: %.2f' % (time.clock() - start))
        # ASYNCHRONOUS: without arg, run the functions together, overlapped.
        # Tasks check in at irregular intervals and run interleaved. 
        start = time.clock()        
        switcher(task1, task2, task3)
        print('all tasks finished, total time: %.2f' % (time.clock() - start))

Expected output, under both 3.X and 2.X:

C:\Code> 1
task1 0 @1
task1 1 @2
task1 2 @3
task1 3 @4
task1 4 @5
task1 5 @6
task2 0 @8
task2 1 @10
task2 2 @12
task3 0 @15
task3 1 @18
all tasks finished, total time: 18.13

task1 0 @1
task2 0 @3
task3 0 @6
task1 1 @7
task2 1 @9
task3 1 @12
task1 2 @13
task2 2 @15
task1 3 @16
task1 4 @17
task1 5 @18
all tasks finished, total time: 18.13

[Home] Books Programs Blog Python Author Training Search Email ©M.Lutz