Coroutines

Python coroutines are similar to generators but instead of producing data, coroutines are mostly used as data consumers. In other words, coroutines are functions that are resumed everytime a value is sent using the send method.

The trick with coroutines is the use of the yield keyword on the right side of an assignment expression. Here’s an example of a coroutine that just prints the values that are sent to it:

  1. def coroutine():
  2.     print('My coroutine')
  3.     while True:
  4.         val = yield
  5.         print('Got', val)
  7. >>> co = coroutine()
  8. >>> next(co)
  9. My coroutine
  10. >>> co.send(1)
  11. Got 1
  12. >>> co.send(2)
  13. Got 2
  14. >>> co.send(3)
  15. Got 3

The initial call to next is required to move the coroutine forward. You can see that it executes the print statement. Eventually, the function reaches the yield expression where it will wait to be resumed. Then, everytime a value is sent (with send), the coroutine function resumes from the yield, copies the value to val and prints it.

Coroutines can be closed with the close() method.

  1. >>> co.close()
  2. >>> co.send(4)
  3. Traceback (most recent call last):
  4.   File "<stdin>", line 1, in <module>
  5. StopIteration

Exercises with coroutines

  1. Create a coroutine named “square” that prints the square of any sent value.

  2. Implement the “minimize” coroutine that keeps and prints the minimum value that is sent to the function.

Pipelines

Coroutines can be used to implement data pipelines where one coroutine will send data to the next coroutine in the pipeline. Coroutines push data into the pipeline using the send() method.

Coroutines - 图1

Here’s an example of a small pipeline where the values sent to the producer coroutine are squared and sent to the consumer coroutine for printing:

  1. def producer(consumer):
  2.     print("Producer ready")
  3.     while True:
  4.         val = yield
  5.         consumer.send(val * val)
  7. def consumer():
  8.     print("Consumer ready")
  9.     while True:
  10.         val = yield
  11.         print('Consumer got', val)

As above, coroutines must be “primed” with next before any value can be sent.

  1. >>> cons = consumer()
  2. >>> prod = producer(cons)
  3. >>> next(prod)
  4. Producer ready
  5. >>> next(cons)
  6. Consumer ready
  8. >>> prod.send(1)
  9. Consumer got 1
  10. >>> prod.send(2)
  11. Consumer got 4
  12. >>> prod.send(3)
  13. Consumer got 9

Also, with coroutines, data can be sent to multiple destinations.The following example implements two consumers where the first only prints numbers in 0..10 and the second only print numbers in 10..20:

  1. def producer(consumers):
  2.     print("Producer ready")
  3.     try:
  4.         while True:
  5.             val = yield
  6.             for consumer in consumers:
  7.                 consumer.send(val * val)
  8.     except GeneratorExit:
  9.         for consumer in consumers:
  10.             consumer.close()
  12. def consumer(name, low, high):
  13.     print("%s ready" % name)
  14.     try:
  15.         while True:
  16.             val = yield
  17.             if low < val < high:
  18.                 print('%s got' % name, val)
  19.     except GeneratorExit:
  20.         print("%s closed" % name)

As before, coroutines must be “primed” before any value can be sent.

  1. >>> con1 = consumer('Consumer 1', 00, 10)
  2. >>> con2 = consumer('Consumer 2', 10, 20)
  3. >>> prod = producer([con1, con2])
  5. >>> next(prod)
  6. Producer ready
  7. >>> next(con1)
  8. Consumer 1 ready
  9. >>> next(con2)
  10. Consumer 2 ready
  12. >>> prod.send(1)
  13. Consumer 1 got 1
  14. >>> prod.send(2)
  15. Consumer 1 got 4
  16. >>> prod.send(3)
  17. Consumer 1 got 9
  18. >>> prod.send(4)
  19. Consumer 2 got 16
  21. >>> prod.close()
  22. Consumer 1 closed
  23. Consumer 2 closed

The data is sent to all consumers but only the second executes the print statement. Notice the use of the GeneratorExit exception. Sometimes it can be useful to catch the exception and inform the downstream coroutines that the pipeline is no longer useful.

Coroutines - 图2

Exercises with coroutine pipelines

  1. Implement a producer-consumer pipeline where the values squared by the producer are sent to two consumers. One should store and print the minimum value sent so far and the other the maximum value.

  2. Implement a producer-consumer pipeline where the values squared by the producer are dispatched to two consumers, one at a time. The first value should be sent to consumer 1, the second value to consumer 2, third value to consumer 1 again, and so on. Closing the producer should force the consumers to print a list with the numbers that each one obtained.