In this section, you’ll learn how to do parallel programming in Python using functional programming principles and the multiprocessing
module. You’ll take the example data set based on an immutable data structure that you previously transformed using the built-in map()
function. But this time you’ll process the data in parallel, across multiple CPU cores using the Python multiprocessing
module available in the standard library.
You’ll see, step by step, how to parallelize an existing piece of Python code so that it can execute much faster and leverage all of your available CPU cores. You’ll also learn how to use the multiprocessing.Pool
class and its parallel map
implementation that makes parallelizing most Python code that’s written in a functional style a breeze.
First, you’ll build a little testbed program that we can use to measure the execution time with the time.time()
function, so that we can compare the single-threaded and multithreaded implementations of the same algorithm.