Parallel

class pocomc.parallel.MPIPool(comm=None, use_dill=True)

A processing pool that distributes tasks using MPI. With this pool class, the master process distributes tasks to worker processes using an MPI communicator.

Parameters:

• comm (mpi4py.MPI.Comm, optional) – An MPI communicator to distribute tasks with. If None, this uses MPI.COMM_WORLD by default.

• use_dill (bool, optional) – If True, use dill for pickling objects. This is useful for pickling functions and objects that are not picklable by the default pickle module. Default is True.

Notes

This implementation is inspired by @juliohm in this module and was adapted from schwimmbad.

close()

Tell all the workers to quit.

map(worker, tasks)

Evaluate a function or callable on each task in parallel using MPI. The callable, worker, is called on each element of the tasks iterable. The results are returned in the expected order.

Parameters:

• worker (callable) – A function or callable object that is executed on each element of the specified tasks iterable. This object must be picklable (i.e. it can’t be a function scoped within a function or a lambda function). This should accept a single positional argument and return a single object.

• tasks (iterable) – A list or iterable of tasks. Each task can be itself an iterable (e.g., tuple) of values or data to pass in to the worker function.

Returns:

results – A list of results from the output of each worker() call.

Return type:

list

wait()

Tell the workers to wait and listen for the master process. This is called automatically when using MPIPool.map() and doesn’t need to be called by the user.