mapper - Parallel processing implementations

AMQPMapper

MPIMapper

MPMapper

SerialMapper

can_pickle

Returns True if problem can be pickled.

nice

setpriority

Set The Priority of a Windows Process.

Parallel and serial mapper implementations.

The API is a bit crufty since interprocess communication has evolved from the original implementation. And the names are misleading.

Usage:

Mapper.start_worker(problem)
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
...
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
Mapper.stop_mapper(mapper)
class bumps.mapper.AMQPMapper[source]

Bases: object

static start_mapper(problem, modelargs=None, cpus=0)[source]
static start_worker(problem)[source]
static stop_mapper(mapper)[source]
class bumps.mapper.MPIMapper[source]

Bases: object

static start_mapper(problem, modelargs=None, cpus=0)[source]
static start_worker(problem)[source]

Start the worker process.

For the main process this does nothing and returns immediately. The worker processes never return.

Each worker sits in a loop waiting for the next batch of points for the problem, or for the next problem. Set t problem is set to None, then exit the process and never

static stop_mapper(mapper)[source]
class bumps.mapper.MPMapper[source]

Bases: object

manager = None
namespace = None
pool = None
problem_id = 0
static start_mapper(problem, modelargs=None, cpus=0)[source]
static start_worker(problem)[source]
static stop_mapper(mapper)[source]
class bumps.mapper.SerialMapper[source]

Bases: object

static start_mapper(problem, modelargs=None, cpus=0)[source]
static start_worker(problem)[source]
static stop_mapper(mapper)[source]
bumps.mapper.can_pickle(problem, check=False)[source]

Returns True if problem can be pickled.

If this method returns False then MPMapper cannot be used and SerialMapper should be used instead.

If check is True then call nllf() on the duplicated object as a “smoke test” to verify that the function will run after copying. This is not foolproof. For example, access to a database may work in the duplicated object because the connection is open and available in the current process, but it will fail when trying to run on a remote machine.

bumps.mapper.nice()[source]
bumps.mapper.setpriority(pid=None, priority=1)[source]

Set The Priority of a Windows Process. Priority is a value between 0-5 where 2 is normal priority and 5 is maximum. Default sets the priority of the current python process but can take any valid process ID.