mapper - Parallel processing implementations¶
AMQPMapper |
|
MPIMapper |
|
MPMapper |
|
SerialMapper |
|
can_pickle |
Returns True if problem can be pickled. |
nice |
|
setpriority |
Set The Priority of a Windows Process. |
Parallel and serial mapper implementations.
The API is a bit crufty since interprocess communication has evolved from the original implementation. And the names are misleading.
Usage:
Mapper.start_worker(problem)
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
...
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
Mapper.stop_mapper(mapper)
-
class
bumps.mapper.
MPIMapper
[source]¶ Bases:
object
-
static
start_worker
(problem)[source]¶ Start the worker process.
For the main process this does nothing and returns immediately. The worker processes never return.
Each worker sits in a loop waiting for the next batch of points for the problem, or for the next problem. Set t problem is set to None, then exit the process and never
-
static
-
class
bumps.mapper.
MPMapper
[source]¶ Bases:
object
-
manager
= None¶
-
namespace
= None¶
-
pool
= None¶
-
problem_id
= 0¶
-
-
bumps.mapper.
can_pickle
(problem, check=False)[source]¶ Returns True if problem can be pickled.
If this method returns False then MPMapper cannot be used and SerialMapper should be used instead.
If check is True then call nllf() on the duplicated object as a “smoke test” to verify that the function will run after copying. This is not foolproof. For example, access to a database may work in the duplicated object because the connection is open and available in the current process, but it will fail when trying to run on a remote machine.