在此页面上,多处理模块的作者Jesse Noller显示正确的处理方法
KeyboardInterrupt是让子流程返回-
不会引发异常。这允许主进程终止池。
但是,如下面的代码所示,主进程只有在运行所有生成的任务 之后 才能到达该
exceptKeyboardInterrupt块。这就是为什么(我相信)在按下后会看到对辅助函数的额外调用。
__
pool.map``run_nlin``Ctrl-C
一种可能的解决方法是,如果
multiprocessing.Event设置了a,则对所有工作程序功能进行测试。如果事件已发生,则请工人提早纾困,否则,请继续进行长时间的计算。
import loggingimport multiprocessing as mpimport timelogger = mp.log_to_stderr(logging.WARNING)def worker(x): try: if not terminating.is_set(): logger.warn("Running worker({x!r})".format(x = x)) time.sleep(3) else: logger.warn("got the message... we're terminating!") except KeyboardInterrupt: logger.warn("terminating is set") terminating.set() return xdef initializer(terminating_): # This places terminating in the global namespace of the worker subprocesses. # This allows the worker function to access `terminating` even though it is # not passed as an argument to the function. global terminating terminating = terminating_def main(): terminating = mp.Event() result = [] pool = mp.Pool(initializer=initializer, initargs=(terminating, )) params = range(12) try: logger.warn("starting pool runs") result = pool.map(worker, params) pool.close() except KeyboardInterrupt: logger.warn("^C pressed") pool.terminate() finally: pool.join() logger.warn('done: {r}'.format(r = result))if __name__ == '__main__': main()
运行脚本将产生:
% test.py[WARNING/MainProcess] starting pool runs[WARNING/PoolWorker-1] Running worker(0)[WARNING/PoolWorker-2] Running worker(1)[WARNING/PoolWorker-3] Running worker(2)[WARNING/PoolWorker-4] Running worker(3)
在这里按Ctrl-C;每个工人都设置
terminating事件。我们确实只需要设置它,但是尽管效率很低,但它仍然有效。
C-c C-c[WARNING/PoolWorker-4] terminating is set[WARNING/PoolWorker-2] terminating is set[WARNING/PoolWorker-3] terminating is set[WARNING/PoolWorker-1] terminating is set
现在,所有其他排队的任务
pool.map都将运行:
[WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-3] got the message... we're terminating!
最终,主要过程到达
except KeyboardInterrupt块。
[WARNING/MainProcess] ^C pressed[WARNING/MainProcess] done: []
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)