multiprocessing.Pool在LinuxPython2.7上的terminate()之后生成新的子对象?

multiprocessing.Pool在LinuxPython2.7上的terminate()之后生成新的子对象?,第1张

multiprocessing.Pool在Linux / Python2.7上的terminate()之后生成新的子对象?

在此页面上,多处理模块的作者Jesse Noller显示正确的处理方法

KeyboardInterrupt
是让子流程返回-
不会引发异常。这允许主进程终止池。

但是,如下面的代码所示,主进程只有在运行所有生成的任务 之后 才能到达该

exceptKeyboardInterrupt
块。这就是为什么(我相信)在按下后会看到对辅助函数的额外调用。
__
pool.map``run_nlin``Ctrl-C

一种可能的解决方法是,如果

multiprocessing.Event
设置了a,则对所有工作程序功能进行测试。如果事件已发生,则请工人提早纾困,否则,请继续进行长时间的计算。


import loggingimport multiprocessing as mpimport timelogger = mp.log_to_stderr(logging.WARNING)def worker(x):    try:        if not terminating.is_set(): logger.warn("Running worker({x!r})".format(x = x)) time.sleep(3)        else: logger.warn("got the message... we're terminating!")    except KeyboardInterrupt:        logger.warn("terminating is set")     terminating.set()    return xdef initializer(terminating_):    # This places terminating in the global namespace of the worker subprocesses.    # This allows the worker function to access `terminating` even though it is    # not passed as an argument to the function.    global terminating    terminating = terminating_def main():    terminating = mp.Event()        result = []    pool = mp.Pool(initializer=initializer, initargs=(terminating, ))    params = range(12)    try:         logger.warn("starting pool runs")         result = pool.map(worker, params)         pool.close()    except KeyboardInterrupt:        logger.warn("^C pressed")        pool.terminate()    finally:        pool.join()        logger.warn('done: {r}'.format(r = result))if __name__ == '__main__':    main()

运行脚本将产生:

% test.py[WARNING/MainProcess] starting pool runs[WARNING/PoolWorker-1] Running worker(0)[WARNING/PoolWorker-2] Running worker(1)[WARNING/PoolWorker-3] Running worker(2)[WARNING/PoolWorker-4] Running worker(3)

在这里按Ctrl-C;每个工人都设置

terminating
事件。我们确实只需要设置它,但是尽管效率很低,但它仍然有效。

  C-c C-c[WARNING/PoolWorker-4] terminating is set[WARNING/PoolWorker-2] terminating is set[WARNING/PoolWorker-3] terminating is set[WARNING/PoolWorker-1] terminating is set

现在,所有其他排队的任务

pool.map
都将运行:

[WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-3] got the message... we're terminating!

最终,主要过程到达

except KeyboardInterrupt
块。

[WARNING/MainProcess] ^C pressed[WARNING/MainProcess] done: []


欢迎分享,转载请注明来源:内存溢出

原文地址: http://outofmemory.cn/zaji/5649199.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-12-16
下一篇 2022-12-16

发表评论

登录后才能评论

评论列表(0条)

保存