returnn.util.multi_proc_non_daemonic_spawn

A multiprocessing context where proc.daemon is always False, and setting it to True is ignored. All processes will still be cleaned up though. For cleaning up, we use SIGINT (instead of the standard SIGTERM) such that the processes can do proper cleanup themselves.

See NonDaemonicSpawnProcess and NonDaemonicSpawnContext.

This is when you don’t want to use the default multiprocessing fork start method, but the spawn start method, but you also want that the started processes are never daemonic.

References:

https://github.com/rwth-i6/returnn/issues/1494 https://github.com/rwth-i6/returnn/issues/1495 https://github.com/pytorch/pytorch/issues/15950

class returnn.util.multi_proc_non_daemonic_spawn.NonDaemonicSpawnProcess(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

This process is always non-daemon, even if proc.daemon=True is executed (like https://stackoverflow.com/a/8963618/133374).

Still, we make sure that the proc is cleaned up at exit. Instead of using SIGTERM as Python does for normal daemonic threads, we use SIGINT, to allow the subprocess to do proper cleanup, e.g. like cleaning up sub-sub procs. So the sub proc would not leave its children orphaned. Note, if SIGINT does nothing on the subproc, this will hang.

property daemon

Return whether process is a daemon

pre_init_func: Callable[[], None] | None = None

start()

terminate()

kill()

join(timeout=None)

close()

class returnn.util.multi_proc_non_daemonic_spawn.NonDaemonicSpawnContext(*, process_pre_init_func: Callable[[], None] | None = None)

Spawn start methods, where all procs are non-daemonic.

Process(*args, **kwargs)

create a new process