Bug 15471 - parallel::stopCluster(...) leaves zombie processes behind for FORK clusters
Summary: parallel::stopCluster(...) leaves zombie processes behind for FORK clusters
Status: CLOSED FIXED
Alias: None
Product: R
Classification: Unclassified
Component: Low-level (show other bugs)
Version: R 3.0.1
Hardware: All Linux
: P5 normal
Assignee: R-core
URL:
Depends on:
Blocks:
 
Reported: 2013-09-24 08:29 UTC by computermacgyver
Modified: 2018-01-10 09:39 UTC (History)
1 user (show)

See Also:


Attachments
updated mclapply function (5.64 KB, text/plain)
2013-10-28 20:25 UTC, dreiss.isb
Details

Note You need to log in before you can comment on or make changes to this bug.
Description computermacgyver 2013-09-24 08:29:38 UTC
Executing the following code leaves 4 processes in the process table as zombies. These are created because the exit status of the child processes is never queried.

library(parallel)
library(doParallel)

cl <- makeForkCluster(4)
registerDoParallel(cl, cores = NUM.CORES)
#...do something
stopCluster(cl)

These zombie processes are minor as they have released all resources, but it seems that either:
(a) the stopCluster(...) call should check the status of the child processes
or 
(b) the makeForkCluster(...) call should explicitly ignore SIGCHLD by setting its handler to SIG_IGN.

As a workaround, (a) can be implemented by calling the wait() method of the fork package. e.g.,
library(fork)
wait()
wait()
wait()
wait()
Removes the four zombie processes from the process table.
Comment 1 computermacgyver 2013-09-24 08:30:29 UTC
Note that this is reported by other users on StackOverflow as well

http://stackoverflow.com/questions/9486952/remove-zombie-processes-using-parallel-package
Comment 2 dreiss.isb 2013-10-28 20:25:16 UTC
Created attachment 1501 [details]
updated mclapply function
Comment 3 dreiss.isb 2013-10-28 20:26:55 UTC
Comment on attachment 1501 [details]
updated mclapply function

replacing "collect(children(jobs)...)" with "collect(jobs...) on line 17 fixes this issue for me.
Comment 4 Tomas Kalibera 2018-01-10 09:39:05 UTC
The problem of zombies left behind by forking in "parallel" has been addressed in R-devel in 73775. 73775 very likely resolves also the issue reported here, yet it can't be easily tested as this report did not include a complete reproducible example (and the linked SO example did not leave a zombie on my system even in older versions of R). If you still see a zombie left behind in R newer than 73775 (but note that non-detached jobs have to be collected), please submit a new bug report with a complete, self-contained reproducible example that does not depend on external packages.