Bug 17122 - mclapply drops warnings when options(warn=0)
Summary: mclapply drops warnings when options(warn=0)
Status: UNCONFIRMED
Alias: None
Product: R
Classification: Unclassified
Component: Low-level (show other bugs)
Version: R-devel (trunk)
Hardware: All Linux
: P5 enhancement
Assignee: R-core
URL:
Depends on:
Blocks:
 
Reported: 2016-07-20 19:56 UTC by Bill Denney
Modified: 2017-03-24 02:41 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bill Denney 2016-07-20 19:56:58 UTC
As found in the PKNCA package and in this StackOverflow question [1], mclapply will drop warnings if they are not set to immediate (options(warn=1)) or an error (options(warn=2)).

The simplest test case is given on StackOverflow and reproduced here:

library(parallel) 
options(warn=0)
mclapply(1:3, function(x) warning(x))
# No warnings

options(warn=1)
mclapply(1:3, function(x) warning(x))
# 3 warnings

[1] http://stackoverflow.com/questions/21486658/warnings-suppressed-with-mclapply-in-r
Comment 1 Bill Denney 2017-02-21 22:56:57 UTC
Updated as the bug is also confirmed in the latest released version of R.
Comment 2 Tomas Kalibera 2017-03-17 16:03:26 UTC
I think this is a feature request, not a bug. The documentation does not suggest that warnings would be propagated (as opposed to errors). Specifically, mc.silent (mentioned in the SO posts) is not related to warnings. To propagate warnings, one would have to collect them in child processes, copy back to master, and then somehow report - and this is not implemented at the moment.
Comment 3 Bill Denney 2017-03-17 16:43:59 UTC
(In reply to Tomas Kalibera from comment #2)
> I think this is a feature request, not a bug. The documentation does not
> suggest that warnings would be propagated (as opposed to errors).
> Specifically, mc.silent (mentioned in the SO posts) is not related to
> warnings. To propagate warnings, one would have to collect them in child
> processes, copy back to master, and then somehow report - and this is not
> implemented at the moment.

In my mind bug is that warnings are dropped without any documented notification to users.  The current default behavior is that if options(warn=0) calls to mclapply are as though options(warn=-1) is set.  To me (and to the SO questioner) the expected behavior is more like converting options(warn=0) to options(warn=1).

The documentation for mclapply indicates "mclapply is a parallelized version of lapply..."  That suggests to me that it is a drop-in replacement for mclapply.  On Windows, mclapply is simply a call to lapply dropping extraneous arguments.  That causes warnings to be handled platform-specifically within the call to mclapply.
Comment 4 Simon Urbanek 2017-03-18 12:49:34 UTC
Sorry, but if you actually read the documentation it is very adamant about the fact that this is NOT a drop-in replacement for lapply() due to the parallel nature of execution. That should be also clear by definition due to the parallelization of side-effects. If you want warning propagated you can send them back - just as the errors are sent back by default.
Comment 5 Bill Denney 2017-03-24 02:41:42 UTC
In the below response, I'm not trying to be argumentative, and I'm sorry if it comes across that way.  I'm trying to understand how mclapply works and how I can learn about the differences from lapply.

(In reply to Simon Urbanek from comment #4)
> Sorry, but if you actually read the documentation it is very adamant about
> the fact that this is NOT a drop-in replacement for lapply() due to the
> parallel nature of execution. That should be also clear by definition due to
> the parallelization of side-effects.

The documentation on Windows (my primary development machine) makes no such notes:  "These are simple serial versions of mclapply, mcmapply, mcMap and pvec for Windows where forking is not available."

When I just checked the documentation online, it notes "mclapply is a parallelized version of lapply".  That is not precisely saying "drop-in replacement", but it is suggestive to the naive reader (me).

These are suggestive that similar behavior could be observed for lapply with the exceptions that are noted below there in the documentation.

When reading the documentation, these are the differences that are apparent between mclapply and lapply:
* It runs in parallel (I'd assumed that the parallelization of side effects included effects external to the computation, race conditions, writing to files, etc.).
    * I would assume that the order of warnings may be different between runs, but I see no indication that the existence of warnings is changed.
* mc.silent mutes stdout unless set to FALSE.  I would have assumed that warnings go to stderr, so I thought this didn't apply.
* Random number handling is different (and therefore may not be reproducible between runs as a parallel race condition item).

I don't see anything in that list that is obvious to the intermediate R user to indicate warnings would be silently dropped.  Can you point me to what I'm missing?

> If you want warning propagated you can
> send them back - just as the errors are sent back by default.

What is the preferred way to propagate them back?

I came across this issue due to the differences between Windows and Linux handling of warnings within mclapply.  If this is considered a feature request my main request would be:  Could the platforms be made to handle them the same way?  (This could be as simple as changing options(warn=-1) just before the lapply call within mclapply on Windows if options("warn") == 0 and resetting it afterward.)

My secondary request would be:  Could warnings be handled within mclapply the same as they are within other code execution in the case of options(warn=0)?