From: Prof Brian Ripley <ripley@stats.ox.ac.uk>
What did your actual application do?  This seems a very strange thing 
to do, and the segfault is in trying to construct the traceback.

Only by using do.call on the object (and not even by name) do I get 
this error.  E.g.

> `[.data.frame`(1:10, 3)
Error in NextMethod("[") : object not specified
> do.call("[.data.frame", list(1:10, 3))
Error in NextMethod("[") : object not specified

are fine.

Obviously it would be nice to fix this, but I'd like to understand the 
actual circumstances: there is more to it than the subject line.

On Thu, 29 Jan 2009, brechbuehler@gmail.com wrote:

> Full_Name: Christian Brechbuehler
> Version: 2.7.2, 2.8.1
> OS: linux-gnu
> Submission from: (NULL) (24.128.51.18)
>
>
> Calling [.data.frame on an object that's not a data frame, specifically 1:10,
> causes segmentation fault.
>
> Context
> =======
> We can subscript with a number of different notations:
>
>   > (1:10)[3]
>   [1] 3
>   > do.call(get("[",pos="package:base"),list(1:10,3))
>   [1] 3
>   > do.call(get("[.numeric_version",pos="package:base"),list(1:10,3))
>   [1] 3
>
> Problem
> =======
> If we mistakenly believe the object is a data frame (as we did in a much more
> complicated real situation), this happens:
>
>   > do.call(get("[.data.frame",pos="package:base"),list(1:10,3))
>   Error in NextMethod("[") :
>     no calling generic was found: was a method called directly?
>
>    *** caught segfault ***
>   address (nil), cause 'unknown'
>
>   Process R:2 segmentation fault (core dumped) at Thu Jan 29 09:26:29 2009
>
> The Error message is appropriate.  But the segmentation fault is unexpected.
>
>
> Versions
> ========
> I reproduced the problem on R 2.7.2 and 2.8.1.  Details:
>
>> version
>               _
> platform       x86_64-unknown-linux-gnu
> arch           x86_64
> os             linux-gnu
> system         x86_64, linux-gnu
> status         Patched
> major          2
> minor          7.2
> year           2008
> month          09
> day            20
> svn rev        46776
> language       R
> version.string R version 2.7.2 Patched (2008-09-20 r46776)
>
> ==========================================================
>
>> version
>               _
> platform       x86_64-unknown-linux-gnu
> arch           x86_64
> os             linux-gnu
> system         x86_64, linux-gnu
> status         Patched
> major          2
> minor          8.1
> year           2009
> month          01
> day            26
> svn rev        47743
> language       R
> version.string R version 2.8.1 Patched (2009-01-26 r47743)
>
> ______________________________________________
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

From: Christian Brechbühler <brechbuehler@gmail.com>
On Thu, Jan 29, 2009 at 4:44 PM, Prof Brian Ripley <ripley@stats.ox.ac.uk>wrote:

> What did your actual application do?  This seems a very strange thing to
> do, and the segfault is in trying to construct the traceback.
>
> Only by using do.call on the object (and not even by name) do I get this
> error.  E.g.
>
>  `[.data.frame`(1:10, 3)
>>
> Error in NextMethod("[") : object not specified
>
>> do.call("[.data.frame", list(1:10, 3))
>>
> Error in NextMethod("[") : object not specified
>
> are fine.
>
> Obviously it would be nice to fix this, but I'd like to understand the
> actual circumstances: there is more to it than the subject line.


Yes, there is more.  For reporting the problem, we tried to pare it down to
a concise, self-contained test case.

My boss was debugging an issue in our R code.  We have our own "[...."
functions, because stock R drops names when subscripting.  To bypass our
now-suspect functions and get the "real" subscripting method, he used "get"
from package:base.  He was examining a large object, and believing it was a
data frame, chose "[.data.frame".  As it turns out, that object was not a
data frame, and he got an unexpected segfault.  I think it was a matrix.
But it doesn't matter -- a vector as in the test case will give the same.

We have since fixed the bug in our replacement subscripting function, so the
issue might not affect us any more.

Thanks,
/Christian


> On Thu, 29 Jan 2009, brechbuehler@gmail.com wrote:
>
>  Full_Name: Christian Brechbuehler
>> Version: 2.7.2, 2.8.1
>> OS: linux-gnu
>>
>> If we mistakenly believe the object is a data frame (as we did in a much
>> more
>> complicated real situation), this happens:
>>
>>  > do.call(get("[.data.frame",pos="package:base"),list(1:10,3))
>>  Error in NextMethod("[") :
>>    no calling generic was found: was a method called directly?
>>
>>   *** caught segfault ***
>>  address (nil), cause 'unknown'
>>
>>  Process R:2 segmentation fault (core dumped) at Thu Jan 29 09:26:29 2009
>>
>> The Error message is appropriate.  But the segmentation fault is
>> unexpected.
>>
>

	[[alternative HTML version deleted]]

NOTES:
 Error is in forming traceback

Audit (from Jitterbug):
Sun Feb  1 11:58:01 2009	ripley	changed notes

As Brian alludes to, a shorter way to trigger the seg.fault is

  do.call(`[.data.frame`, list(2,3))

the problem is still with us today,
and even if stemming from very improbable user code use, it's bug we should squish...

(all this is mostly to trigger a reminder to all of us)

I have spent an hour or two looking for this in the past.  Yes, we 
should squash it *if* we have enough resources -- I don't for now.

On Wed, 14 Jul 2010, r-bugs@r-project.org wrote:

> https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=13487
>
>
> Martin Maechler <maechler@stat.math.ethz.ch> changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                 CC|                            |maechler@stat.math.ethz.ch
>
>
>
>
> --- Comment #5 from Martin Maechler <maechler@stat.math.ethz.ch>  2010-07-14 06:01:55 ---
> As Brian alludes to, a shorter way to trigger the seg.fault is
>
>  do.call(`[.data.frame`, list(2,3))
>
> the problem is still with us today,
> and even if stemming from very improbable user code use, it's bug we should
> squish...
>
> (all this is mostly to trigger a reminder to all of us)
>
> -- 
> Configure bugmail: https://bugs.r-project.org/bugzilla3/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are the assignee for the bug.
>
> _______________________________________________
> R-core list: https://stat.ethz.ch/mailman/listinfo/r-core
>


-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

This bug has ruined my weekend :-), but I believe I have something that will help put it behind us. The summary below is terse. I have many more details and notes if necessary.

`[.data.frame` calls NextMethod("["), which calls do_nextmethod. do_nextmethod then gets the context (RCNTXT) of the function that called NextMethod from the context stack. do_nextmethod expects the CDR(RCNTXT->call) member to be a symbol, then uses R_LookupMethod to get the closure of the caller. When we execute do.call(`[.data.frame`, list(1,1)), the call in the context _is already a closure_. That is, CDR(RCNTXT->call) is a CLOSXP. do_nextmethod doesn't check this, and still passes CDR(RCNTXT->call) to R_LookupMethod, which (apparently) corrupts the CLOSXP. In particular, the FORMALS of the CLOSXP become corrupted. When do_nextmethod then calls error(), the corrupted CLOSXP is eventually deparsed, resulting in a call to args2buff, where the actual segfault occurs.

Supposing that the RCNTXT->call is constructed correctly, the next step is to decide the action of do_nextmethod when it encounters a CLOSXP in CDR(RCNTXT->call), rather than an SYMSXP. I corrected the immediate problem by making do_nextmethod aware that CDR(RCNTXT->call) might be the the calling CLOSXP, rather than a SYMBOL for it:

Index: src/main/objects.c
===================================================================
--- src/main/objects.c	(revision 52708)
+++ src/main/objects.c	(working copy)
@@ -547,7 +547,10 @@
     else if (defenv == R_UnboundValue) defenv = R_GlobalEnv;
 
     /* set up the arglist */
-    s = R_LookupMethod(CAR(cptr->call), env, callenv, defenv);
+    if (TYPEOF(CAR(cptr->call)) == CLOSXP)
+        s = CAR(cptr->call);
+    else 
+        s = R_LookupMethod(CAR(cptr->call), env, callenv, defenv);
     if (TYPEOF(s) == SYMSXP && s == R_UnboundValue)
 	error(_("no calling generic was found: was a method called directly?"));
     if (TYPEOF(s) != CLOSXP){ /* R_LookupMethod looked for a function */

This patch renders our offending function call (first of three below) equivalent to similar calls

> do.call(`[.data.frame`,list(1,1))
Error in NextMethod("[") : object not specified
> do.call("[.data.frame",list(1,1))
Error in NextMethod("[") : object not specified
> `[.data.frame`(1,1)
Error in NextMethod("[") : object not specified

> do.call(`[.data.frame`,list(data.frame(1),1))
  X1
1  1
> do.call("[.data.frame",list(data.frame(1),1))
  X1
1  1
> `[.data.frame`(data.frame(1),1)
  X1
1  1

However, I think it would be prudent, at this time, for someone more familiar with the S3 internals to review these findings. I would be grateful if any further discussion of this (if not here) were copied to my address.

Regards,
Matt Shotwell

Further illustrations:

This bug is not related specific to data frames. The segfault occurs when do.call is used with a function argument (as opposed to character string), which calls NextMethod. For example:

-------------------------
Example 1.
-------------------------
> do.call(function(x) NextMethod('foo'),list())
Error in NextMethod("foo") : 
  no calling generic was found: was a method called directly?

 *** caught segfault ***
address 0x3ad992, cause 'memory not mapped'
Segmentation fault


-------------------------
Example 2.
-------------------------
> f <- function(x) NextMethod('foo')
> do.call(f,list())
Error in NextMethod("foo") : 
  no calling generic was found: was a method called directly?

 *** caught segfault ***
address 0x3abf82, cause 'memory not mapped'
Segmentation fault

However, passing the string name of the function is OK, no segfault:

-------------------------
Example 3.
-------------------------
> f <- function(x) NextMethod('foo')
> do.call("f",list())
Error in NextMethod("foo") : object not specified

Also notice that supplying a function without an argument, but still calling NextMethod does not result in segfault. This reflects the fact that segfault occurs in deparse of FORMALS.

-------------------------
Example 4.
-------------------------
> f <- function() NextMethod('foo')
> do.call(f,list())
Error in NextMethod("foo") : 
  no calling generic was found: was a method called directly?


Applying the patch I mentioned before corrects this behavior.

-Matt Shotwell

(In reply to comment #8)

Thank you, Matt.  This is very convincing to me.
I'm currently testing your patch and will commit it unless I see problems in the standard checks.
Martin

(spam comment removed)