Bug 1662 - fisher.test FEXACT memory bug "should not occur"
Summary: fisher.test FEXACT memory bug "should not occur"
Status: NEW
Alias: None
Product: R
Classification: Unclassified
Component: Add-ons (show other bugs)
Version: R 3.1.1
Hardware: x86_64/x64/amd64 (64-bit) All
: P5 normal
Assignee: Jitterbug compatibility account
URL:
Depends on:
Blocks:
 
Reported: 2002-06-13 11:21 UTC by Jitterbug compatibility account
Modified: 2015-04-26 10:18 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jitterbug compatibility account 2002-06-13 11:21:50 UTC
From: Martin Maechler <maechler@stat.math.ethz.ch>
This is a bad bug as reported by Robin Hankin,
it is still in "R-patched" ...

##- From: Robin Hankin <r.hankin@auckland.ac.nz>
##- To: r-help@stat.math.ethz.ch
##- Subject: [R] possum sleeping: thanks and fisher.test() FEXACT error
##- Date: Thu, 13 Jun 2002 16:46:26 +1200

## .....

## Example slighlty modified (MM)

d4 <- matrix(c(0, 0, 0, 0, 0,  0, 3, 0, 1, 0,  0, 0, 0, 0, 0,
               1, 0, 0, 0, 0,  1, 0, 0, 2, 0,  0, 0, 1, 0, 0,
               0, 1, 0, 1, 0,  4, 0, 2, 0, 0,  0, 1, 0, 0, 0,  0, 0, 0, 0, 0,
               0, 1, 0, 0, 2,  0, 0, 0, 2, 2,  0, 1, 0, 0, 0,
               0, 0, 1, 1, 0,  0, 0, 0, 0, 0,  0, 1, 0, 0, 0,
               1, 0, 0, 0, 2,  0, 0, 0, 3, 0,  0, 0, 0, 0, 1,  0, 0, 0, 0, 0,
               2, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 1, 0, 1,
               0, 0, 0, 0, 2,  0, 0, 0, 0, 8,  0, 0, 0, 3, 0,
               0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 1,
               0, 0, 1, 0, 0,  0, 0, 0, 0, 0,  2, 0, 0, 1, 0,
               0, 2, 0, 0, 0,  0, 2, 0, 0, 1,  3, 0, 0, 0, 0,
               0, 0, 0, 0, 0,  0, 0, 0, 0, 1,  0, 0, 1, 0, 0,  4, 0, 0, 0, 0),
             nr=50)

fisher.test(d4)
##- Error in fisher.test(alldata[, 2:5]) : FEXACT error 30.
##- Stack length exceeded in f3xact.
##- This problem should not occur.

Comment 1 Jitterbug compatibility account 2002-06-13 11:56:55 UTC
From: Uwe Ligges <ligges@statistik.uni-dortmund.de>
maechler@stat.math.ethz.ch wrote:
> 
> This is a bad bug as reported by Robin Hankin,
> it is still in "R-patched" ...
> 
> ##- From: Robin Hankin <r.hankin@auckland.ac.nz>
> ##- To: r-help@stat.math.ethz.ch
> ##- Subject: [R] possum sleeping: thanks and fisher.test() FEXACT error
> ##- Date: Thu, 13 Jun 2002 16:46:26 +1200
> 
> ## .....
> 
> ## Example slighlty modified (MM)
> 
> d4 <- matrix(c(0, 0, 0, 0, 0,  0, 3, 0, 1, 0,  0, 0, 0, 0, 0,
>                1, 0, 0, 0, 0,  1, 0, 0, 2, 0,  0, 0, 1, 0, 0,
>                0, 1, 0, 1, 0,  4, 0, 2, 0, 0,  0, 1, 0, 0, 0,  0, 0, 0, 0, 0,
>                0, 1, 0, 0, 2,  0, 0, 0, 2, 2,  0, 1, 0, 0, 0,
>                0, 0, 1, 1, 0,  0, 0, 0, 0, 0,  0, 1, 0, 0, 0,
>                1, 0, 0, 0, 2,  0, 0, 0, 3, 0,  0, 0, 0, 0, 1,  0, 0, 0, 0, 0,
>                2, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 1, 0, 1,
>                0, 0, 0, 0, 2,  0, 0, 0, 0, 8,  0, 0, 0, 3, 0,
>                0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 1,
>                0, 0, 1, 0, 0,  0, 0, 0, 0, 0,  2, 0, 0, 1, 0,
>                0, 2, 0, 0, 0,  0, 2, 0, 0, 1,  3, 0, 0, 0, 0,
>                0, 0, 0, 0, 0,  0, 0, 0, 0, 1,  0, 0, 1, 0, 0,  4, 0, 0, 0, 0),
>              nr=50)
> 
> fisher.test(d4)
> ##- Error in fisher.test(alldata[, 2:5]) : FEXACT error 30.
> ##- Stack length exceeded in f3xact.
> ##- This problem should not occur.

Just for the record:
  repeat try(fisher.test(d4))
results in a crash after a few iterations (R-1.5.0 patched on WinNT
4.0).

Uwe Ligges
Comment 2 Jitterbug compatibility account 2002-06-13 15:15:53 UTC
From: Martin Maechler <maechler@stat.math.ethz.ch>
(CC'ed to R-bugs  ``for the record'')

>>>>> "BDR" == Prof Brian D Ripley <ripley@stats.ox.ac.uk> writes:

    BDR> On Thu, 13 Jun 2002, Martin Maechler wrote:
    >> >>>>> "MM" == Martin Maechler
    >> <maechler@stat.math.ethz.ch> writes:
    >> 
    >> >>>>> "BDR" == Brian D Ripley <ripley@stats.ox.ac.uk>
    >> writes:
    BDR> Martin, What makes this a `bad bug'?  Are you getting a
    BDR> seg fault?
    >>
    MM> yes, always in the half a dozen restarts I tried.
    >>  (in the mean time I had one case where it did not ..)
    >> 
    BDR> Like Uwe this works for me the first few times and I do
    BDR> then get a backtrace (in memory.c, so this is almost
    BDR> certainly an earlier overrun).
    >>
    MM> I see.  This has been different for me.
    >>
    BDR> I think we've had problems before with FEXACT
    BDR> incorrectly specifying the required sizes of its
    BDR> workspaces.
    >>  (I'm not so sure this is the problem here.)
    >> 
    >> I found that
    >> 
    >> fisher.test(d4[1:30,])
    >> 
    >> gives a direct segmentation fault (without any error
    >> message) for me.
    >> 

    BDR> I've had to compile up gdb 5.2 to get the correct
    BDR> information out of R on Linux compiled under gcc-3.1.
    BDR> That shows (on the original problem)

    BDR> Loaded symbols for
    BDR> /users/ripley/R/R-patched/library/ctest/libs/ctest.so
    BDR> #0 f3xact (nrow=0x40051074, irow=0x4005113c,
    BDR> ncol=0x40051000, icol=0x40051120, dlp=0x4041b110,
    BDR> mm=0x40050fc0, fact=0x4035f020, ico=0x4035f4dc,
    BDR> iro=0x4035f5a4, it=0x4035f66c, lb=0x4035f734,
    BDR> nr=0x4035f7fc, nt=0x4035f8c4, nu=0x4035f98c,
    BDR> itc=0x4035fa54, ist=0x40360094, stv=0x40364f18,
    BDR> alen=0x40365ba0, tol=0x4004f878) at
    BDR> /users/ripley/R/cvs/R-patched/src/library/ctest/src/fexact.c:1125
    BDR> 1125 ist[itp] = -1; (gdb) print itp $1 = -3544

    BDR> The same with your variant (except the value of itp).

Okay, I found a much smaller table  --- and a different place
for the seg.fault :
> fisher.test(cbind(0,c(0,0,1)))

Program received signal SIGSEGV, Segmentation fault.
f2xact (nrow=0x8a12ec0, ncol=0x8a12ee0, table=0x8b7bb38, ldtabl=0x8a12f00, 
    expect=0x89dad68, percnt=0x89dad90, emin=0x89dadb8, prt=0x89dade0, 
    pre=0x89dae08, fact=0x403b3020, ico=0x403b302c, iro=0x403b3038, 
    kyy=0x403b3044, idif=0x403b3050, irn=0x403b3058, key=0x403b49d4, 
    ldkey=0xbfffc398, ipoin=0x403b5d44, stp=0x403b70b0, ldstp=0xbfffc39c, 
    ifrq=0x403ffef4, dlp=0x4046d450, dsp=0x4046fb30, tm=0x40472210, 
    key2=0x404748f4, iwk=0x403b3060, rwk=0x403b3d30)
    at ../../../../../R/src/library/ctest/src/fexact.c:524
524		obs += fact[ico[j]] - dd;

Martin


Comment 3 Jitterbug compatibility account 2002-06-13 16:14:46 UTC
From: Prof Brian D Ripley <ripley@stats.ox.ac.uk>
This occurred because integer overflow was giving negative keys.

Append the following line at line 1083 of fexact.c

      if (ipn < 1) ipn += ldst; /* because key might be negative */


> fisher.test(d4)

      Fisher's Exact Test for Count Data

data:  d4
p-value = < 2.2e-16
alternative hypothesis: two.sided



On Thu, 13 Jun 2002 maechler@stat.math.ethz.ch wrote:

> This is a bad bug as reported by Robin Hankin,
> it is still in "R-patched" ...
>
> ##- From: Robin Hankin <r.hankin@auckland.ac.nz>
> ##- To: r-help@stat.math.ethz.ch
> ##- Subject: [R] possum sleeping: thanks and fisher.test() FEXACT error
> ##- Date: Thu, 13 Jun 2002 16:46:26 +1200
>
> ## .....
>
> ## Example slighlty modified (MM)
>
> d4 <- matrix(c(0, 0, 0, 0, 0,  0, 3, 0, 1, 0,  0, 0, 0, 0, 0,
>                1, 0, 0, 0, 0,  1, 0, 0, 2, 0,  0, 0, 1, 0, 0,
>                0, 1, 0, 1, 0,  4, 0, 2, 0, 0,  0, 1, 0, 0, 0,  0, 0, 0, 0, 0,
>                0, 1, 0, 0, 2,  0, 0, 0, 2, 2,  0, 1, 0, 0, 0,
>                0, 0, 1, 1, 0,  0, 0, 0, 0, 0,  0, 1, 0, 0, 0,
>                1, 0, 0, 0, 2,  0, 0, 0, 3, 0,  0, 0, 0, 0, 1,  0, 0, 0, 0, 0,
>                2, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 1, 0, 1,
>                0, 0, 0, 0, 2,  0, 0, 0, 0, 8,  0, 0, 0, 3, 0,
>                0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 0,  0, 0, 0, 0, 1,
>                0, 0, 1, 0, 0,  0, 0, 0, 0, 0,  2, 0, 0, 1, 0,
>                0, 2, 0, 0, 0,  0, 2, 0, 0, 1,  3, 0, 0, 0, 0,
>                0, 0, 0, 0, 0,  0, 0, 0, 0, 1,  0, 0, 1, 0, 0,  4, 0, 0, 0, 0),
>              nr=50)
>
> fisher.test(d4)
> ##- Error in fisher.test(alldata[, 2:5]) : FEXACT error 30.
> ##- Stack length exceeded in f3xact.
> ##- This problem should not occur.
>
>
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-devel-request@stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Comment 4 Jitterbug compatibility account 2002-06-13 20:56:20 UTC
From: ripley@stats.ox.ac.uk
On Thu, 13 Jun 2002, Martin Maechler wrote:

> Okay, I found a much smaller table  --- and a different place
> for the seg.fault :
> > fisher.test(cbind(0,c(0,0,1)))


That one is easy:  You (MM) have calculated fact[2], but fact is only of
length 2 (for values 0 and 1), so ico gets overwritten.

I now suspect that my fix for the original bug is merely a fix for a
symptom, so am going to replace it by an error message.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

Comment 5 Jitterbug compatibility account 2002-06-13 22:15:00 UTC
NOTES:
 The supplementary (table of sum one) is fixed for 1.5.1.
Detection code for the first problem has been added to 1.5.1 which will stop
the crash, but the underlying cause is still open.
Comment 6 Jitterbug compatibility account 2002-06-14 00:15:42 UTC
Audit (from Jitterbug):
Thu Jun 13 19:15:42 2002	ripley	changed notes
Thu Jun 13 19:15:42 2002	ripley	foobar
Thu Jun 13 19:15:42 2002	ripley	moved from incoming to Add-ons
Comment 7 Russell 2014-09-11 15:37:33 UTC
This error still exists as of R-3.1.1.

R version 3.1.1 (2014-07-10) -- "Sock it to Me"
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

> tab <- structure(list(V1 = c(1, 0, 0, 0, 0, 0), V2 = c(323, 4, 1, 0,
+ 0, 22), V3 = c(3, 0, 0, 0, 0, 1), V4 = c(2, 0, 1, 0, 1, 3), V5 = c(1,
+ 0, 0, 0, 0, 4), V6 = c(1, 0, 0, 0, 0, 0), V7 = c(0, 0, 0, 1,
+ 0, 1), V8 = c(96, 0, 0, 0, 0, 2)), .Names = c("V1", "V2", "V3",
+ "V4", "V5", "V6", "V7", "V8"), class = "data.frame", row.names = c(NA,
+ -6L))
> fisher.test(tab)
Error in fisher.test(tab) : Bug in FEXACT: gave negative key

Increasing the workspace appears to have no effect.
Comment 8 Oliver Keyes 2015-04-26 10:18:49 UTC
Also still around as of 3.2.0, using the code reported by Robin via Martin in 2002.