Bug 17135 - Event loop deadlock + libcurl problem
Summary: Event loop deadlock + libcurl problem
Alias: None
Product: R
Classification: Unclassified
Component: I/O (show other bugs)
Version: R 3.3.*
Hardware: Other Other
: P5 enhancement
Assignee: R-core
Depends on:
Reported: 2016-08-25 10:57 UTC by Jeroen Ooms
Modified: 2016-08-31 13:28 UTC (History)
2 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Jeroen Ooms 2016-08-25 10:57:16 UTC
A complex problem: On Unix, R deadlocks when it connects to the local httpd (either help-server or httpuv), even when the client calls R_CheckUserInterrupt occasionally to let the server process events. 

On Windows, this does not happen for clients that call R_CheckUserInterrupt. However unfortunately the default 'libcurl' download method does not seem to do this, so we still freeze.

The following example freezes R on all platforms:

	port <- tools::startDynamicHelp(NA)
	u <- paste0("", port)

The following example freezes on Unix but not on Windows:

	port <- tools::startDynamicHelp(NA)
	u <- paste0("", port)

	# non blocking clients
	readLines(url(u, method = "internal"))

The 'blocking' parameter in base::url() does not make any difference.
Comment 1 Luke Tierney 2016-08-25 17:56:17 UTC
I'll leave this open for someone more familiar with Windows and the http server to finalize. But it does not seem reasonable to me to expect this to work since R is single-threaded. I suspect you will get a deadlock on Windows as well if the http response writes enough data to fill the buffer and has to block waiting for a read. Different behavior on Windows and Unix reflects implementation differences; I don't see that as a bug. Getting this sort of thing to work without multiple threads doesn't seem realistic. R could refuse to connect to itself, though detecting that reliably looks like a fair bit of work.
Comment 2 Mikko Korpela 2016-08-31 13:28:20 UTC
When I run the second example, interrupting with Ctrl+C after each stalled readLines(), I get a segfault. Here is the output when debugging with valgrind, R-devel r71178. The version of the curl package was 1.2 and the version of libcurl was 7.50.1. I have no idea if the problem is in R, the curl package, or elsewhere.

    ==10957== Memcheck, a memory error detector
    ==10957== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
    ==10957== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
    ==10957== Command: /home/mvkorpel/root_R-devel-r71178/lib/R/bin/exec/R --vanilla
    R Under development (unstable) (2016-08-31 r71178) -- "Unsuffered Consequences"
    Copyright (C) 2016 The R Foundation for Statistical Computing
    Platform: x86_64-pc-linux-gnu (64-bit)
    R is free software and comes with ABSOLUTELY NO WARRANTY.
    You are welcome to redistribute it under certain conditions.
    Type 'license()' or 'licence()' for distribution details.
      Natural language support but running in an English locale
    R is a collaborative project with many contributors.
    Type 'contributors()' for more information and
    'citation()' on how to cite R or R packages in publications.
    Type 'demo()' for some demos, 'help()' for on-line help, or
    'help.start()' for an HTML browser interface to help.
    Type 'q()' to quit R.
    > port <- tools::startDynamicHelp(NA)
    starting httpd help server ... done
    > u <- paste0("", port)
    > readLines(url(u, method = "internal"))
    > readLines(curl::curl(u))
    ^C==10957== Jump to the invalid address stated on the next line
    ==10957==    at 0x14000000: ???
    ==10957==    by 0x59DF28: R_SelectEx (sys-std.c:125)
    ==10957==    by 0xC77FCFF: ???
    ==10957==    by 0xC77FC9B: ???
    ==10957==    by 0x1356469B: fetchdata (curl.c:113)
    ==10957==    by 0x1356485E: rcurl_open (curl.c:203)
    ==10957==    by 0x466857: do_readLines (connections.c:3596)
    ==10957==    by 0x4B0D5B: bcEval (eval.c:5696)
    ==10957==    by 0x4BF56F: Rf_eval (eval.c:616)
    ==10957==    by 0x4C0D66: Rf_applyClosure (eval.c:1149)
    ==10957==    by 0x4BF6C8: Rf_eval (eval.c:732)
    ==10957==    by 0x4E54C0: Rf_ReplIteration (main.c:258)
    ==10957==  Address 0x14000000 is not stack'd, malloc'd or (recently) free'd
     *** caught segfault ***
    address 0x14000000, cause 'memory not mapped'
     1: readLines(curl::curl(u))

A similar error also occurred with R 3.3.1 patched r71178, libcurl 7.35.0.

    ==11011== Use of uninitialised value of size 8
    ==11011==    at 0x59B517: R_SelectEx (sys-std.c:129)
    ==11011==    by 0x55ACD4F: ??? (in /lib/x86_64-linux-gnu/libreadline.so.6.3)
    ==11011==    by 0xF09EB1F: ???
    ==11011==    by 0x55ACD4F: ??? (in /lib/x86_64-linux-gnu/libreadline.so.6.3)
    ==11011==    by 0xDCDA24F: ???
    ==11011==    by 0xDCDA367: ???
    ==11011==    by 0xC712F77: ???
    ==11011==    by 0xDC40071: ???
    ==11011==    by 0xEF0AB9DC2D340CFF: ???
    ==11011==    by 0xA78FE1F: ???
    ==11011==    by 0xF00F40F: ???
    ==11011==    by 0xDCDA367: ???
    ==11011== Invalid write of size 4
    ==11011==    at 0x59B39D: readline_handler (sys-std.c:629)
    ==11011==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
     *** caught segfault ***
    address (nil), cause 'memory not mapped'
     1: readLines(curl::curl(u))