Bug 14866 - Bug when updating recommended packages in R_HOME/library without root access (with patch)
Bug when updating recommended packages in R_HOME/library without root access ...
Status: CLOSED FIXED
Product: R
Classification: Unclassified
Component: Low-level
R 2.14.1 patched
All Linux
: P4 normal
Assigned To: R-core
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-30 18:52 UTC by Denis Dupeyron
Modified: 2012-04-04 10:58 UTC (History)
2 users (show)

See Also:


Attachments
R-2.14.2-library-writability.patch (1.99 KB, application/octet-stream)
2012-03-30 18:52 UTC, Denis Dupeyron
Details
R-2.14.2-prune-package-update.patch (1.15 KB, patch)
2012-03-30 19:23 UTC, Denis Dupeyron
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Denis Dupeyron 2012-03-30 18:52:30 UTC
Created attachment 1285 [details]
R-2.14.2-library-writability.patch

Here's what I think is bug and the corresponding patch about handling of the writability of libraries when updating packages. The situation is the following:

User does not have root access on her Linux workstation, she thus does not have write permission to where the base and recommended packages are installed (i.e. R_HOME/library). She however needs to update some of the recommended default packages (installed in R_HOME/library by the distribution's package manager) to versions more recent than those in the latest release of R, due to some other non-base package requiring it. The following is done as an example on a fresh install of R 2.14.1 for a user who doesn't (yet) have a personal library in her HOME directory.

> update.packages()
--- Please select a CRAN mirror for use in this session ---
Loading Tcl/Tk interface ... done
boot :
 Version 1.3-3 installed in /usr/lib64/R/library
 Version 1.3-4 available at http://ftp.osuosl.org/pub/cran
Update (y/N/c)?  y
[...snip...]
survival :
 Version 2.36-10 installed in /usr/lib64/R/library
 Version 2.36-12 available at http://ftp.osuosl.org/pub/cran
Update (y/N/c)?  y
Warning in install.packages(update[instlib == l, "Package"], l,
contriburl = contriburl,  :
 'lib = "/usr/lib64/R/library"' is not writable
Would you like to create a personal library
~/R/x86_64-pc-linux-gnu-library/2.14
to install packages into?  (y/n) y

The user is offered to create a personal library in her HOME directory and the update of base packages works. Once that is done the versions of base packages which will be automatically picked up by library() or require() are the updated ones in her personal library, and everything will work as expected.

If, however, the user already had a personal library at ~/R/x86_64-pc-linux-gnu-library/2.14, here is what the end of the same update session would look like:

Warning in install.packages(update[instlib == l, "Package"], l,
contriburl = contriburl,  :
 'lib = "/usr/lib64/R/library"' is not writable
Error in install.packages(update[instlib == l, "Package"], l,
contriburl = contriburl,  :
 unable to install packages

The user is no longer offered to create a personal library and the update fails. The attached patch fixes this issue by offering to use a personal library when the library the user needs to install to isn't writable. Then, if the user doesn't have a personal library, she is offered to create one as the current code does.

Here is what happens with the patch and without a pre-existing personal library:

Warning in install.packages(update[instlib == l, "Package"], l,
contriburl = contriburl,  :
 'lib = "/usr/lib64/R/library"' is not writable
Would you like to use a personal library instead?  (y/n) y
Would you like to create a personal library
~/R/x86_64-pc-linux-gnu-library/2.14
to install packages into?  (y/n) y

And here is the same thing with a pre-existing personal library:

Warning in install.packages(update[instlib == l, "Package"], l,
contriburl = contriburl,  :
 'lib = "/usr/lib64/R/library"' is not writable
Would you like to use a personal library instead?  (y/n) y

Once the personal library is created the installation can proceed as usual.

Note 1. The non-interactive behavior isn't changed by this patch. Compatibility with existing code should be maintained.

Note 2. This patch has been in use in Gentoo for almost 4 weeks (see [1]) and no bug has been reported yet.

Note 3. I am a Gentoo developer and this patch is now used in Gentoo but as far as I know the bug should occur on all Linux distributions and the attached patch is distribution-agnostic.

Note 4. Fixing this behavior uncovers another bug in update.packages() which is the subject of my next patch.

Thanks,
Denis Dupeyron.

[1] http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-lang/R/files/R-2.14.2-library-writability.patch?revision=1.1&view=markup
Comment 1 Denis Dupeyron 2012-03-30 19:23:36 UTC
Created attachment 1286 [details]
R-2.14.2-prune-package-update.patch

Here is the promised second patch and why it is needed. I am using the same bug entry as before since both issues are related.

The situation occurs when a recommended package, originally installed in R_HOME/library by the distribution package manager, is at some point updated in the user's personal library because the user does not have root access on her workstation. When running update.packages() later to check for newly updatable packages, R will keep wanting to update packages which have already been updated in the user's personal library.

The attached patch lets update.packages() automatically skip packages detected as updatable but for which a newer version is available in the search path and when this newer version is the one which is going to be automatically picked up by library() or require(). When skipping a package update the new code will produce a warning.

Here is an example session using R-2.14.2:

> update.packages()
--- Please select a CRAN mirror for use in this session ---
Loading Tcl/Tk interface ... done
Warning: package 'lattice' in library '/usr/lib64/R/library' will not be updated
Warning: package 'Matrix' in library '/usr/lib64/R/library' will not be updated
Warning: package 'rpart' in library '/usr/lib64/R/library' will not be updated

In this example there are already installed and newer versions of packages lattice, Matrix and rpart, so these updates will be skipped.

It is still possible to force the update of these three packages like this:

> update.packages(lib.loc = "/usr/lib64/R/library")
lattice :
 Version 0.20-0 installed in /usr/lib64/R/library 
 Version 0.20-6 available at http://ftp.osuosl.org/pub/cran
Update (y/N/c)?  y
[...]

If, like in this example, the library in which the packages updates are forced is not user-writable, then the user will be offered to do these updates in her personal library (see previous patch).

This patch has also been in use in Gentoo for almost 4 weeks (see [1]) and
no bug has been reported yet. This is again a distribution-agnostic patch which should be useful to all linux distributions.

Thanks,
Denis Dupeyron.

[1] http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-lang/R/files/R-2.14.2-prune-package-update.patch?revision=1.1&view=markup
Comment 2 Martin Maechler 2012-04-03 16:28:37 UTC
Your analysis and patch sound sound :-)
So I've applied a version of your patch  to R-devel  (to be R 2.16.0).
If no problems are found in a while, it should be ported to
R-2.15.0 patched
Comment 3 Martin Maechler 2012-04-03 17:01:58 UTC
(In reply to comment #2)
> Your analysis and patch sound sound :-)
> So I've applied a version of your patch  to R-devel  (to be R 2.16.0).
> If no problems are found in a while, it should be ported to
> R-2.15.0 patched

This applies to the first patch only, for now.
I'm doing the 2nd one as well,
but no more today.

Martin
Comment 4 Denis Dupeyron 2012-04-03 18:22:23 UTC
(In reply to comment #2)
> Your analysis and patch sound sound :-)
> So I've applied a version of your patch  to R-devel  (to be R 2.16.0).
> If no problems are found in a while, it should be ported to
> R-2.15.0 patched

Thanks. I forgot to add that just after I posted this I saw R 2.15.0 was released. So since I maintain R in Gentoo I updated it and the patches apply cleanly. So far we still don't have any complaints in our bugzilla about these patches and R 2.15.0.

I'm not an R user myself but one of my coworkers uses it heavily. He's been using these patches for more than a month and seems to be pretty happy about them.

Denis.