Bug 16362 - Repeated CRAN mirror queries, and wish to install tar.gz files (was: R 3.2.0 and 3.2.0 patched package update issue)
Summary: Repeated CRAN mirror queries, and wish to install tar.gz files (was: R 3.2.0 ...
Alias: None
Product: R
Classification: Unclassified
Component: Windows GUI / Window specific (show other bugs)
Version: R 3.2.0
Hardware: x86_64/x64/amd64 (64-bit) Windows 64-bit
: P5 normal
Assignee: R-core
Depends on:
Reported: 2015-05-04 01:28 UTC by tjlawrence
Modified: 2015-05-13 13:30 UTC (History)
1 user (show)

See Also:

Output from installing/updating packages (4.40 KB, text/plain)
2015-05-05 04:47 UTC, tjlawrence

Note You need to log in before you can comment on or make changes to this bug.
Description tjlawrence 2015-05-04 01:28:34 UTC
The simple description of the bug is that in R3.2.0 and R3.2.0 patched, the standard Windows interface will download the tar file of a package when the zip file is not available.  I would prefer it only download zip files.

The detailed description is this: I use the standard Windows interface for R and use update.packages to update my packages. In R3.1.2 and earlier versions, the interface asks me to select a CRAN mirror, then once I do that tells me what packages I can update, I typically selects them all and then it downloads the zip files.

However in R3.2.0 and 3.2.0 patched, there are numerous issues.  It seems to be that the Windows interface decides to download the tar file of a package if the zip file isn't yet available.  When R decides to download a tar file, I go through a process in which I told the source and binary versions are different, and do I wish to proceed.  This question points out that C/C++/Fortran compiling may be required and that does happen, generally giving a lot of output I don't understand and can't act on.  On a couple of occasions, the output has included warnings.  (Presumably if the compiling does that, the package shouldn't be available for download!)

During this process I have to select my CRAN mirror 3 or 4 times.

After downloading R3.2.0 patched, I did as usual and tried to install all the packages that I had previously installed on earlier versions of R.  (I make a point of saving all the downloaded zip files somewhere I can get to them easily - but when R3.2.0 started downloading tar files I made a point of keeping the most recent zip file even though they were for earlier versions of the relevant packages).  The installer generated an error every time it came across a tar file.  As for each tar file I had retained the zip file for the previous version of the package, I used the package updater and this time all the zip files except one got downloaded.  I had to go through the same process as described above to download the tar file instead.  It was after doing that that I searched the R website and found for that package, the tar file was available but the zip file wasn't available yet.  I hadn't yet uninstalled R3.2.0 but I ran the package updater on it and it did not download and install zip files for packages for which it had previously downloaded the tar file.  Given the package installer from local zip files can't handle tar files either the installer or better still the package updater should be fixed.
Comment 1 Duncan Murdoch 2015-05-04 09:03:18 UTC
The change to allow "both" as the package type to download is in the NEWS.  One way to stop this is to set options(pkgType = "binary").

I won't close this yet, as your description of what happens (multiple requests for the mirror) sounds like a bug, and the ability to install source packages (*.tar.gz) makes sense.  

Please give a reproducible example, i.e. tell us which package leads to the multiple requests.
Comment 2 tjlawrence 2015-05-04 09:46:10 UTC
The current version gpclib package is the best example I can give.

There were about eight or so packages that triggered this issue in R3.2.0.  By the time I installed R3.2.0 patched, I found that of them only gpclib's zip file was not yet available.  Thus, having installed the older version zip files of all eight or so packages into R3.2.0 patched from my hard drive, and made a call to the package updater, only gpclib triggered the issues that all eight packages did in R3.2.0.
Comment 3 Duncan Murdoch 2015-05-04 11:26:34 UTC
The repeated requests to set the CRAN mirror are now fixed.

I've left this open as a wishlist item, to allow installation in the Windows GUI of a local tar.gz file.

I don't see anything else in the original report that we can deal with.
Comment 4 Duncan Murdoch 2015-05-04 11:46:35 UTC
The Windows GUI can install .tar.gz files now.  It was easier than I expected, but users who don't understand the error messages may be confused by the new option, so I've only put it in R-devel for now.

Please try it out, and if it makes sense to you, I'll put it in R-patched as well.

You can download R-devel from http://cran.r-project.org/bin/windows/base/rdevel.html.  Wait until it shows "r68319" or later (probably available tomorrow).
Comment 5 tjlawrence 2015-05-05 04:47:27 UTC
Created attachment 1823 [details]
Output from installing/updating packages

This is the output from me testing the bug repair.  I installed the abind and stabledist zip files from my computer, and the gpclib tar file too, before updating the packages and installing tar files of the newer versions of both abind and gpclib.  As you can see, the same issues arise as before regarding multiple requests to choose a CRAN mirror, and generating a large amount of output as a tar folder is installed.  The real issue with the output is that much of it appears to be associated with compiling rather than installing the package.

Also, when I initially tried to install multiple packages, which triggered the error messages in the first few lines of output.  I don't know if this is another bug or is a standard facet of developmental versions of R.

Having said that, it's great that the Windows GUI now installs tar files, which will help people and teams who use R on both Windows and Linux machines.

However, this is my personal opinion but I would have preferred the Windows GUI package updater be altered so that when it contacts CRAN for package updates it only accepts zip files.  I have a gut feeling that if that were possible, then the bugs I have found would be resolved.
Comment 6 tjlawrence 2015-05-05 04:49:02 UTC
Thanks for the quick response.  I have tested R-devel 2015-05-05 r68320 but believe the issues are not entirely resolved.  I have attached the output from some of my testing, and detailed comments are in the comments section of the attachment page.
Comment 7 Duncan Murdoch 2015-05-05 09:42:45 UTC
Sorry, the same "multiple request" bug appeared in several places, and I only fixed one of them the first time.  I think I have caught them all now.

The voluminous output is normal:  you *are* compiling the code in the package when you install a package from source.  You won't see that for all packages, but for packages with C, Fortran or C++ code, you will.

Regarding the default to look for source files:  I told you how to change that in comment 1.  The reason for it is that people may want the most recent version of a package, even if the binary is not available:  that's what everyone gets on Linux, so we'd like to offer the possibility on Windows.  

Thanks for your persistence in reporting these bugs.  Hopefully I've caught everything now (in revision 68321 or later).
Comment 8 tjlawrence 2015-05-06 07:07:53 UTC
Thanks for your help thus far.  I will use your advice from comment 1 in future - sorry for overlooking it in a previous comment.

As regards the original bug, it appears to be fixed.  I tried installing a older version of gpclib from local zip file on Rdevel 2015-05-05 r68332 and then using the package updater, which downloaded the current version's tar file.  (Interestingly at least one package for which a new version came out later than gpclib - stabledist - now has a zip file available, while gpclib doesn't).  This time the CRAN mirror issue appeared to be resolved as I only asked for it once.

The issue I found with installing multiple packages from local zip files at once also appears to be resolved.  However the package updater seems to have a bug in that every time I call it, it wants to update all the packages which I have previously downloaded and installed for which the most recent version I have is as a zip file.  This occurs even though (with the exception of gpclib, which it doesn't want to update) all the zip files for packages I have are the latest version.  The versions the updater downloads are also the latest version.  For that reason alone I am reopening the bug.  Sorry about that!
Comment 9 Duncan Murdoch 2015-05-06 11:07:05 UTC
Re your problem with unnecessary updates.  Could you please do the following for one of those packages, and post a copy of the log of commands you used and the output they produced?  Supposing the package name is "pkg":

1.  Install it from your local pkg_*.zip file.

2.  Run packageDescription("pkg").

3.  Run update.packages to trigger the unwanted update.

4.  Run packageDescription("pkg") again.

There are several fields in the DESCRIPTION file that update.packages looks at to decide if a package needs updating; I'd like to see what your package contains there to see why it is being updated.
Comment 10 tjlawrence 2015-05-13 04:10:46 UTC
I downloaded the new R-patched version today (2015-05-08 r68346)and the package installation from local zip files worked perfectly, as did updating packages from CRAN.  However, when I tried to install the tar folder for the gpclib package I had downloaded from CRAN some time ago (version 1.5-5), R could not do so - instead it gave an error.

However, R was able to install the latest zip file I had for gpclib (1.5-1) that I had previously downloaded, and the updater was able to download and install the newer tar folder (i.e. version 1.5-5).
Comment 11 Duncan Murdoch 2015-05-13 13:30:29 UTC
Sounds fixed then.  Problems compiling gpclib are likely problems with that package or with prerequisites on your system -- they should be taken up with the gpclib maintainer if the .zip is insufficient.