Bug 17213 - [patch] detectCores() does not see multiple sockets on Linux
Summary: [patch] detectCores() does not see multiple sockets on Linux
Status: UNCONFIRMED
Alias: None
Product: R
Classification: Unclassified
Component: System-specific (show other bugs)
Version: R-devel (trunk)
Hardware: x86_64/x64/amd64 (64-bit) Linux
: P5 minor
Assignee: R-core
URL:
Depends on:
Blocks:
 
Reported: 2017-01-17 13:25 UTC by Mikko Korpela
Modified: 2017-01-17 13:29 UTC (History)
0 users

See Also:


Attachments
Proposed patch (1.17 KB, patch)
2017-01-17 13:25 UTC, Mikko Korpela
Details | Diff
Shell script for testing, reads from stdin (192 bytes, application/x-shellscript)
2017-01-17 13:26 UTC, Mikko Korpela
Details
Stripped-down /proc/cpuinfo examples for testing (2.57 KB, application/zip)
2017-01-17 13:29 UTC, Mikko Korpela
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mikko Korpela 2017-01-17 13:25:23 UTC
Created attachment 2217 [details]
Proposed patch

When running 'parallel::detectCores(logical = FALSE)' on Linux systems with multiple CPU sockets, only the cores in one socket are reported. For example, on a system with 2 sockets, each with 8 cores, only 8 of the total 16 cores are detected.

This occurs on R 3.3.2 patched r71996 and R-devel r71996.

The attached patch is an attempt to fix the issue. The idea is to look at both "cpu cores" and "physical id" when inspecting /proc/cpuinfo. Unique combinations of core count and physical id are kept. Finally, the sum of these core counts is returned. The patch also fixes the wrong result returned on Raspberry Pi (ARMv6) systems with 'detectCores(logical = TRUE)' (the default), <https://stat.ethz.ch/pipermail/r-devel/2016-December/073486.html>.

Test files are included in a zip archive. All of these are stripped down versions of actual (files 1-6) or fictional (files 7-8) /proc/cpuinfo entries, with only (what I think are) relevant lines kept. The Raspberry Pi examples (test files 5 and 6) are from the Raspberry Pi forum, user DougieLawson <https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=146742>. Other test files represent x86 (64-bit) systems. With 'logical = FALSE', the current implementation of detectCores() is inaccurate in the multiple socket test cases (1, 2, 3). With 'logical = TRUE', it is inaccurate on single-core Raspberry Pi systems, test case 6. The proposed patch fixes these cases and returns identical results in the other included test cases.

A stand-alone shell script "detectCores2.sh" is also included. The code in the script is otherwise identical to the 'logical = FALSE' branch of the patched detectCores() code, but the stand-alone script gets its input from stdin.

The patch and the stand-alone script were tested on Ubuntu 14.04.5 LTS, with command interpreters dash (0.5.7-4ubuntu1) and bash (4.3-7ubuntu1.5). Also the following standard command line tools are used: cut, grep, paste, sed, sort.
Comment 1 Mikko Korpela 2017-01-17 13:26:45 UTC
Created attachment 2218 [details]
Shell script for testing, reads from stdin
Comment 2 Mikko Korpela 2017-01-17 13:29:24 UTC
Created attachment 2219 [details]
Stripped-down /proc/cpuinfo examples for testing