Bug 16792 - parallel:parLapplyLB does not load balance
Summary: parallel:parLapplyLB does not load balance
Alias: None
Product: R
Classification: Unclassified
Component: Add-ons (show other bugs)
Version: R 3.2.3
Hardware: Other Other
: P5 normal
Assignee: R-core
Depends on:
Reported: 2016-03-31 14:30 UTC by Daniel Schlaepfer
Modified: 2016-03-31 14:30 UTC (History)
0 users

See Also:

Reproducible example and suggested fix (3.89 KB, text/plain)
2016-03-31 14:30 UTC, Daniel Schlaepfer

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Schlaepfer 2016-03-31 14:30:51 UTC
Created attachment 2051 [details]
Reproducible example and suggested fix

The function 'parLapplyLB' does not perform load balancing (despite the documentation) because it creates a list of jobs with the same length as the length of the cluster (instead of an integer vector or of a list that is longer than the cluster).

The function 'clusterApplyLB' performs load balancing if its argument 'x' is a vector of integer indices. 

Load balancing is achieved through the function 'dynamicClusterApply' by the line "if (j <= n) submit(d$node, j)", i.e., the list of jobs that 'parLapplyLB' passes to 'dynamicClusterApply' via 'clusterApplyLB' should be longer than the cluster for j to become smaller than n and load balancing to occur.

Attached a reproducible example, one suggestion to fix 'parLapplyLB', and some timing examples.