I noticed that the recursive download of a sharepoint folder isn't really recursive when parallel = TRUE, it stops at second level of recursion. I checked the function definition of the download method and i noticed that simply the recursive argument isn't passed when parallel = TRUE to subsequent calls to download:
function (dest = self$properties$name, overwrite = FALSE, recursive = FALSE,
parallel = FALSE)
{
if (self$is_folder()) {
children <- self$list_items()
isdir <- children$isdir
if (!is.character(dest))
stop("Must supply a destination folder", call. = FALSE)
dest <- normalizePath(dest, mustWork = FALSE)
dir.create(dest, showWarnings = FALSE)
if (isTRUE(parallel))
parallel <- 5
if (is.numeric(parallel)) {
parallel <- parallel::makeCluster(parallel)
on.exit(parallel::stopCluster(parallel))
}
if (inherits(parallel, "cluster")) {
files <- children$name[!isdir]
dirs <- children$name[isdir]
parallel::parLapply(parallel, files, function(f,
item, dest, overwrite) {
item$get_item(f)$download(file.path(dest, f),
overwrite = overwrite)
}, item = self, dest = dest, overwrite = overwrite)
if (recursive)
for (d in dirs) self$get_item(d)$download(file.path(dest,
d), overwrite = overwrite, parallel = parallel) # recursive isn't passed to download!!!!!
}
else if (isFALSE(parallel)) {
if (!recursive)
children <- children[!isdir, , drop = FALSE]
for (f in children$name) self$get_item(f)$download(file.path(dest,
f), overwrite = overwrite, recursive = recursive,
parallel = parallel)
}
else stop("Unknown value for 'parallel' argument", call. = FALSE)
}
else private$download_file(dest, overwrite)
}
I noticed that the recursive download of a sharepoint folder isn't really recursive when parallel = TRUE, it stops at second level of recursion. I checked the function definition of the download method and i noticed that simply the recursive argument isn't passed when parallel = TRUE to subsequent calls to download:
It should be a pretty easy fix