Skip to content

Recursive download is not really recursive when parallel = TRUE #228

@Upipa

Description

@Upipa

I noticed that the recursive download of a sharepoint folder isn't really recursive when parallel = TRUE, it stops at second level of recursion. I checked the function definition of the download method and i noticed that simply the recursive argument isn't passed when parallel = TRUE to subsequent calls to download:


function (dest = self$properties$name, overwrite = FALSE, recursive = FALSE, 
    parallel = FALSE) 
{
    if (self$is_folder()) {
        children <- self$list_items()
        isdir <- children$isdir
        if (!is.character(dest)) 
            stop("Must supply a destination folder", call. = FALSE)
        dest <- normalizePath(dest, mustWork = FALSE)
        dir.create(dest, showWarnings = FALSE)
        if (isTRUE(parallel)) 
            parallel <- 5
        if (is.numeric(parallel)) {
            parallel <- parallel::makeCluster(parallel)
            on.exit(parallel::stopCluster(parallel))
        }
        if (inherits(parallel, "cluster")) {
            files <- children$name[!isdir]
            dirs <- children$name[isdir]
            parallel::parLapply(parallel, files, function(f, 
                item, dest, overwrite) {
                item$get_item(f)$download(file.path(dest, f), 
                  overwrite = overwrite)
            }, item = self, dest = dest, overwrite = overwrite)
            if (recursive) 
                for (d in dirs) self$get_item(d)$download(file.path(dest, 
                  d), overwrite = overwrite, parallel = parallel) # recursive isn't passed to download!!!!!
        }
        else if (isFALSE(parallel)) {
            if (!recursive) 
                children <- children[!isdir, , drop = FALSE]
            for (f in children$name) self$get_item(f)$download(file.path(dest, 
                f), overwrite = overwrite, recursive = recursive, 
                parallel = parallel)
        }
        else stop("Unknown value for 'parallel' argument", call. = FALSE)
    }
    else private$download_file(dest, overwrite)
}

It should be a pretty easy fix

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions