Skip to content

path and to.dfs with R launched from two different directories #185

@nturenne

Description

@nturenne

Hi !

i use rmr2 and hadoop as a newbie on windows 7 with hadoop 2.7.1 and R 3.3.2

This are my parameters :

#init env R
Sys.setenv(HADOOP_CMD="D:/Soft/hadoop/bin/hadoop")
Sys.setenv(HADOOP_HOME="D:/Soft/hadoop/")
Sys.setenv("HADOOP_PREFIX"="D:/Soft/hadoop/")
Sys.setenv(HADOOP_STREAMING="D:/Soft/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar")

Sys.setenv(TMP = 'D:\Soft\hadoop')
library(rhdfs)
library(rmr2)
library(ravro)
.jinit()
hdfs.init()

At this point all is ok

if i run with the path on which R is installed all is ok :

getwd()
[1] "C:/Program Files/R/R-3.3.2/bin/x64"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
17/11/16 17:16:08 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib l
ibrary
17/11/16 17:16:08 INFO compress.CodecPool: Got brand-new compressor [.deflate]

BUT if i run on another path there a path problem somewhere and i dont know why

getwd()
[1] "D:/Utilisateurs/turenn"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
Le chemin d'accès spécifié est introuvable.
Warning message:
l'exécution de la commande 'D:/Soft/hadoop/bin/hadoop jar D:/Soft/hadoop/share/h
adoop/tools/lib/hadoop-streaming-2.7.1.jar loadtb /tmp/file30a0419d6021 < /Temp/
RtmpAL2rCq/file30a0240f658e' renvoie un statut 1

why such problem about path when calling the same Rterm from two different directories
did i miss something ?

thank you for help
Nico

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions