i use distcp copy list of files (> 1k files) hdfs. have stored list of files in local directory, can use -f copy files? if yes format have maintain in files list file? or there other better way?
you don't have use distcp if use-case copying data local filesystem (say linux) hdfs. can use hdfs dfs -put command same. here syntax.
hdfs dfs -put /path/to/local/dir/* /path/on/hdfs/
e.g.
hdfs dfs -mkdir /user/hduser/destination-dir/
hdfs dfs -put /home/abc/mydir/* /user/hduser/destination-dir/
you have created file containing list of file paths not @ needed. it's used (for distcp) when copying data 1 cluster other cluster
Comments
Post a Comment