Mv
(since EVL 1.0)
Move <source> to <dest>, or multiple <source>(s) to <dest> directory.
<source> and <dest> might be one of:
<local_path> gdrive://<path> gs://<bucket>/<path> hdfs://<path> s3://<bucket>/<path> sftp://<path> smb://<path>
So for example when argument starts with ‘hdfs://’, then it is supposed to be on HDFS file system and calls the function ‘evl_hdfs_mv’, which is by default ‘hadoop fs -mv’.
Or when argument starts with ‘s3://’, then it is supposed to be on S3 file system and calls the function ‘evl_s3_mv’, which is by default ‘aws s3 mv’.
Otherwise act as usual ‘mv’ command.
So far it can move to/from local path from/to some specific one or within one URI type. So not yet possible to move dirrectly for example from S3 to G-Drive.
#But when <source> and/or <dest> contain URI like ‘hdfs://’, ‘s3://’,
#‘gs://’ or ‘sftp://’ (i.e. Hadoop FS, Amazon S3, Google Storage and SFTP), #then it
calls appropriate commands to move (or copy and delete) from/to such source/destination.
When more than one <source> is specified, then URI prefix must be the same for all of them.
- Mv
-
is to be used in EVS job structure definition file or in EWS workflow structure definition.
- evl mv
-
is intended for standalone usage, i.e. to be invoked from command line.
Using this command might keep your code clean, but for more complex copying better use appropriate commands directly, as it gives you all the available options for particular storage or protocol type.
Synopsis
Mv [-f|--force] <source>... <dest> evl mv [-f|--force] <source>... <dest> [-v|--verbose] evl mv ( --help | --usage | --version )
Options
- -f, --force
-
destination will be overwritten if exists
Standard options:
- --help
-
print this help and exit
- --usage
-
print short usage information and exit
- -v, --verbose
-
print to stderr info/debug messages of the component
- --version
-
print version and exit
Examples
- This line in EVL job (an ‘evs’ file):
Mv hdfs:///some/path/to/file /some/local/path/
will call:
evl_hdfs_get hdfs:///some/path/file /some/local/path/ evl_hdfs_rm hdfs:///some/path/file
- This line in EVL job (an ‘evs’ file):
Mv /some/local/path/file hdfs:///some/path/
will call:
evl_hdfs_put /some/local/path/file hdfs:///some/path/ rm /some/local/path/file
- This line in EVL job (an ‘evs’ file):
Mv hdfs:///some/path/file hdfs:///other/path/
will call:
evl_hdfs_cp hdfs:///some/path/file hdfs:///other/path/ evl_hdfs_rm hdfs:///some/path/file
- This line in EVL job (an ‘evs’ file):
Mv /some/local/path/file /other/local/path/
will call:
mv /some/local/path/file /other/local/path/
Where defauls for ‘evl_hdfs_*’ variables are:
function evl_hdfs_cp { hdfs dfs -cp ; }
function evl_hdfs_get { hdfs dfs -get ; }
function evl_hdfs_put { hdfs dfs -put ; }
function evl_hdfs_rm { hdfs dfs -rm ; }