globus transfer - Submit a transfer task (asynchronous)
globus transfer [OPTIONS] — SOURCE_ENDPOINT_ID:SOURCE_PATH DEST_ENDPOINT_ID:DEST_PATH
globus transfer [OPTIONS] SOURCE_ENDPOINT_ID DEST_ENDPOINT_ID --batch < INPUT_LINE(s)…
The globus transfer command submits an asynchronous task that copies files and/or directories between endpoints.
globus transfer has two modes. Single target, which transfers one file or one directory, and batch, which takes in several lines to transfer multiple files or directories. See "Batch Input" below for more information.
globus transfer will always place the dest files in a consistent, deterministic location. The contents of a source directory will be placed inside the dest directory. A source file will be copied to the dest file path, which must not be an existing directory. All intermediate / parent directories on the dest will be automatically created if they don’t exist.
If the files or directories given as input are symbolic links, they are followed. However, no other symbolic links are followed and no symbolic links are ever created on the dest.
Automatic Endpoint Activation
This command requires all endpoints it uses to be activated. It will attempt to auto-activate any endpoints that are not active, but if auto-activation fails, you will need to manually activate the endpoint. See globus endpoint activate for more details.
If you use SOURCE_PATH and DEST_PATH without the --batch flag, you will submit a single-file or single-directory transfer task. This has behavior similar to cp and cp -r across endpoints.
Using --batch, globus transfer can submit a task which transfers multiple files or directories. Paths to transfer are taken from stdin. Lines are split on spaces, respecting quotes, and every line is treated as a file or directory to transfer.
Lines are of the form [--recursive] SOURCE_PATH DEST_PATH
Empty lines and comments beginning with # are ignored.
Batch only requires a SOURCE_ENDPOINT and a DEST_ENDPOINT before passing lines, on stdin, but if you pass SOURCE_PATH and/or DEST_PATH in the original command, these paths will be used as prefixes to their respective paths on stdin.
Starts batch mode input for transfers, which takes lines in the form [--recursive] SOURCE_PATH DEST_PATH from stdin to transfer multiple files and/or directories. See "Batch Input" above for more information.
Mutually exclusive with --recursive. i.e. This is invalid: globus transfer --recursive --batch …. Use of --recursive inside of batch input is distinct, and allowed.
- -r, --recursive
If SOURCE_PATH and DEST_PATH are both directories, do a recursive transfer of all files and directories inside of the source directory to the destination directory. Note that for batch input, each directory must be marked as recursive individually unlike globus delete.
Mutually exclusive with --batch. If you want to recursively transfer directories in --batch input, see "Batch Input" above.
- -s, --sync-level [exists|size|mtime|checksum]
Set the level at which the transfer task will decide to replace a file at the destination with a file at the source with the same name. If not set, files at the destination will be replaced by default.
exists will only copy files that do not exist at the destination
size will replace a file if the size of the destination file does not match the size of the source file
mtime will replace a file if the timestamp of the destination file is older than the timestamp of the source file
checksum will replace a file if the source and destination files have different checksums.
Preserve the modification time of files transfered. Note that directories will not have their modification times preserved.
- --verify-checksum / --no-verify-checksum
After transfer, verify that the source and destination file checksums match. If they don’t, re-transfer the entire file and keep trying until it succeeds. Defaults to True.
Encrypt the transfer data channel. If either the source or the destination endpoints has force_encryption set this will be done automatically.
Delete extraneous files and directories in the dest directory. Only applies to recursive directory transfers. For example, a recursive transfer of /some/source/dir/ to /tmp/mydir/ will ensure /tmp/mydir/ will have no files not in /some/source/dir/. All other files in /tmp outside of /tmp/mydir/ are left alone.
Set a deadline in ISO time format for this to be canceled if not completed by. ISO format is YYYY-MM-DD or YYYY-MM-DD HH:MM:SS If hours minutes and seconds are not given they are assumed to be 00:00:00. This time must not be in the past or the task will not be submitted.
Don’t submit the task, but print submission data instead.
Set an optional label for the task. These are meant for human readability and ease of filtering the task list
Specify a submission id to use with this task. Submission id’s are generated using globus task generate-submission-id and prevent re-submission of a task with the same submission id in case of unreliable networks. Note that this is not the same id as the task’s id once it is submitted. If not specified the Globus CLI will still generate a submission id and safely resubmit tasks on network errors.
Submit the task even if one or more of the target endpoints are not currently activated. Once all target endpoints have been activated the task will begin normally. If this option is given, auto-activation will not be attempted on the target endpoints.
Comma separated list of task events which notify by email. on and off may be used to enable or disable notifications for all event types. Otherwise, use succeeded, failed, or inactive. e.g. --notify failed,inactive to skip notifications on successful tasks.
- --map-http-status TEXT
Map non success HTTP response codes to exit codes other than 1. e.g. "--map-http-satus 403=0,404=0" would exit with 0 even if a 403 or 404 http error code was received. Valid exit codes are 0,1,50-99.
- -F, --format [json|text]
Set the output format for stdout. Defaults to "text".
- --jq, --jmespath EXPR
Supply a JMESPath expression to apply to json output. Takes precedence over any specified --format and forces the format to be json processed by this expression.
A full specification of the JMESPath language for querying JSON structures may be found at https://jmespath.org/
- -h, --help
Show help text for this command.
- -v, --verbose
Control the level of output.
Use -v or --verbose to show warnings and any additional text output.
Use -vv to add informative logging.
Use -vvv to add debug logging and full stack on any errors. (equivalent to -v --debug)
Transfer a single file:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/godata/file1.txt $dest_ep:~/mynewfile.txt
Transfer a directory recursively:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/godata/ $dest_ep:~/mynewdir/ --recursive
Use the batch input method to transfer multiple files and directories:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep $dest_ep --batch # lines starting with '#' are comments # and blank lines (for spacing) are allowed # files in the batch /share/godata/file1.txt ~/myfile1.txt /share/godata/file2.txt ~/myfile2.txt /share/godata/file3.txt ~/myfile3.txt # these are recursive transfers in the batch # you can use -r, --recursive, and put the option before or after /share/godata ~/mygodatadir -r --recursive godata mygodatadir2 <EOF>
Use the batch input method to transfer multiple files and directories, with a prefix on the source and destination endpoints (this is identical to the case above, but much more concise):
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/ $dest_ep:~/ --batch godata/file1.txt myfile1.txt godata/file2.txt myfile2.txt godata/file3.txt myfile3.txt godata mygodatadir -r --recursive godata mygodatadir2 <EOF>
Consume a batch of files to transfer from a data file, submit the transfer
task, get back its task ID for use in
globus task wait, wait for up to 30
seconds for the task to complete, and then print a success or failure message.
$ cat my_file_batch.txt /share/godata/file1.txt ~/myfile1.txt /share/godata/file2.txt ~/myfile2.txt /share/godata/file3.txt ~/myfile3.txt
source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec task_id="$(globus transfer $source_ep $dest_ep \ --jmespath 'task_id' --format=UNIX \ --batch < my_file_batch.txt)" echo "Waiting on 'globus transfer' task '$task_id'" globus task wait "$task_id" --timeout 30 if [ $? -eq 0 ]; then echo "$task_id completed successfully"; else echo "$task_id failed!"; fi
0 on success.
1 if a network or server error occurred, unless --map-http-status has been used to change exit behavior on http error codes.
2 if the command was used improperly.