Copy a file or directory from one endpoint to another as an asynchronous task.
'globus transfer' has two modes. Single target, which transfers one file or one directory, and batch, which takes in several lines to transfer multiple files or directories. See "Batch Input" below for more information.
'globus transfer' will always place the dest files in a consistent, deterministic location. The contents of a source directory will be placed inside the dest directory. A source file will be copied to the dest file path, which must not be an existing directory. All intermediate / parent directories on the dest will be automatically created if they don’t exist.
If the files or directories given as input are symbolic links, they are followed. However, no other symbolic links are followed and no symbolic links are ever created on the dest.
If you use
DEST_PATH without the
--batch flag, you
will submit a single-file or single-directory transfer task.
globus transfer can submit a task which transfers multiple
specified files or directories.
Each line of
--batch input is treated as a path to a file or directory to
Lines are of the form [--recursive] [--external-checksum TEXT] SOURCE_PATH DEST_PATH
Skips empty lines and allows comments beginning with "#".
If you use
--batch and a commandline SOURCE_PATH and/or DEST_PATH, these
paths will be used as dir prefixes to any paths read from the
Sync Levels are ways to decide whether or not files are copied, with the following definitions:
EXISTS: Determine whether or not to transfer based on file existence. If the destination file is absent, do the transfer.
SIZE: Determine whether or not to transfer based on the size of the file. If destination file size does not match the source, do the transfer.
MTIME: Determine whether or not to transfer based on modification times. If source has a newer modififed time than the destination, do the transfer.
CHECKSUM: Determine whether or not to transfer based on checksums of file contents. If source and destination contents differ, as determined by a checksum of their contents, do the transfer.
If a transfer fails, CHECKSUM must be used to restart the transfer. All other levels can lead to data corruption.
--exclude options are evaluated in order together
to determine which files are transferred during recursive transfers.
--exclude options have priority over later such
options, with the first option that matches the name of a file being
applied. A file that does not match any
is included by default, making the
--include option only useful for
globus transfer --include *.txt --exclude * … will
only transfer files ending in .txt found within the directory structure.
Submit the task even if the endpoint(s) aren’t currently activated.
Set a deadline for this to be canceled if not completed by.
Set a label for this task.
Task submission ID, as generated by
globus task generate-submission-id. Used for safe resubmission in the presence of network failures.
Comma separated list of task events which notify by email. 'on' and 'off' may be used to enable or disable notifications for all event types. Otherwise, use 'succeeded', 'failed', or 'inactive'.
Don’t actually submit the task, print submission data instead
Specify that only new or modified files should be transferred, depending on which setting is provided.
Accept a batch of source/dest path pairs from a file. Use
-to read from stdin.
Uses SOURCE_ENDPOINT_ID and DEST_ENDPOINT_ID as passed on the commandline.
See documentation on "Batch Input" for more information.
- -r, --recursive / --no-recursive
Use --recursive to flag that the paths are directories and should be transferred recursively. Use --no-recursive to flag that the paths are files that must not be transferred recursively. Omit these options to use path type auto-detection.
- --preserve-timestamp, --preserve-mtime
Preserve file and directory modification times.
- --verify-checksum / --no-verify-checksum
Verify checksum after transfer. [default: verify-checksum]
- --encrypt-data, --encrypt
Encrypt data sent through the network.
Skip over source paths that hit permission denied or file not found errors during the transfer.
Cause the task to fail if any quota exceeded errors are hit during the transfer.
Delete extraneous files in the destination directory. Only applies to recursive directory transfers.
An external checksum to verify source file and data transfer integrity. Assumed to be an MD5 checksum if --checksum-algorithm is not given.
Specify an algorithm for --external-checksum or --verify-checksum
Include files found with names that match the given pattern in recursive transfers. Pattern may include "*", "?", or "" for Unix-style globbing. This option can be given multiple times along with --exclude to control which files are transferred, with earlier options having priority.
Exclude files found with names that match the given pattern in recursive transfers. Pattern may include "*", "?", or "" for Unix-style globbing. This option can be given multiple times along with --include to control which files are transferred, with earlier options having priority.
Optional value passed to the source’s identity mapping specifying which local user account to map to. Only usable with Globus Connect Server v5 mapped collections.
Optional value passed to the destination’s identity mapping specifying which local user account to map to. Only usable with Globus Connect Server v5 mapped collections.
- -v, --verbose
Control level of output.
- -h, --help
Show this message and exit.
Output format for stdout. Defaults to text.
A JMESPath expression to apply to json output. Forces the format to be json processed by this expression.
Map HTTP statuses to any of these exit codes: 0,1,50-99. e.g. "404=50,403=51"
Transfer a single file:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/godata/file1.txt $dest_ep:~/mynewfile.txt
Transfer a directory recursively:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/godata/ $dest_ep:~/mynewdir/ --recursive
Use the batch input method to transfer multiple files and directories:
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep $dest_ep --batch - # lines starting with '#' are comments # and blank lines (for spacing) are allowed # files in the batch /share/godata/file1.txt ~/myfile1.txt /share/godata/file2.txt ~/myfile2.txt /share/godata/file3.txt ~/myfile3.txt # these are recursive transfers in the batch # you can use -r, --recursive, and put the option before or after /share/godata ~/mygodatadir -r --recursive godata mygodatadir2 <EOF>
Use the batch input method to transfer multiple files and directories, with a prefix on the source and destination endpoints (this is identical to the case above, but much more concise):
$ source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec $ dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec $ globus transfer $source_ep:/share/ $dest_ep:~/ --batch - godata/file1.txt myfile1.txt godata/file2.txt myfile2.txt godata/file3.txt myfile3.txt godata mygodatadir -r --recursive godata mygodatadir2 <EOF>
Consume a batch of files to transfer from a data file, submit the transfer
task, get back its task ID for use in
globus task wait, wait for up to 30
seconds for the task to complete, and then print a success or failure message.
$ cat my_file_batch.txt /share/godata/file1.txt ~/myfile1.txt /share/godata/file2.txt ~/myfile2.txt /share/godata/file3.txt ~/myfile3.txt
source_ep=ddb59aef-6d04-11e5-ba46-22000b92c6ec dest_ep=ddb59af0-6d04-11e5-ba46-22000b92c6ec task_id="$(globus transfer $source_ep $dest_ep --jmespath 'task_id' --format=UNIX --batch my_file_batch.txt)" echo "Waiting on 'globus transfer' task '$task_id'" globus task wait "$task_id" --timeout 30 if [ $? -eq 0 ]; then echo "$task_id completed successfully"; else echo "$task_id failed!"; fi
0 on success.
1 if a network or server error occurred, unless --map-http-status has been used to change exit behavior on http error codes.
2 if the command was used improperly.
3 if the command was used on the wrong type of object, e.g. a collection command used on an endpoint.
4 if the command has authentication or authorization requirements which were not met, as in ConsentRequired errors or missing logins.