2.1. Obtaining Tokens

2.2. Using Tokens

The Transfer token needs to be passed to the Transfer API in the Authorization header with custom method Bearer:

Authorization: Bearer TOKEN

You can test this by saving the token to a private temporary file, e.g. $HOME/tmp-globus-auth-token.txt, and then passing it to curl with the -H option:

TOKEN=$(cat $HOME/tmp-globus-auth-token.txt)
curl -H "Authorization : Bearer $TOKEN" \
    'https://transfer.api.globusonline.org/v0.10/task_list'

You could also paste the token directly but then it will be stored in your shell history, which is often not desirable.

2.3. Terminology

2.4. Base URL

All the URLs in the examples below should be taken relative to the Transfer API root:

https://transfer.api.globusonline.org/v0.10

so the full URL to /task_list will be:

https://transfer.api.globusonline.org/v0.10/task_list

Clients should store the base URL in one place and use it when constructing resource URLs, to simplify changing versions.

2.5. URL Patterns and Links

The API exposes lists of resources and allows fetching single resources by name. For example, a list of tasks is available at /task_list, and a task with id 123-abc is accessed with /task/123-abc. This convention is used for all resource URL patterns.

Many resources provide links to related resources; these can be used instead of hard-coding URL patterns, making the client more robust to changes in future versions of the API.

2.6. Paged Lists

The task_list, event_list, and endpoint_search resources support pagination via query parameters. By default you will only get the first 10 records. Getting all records is currently not supported. Different records can be selected using the limit and offset query parameters. See the Paging section for details.

2.7. Document Formats

The API uses json for all input and output, including error documents. Some resources have legacy support for html, but that is deprecated and will be removed in the future.

Note that application/x-www-form-urlencoded is not supported. The body should contain the actual JSON data, not a form encoded version of that data.

The json representation uses a "DATA_TYPE" key to specify the type of resource and a "DATA" key containing a list of sub-documents, if any. For example, the endpoint document type is described in detail here:

2.8. Errors

When an error occurs an HTTP status code >=400 will be returned. The body of the response will be a JSON document with details about the error, including a code field. The error code will also be provided in the "X-Transfer-API-Error" header. Note that requests outside the API path version prefix may return an HTML or plaintext error body instead. Here is an example error returned when an endpoint is not found:

{
  "code": "EndpointNotFound",
  "message": "No such endpoint '23c1a962-7e68-11e5-ac37-f0def10a689e'",
  "request_id": "HrbjJy3QJ",
  "resource": "/endpoint/23c1a962-7e68-11e5-ac37-f0def10a689e"
}

A 404 status code is used for this response. The code field has the same value as the X-Transfer-API-Error header, for convenient access.

3.1. Conventions

The convention used for examples in this document is similar to raw HTTP requests and responses, with the URL shortened and most headers omitted. As an example, to get a task_list for the logged in user, the request is described as:

GET /task_list

This means that a GET request must be made to the task_list resource, which actually has the URL https://transfer.api.globusonline.org/v0.10/task_list for version v0.10. This is BASE_URL + /task_list. As discussed above, the BASE_URL should be set in one place and re-used, not hard coded into each request. The actual raw HTTP request will typically include many headers:

GET /v0.10/task_list HTTP/1.1
Host: transfer.api.globusonline.org
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Iceweasel/4.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: UTF-8,*
Keep-Alive: 115
Connection: keep-alive

Most of these headers were added by the browser (Firefox); the developer will not normally need to deal with them.

For examples that involve sending data, the body is included inline, just like it would be in an HTTP request. For example endpoint creation is described like this:

POST /endpoint
Content-Type: application/json

{
  "display_name": "ACME University shared storage",
  "DATA_TYPE": "endpoint",
  "description": "Example gridftp endpoint."
  "DATA": [
    {
      "DATA_TYPE": "server",
      "hostname": "gridftp.example.org",
      "scheme": "gsiftp",
      "port": 2811,
    }
  ],
}

This means that to create an endpoint, a request using method POST can be made to BASE_URL + /endpoint, with header content-type set to "application/json", and having as the request body the JSON data describing the endpoint. Other headers are required for authentication, but they are not specific to this request.

This format is used to provide a quick description of how to make a request, independent of the client used. The Python and Java examples hide many of the details involved in accessing the API; this document is focused on describing the API itself including those details.

3.2. Monitoring

3.3. Endpoint Management

200 OK
X-Transfer-API-KOA-Version: 4.5
Content-Type: application/json

{
 u'DATA_TYPE': u'endpoint_list',
 u'has_next_page': False,
 u'limit': 3,
 u'offset': 0
 u'DATA': [
   {u'_rank': 0.421588,
    u'acl_available': False,
    u'acl_editable': False,
    u'activated': False,
    u'canonical_name': u'arnoldg#gordon',
    u'contact_email': None,
    u'contact_info': None,
    u'default_directory': u'/~/',
    u'department': None,
    u'description': u'Mirrors xsede#gordon',
    u'disable_verify': False,
    u'display_name': None,
    u'expire_time': None,
    u'expires_in': 0,
    u'force_encryption': False,
    u'gcp_connected': None,
    u'gcp_paused': None,
    u'globus_connect_setup_key': None,
    u'host_endpoint': None,
    u'host_endpoint_display_name': None,
    u'host_endpoint_id': None,
    u'host_path': None,
    u'id': u'cbfb19f5-6d04-11e5-ba46-22000b92c6ec',
    u'in_use': False,
    u'info_link': None,
    u'is_globus_connect': False,
    u'is_go_storage': False,
    u'keywords': None,
    u'location': u'Automatic',
    u'max_concurrency': 4,
    u'max_parallelism': 8,
    u'my_effective_roles': [],
    u'myproxy_dn': None,
    u'myproxy_server': None,
    u'name': u'gordon',
    u'network_use': u'normal',
    u'oauth_server': u'cilogon.org',
    u'organization': None,
    u'preferred_concurrency': 2,
    u'preferred_parallelism': 4,
    u'public': True,
    u's3_owner_activated': False,
    u's3_url': None,
    u'shareable': True,
    u'sharing_target_endpoint': None,
    u'sharing_target_root_path': None,
    u'subscription_id': None,
    u'username': u'arnoldg'},
   {u'_rank': 0.421588,
    u'acl_available': False,
    u'acl_editable': False,
    u'activated': False,
    u'canonical_name': u'vyekkirala#gordon',
    u'contact_email': None,
    u'contact_info': None,
    u'default_directory': None,
    u'department': None,
    u'description': u'Mirrors xsede#gordon except that this uses test-oa4mp.iu.xsede.org for authentication/delegation.',
    u'disable_verify': False,
    u'display_name': None,
    u'expire_time': None,
    u'expires_in': 0,
    u'force_encryption': False,
    u'gcp_connected': None,
    u'gcp_paused': None,
    u'globus_connect_setup_key': None,
    u'host_endpoint': None,
    u'host_endpoint_display_name': None,
    u'host_endpoint_id': None,
    u'host_path': None,
    u'id': u'cf08f264-6d04-11e5-ba46-22000b92c6ec',
    u'in_use': False,
    u'info_link': None,
    u'is_globus_connect': False,
    u'is_go_storage': False,
    u'keywords': None,
    u'location': u'Automatic',
    u'max_concurrency': 4,
    u'max_parallelism': 8,
    u'my_effective_roles': [],
    u'myproxy_dn': None,
    u'myproxy_server': None,
    u'name': u'gordon',
    u'network_use': u'normal',
    u'oauth_server': u'test-oa4mp.iu.xsede.org',
    u'organization': None,
    u'preferred_concurrency': 2,
    u'preferred_parallelism': 4,
    u'public': True,
    u's3_owner_activated': False,
    u's3_url': None,
    u'shareable': True,
    u'sharing_target_endpoint': None,
    u'sharing_target_root_path': None,
    u'subscription_id': None,
    u'username': u'vyekkirala'},
   {u'_rank': 0.396413,
    u'acl_available': False,
    u'acl_editable': False,
    u'activated': False,
    u'canonical_name': u'xsede#gordon',
    u'contact_email': None,
    u'contact_info': None,
    u'default_directory': None,
    u'department': None,
    u'description': None,
    u'disable_verify': False,
    u'display_name': None,
    u'expire_time': u'2015-08-25T21:14:17+00:00',
    u'expires_in': 0,
    u'force_encryption': False,
    u'gcp_connected': None,
    u'gcp_paused': None,
    u'globus_connect_setup_key': None,
    u'host_endpoint': None,
    u'host_endpoint_display_name': None,
    u'host_endpoint_id': None,
    u'host_path': None,
    u'id': u'c5e7e362-6d04-11e5-ba46-22000b92c6ec',
    u'in_use': False,
    u'info_link': None,
    u'is_globus_connect': False,
    u'is_go_storage': False,
    u'keywords': None,
    u'location': u'Automatic',
    u'max_concurrency': 4,
    u'max_parallelism': 8,
    u'my_effective_roles': [],
    u'myproxy_dn': None,
    u'myproxy_server': u'myproxy.xsede.org',
    u'name': u'gordon',
    u'network_use': u'normal',
    u'oauth_server': u'oa4mp.xsede.org',
    u'organization': None,
    u'preferred_concurrency': 2,
    u'preferred_parallelism': 4,
    u'public': True,
    u's3_owner_activated': False,
    u's3_url': None,
    u'shareable': True,
    u'sharing_target_endpoint': None,
    u'sharing_target_root_path': None,
    u'subscription_id': u'1813a867-5f94-11e4-b64e-12313940394d',
    u'username': u'xsede'}],
}

3.4. Endpoint Directory Listing

GET /endpoint/ID/activation_requirements

POST /endpoint/ID/activate

GET /endpoint/ID/ls?path=/~/directory

3.5. Creating Directories

To create a directory on an endpoint, submit a mkdir document to POST /operation/endpoint/ID/mkdir

{
  "path": "/~/newdir",
  "DATA_TYPE": "mkdir"
}

If the path field does not contain an absolute path, it’s assumed to be relative to the user’s home directory (~).

A standard error document is returned on failure; on sucess a "mkdir_result" is returned, with status 202 and code DirectoryCreated:

{
  "message": "The directory was created successfully",
  "code": "DirectoryCreated",
  "resource": "/operation/endpoint/427a0454-77dd-45d4-89d3-282c431c6bfe/mkdir",
  "DATA_TYPE": "mkdir_result",
  "request_id": "abc123"
}

Note that recursive transfers implicitly create directories as needed at the destination; the purpose of the mkdir resource is to provide explicit creation.

3.6. Transfer Submission

A transfer is a request to copy files and directories from a source endpoint to a destination endpoint. The request document is essentially a list of transfer items containing source / destination path pairs, with flags to indicate if the path is a directory to be copied recursively or a single file to be transfered. To fullfill the request, the service creates a task, which can be monitored using the task_id.

For recursive (directory) transfer items, the contents of the source directory are copied to the destination directory, including any subdirectories. Any intermediate/parent directories that don’t exist on the destination will be created.

For non-recursive (file) transfer items, the source file is copied to the file path specified as the destination. The destination path can’t be a directory, This is to avoid inconsistent behavior depending on whether or not the destination exists, so when run repeatedly (for example to keep two copies in sync) it performs the same operation each time.

Both endpoints need to be activated before the transfer is submitted. If an endpoint expires before the transfer is complete, the endpoints can be re-activated to allow it to continue, up until the deadline (which defaults to 24 hours after the request time).

When submitting a transfer, you must first get a submission_id:

GET /submission_id

The submission id should be saved in case the submission is interrupted before a result is received from the server. The transfer can then be resubmitted, and if the original request was successful it will not double submit, it will simply return a result indicating that it’s a duplicate id, with the id of the task created to fulfill the request.

The transfer itself is submitted via POST /transfer:

{
  "DATA_TYPE": "transfer",
  "submission_id": "VAwPR1dFRhAHQn93dmd3EkETBSs2ejJnVQRWIyp6YytFUl8O",
  "source_endpoint": "d561f96b-6161-4abd-96ad-2b14612f9fe6",
  "destination_endpoint": "e0d7e8a7-6347-40af-b5bb-df0c84731dd4",
  "label": "example transfer label",
  "sync_level": null,
  "DATA": [
    {
      "source_path": "/~/file1.txt",
      "destination_path": "/~/dir1/file1copy.txt",
      "recursive": false,
      "DATA_TYPE": "transfer_item"
    }
    {
      "source_path": "/~/some_directory/",
      "destination_path": "/~/some_directory_copy/",
      "recursive": true,
      "DATA_TYPE": "transfer_item",
    }
  ]
}

and returns a transfer_result:

{
  "submission_id": "UAlfRFdDQEsHQn8tJGd3EkETBStoemJnVQRWIyp6YytFUl8O",
  "code": "Accepted",
  "resource": "/transfer",
  "task_id": "5f63266a-f6ba-11e0-a861-f0def10a689e",
  "DATA_TYPE": "transfer_result",
  "request_id": "abc123",
  "message": "Transfer submission accepted.",
}

sync_level can be used to request that only modified files are transferred, using different mechanisms to determine modification. See the transfer document type for details on the different sync levels. If sync_level is not included or null, all files will be transferred.

3.7. Task Monitoring

To track the progress of a newly submitted task, use the task_id field of the returned result document.

GET /task/TASK_ID

This returns a task document.

A request to cancel the task can be submitted like this:

POST /task/TASK_ID/cancel

It is possible that the transfer will finish before the cancellation goes through; a result document type is returned with a message describing what happened.

3.8. Delete Submission

Remote files and directories can be deleted on an endpoint by submitting a delete document to POST /delete:

{
  "submission_id": "AA1bFgMUEBgHQn8ufWd3EkETBSgzdGZnAgYBd39zYn0RCANT",
  "endpoint": "ddb59af0-6d04-11e5-ba46-22000b92c6ec",
  "recursive": false,
  "DATA_TYPE": "delete",
  "label": "example delete label",
  "length": 2,
  "ignore_missing": false,
  "DATA": [
    {
      "path": "/~/bashrc_copy_example",
      "DATA_TYPE": "delete_item"
    }
  ]
}

The submission_id, label, and deadline fields behave just like the same fields in a transfer document, and the delete_result returned after submission is the same as a transfer_result.

If any of the paths point to a directory, recursive must be set to true and the entire directory contents will be deleted. Deleting a directory only if it is empty is not supported.

If ignore_missing is not set, the job will fail and stop deleting paths if one of the paths does not exist.

To avoid breaking backward compatibility in 0.10, delete tasks are not included by default in task_list. To include delete tasks, use filter=type:TRANSFER,DELETE.

4.1. Paging

List resources which use paging can be controlled with the offset and limit query parameters. The default offset is 0, while the default limit and maximum offset and limit vary among resources. Most list resources have a default limit of 10 and a maximum of 1000. Typical usage involves starting with offset 0, choosing a page size (limit=PAGE_SIZE), and incrementing offset by PAGE_SIZE to display successive pages.

For example, with a page size of 50:

# page 1
GET /task_list?offset=0&limit=50

# page 2
GET /task_list?offset=50&limit=50

# page 3
GET /task_list?offset=100&limit=50

4.2. Filtering

Only certain fields support filtering; see the reference documentation for a full list. There are several types of filters, including date range, a single value, or a list of values. See the field documentation for descriptions and examples.

This example for the task list returns ACTIVE and SUCCESSFUL tasks submitted before December 20 2010:

GET /task_list?filter=status:ACTIVE,SUCCESSFUL/request_time:,2010-12-20 00:00:00

The new convention for filters is to use separate parameters for each, of the form filter_NAME - see Endpoint Search for an example.

4.3. Sorting

The orderby parameter sets a sort field and direction. Only fields which support filtering are sortable. The value is a comma separated list of field names, with an optional direction specifier. For example:

GET /task_list?orderby=status,request_time desc

returns tasks first ordered by status, in ascending alphabetical order, then within tasks with the same status sorts by request_time, with newer tasks first (descending).

4.4. Limiting Result Fields

The fields query parameter can be used to limit which fields are included in the response, for example:

GET /task_list?fields=task_id,status

will return a task list with only task_id and status fields in each task. This can save bandwidth and parsing time if you know you only need certain fields.

Field selection can also be done on sub-documents, by prefixing the field name with the document type name. For example:

GET /endpoint_search?filter_scope=my-endpoints&fields=id,display_name

will include only the id and display_name of each endpoint.