Task Monitoring

1. Overview

Transfer and delete are asynchronous operations, and result in a background task being created. The task id is returned from successful submission, and can be used to monitor the progress of the task.

Note

Detailed per-file status, errors, and information events are retained for 30 days and then deleted. When this happens, the history_deleted flag will be set on the task document.

1.1. Task Visibility

A task will be visible if the current primary identity is the same as the primary identity when the task was submitted. The only time this will not be the case is when two separate identities with existing task history are linked. When that happens, the task history for the linked identity that is no longer primary will not be visible. It can still be recovered and viewed by unlinking the identity and signing in to that identity separately.

2. Document Types

2.1. Task Document

The "task" document type represents a single transfer or delete submission. Other task types may be added in the future.

2.1.1. Task Fields

Field Name	JSON Type	Description
DATA_TYPE	string	Always has value "task" to indicate this document type.
task_id	string	Globally unique uuid for this task.
type	string	The type of task - either "TRANSFER" or "DELETE".
status	string	One word indicating the progress of the task: "ACTIVE" The task is in progress. "INACTIVE" The task has been suspended and will not continue without intervention. Currently, only credential expiration will cause this state. "SUCCEEDED" The task completed successfully. "FAILED" The task or one of its subtasks failed, expired, or was canceled.
fatal_error	object	For tasks with "FAILED" status, this is an object with code and description strings from the fatal event. Otherwise, this is null. Note: This field is valid only for tasks submitted after July 1, 2017.
label	string	User defined label to make finding tasks simpler.
username	string	[DEPRECATED] Use owner_id instead. The globus username of the user who created the task. For non-globusid.org identities, this will be a meaningless string starting with 'u_', and for globusid.org identities it will be the globusid.org username.
owner_id	string	Identity id of the task owner. If the task was submitted with multiple linked identities, the owner will be the primary identity.
request_time	string	The date and time the task was created, in ISO 8601 format.
completion_time	string	The date and time the task was completed or failed, in ISO 8601 format. If the task is still in progress (status "ACTIVE" or "INACTIVE"), this will be null.
deadline	string	If set when the task is created, cancel the task if it’s not finished when this date and time is reached. By default transfer tasks will not be canceled as long as they are making progress. Delete tasks default to 24 hours after task submission.
source_endpoint	string	[DEPRECATED] Use source_endpoint_id instead. Canonical name of the source endpoint for type "TRANSFER" and the name of the only endpoint for type "DELETE".
source_endpoint_id	string	id of the source endpoint.
source_endpoint_display_name	string	display_name of the source endpoint at time of submit, or the canonical_name if display_name is not set. For tasks submitted prior to August 2015 (version 4.4), this will be the same as source_endpoint.
destination_endpoint	string	[DEPRECATED] Use destination_endpoint_id instead. Canonical name of the destination endpoint for type "TRANSFER", null for type "DELETE".
destination_endpoint_id	string	id of the destination endpoint if any (null for "DELETE" tasks).
destination_endpoint_display_name	string	Like source_endpoint_display_name but for the destination, or null for "DELETE" tasks.
sync_level	integer	The sync level used for the transfer, or null if not used. Always null for delete tasks.
encrypt_data	boolean	For transfer tasks, this will be true if the data channel is encrypted. This can happen in two ways: if one of the endpoints or host endpoints involved in the transfer has force_encryption set to true, or if the transfer request has the encrypt_data option set to true. Always false for delete tasks.
verify_checksum	boolean	The verify_checksum option used for the transfer. Always false for delete tasks.
delete_destination_extra	boolean	The delete_destination_extra option used for the transfer. Always false for delete tasks.
recursive_symlinks	string	The recursive_symlinks option used for transfer tasks. Always null for delete tasks.
preserve_timestamp	boolean	The preserve_timestamp option used for the transfer. Always false for delete tasks.
command	string	If the task was submitted via the CLI, this field will contain the original command line, including options, that created the task. If submitted via the Globus website or directly via the Transfer API, this will contain the string "API", followed by the API version, and optionally followed by a short string representing the client that submitted the request. The format of this field is subject to change and should not be relied upon.
history_deleted	boolean	This flag will be set for tasks older than one month, and indicates that details of the task are no longer available. In particular, subtasks and successful transfers will not be available for the task if this is true, so clients should always check this before querying for subtasks and successful transfers.
faults	int	The number of errors this task encountered. Note that certain types of faults are not fatal (for example, network communication errors) and can be successfully retried. A CANCELED or EXPIRED event is not included in this fault count.
files	int	The total number of files in a transfer task. This can grow as the task recursively scans directories.
directories	int	The total number of directories in a transfer task. This can grow as the task recursively scans directories.
symlinks	int	The total number of kept symlinks (not copied or ignored) in a transfer task. This can grow as the task recursively scans directories.
files_skipped	int	The number of files skipped because no changes were detected. This will always be zero for non-sync transfer tasks (with null sync_level) and for all delete tasks.
files_transferred	int	The number of files actually transferred over the network. For a successful transfer, files = files_skipped + files_transferred. Not used for delete tasks.
subtasks_total	int	Total number of subtasks. This includes file transfer or delete subtasks and helper subtasks such as directory expansion, so is not a reliable measure of the number of files being transferred. It can also grow over time as new files and directories are discovered in directory expansions.
subtasks_pending	int	Number of subtasks which are still in progress.
subtasks_retrying	int	This field is deprecated, and may be 0 or removed in a future release.
subtasks_succeeded	int	Number of subtasks which have completed successfully.
subtasks_expired	int	Number of subtasks which expired and were not completed.
subtasks_canceled	int	Number of subtasks which were canceled.
subtasks_failed	int	Number of subtasks which failed for reasons other than expiring or being cancelled.
bytes_transferred	int	The total number of bytes transferred summed across all subtasks.
bytes_checksummed	int	If sync level 3 is used, the number of bytes checksummed while determining which files need to be transferred.
effective_bytes_per_second	int	A simplistic calculation of bytes/second based on the start time of the task and its completion time, if applicable, or the current time. Valid for transfer tasks. Always 0 for other task types.
nice_status	string	[ALPHA] For tasks with status ACTIVE or INACTIVE, a string indicating a more detailed status of the task. For completed tasks, this will always be empty or null. If it has value OK or Queued, the task is waiting for other tasks to complete or proceeding normally, and no intervention should be required. All other values indicate that some error is being encountered, which may or may not resolve on it’s own. Examples: Creds Expired, PERMISSION_DENIED, ENDPOINT_ERROR, CONNECT_FAILED, PAUSED_BY_ADMIN.
nice_status_details	string	DEPRECATED This field is always null. Use the task event_list instead.
nice_status_short_description	string	[ALPHA] 3-4 word description of nice_status code
nice_status_expires_in	string	[ALPHA] Seconds until any credential required for the task expires (-1=never, 0=expired)
canceled_by_admin	string	[ALPHA] If the task completes successfully, is canceled by the task owner using the standard cancel resource, or hits the deadline before completing, this will be null. It is set only for tasks canceled by an endpoint administrator using the Advanced Endpoint Management API. For such tasks, it will have one of the following values: "SOURCE" An administrator of the source endpoint of the task canceled the task. Also set for "DELETE" type tasks which involve only one endpoint "DESTINATION" An administrator of the destination endpoint of the task canceled the task "BOTH" An administrator of both the source and destination endpoint canceled the task.
canceled_by_admin_message	string	[ALPHA] For tasks with canceled_by_admin set to a non-null value, this will contain a message from the administrator who canceled the task.
is_paused	boolean	"true" if the task is in progress (status "ACTIVE" or "INACTIVE") and has been paused by the administrator of the source or destination endpoint, "false" if the task has not been paused or is complete (status "SUCCEEDED" or "FAILED"). Use Get task pause info to get information about why the task is paused.

Example Task Document

{
    "DATA_TYPE": "task",
    "bytes_checksummed": 4,
    "bytes_transferred": 0,
    "canceled_by_admin": null,
    "canceled_by_admin_message": null,
    "command": "API",
    "completion_time": "2017-08-15 15:23:31+00:00",
    "deadline": "2017-08-16 15:23:30+00:00",
    "delete_destination_extra": false,
    "destination_endpoint": "go#ep2",
    "destination_endpoint_display_name": "Test Endpoint",
    "destination_endpoint_id": "5d3c6d21-5244-11e5-84dd-22000bb3f45d",
    "directories": 0,
    "effective_bytes_per_second": 0,
    "encrypt_data": false,
    "fatal_error": null,
    "faults": 0,
    "files": 1,
    "files_skipped": 1,
    "files_transferred": 0,
    "history_deleted": false,
    "is_paused": false,
    "label": null,
    "nice_status": null,
    "nice_status_expires_in": null,
    "nice_status_short_description": null,
    "owner_id": "e7f4cc10-cb98-11e5-a345-cfce8e17a360",
    "preserve_timestamp": false,
    "recursive_symlinks": "ignore",
    "request_time": "2017-08-15 15:23:30+00:00",
    "source_endpoint": "go#ep1",
    "source_endpoint_display_name": "Test Endpoint",
    "source_endpoint_id": "5d3c6c59-5244-11e5-84dd-22000bb3f45d",
    "status": "SUCCEEDED",
    "subtasks_canceled": 0,
    "subtasks_expired": 0,
    "subtasks_failed": 0,
    "subtasks_pending": 0,
    "subtasks_retrying": 0,
    "subtasks_succeeded": 1,
    "subtasks_total": 1,
    "symlinks": 0,
    "sync_level": 3,
    "task_id": "b156d99b-81cd-11e7-a32b-0eff1506ce8e",
    "type": "TRANSFER",
    "verify_checksum": true
}

2.2. Event Document

Events are logged as a task makes progress or runs into errors.

Example Event Document

{
    "DATA_TYPE": "event",
    "code": "PERMISSION_DENIED",
    "description": "Permission denied",
    "details": "Error (transfer)\nServer: ballen#uc-laptop (Globus Connect)\nFile: /~/Downloads/plus-plan-exposure.png\nCommand: STOR ~/Downloads/plus-plan-exposure.png\nMessage: Fatal FTP response\n---\n500 Command failed. : Path not allowed.\n",
    "is_error": true,
    "time": "2014-07-08 18:50:18+00:00"
}

2.2.1. Event Fields

Field Name	JSON Type	Description
DATA_TYPE	string	Always has value "event" to indicate this document type.
code	string	A code indicating the type of the event.
is_error	boolean	true if event is an error event
description	string	A description of the event.
details	string	Type specific details about the event.
time	string	The date and time the event occurred, in ISO 8601 format (YYYY-MM-DD HH:MM:SS) and UTC.

List of Event Codes

The full list of event codes is below.

Note that ABEND and FILE_ACCESS are deprecated. Modern endpoints are unlikely to return them.

ABEND
ACL_CHANGED
AMBIGUOUS_PATH
AUTH
CANCELED
CLOCK
CONNECT_FAILED
CONNECTION_BROKEN
CONNECTION_RESET
ENDPOINT_ERROR
ENDPOINT_TOO_BUSY
EXPIRED
EXTERNAL_CHECKSUM_MISMATCH
FILE_ACCESS
FILE_NOT_FOUND
FILE_SIZE_CHANGED
GC_NOT_CONNECTED
GC_PAUSED
GSI_DN_MISMATCH
INTERNAL_ERROR
INVALID_PATH_NAME
INVALID_SERVICE_CREDENTIAL
INVALID_SYMLINK
IS_A_DIRECTORY
IS_A_FILE
LIMIT_EXCEEDED
MPU_NOT_FOUND
NO_APPEND_FILESYSTEM
NO_CREDENTIALS
NO_PHYSICALS
NO_SPACE_LEFT
NOT_A_DIRECTORY
NOT_A_FILE
NOT_A_SYMLINK
PAUSED
PERMISSION_DENIED
PROGRESS
QUOTA_EXCEEDED
STARTED
SUCCEEDED
TIMEOUT
UNKNOWN
UNPAUSED
VERIFY_CHECKSUM

2.3. Limited pause rule document

The limited pause rule document is a subset of the full pause rule document. It does not contain sensitive fields, in particular modified_by, which is only viewable by endpoint administrators using the Advanced Endpoint Management API. The DATA_TYPE is "pause_rule_limited" instead of "pause_rule", to indicate the exclusions. See pause_rule document for details.

3. Path Arguments

Name	Type	Description
task_id	string	Unique id string of a task.

4. Common Query Parameters

Name	Type	Description
fields	string	Comma separated list of fields to include in the response. This can be used to save bandwidth on large list responses when not all fields are needed.
limit	int	For paged resources, change the page size. For event_list and task_list, the default is 10 and the max is 1000.
offset	int	For paged resources, specify an offset within the full result set. Typically a fixed page size is specified with limit, and offset is incremented by the page size to fetch each page.
orderby	string	For paged resources, a comma separated list of order by options. Each order by option is either a field name, or a field name followed by space and 'ASC' or 'DESC' for ascending and descending; ascending is the default. Note that only certain fields are supported for ordering; see the specific operation documentation for details.
filter	string	For paged resources, return only resources that match all of the specified filter criteria. The value must be a "/" separated list of "FIELD_NAME:FIELD_FILTER" strings. See the event_list and task_list documentation for details on what fields are supported for each.

5. Common Errors

Code	HTTP Status	Description
TaskNotFound	404	If task specified by <task_id> is not found
Conflict	409	If task is complete and can’t be updated.
ServiceUnavailable	503	If the service is down for maintenance.

6. Operations

6.1. Get task list

Get a paged list of tasks submitted by the current user. Returns only returns TRANSFER tasks by default; clients should pass query parameter filter=type:TRANSFER,DELETE to get all tasks.

Note

In the next release, this will be changed so all tasks of any type will be returned by default. The explicit filter including both types will continue to work.

URL	/task_list?filter=type:TRANSFER,DELETE
Method	GET
Response Body	{ "DATA_TYPE": "task_list", "length": 2, "limit": 10, "offset": 20, "total": 125, "DATA": [<task document>, ...] ] }

6.1.1. Filter and Order By Options

Fields allowed in the filter and orderby query parameters.

Task List Filter Example

GET /task_list?filter=status:ACTIVE,INACTIVE/label:~experiment1*

Get tasks that are still running (status ACTIVE or INACTIVE), and have a label that begins with the string "experiment1".

Name	Type	Description
task_id	string list	Comma separated list of UUID strings. Return only tasks with the specified task ids. Only returns tasks owned by the current user.
type	string list	Comma separated list of type names (TRANSFER, DELETE). Return only tasks of the specified type(s). The default is currently TRANSFER, but this will be changing in the next release so both TRANSFER and DELETE tasks are included by default.
status	string list	Comma separated list of status codes (ACTIVE, INACTIVE, FAILED, SUCCEEDED). Return only tasks with one of the specified statuses.
label	pattern list	Comma separated list of patterns to match against the label field. Returns tasks that match any of the patterns. Each pattern is an operator, followed by a string. The operator is one of !, =, ~, or !~. The = operator requires an exact match, and ! matches anything other than the specified string. The ~ operator is a case-insentive match and allows the use of * as a wildcard. The !~ operator matches anything that does not match the pattern. For both ~ operators, a literal * can be escaped as \*, and a literal backslash can be escaped as \\. As a shortcut, if no operator is specified, = (exact match) is assumed.
request_time	datetime range	Accepts a time range, specified by a comma separated list of two ISO 8601 date/time strings. If one of the dates is ommited, it forms an open range, so "dt," returns all records with date greater or equal to dt, and ",dt" returns all records with dates less than or equal to dt. If there is no comma, it is treated in the same way as "dt,". If the time is omitted from a date/time, it’s assumed to be 00:00.
completion_time	datetime range	Like the request time filter, but for the completion time.

6.2. Get task by id

Get a single task by task id. All fields are included by default, but the fields query parameter can be used to fetch only specific fields.

URL	/task/<task_id>
Method	GET
Response Body	Task document

6.3. Update task by id

Update a single task by task id. Only the label and deadline fields can be updated, and only on tasks that are still running. If the task is complete a "Conflict" error will be returned. A copy of the task body with one of those fields modified can be used, or a partial document containing only DATA_TYPE and the fields to be modified.

URL	/task/<task_id>
Method	PUT
Response Body	Result resource

6.4. Cancel task by id

Submit a cancel request for an active task, by id. Cancel requests are processed asynchronously, but this API call will wait up to 10 seconds for the cancel request to be completed before returning a response. If the task was already complete, result code "TaskComplete" is returned. If the cancel request is processed within 10 seconds, result code "Canceled" is returned. Note that when "Canceled" is returned, it’s still possible that the task completed successfully just as the request was processed but after the check was made to see if the task was already complete. Clients should always check the status field of the task to verify what happened if they care about whether the task succeeded or failed. What will always be true when "Canceled" or "TaskComplete" is returned is that the task is no longer active. If the cancel request can’t be processed in 10 seconds, code "CancelAccepted" is returned, and the client can use "Get task by id" to fetch the task and see when it’s status changes from "ACTIVE" to "FAILED" or "SUCCEEDED".

Only the owner of a task can cancel it via this API resource. If the owner is an administrator on one of the endpoints involved in the task, tasks canceled with this resource will still NOT be marked as as canceled_by_admin. This resource is designed for when the user is acting as a normal user, regardless of any higher level authority they have been granted.

URL	/task/<task_id>/cancel
Method	POST
Response Body	`{ "DATA_TYPE": "result", "code": "Canceled", "message": "The task has been cancelled successfully.", "resource": "/endpoint/user#ep1/cancel", "request_id": "ABCdef789" }`

6.5. Get event list

Get a paged list of all events, including error and info events. The results are ordered by time descending (newest first).

URL	/task/<task_id>/event_list
Method	GET
Response Body	{ "DATA_TYPE": "event_list", "length": 2, "limit": 2, "offset": 0, "total": 125, "DATA": [ { "code": "PROGRESS", "description": null, "DATA_TYPE": "event", "is_error": false, "details": null, "time": "2000-01-02 03:45:06+00:00" }, { "code": "FILE_NOT_FOUND", "description": "No such file or directory", "DATA_TYPE": "event", "is_error": true, "details": "Error (transfer)\nServer: go#ep1 (endpoint1.tutorial.globusonline.org:2811)\nFile: /~/doesnotexist\nCommand: MLST ~/doesnotexist\nMessage: Fatal FTP response\n---\n500-Command failed : System error in stat: No such file or directory\r\n500-A system call failed: No such file or directory\r\n500 End.\n", "time": "2014-07-28 02:35:10+00:00" } ] }

URL

/task/<task_id>/event_list

Method

GET

Response Body

{

    "DATA_TYPE": "event_list",
    "length": 2,
    "limit": 2,
    "offset": 0,
    "total": 125,
    "DATA": [
        {
            "code": "PROGRESS",
            "description": null,
            "DATA_TYPE": "event",
            "is_error": false,
            "details": null,
            "time": "2000-01-02 03:45:06+00:00"
        },
        {
            "code": "FILE_NOT_FOUND",
            "description": "No such file or directory",
            "DATA_TYPE": "event",
            "is_error": true,
            "details": "Error (transfer)\nServer: go#ep1 (endpoint1.tutorial.globusonline.org:2811)\nFile: /~/doesnotexist\nCommand: MLST ~/doesnotexist\nMessage: Fatal FTP response\n---\n500-Command failed : System error in stat: No such file or directory\r\n500-A system call failed: No such file or directory\r\n500 End.\n",
            "time": "2014-07-28 02:35:10+00:00"
        }
    ]
}

Fields allowed in the filter query parameter.

Event List Filter Example

GET /event_list?filter=is_error:1

Get only error events.

Name	Type	Description
is_error	boolean	"1" for true, "0" for false. If true, return only events that are classified as errors. If false, return only events that are classified as non-error (informational). By default, returns all events.

6.5.2. Errors

Code	HTTP Status	Description
HistoryDeleted	409	After 30 days, the event list for a task is no longer available and this error will be returned.

6.6. Get task successful transfers

For "TRANSFER" tasks that are completed (have status "SUCCEEDED" or "FAILED"), get a list of files that were successfully transferred. The list includes the source and destination paths of each file. Note that this does not include files that were checked but skipped as part of a sync transfer, only files that were actually transferred, and does not include any directories.

A result set may be too large to fit into a single response. In this case, the response will have the next_marker field set to a non-empty, opaque integer token. Pass this token as the marker query parameter in another request to get the next chunk of data. A client must not attempt to generate or assume knowledge of a marker’s format. Note that this differs from the offset based paging supported by some other resources.

Returns 404 "ClientError.NotFound" if history has been deleted (history is kept for only one month ). Returns 400 "ClientError.BadRequest" if the task is not yet complete or is not of type "TRANSFER".

Note

The paths are relative to the shared endpoint root, as submitted in the transfer request, not relative to the host endpoint root.

URL	/task/<task_id>/successful_transfers [?marker=MARKER]
Method	GET
Response Body	`{ "DATA_TYPE": "successful_transfers", "marker": 0, "next_marker": null, "DATA": [ { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file1.txt", "destination_path": "/~/test_godata_copy/file1.txt" }, { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file2.txt", "destination_path": "/~/test_godata_copy/file2.txt" }, { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file3.txt", "destination_path": "/~/test_godata_copy/file3.txt" } ] }`

URL

/task/<task_id>/successful_transfers [?marker=MARKER]

Method

GET

Response Body

{
  "DATA_TYPE": "successful_transfers",
  "marker": 0,
  "next_marker": null,
  "DATA": [
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file1.txt",
      "destination_path": "/~/test_godata_copy/file1.txt"
    },
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file2.txt",
      "destination_path": "/~/test_godata_copy/file2.txt"
    },
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file3.txt",
      "destination_path": "/~/test_godata_copy/file3.txt"
    }
  ]
}

6.7. Get task pause info

Get details about why a task is paused (or possibly about to be paused). This includes pause rules on source and destination endpoints (both host and shared endpoint) that affect the owner of the task, and per-task pause flags set by source endpoint and destination endpoint or source and destination shared endpoint administrators. Any pause rules that have been overridden by an administrator are not listed.

If the task is not paused, this may still return pause rules that have been created but not yet applied to the task. This is because pause rules are processed asynchronously.

If the task is complete, this will return an empty result set, meaning that pause_rules list will be empty and both pause messages will be null.

Authorization

Requires the user to be the owner of the task. To access pause info as an administrator with endpoint manager privileges, use the endpoint manager pause info operation.

Pause Rule

A pause rule is set by the administrator of an endpoint and causes all matching tasks to or from that endpoint to be paused. The rules returned by this operation have some sensitive fields removed, see the pause_rule_limited document.

URL	/task/<task_id>/pause_info
Method	GET
Response Body	{ "DATA_TYPE": "pause_info_limited", "pause_rules": [... list of pause_rule_limited documents...], "source_pause_message": null, "destination_pause_message": "Disk problems, pausing all tasks until we resolve", "source_pause_message_share": null, "destination_pause_message_share": null }

URL

/task/<task_id>/pause_info

Method

GET

Response Body

{
    "DATA_TYPE": "pause_info_limited",
    "pause_rules": [... list of pause_rule_limited documents...],
    "source_pause_message": null,
    "destination_pause_message": "Disk problems, pausing all tasks until we resolve",
    "source_pause_message_share": null,
    "destination_pause_message_share": null
}

Transfer API Menu

Task Monitoring

1. Overview

1.1. Task Visibility

2. Document Types

2.1. Task Document

2.1.1. Task Fields

2.2. Event Document

2.2.1. Event Fields

List of Event Codes

2.3. Limited pause rule document

3. Path Arguments

4. Common Query Parameters

5. Common Errors

6. Operations

6.1. Get task list

6.1.1. Filter and Order By Options

6.2. Get task by id

6.3. Update task by id

6.4. Cancel task by id

6.5. Get event list

6.5.1. Filter Options

6.5.2. Errors

6.6. Get task successful transfers

6.7. Get task pause info