Task Monitoring

1. Overview

Transfer and delete are asynchronous operations, and result in a background task being created. The task id is returned from successful submission, and can be used to monitor the progress of the task.

Note:Detailed per-file status, errors, and information events are retained for 30 days and then deleted. When this happens, the history_deleted flag will be set on the task document.

1.1. Task Visibility

A task will be visible if the current primary identity is the same as the primary identity when the task was submitted. The only time this will not be the case is when two separate identities with existing task history are linked. When that happens, the task history for the linked identity that is no longer primary will not be visible. It can still be recovered and viewed by unlinking the identity and signing in to that identity separately.

2. Document Types

2.1. Task Document

The "task" document type represents a single transfer or delete submission. Other task types may be added in the future.

2.1.1. Task Fields

Field Name	JSON Type	Description
DATA_TYPE	string	Always has value "task" to indicate this document type.
task_id	string	Globally unique uuid for this task.
type	string	The type of task - either "TRANSFER" or "DELETE".
status	string	One word indicating the progress of the task: "ACTIVE" The task is in progress. "INACTIVE" The task has been suspended and will not continue without intervention. Currently, only credential expiration will cause this state. "SUCCEEDED" The task completed successfully. "FAILED" The task or one of its subtasks failed, expired, or was canceled.
fatal_error	object	For tasks with "FAILED" status, this is an object with `code` and `description` strings from the fatal event. Otherwise, this is null. Note: This field is valid only for tasks submitted after July 1, 2017.
label	string	User defined label to make finding tasks simpler.
username	string	[DEPRECATED] Use `owner_id` instead. The globus username of the user who created the task. For non-globusid.org identities, this will be a meaningless string starting with u_, and for globusid.org identities it will be the globusid.org username.
owner_id	string	Identity id of the task owner. If the task was submitted with multiple linked identities, the owner will be the primary identity.
request_time	string	The date and time the task was created, in ISO 8601 format.
completion_time	string	The date and time the task was completed or failed, in ISO 8601 format. If the task is still in progress (status "ACTIVE" or "INACTIVE"), this will be null.
deadline	string	If set when the task is created, cancel the task if it’s not finished when this date and time is reached. By default tasks will not be canceled as long as they are making progress.
source_endpoint	string	[DEPRECATED] Use `source_endpoint_id` instead. Canonical name of the source endpoint for `type` "TRANSFER" and the name of the only endpoint for `type` "DELETE".
source_endpoint_id	string	`id` of the source endpoint.
source_endpoint_display_name	string	`display_name` of the source endpoint at time of submit, or the `canonical_name` if `display_name` is not set. For tasks submitted prior to August 2015 (version 4.4), this will be the same as `source_endpoint`.
destination_endpoint	string	[DEPRECATED] Use `destination_endpoint_id` instead. Canonical name of the destination endpoint for `type` "TRANSFER", null for `type` "DELETE".
destination_endpoint_id	string	`id` of the destination endpoint if any (null for "DELETE" tasks).
destination_endpoint_display_name	string	Like `source_endpoint_display_name` but for the destination, or null for "DELETE" tasks.
sync_level	integer	The sync level used for the transfer, or null if not used. Always null for delete tasks.
encrypt_data	boolean	For transfer tasks, this will be true if the data channel is encrypted. This can happen in two ways: if one of the endpoints or host endpoints involved in the transfer has `force_encryption` set to true, or if the transfer request has the `encrypt_data` option set to true. Always false for delete tasks.
verify_checksum	boolean	The verify_checksum option used for the transfer. Always false for delete tasks.
delete_destination_extra	boolean	The delete_destination_extra option used for the transfer. Always false for delete tasks.
recursive_symlinks	string	The `recursive_symlinks` option used for transfer tasks. Always null for delete tasks.
preserve_timestamp	boolean	The preserve_timestamp option used for the transfer. Always false for delete tasks.
command	string	If the task was submitted via the CLI, this field will contain the original command line, including options, that created the task. If submitted via the Globus website or directly via the Transfer API, this will contain the string "API", followed by the API version, and optionally followed by a short string representing the client that submitted the request. The format of this field is subject to change and should not be relied upon.
history_deleted	boolean	This flag will be set for tasks older than one month, and indicates that details of the task are no longer available. In particular, subtasks and successful transfers will not be available for the task if this is true, so clients should always check this before querying for subtasks and successful transfers.
faults	int	The number of errors this task encountered. Note that certain types of faults are not fatal (for example, network communication errors) and can be successfully retried. A CANCELED or EXPIRED event is not included in this fault count.
files	int	The total number of files in a transfer task. This can grow as the task recursively scans directories.
directories	int	The total number of directories in a transfer task. This can grow as the task recursively scans directories.
symlinks	int	The total number of kept symlinks (not copied or ignored) in a transfer task. This can grow as the task recursively scans directories.
files_skipped	int	The number of files skipped because no changes were detected. This will always be zero for non-sync transfer tasks (with null `sync_level`) and for all delete tasks.
files_transferred	int	The number of files actually transferred over the network. For a successful transfer, `files` = `files_skipped` + `files_transferred`. Not used for delete tasks.
subtasks_total	int	Total number of subtasks. This includes file transfer or delete subtasks and helper subtasks such as directory expansion, so is not a reliable measure of the number of files being transferred. It can also grow over time as new files and directories are discovered in directory expansions.
subtasks_pending	int	Number of subtasks which are still in progress.
subtasks_retrying	int	This field is deprecated, and may be 0 or removed in a future release.
subtasks_succeeded	int	Number of subtasks which have completed successfully.
subtasks_expired	int	Number of subtasks which expired and were not completed.
subtasks_canceled	int	Number of subtasks which were canceled.
subtasks_failed	int	Number of subtasks which failed for reasons other than expiring or being cancelled.
bytes_transferred	int	The total number of bytes transferred summed across all subtasks.
bytes_checksummed	int	If sync level 3 is used, the number of bytes checksummed while determining which files need to be transferred.
effective_bytes_per_second	int	A simplistic calculation of bytes/second based on the start time of the task and its completion time, if applicable, or the current time. Valid for transfer tasks. Always 0 for other task types.
nice_status	string	[ALPHA] For tasks with status `ACTIVE` or `INACTIVE`, a string indicating a more detailed status of the task. For completed tasks, this will always be empty or null. If it has value `OK` or `Queued`, the task is waiting for other tasks to complete or proceeding normally, and no intervention should be required. All other values indicate that some error is being encountered, which may or may not resolve on it’s own. Examples: `Creds Expired`, `PERMISSION_DENIED`, `ENDPOINT_ERROR`, `CONNECT_FAILED`, `PAUSED_BY_ADMIN`.
nice_status_details	string	[ALPHA] Raw error output
nice_status_short_description	string	[ALPHA] 3-4 word description of nice_status code
nice_status_expires_in	string	[ALPHA] Seconds until any credential required for the task expires (-1=never, 0=expired)
canceled_by_admin	string	[ALPHA] If the task completes successfully, is canceled by the task owner using the standard cancel resource, or hits the deadline before completing, this will be null. It is set only for tasks canceled by an endpoint administrator using the Advanced Endpoint Management API. For such tasks, it will have one of the following values: "SOURCE" An administrator of the source endpoint of the task canceled the task. Also set for "DELETE" type tasks which involve only one endpoint "DESTINATION" An administrator of the destination endpoint of the task canceled the task "BOTH" An administrator of both the source and destination endpoint canceled the task.
canceled_by_admin_message	string	[ALPHA] For tasks with `canceled_by_admin` set to a non-null value, this will contain a message from the administrator who canceled the task.
is_paused	boolean	"true" if the task is in progress (status "ACTIVE" or "INACTIVE") and has been paused by the administrator of the source or destination endpoint, "false" if the task has not been paused or is complete (status "SUCCEEDED" or "FAILED"). Use Get task pause info to get information about why the task is paused.

Example Task Document

{
    "DATA_TYPE": "task",
    "bytes_checksummed": 4,
    "bytes_transferred": 0,
    "canceled_by_admin": null,
    "canceled_by_admin_message": null,
    "command": "API",
    "completion_time": "2017-08-15 15:23:31+00:00",
    "deadline": "2017-08-16 15:23:30+00:00",
    "delete_destination_extra": false,
    "destination_endpoint": "go#ep2",
    "destination_endpoint_display_name": "Test Endpoint",
    "destination_endpoint_id": "5d3c6d21-5244-11e5-84dd-22000bb3f45d",
    "directories": 0,
    "effective_bytes_per_second": 0,
    "encrypt_data": false,
    "fatal_error": null,
    "faults": 0,
    "files": 1,
    "files_skipped": 1,
    "files_transferred": 0,
    "history_deleted": false,
    "is_paused": false,
    "label": null,
    "nice_status": null,
    "nice_status_details": null,
    "nice_status_expires_in": null,
    "nice_status_short_description": null,
    "owner_id": "e7f4cc10-cb98-11e5-a345-cfce8e17a360",
    "preserve_timestamp": false,
    "recursive_symlinks": "ignore",
    "request_time": "2017-08-15 15:23:30+00:00",
    "source_endpoint": "go#ep1",
    "source_endpoint_display_name": "Test Endpoint",
    "source_endpoint_id": "5d3c6c59-5244-11e5-84dd-22000bb3f45d",
    "status": "SUCCEEDED",
    "subtasks_canceled": 0,
    "subtasks_expired": 0,
    "subtasks_failed": 0,
    "subtasks_pending": 0,
    "subtasks_retrying": 0,
    "subtasks_succeeded": 1,
    "subtasks_total": 1,
    "symlinks": 0,
    "sync_level": 3,
    "task_id": "b156d99b-81cd-11e7-a32b-0eff1506ce8e",
    "type": "TRANSFER",
    "verify_checksum": true
}

2.2. Event Document

Events are logged as a task makes progress or runs into errors.

Example Event Document

{
    "DATA_TYPE": "event",
    "code": "PERMISSION_DENIED",
    "description": "Permission denied",
    "details": "Error (transfer)\nServer: ballen#uc-laptop (Globus Connect)\nFile: /~/Downloads/plus-plan-exposure.png\nCommand: STOR ~/Downloads/plus-plan-exposure.png\nMessage: Fatal FTP response\n---\n500 Command failed. : Path not allowed.\n",
    "is_error": True,
    "time": "2014-07-08 18:50:18+00:00"
}

2.2.1. Event Fields

Field Name	JSON Type	Description
DATA_TYPE	string	Always has value "event" to indicate this document type.
code	string	A code indicating the type of the event.
is_error	boolean	true if event is an error event
description	string	A description of the event.
details	string	Type specific details about the event.
time	string	The date and time the event occurred, in ISO 8601 format (YYYY-MM-DD HH:MM:SS) and UTC.

2.3. Limited pause rule document

The limited pause rule document is a subset of the full pause rule document. It does not contain sensitive fields, in particular modified_by, which is only viewable by endpoint administrators using the Advanced Endpoint Management API. The DATA_TYPE is "pause_rule_limited" instead of "pause_rule", to indicate the exclusions. See pause_rule document for details.

3. Path Arguments

Name	Type	Description
task_id	string	Unique id string of a task.

4. Common Query Parameters

Name	Type	Description
fields	string	Comma separated list of fields to include in the response. This can be used to save bandwidth on large list responses when not all fields are needed.
limit	int	For paged resources, change the page size. For `event_list` and `task_list`, the default is 10 and the max is 1000.
offset	int	For paged resources, specify an offset within the full result set. Typically a fixed page size is specified with limit, and offset is incremented by the page size to fetch each page.
orderby	string	For paged resources, a comma separated list of order by options. Each order by option is either a field name, or a field name followed by space and ASC or DESC for ascending and descending; ascending is the default. Note that only certain fields are supported for ordering; see the specific operation documentation for details.
filter	string	For paged resources, return only resources that match all of the specified filter criteria. The value must be a "/" separated list of of "FIELD_NAME:FIELD_FILTER" strings. See the `event_list` and `task_list` documentation for details on what fields are supported for each.

5. Common Errors

Code	HTTP Status	Description
TaskNotFound	404	If task specified by <task_id> is not found
Conflict	409	If task is complete and can’t be updated.
ServiceUnavailable	503	If the service is down for maintenance.

6. Operations

6.1. Get task list

Get a paged list of tasks submitted by the current user. Returns only returns TRANSFER tasks by default; clients should pass query parameter filter=type:TRANSFER,DELETE to get all tasks.

Note:In the next release, this will be changed so all tasks of any type will be returned by default. The explicit filter including both types will continue to work.

URL	/task_list?filter=type:TRANSFER,DELETE
Method	GET
Response Body	{ "DATA_TYPE": "task_list", "length": 2, "limit": 10, "offset": 20, "total": 125, "DATA": [<task document>, ...] ] }

6.1.1. Filter and Order By Options

Fields allowed in the filter and orderby query parameters.

Task List Filter Example

GET /task_list?filter=status:ACTIVE,INACTIVE/label:~experiment1*

Get tasks that are still running (status ACTIVE or INACTIVE), and have a label that begins with the string "experiment1".

Name	Type	Description
task_id	string list	Comma separated list of UUID strings. Return only tasks with the specified task ids. Only returns tasks owned by the current user.
type	string list	Comma separated list of type names (TRANSFER, DELETE). Return only tasks of the specified type(s). The default is currently TRANSFER, but this will be changing in the next release so both TRANSFER and DELETE tasks are included by default.
status	string list	Comma separated list of status codes (ACTIVE, INACTIVE, FAILED, SUCCEEDED). Return only tasks with one of the specified statuses.
label	pattern list	Comma separated list of patterns to match against the label field. Returns tasks that match any of the patterns. Each pattern is an operator, followed by a string. The operator is one of `!`, `=`, , or `!`. The `=` operator requires an exact match, and `!` matches anything other than the specified string. The operator is a case-insentive match and allows the use of as a wildcard. The `!` operator matches anything that does not match the pattern. For both `~` operators, a literal can be escaped as `\*`, and a literal backslash can be escaped as `\\`. As a shortcut, if no operator is specified, `=` (exact match) is assumed.
request_time	datetime range	Accepts a time range, specified by a comma separated list of two ISO 8601 date/time strings. If one of the dates is ommited, it forms an open range, so "dt," returns all records with date greator or equal to dt, and ",dt" returns all records with dates less than or equal to dt. If there is no comma, it is treated in the same way as "dt,". If the time is omitted from a date/time, it’s assumed to be 00:00.
completion_time	datetime range	Like the request time filter, but for the completion time.

6.2. Get task by id

Get a single task by task id. All fields are included by default, but the fields query parameter can be used to fetch only specific fields.

URL	/task/<task_id>
Method	GET
Response Body	Task document

6.3. Update task by id

Update a single task by task id. Only the label and deadline fields can be updated, and only on tasks that are still running. If the task is complete a "Conflict" error will be returned. A copy of the task body with one of those fields modified can be used, or a partial document containing only DATA_TYPE and the fields to be modified.

URL	/task/<task_id>
Method	PUT
Response Body	Result resource

6.4. Cancel task by id

Submit a cancel request for an active task, by id. Cancel requests are processed asynchronously, but this API call will wait up to 10 seconds for the cancel request to be completed before returning a response. If the task was already complete, result code "TaskComplete" is returned. If the cancel request is processed within 10 seconds, result code "Canceled" is returned. Note that when "Canceled" is returned, it’s still possible that the task completed successfully just as the request was processed but after the check was made to see if the task was already complete. Clients should always check the status field of the task to verify what happened if they care about whether the task succeeded or failed. What will always be true when "Canceled" or "TaskComplete" is returned is that the task is no longer active. If the cancel request can’t be processed in 10 seconds, code "CancelAccepted" is returned, and the client can use "Get task by id" to fetch the task and see when it’s status changes from "ACTIVE" to "FAILED" or "SUCCEEDED".

Only the owner of a task can cancel it via this API resource. If the owner is an administrator on one of the endpoints involved in the task, tasks canceled with this resource will still NOT be marked as as canceled_by_admin. This resource is designed for when the user is acting as a normal user, regardless of any higher level authority they have been granted.

URL	/task/<task_id>/cancel
Method	POST
Response Body	{ "DATA_TYPE": "result", "code": "Canceled", "message": "The task has been cancelled successfully.", "resource": "/endpoint/user#ep1/cancel", "request_id": "ABCdef789" }

6.5. Get event list

Get a paged list of all events, including error and info events. The results are ordered by time descending (newest first).

URL	/task/<task_id>/event_list
Method	GET
Response Body	{ "DATA_TYPE": "event_list", "length": 2, "limit": 2, "offset": 0, "total": 125, "DATA": [ { "code": "PROGRESS", "description": null, "DATA_TYPE": "event", "is_error": false, "details": null, "time": "2000-01-02 03:45:06+00:00" }, { "code": "FILE_NOT_FOUND", "description": "No such file or directory", "DATA_TYPE": "event", "is_error": true, "details": "Error (transfer)\nServer: go#ep1 (endpoint1.tutorial.globusonline.org:2811)\nFile: /~/doesnotexist\nCommand: MLST ~/doesnotexist\nMessage: Fatal FTP response\n---\n500-Command failed : System error in stat: No such file or directory\r\n500-A system call failed: No such file or directory\r\n500 End.\n", "time": "2014-07-28 02:35:10+00:00" } ] }

URL

/task/<task_id>/event_list

Method

GET

Response Body

{

    "DATA_TYPE": "event_list",
    "length": 2,
    "limit": 2,
    "offset": 0,
    "total": 125,
    "DATA": [
        {
            "code": "PROGRESS",
            "description": null,
            "DATA_TYPE": "event",
            "is_error": false,
            "details": null,
            "time": "2000-01-02 03:45:06+00:00"
        },
        {
            "code": "FILE_NOT_FOUND",
            "description": "No such file or directory",
            "DATA_TYPE": "event",
            "is_error": true,
            "details": "Error (transfer)\nServer: go#ep1 (endpoint1.tutorial.globusonline.org:2811)\nFile: /~/doesnotexist\nCommand: MLST ~/doesnotexist\nMessage: Fatal FTP response\n---\n500-Command failed : System error in stat: No such file or directory\r\n500-A system call failed: No such file or directory\r\n500 End.\n",
            "time": "2014-07-28 02:35:10+00:00"
        }
    ]
}

Fields allowed in the filter query parameter.

Event List Filter Example

GET /event_list?filter=is_error:1

Get only error events.

Name	Type	Description
is_error	boolean	"1" for true, "0" for false. If true, return only events that are classified as errors. If false, return only events that are classified as non-error (informational). By default, returns all events.

6.5.2. Errors

Code	HTTP Status	Description
HistoryDeleted	409	After 30 days, the event list for a task is no longer available and this error will be returned.

6.6. Get task successful transfers

For "TRANSFER" tasks that are completed (have status "SUCCEEDED" or "FAILED"), get a list of files that were successfully transferred. The list includes the source and destination paths of each file. Note that this does not include files that were checked but skipped as part of a sync transfer, only files that were actually transferred, and does not include any directories.

A result set may be too large to fit into a single response. In this case, the response will have the next_marker field set to a non-empty, opaque integer token. Pass this token as the marker query parameter in another request to get the next chunk of data. A client must not attempt to generate or assume knowledge of a marker’s format. Note that this differs from the offset based paging supported by some other resources.

Returns 404 "ClientError.NotFound" if history has been deleted (history is kept for only one month ). Returns 400 "ClientError.BadRequest" if the task is not yet complete or is not of type "TRANSFER".

Note:The paths are relative to the shared endpoint root, as submitted in the transfer request, not relative to the host endpoint root.

URL	/task/<task_id>/successful_transfers [?marker=MARKER]
Method	GET
Response Body	{ "DATA_TYPE": "successful_transfers", "marker": 0 "next_marker": null, "DATA": [ { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file1.txt" "destination_path": "/~/test_godata_copy/file1.txt", }, { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file2.txt" "destination_path": "/~/test_godata_copy/file2.txt", }, { "DATA_TYPE": "successful_transfer", "source_path": "/share/godata/file3.txt" "destination_path": "/~/test_godata_copy/file3.txt", } ], }

URL

/task/<task_id>/successful_transfers [?marker=MARKER]

Method

GET

Response Body

{
  "DATA_TYPE": "successful_transfers",
  "marker": 0
  "next_marker": null,
  "DATA": [
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file1.txt"
      "destination_path": "/~/test_godata_copy/file1.txt",
    },
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file2.txt"
      "destination_path": "/~/test_godata_copy/file2.txt",
    },
    {
      "DATA_TYPE": "successful_transfer",
      "source_path": "/share/godata/file3.txt"
      "destination_path": "/~/test_godata_copy/file3.txt",
    }
  ],
}

6.7. Get task pause info

Get details about why a task is paused (or possibly about to be paused). This incudes pause rules on source and destination endpoints (both host and shared endpoint) that affect the owner of the task, and per-task pause flags set by source endpoint and destination endpoint or source and destination shared endpoint administrators. Any pause rules that have been overridden by an administrator are not listed.

If the task is not paused, this may still return pause rules that have been created but not yet applied to the task. This is because pause rules are processed asynchronously.

If the task is complete, this will return an empty result set, meaning that pause_rules list will be empty and both pause messages will be null.

Authorization

Requires the user to be the owner of the task. To access pause info as an administrator with endpoint manager privileges, use the endpoint manager pause info operation.

Pause Rule

A pause rule is set by the administrator of an endpoint and causes all matching tasks to or from that endpoint to be paused. The rules returned by this operation have some sensitive fields removed, see the pause_rule_limited document.

URL	/task/<task_id>/pause_info
Method	GET
Response Body	{ "DATA_TYPE": "pause_info_limited", "pause_rules": [... list of pause_rule_limited documents...], "source_pause_message": null, "destination_pause_message": "Disk problems, pausing all tasks until we resolve", "source_pause_message_share": null, "destination_pause_message_share": null }

URL

/task/<task_id>/pause_info

Method

GET

Response Body

{
    "DATA_TYPE": "pause_info_limited",
    "pause_rules": [... list of pause_rule_limited documents...],
    "source_pause_message": null,
    "destination_pause_message": "Disk problems, pausing all tasks until we resolve",
    "source_pause_message_share": null,
    "destination_pause_message_share": null
}

Transfer API Menu

Task Monitoring

1. Overview

1.1. Task Visibility

2. Document Types

2.1. Task Document

2.1.1. Task Fields

2.2. Event Document

2.2.1. Event Fields

2.3. Limited pause rule document

3. Path Arguments

4. Common Query Parameters

5. Common Errors

6. Operations

6.1. Get task list

6.1.1. Filter and Order By Options

6.2. Get task by id

6.3. Update task by id

6.4. Cancel task by id

6.5. Get event list

6.5.1. Filter Options

6.5.2. Errors

6.6. Get task successful transfers

6.7. Get task pause info