Require a Flow for Data Movement

Managing data through its lifecycle requires various actions on the data dictated by several factors ranging from technical constraints on the system to security policy requirements. For example, many archival systems would be better served by creating tar or zip archives before storing files, and secure enclaves may require automated or manual reviews before data egress. Globus Flows provides secure and reliable task automation for data processing. By requiring a flow to be run prior to data transfer, institutions can shield their users from the complexities of preparing data for a specific storage system or ensure compliance with technical or policy requirements, while users can use the same familiar interface to transfer data.

Associate a Flow with your Collection

An administrator can configure a collection to require a flow to be run when a user submits a transfer request. When the transfer request is submitted, the user is directed to perform the transfer by running the flow, thereby ensuring data processing prior to the transfer. The association between a flow and a collection can be set using the Globus Connect Server CLI --flow-transfer-source and --flow-transfer-destination options to collection create or collection update. When a flow is associated with a collection, the Transfer API will reject task submissions not initiated using the flow, specified in the associated_flow_policy documentation.

You must specify the direction of data transfer that requires the flow. All data movement in the configured direction is required to originate from the flow. HTTPS access in the configured direction is disabled. Only transfer tasks, not delete tasks or other operations, can require initiation from a flow.

Flow Input Schema for the File Manager

The Globus Web App simplifies the user interaction by allowing the user to submit a transfer request using the File Manager and then guiding the user through starting the flow.

An example of the File Manager dialog prompting the user to run an associated flow.

In order for the user-selected options from the File Manager to be used as input to your flow, you must include a top-level property in your flow input schema that implements the "globus-transfer-transfer#0.10" format.

An example input schema using "globus-transfer-transfer#0.10" 
{
  "title": "Start a Transfer",
  "type": "object",
  "required": [
    "file_manager_settings"
  ],
  "properties": {
    "file_manager_settings": {
      "type": "object",
      "format": "globus-transfer-transfer#0.10"
    }
  },
  "additionalProperties": false
}

The object provided by the File Manager to your flow will match the Transfer API Task - Transfer and Delete Documents.

It is up to you how the values from the File Manager are used as input to your flow.

Common Errors

Invalid Flow Input Schema

If the flow you associated with your collection does not have an input schema defined with the "globus-transfer-transfer#0.10" format, the Web App will display an error message when users attempt to start your flow from the File Manager. Users may still run your associated flow by navigating to the flow start page, but all File Manager state will be lost.

An example of a the Flow Guided Start page displaying an error message about the flow input schema.