Globus automation services provide secure and reliable task orchestration across a set of heterogeneous resources at scale. These services can be used to automate tasks as simple as replicating data across multiple storage systems, or as intricate as managing multiple conditional data analysis and results distribution tasks, with optional human intervention where needed for say review and confirmation. Globus automation services include the Flows API, which enables composition and deployment of task pipelines called flows, and action providers, which provide functionality for each step of the flow.

The Globus automation overview provides a detailed guide to the ecosystem that delivers these capabilities.


1. Running Flows

When a user starts a flow, it is called a flow run, or simply a run. Users may search for flows to run, provide input parameters, and start a run. Once a run has started, users can monitor and manage the run, including view the status, cancel, and grant permission to others to manage or monitor the run.

How to Start a Run and How to Monitor a Run provide step-by-step instructions for using the Globus Web App to start and monitor runs.

A command line interface is also available to start runs, and monitor them. Runs can also be started automatically using the CLI, including triggering the start of a flow based on file system events. Some examples of starting runs automatically are provided.

Globus provides a couple flows that you can run by using the Globus Web App:

2. Creating Flows

Flows are a series of tasks that are executed in a specified order. The Globus Flows service provides managed and reliable execution of flows. Users may define their own flows and allow other users or groups to see, run, or manage these flows. Globus Auth and Groups services are used for authentication and authorization in Flows.

Each task, or step, in the flow calls an action provider, which can act on a resource in either Globus services or externally managed services. An action provider implements a single task, which users then reference in a flow as a step to create resilient automation pipelines. Several Globus capabilities, including data transfer and permission management, are implemented as Globus-provided action providers and are available for use. Action providers double as a powerful extension point, allowing users to define and run their own custom action providers to expand the set of tools that can be leveraged in flows.

A flow is written in two parts: A flow definition, which defines the steps in a flow and the logic between them, and an input schema, which defines the structure of the input parameters for a run of a flow. Both the flow definition and the input schema are written in either JSON or YAML. Once the flow definition and input schema are written, the flow is registered with Globus using the flow deploy command of the Automate CLI. After a flow is deployed, the CLI or Globus Web App can be used to manage the flow and assign roles to the flow to determine who can see, run, or manage the flow.

How to Create a Flow gives an overview of using the Automate CLI to create, deploy, and manage a flow. We provide example flow definitions to demonstrate some simple operations that can be used as initial templates for developing flows. Our flow authoring guide provides a detailed description of how to write a flow definition and input schema.

4. Creating Action Providers

In addition to Globus-provided action providers, you can create custom action providers to incorporate other applications and capabilities into flows. For example, if you have developed a service to perform a domain-specific analysis, you can create an action provider that allows you and others to apply that analysis as an action in a Globus flow.

Globus Action Provider Tools provide a Python library for implementing custom action providers for use in Globus Flows.

5. Tutorial Resources

You can immediately start experimenting with creating and running a simple flow to move and share data using our Jupyter notebook, or watch an overview of automating instrument data handling or a tutorial on building and running your own flows.

6. CLI, SDK, and APIs

We provide a Python SDK and a command line interface as part of our client tools for use with Globus Flows and the action provider API.

7. Mailing List

Release updates, feature announcements, and discussions for all Globys services take place on the Globus Discuss mailing list: discuss@globus.org

If you have any questions, please contact support@globus.org.