Globus Automation Services

Globus automation services provide secure and reliable task orchestration across a set of heterogeneous resources. The automation services include the Flows service, which allows composition and deployment of task pipelines called flows, and Action Providers, which provide functionality for each step of the flow. An Action Provider implements a single task, which users then reference in flows as a step, to create resilient automation pipelines.

Users may define their own flows and allow other users or groups to run these flows. The Flows service provides managed and reliable execution of flows. For example, an instrument facility may define a flow that is run when data is acquired from a machine: transfer from acquisition machine to project storage, analyze the initial data once transfer succeeds, notify a user and gather a decision from the user to proceed with full analysis, continue or cancel a full analysis based on the user’s choice, and publish the results. Authorized users at the facility can run this flow for their data, and manage and monitor those runs. This provides a reproducible and managed solution for the facility, and a powerful automation capability for the users.

In keeping with the extensible Globus platform, custom Action Providers can be implemented and plugged into the ecosystem. For example, any analysis, or infrastructure service, can be wrapped as an Action Provider, allowing users to use that capability as a step in their flow. Several Globus capabilities, including data transfer and permission management, are implemented as Action Providers and available for use.

Globus Auth and Groups are used for authentication and authorization in the automation services, and fine grained policy for visibility and management of flows, and runs are supported using these IAM services.

Services

Clients

We provide a Python SDK and a command line interface as part of our client tools for use with Globus automation services.

  • Globus Automate Client Guidelines for installing the Python SDK and CLI for interacting with Flows and Action Providers.

  • The Globus Automate Command Line Interface is a text based interface for use with Globus automation services.

  • The Globus Automate Python SDK contains clients for interacting with individual Globus automation services.

    • globus_automate_client.ActionClient is a client class with methods for interacting with Action Provider interface compliant services.

    • globus_automate_client.FlowsClient is a client class for deploying and interacting with Flows in the Flows service.

  • Globus Action Provider SDK A Python SDK for implementing custom Action Providers for use in Globus automation services.

Example Flows

To illustrate some simple flow definitions, we provide three example flows: a flow that transfers data and then deletes the data at the source (Move), a flow that transfers data to one location and then to another location (2 Stage Transfer), and a flow that transfers data and sets access permissions on the data (Transfer Set Permissions). These flows can be used as intial templates for developing your own flows.


If you have any questions, please contact support@globus.org.