Guide for applications adding support for Globus Connect Server v5

Last Updated: March 7, 2022

1. Introduction

Globus Connect Server version 5 has several notable differences from Globus Connect Server version 4 and Globus Connect Personal. This guide is to help developers maintaining applications that use Globus Connect Server version 4 and/or Globus Connect Personal endpoints for data access to update their applications to add support for Globus Connect Server version 5 collections. The scope of this document is primarily limited to data access, and does not cover other administrative or endpoint/collection management capabilities in depth.

2. Terminology

Globus Connect Server version 4 and Globus Connect Personal have two types of entities:

  • An endpoint, which provides both (a) server and network configuration and (b) data access interfaces that use user certificates for authentication and map to a local account for data access.

  • A shared endpoint, which provides a data access interface that uses permissions created in the Transfer service to allow users data access without requiring local accounts.

Globus Connect Server version 5 has four types of entities:

  • An endpoint, which manages server and network configuration across all connectors, and does not have any data access interfaces.

  • A storage gateway, which provides connector specific configuration to collections created on it and does not have any data access interfaces.

  • A mapped collection, which provides a data access interface for users with local accounts. This uses OAuth2 for authentication, and user identity is mapped to a local account.

  • A guest collection, which provides a data access interface that uses permissions created in the Transfer service to allow users data access without requiring local accounts.

The Transfer API does not use Globus Connect Server version 5 terminology and as such generally does not use the term collection even when dealing with Globus Connect Server version 5 collections. For example, a call to Transfer to manage ACL permissions on a Globus Connect Server version 5 guest collection will still have /endpoint in its urls.

The closest equivalent tedundant witho a standard Globus Connect Server version 5 mapped collection is a Globus Connect Server version 4 endpoint, and the equivalent to Globus Connect Server version 5 guest collection is a Globus Connect Server version 4 shared endpoint. There is no direct equivalent in Globus Connect Server version 4 or Globus Connect Personal to a Globus Connect Server version 5 endpoint or storage gateway, but some of the configuration options of Globus Connect Server version 4 and Globus Connect Personal endpoints are available at the endpoint or storage gateway level. Globus Connect Server version 5 endpoint and Storage Gateway configuration is generally done by administrators via globus-connect-server CLI commands and is not covered in this document.

3. Using the GCS Manager API

Globus Connect Server version 5 deployments have a GCS Manager service that provides some management and configuration capability for that particular deployment. Some of the functionality that is managed by Transfer service for Globus Connect Server version 4 and Globus Connect Personal endpoints, is managed by the GCS Manager service for Globus Connect Server version 5 endpoints. As such, Transfer API calls for managing endpoints and roles will fail if made against Globus Connect Server version 5 endpoints as they are read only in Transfer, and your application should instead use corresponding calls to the GCS Manager API.

Unlike the Transfer API, which acts as a central service for all Globus endpoints and thus has a single OAuth scope, the GCS Manager API is operated independently at each Globus Connect Server version 5 deployment. Thus to keep the least-privilege security model, the application needs to manage tokens with specific scopes for each of the Globus Connect Server version 5 endpoints.

This means if your application needs to interact with multiple Globus Connect Server version 5 endpoints, you will need to manage separate tokens for each endpoint you make calls to. SDK users may want to use TokenStorage to help manage tokens across multiple endpoints.

The table below lists the capabilities and calls that use Transfer API for Globus Connect Server version 4 and Globus Connect Personal endpoints, and their corresponding GCS Manager API calls for Globus Connect Server version 5 endpoints. These include endpoint/collection and role management, and corresponding Globus SDK methods are listed as well.

Description Globus Connect Server version 4 - Transfer API call/ SDK TransferClient method Globus Connect Server version 5 - GCS Manager API/ SDK GCSClient method

Create a GCSv4 host endpoint or GCSv5 mapped collection

POST /endpoint

create_endpoint

POST /collections

create_collection

Create a shared endpoint or guest collection

POST /shared_endpoint

create_shared_endpoint

POST /collections

create_collection

Update an endpoint or collection

PUT /endpoint/<endpoint_xic>

update_endpiont

PATCH /collections/{collection_id}

update_collection

Delete an endpoint or collection

DELETE /endpoint/<endpoint_xid>

delete_endpoint

DELETE /collections/{collection_id}

delete_collection

List roles on an endpoint or collection

GET /endpoint/<endpoint_xid>/role_list

endpoint_role_list

GET /roles

get_role_list

Create a role on an endpoint or collection

POST /endpoint/<endpoint_xid>/role

add_endpoint_role

POST /roles

create_role

Get a role

GET /endpoint/<endpoint_xid>/role/<role_id>

get_endpoint_role

GET /role/{role_id}

get_role

Delete a role

DELETE /endpoint/<endpoint_xid>/role/<role_id>

delete_endpoint_role

DELETE /roles/{role_id}

delete_role

Note

Getting the definition of a Globus Connect Server version 5 collection is supported by both the Transfer and GCS Manager APIs, but will return different documents. See GET /endpoint/<endpoint_xid> in the Transfer API and GET /collection/{collection_id} in the GCS Manager API for details.

4. Determining endpoint or collection type

If you application needs to support a mix of Globus Connect Server version 4, Globus Connect Server version 5, and Globus Connect Personal endpoints and collections, then you may need to be able to determine what type of endpoint or collection you are dealing with to make the correct API calls to Transfer or the GCS Manager.

The most reliable way to determine what type an endpoint or collection your application is dealing with is a Transfer endpoint document.

If your application is using Transfer’s endpoint search (TransferClient.endpoint_search in the SDK) to discover endpoints and collections, then the documents returned will be sufficient.

If your application only has the UUID of an endpoint or collection, then you can make a GET /endpoint/<endpoint_xid> API call (TransferClient.get_endpoint in the SDK) to get a Transfer endpoint document for that endpoint or collection.

Once you have a Transfer endpoint document, the endpoint or collection type can be determined from specific fields within as described in the Transfer API’s endpoint types documentation

5. Application changes to use Globus Connect Server version 5 mapped collections

While both Globus Connect Server version 4 host endpoints and Globus Connect Server version 5 mapped collections require mapping a user’s identity to local account to determine access, using X.509 based authentication to OAuth2 brings some significant changes in how data access occurs through the Transfer API. The calls for submitting tasks or doing synchronous data access operations remain the same, but have different prerequisites.

As mentioned in the terminology section, Globus Connect Server version 5 mapped collections use OAuth2 tokens instead of X509 certificates for user authentication. Your application does not need to handle Endpoint Activation of any kind when using Globus Connect Server version 5 collections.

Note

High assurance Globus Connect Server version 5 mapped collections do not require a data_access consent since they have additional session requirements. For more information see the High Assurance Collections guide.

Instead, users will need to consent to the Transfer service accessing the specific mapped collection on behalf of the user. This is implemented as a dependent scope ("data_access") for the specific mapped collection, on the transfer scope. Transfer API access will need to present tokens that have the dependent scope for the specific mapped collection. These consents are not time limited, and are granted once per application, unless explicitly revoked by the user.

If you know ahead of time which mapped collections your application will be using, you should add their data_access scopes to the Authorization Grant’s scope field when users authenticate to your application. The scopes will be in the format urn:globus:auth:scope:transfer.api.globus.org:all[*https://auth.globus.org/scopes/COLLECTION_UUID/data_access] where COLLECTION_UUID is the uuid of the mapped collection to which the user is consenting.

If you don’t know which mapped collections your application will be using, you will need to respond reactively to ConsentRequired errors raised when calls are made to mapped collections users have not yet consented to. These error responses include the required scopes in their JSON bodies, which should then be used in a new user authentication flow to get the user’s consent and new tokens.

Note

If your application uses a Client Credentials Grant to get tokens to act as itself, add the required data access scopes to the scope filed requested in that grant. Client Identities consent to scopes just by requesting them.

5.2. Authentication Timeouts

Unlike X509 certificates, OAuth2 consents do not expire. To maintain the same level of security as Globus Connect Server version 4 in Globus Connect Server version 5, mapped collections are configured with an authentication timeout, visible as the authentication_timeout_mins in the Globus Connect Server version 5 Mapped Collection document document or Transfer Endpoint document.

Users must have authenticated with an identity from one of the mapped collection’s required domains within that timeout in order to use that mapped collection. Otherwise some form of Permission Denied error will be returned. These errors will be in the session error format with an authorization_parameters field containing the values needed for the user to re-authenticate. Only high assurance collections will enforce that the authentication is in the same session as your application however.

Unlike with Globus Connect Server version 4, this requirement is only enforced at the time data access is initiated. Users will not have to reauthenticate once the transfer is submitted.

Note

Authentication timeouts do not apply to client identities. If your application uses a Client Credentials Grant to get tokens to act as itself rather than as a user, then you do not need to handle timeouts when using those tokens.

5.3. User Impact

The main differences users will notice when using your application against Globus Connect Server version 5 endpoints will be with re-authentication. They may see a different login flow that prompts them to log in as a specific identity requested by the mapped collection. The images below show what such a flow generally looks like from a user perspective.

File Manager
Authentication Required using Globus
Consent Request using Globus

Users may also notice that they don’t have to reactivate long lived tasks that are against Globus Connect Server version 5 endpoints unlike Globus Connect Server version 4 which require reactivation when x509 credentials expire.

If users who are used to proactively deactivating their Globus Connect Server version 4 endpoints wish to do the same for Globus Connect Server version 5 endpoints, they should revoke their data_access consent for that specific endpoint. They can either do this directly on the Globus Auth Consent page or using the "Manage Consent" shortcut on the top right of the collection overview page as shown below.

Collection Overview

6. Application changes to use Globus Connect Server version 5 guest collections

Globus Connect Server version 4 shared endpoints, Globus Connect Personal shared endpoints, and Globus Connect Server version 5 guest collections all use permissions created in the Globus Transfer service to allow users access, so very little needs to change in your application to support managing permissions and accessing data on shared endpoints or guest collections. If your application also needs to create guest collections or manage roles on them, then you will need to make calls to the GCS Manager API as described in the above Using the GCS Manager API section.

As with mapped collections, the Transfer API calls for submitting tasks and other data access operations are the same between shared endpoints and guest collections.

Permission management in the Transfer API is also identical between shared endpoints and guest collections.

6.1. User Impact

Users likely won’t notice any difference between using shared endpoints and guest collections other than the change in terminology.