Guide for applications adding support for Globus Connect Server v5
Last Updated: Oct 31, 2022
1. Introduction
Globus Connect Server version 5 has several notable differences from Globus Connect Server version 4 and Globus Connect Personal. This guide is to help developers maintaining applications that use Globus Connect Server version 4 and/or Globus Connect Personal endpoints for data access to update their applications to add support for Globus Connect Server version 5 collections. The scope of this document is primarily limited to data access, and does not cover other administrative or endpoint/collection management capabilities in depth.
2. Terminology
Globus Connect Server version 4 has two types of entities:
-
A host endpoint, which provides both (a) server and network configuration and (b) data access interfaces that use user certificates for authentication and map to a local account for data access.
-
A shared endpoint, which provides a data access interface that uses permissions created in the Transfer service to allow users data access without requiring local accounts.
Globus Connect Personal has two types of entities:
-
A mapped collection, which provides a data access interface only for the owner of the Globus Connect Personal.
-
A guest collection, which provides a data access interface that uses permissions created in the Transfer service to allow other users data access.
Globus Connect Server version 5 has four types of entities:
-
An endpoint, which manages server and network configuration across all connectors, and does not have any data access interfaces.
-
A storage gateway, which provides connector specific configuration to collections created on it and does not have any data access interfaces.
-
A mapped collection, which provides a data access interface for users with local accounts. This uses OAuth2 for authentication, and user identity is mapped to a local account.
-
A guest collection, which provides a data access interface that uses permissions created in the Transfer service to allow users data access without requiring local accounts.
The Transfer API does not use Globus Connect Server version 5 terminology and as such generally
does not use the term collection even when dealing with collections.
For example, a call to Transfer to manage permissions on a Globus Connect Server version 5 guest
collection will still have /endpoint in its urls. An exception
is the entity_type field in the Transfer endpoint document, which allows
one to determine which of the above entities the document refers to. See
entity types
in the Transfer API documentation for details.
The closest equivalent to a Globus Connect Server version 5 mapped collection is a Globus Connect Server version 4 endpoint, and the equivalent to a Globus Connect Server version 5 guest collection is a Globus Connect Server version 4 shared endpoint. There is no direct equivalent in Globus Connect Server version 4 or Globus Connect Personal to a Globus Connect Server version 5 endpoint or storage gateway, but some of the configuration options of Globus Connect Server version 4 and Globus Connect Personal endpoints are available at the endpoint or storage gateway level. Globus Connect Server version 5 endpoint and Storage Gateway configuration is generally done by administrators via globus-connect-server CLI commands and is not covered in this document.
3. Using the GCS Manager API
Globus Connect Server version 5 deployments have a GCS Manager service that provides some management and configuration capability for that particular deployment. Some of the functionality that is managed by Transfer service for Globus Connect Server version 4 and Globus Connect Personal endpoints, is managed by the GCS Manager service for Globus Connect Server version 5 endpoints. As such, Transfer API calls for managing endpoints and roles will fail if made against Globus Connect Server version 5 endpoints as they are read only in Transfer, and your application should instead use corresponding calls to the GCS Manager API.
The Globus SDK supports making calls to the GCS Manager API with the GCSClient and its helper methods. For example usage, see the Globus Connect Server version 5 guest collection creation example script.
Unlike the Transfer API, which acts as a central service for all Globus endpoints and thus has a single OAuth scope, the GCS Manager API is operated independently at each Globus Connect Server version 5 deployment. Thus to keep the least-privilege security model, the application needs to manage tokens with specific scopes for each of the Globus Connect Server version 5 endpoints.
This means if your application needs to interact with multiple Globus Connect Server version 5 endpoints, you will need to manage separate tokens for each endpoint you make calls to. SDK users may want to use TokenStorage to help manage tokens across multiple endpoints.
The table below lists the capabilities and calls that use Transfer API for Globus Connect Server version 4 and Globus Connect Personal endpoints, and their corresponding GCS Manager API calls for Globus Connect Server version 5 endpoints. These include endpoint/collection and role management, and corresponding Globus SDK methods are listed as well.
| Description | Globus Connect Server version 4 - Transfer API call/ SDK TransferClient method | Globus Connect Server version 5 - GCS Manager API/ SDK GCSClient method |
|---|---|---|
Create a GCSv4 host endpoint or GCSv5 mapped collection |
POST /endpoint |
|
Create a shared endpoint or guest collection |
POST /shared_endpoint |
|
Update an endpoint or collection |
PUT /endpoint/<endpoint_or_collection_id> |
|
Delete an endpoint or collection |
DELETE /endpoint/<endpoint_or_collection_id> |
|
List roles on an endpoint or collection |
||
Create a role on an endpoint or collection |
||
Get a role |
||
Delete a role |
Getting the definition of a Globus Connect Server version 5 collection is supported by both the Transfer and GCS Manager APIs, but will return different documents. See GET /endpoint/<endpoint_id> in the Transfer API and GET /collection/{collection_id} in the GCS Manager API for details.
4. Application changes to use Globus Connect Server version 5 mapped collections
While both Globus Connect Server version 4 host endpoints and Globus Connect Server version 5 mapped collections require mapping a user’s identity to local account to determine access, using X.509 based authentication to OAuth2 brings some significant changes in how data access occurs through the Transfer API. The calls for submitting tasks or doing synchronous data access operations remain the same, but have different prerequisites.
4.1. Activation is replaced by Consent
As mentioned in the terminology section, Globus Connect Server version 5 mapped collections use OAuth2 tokens instead of X509 certificates for user authentication. Your application does not need to handle endpoint activation of any kind when using Globus Connect Server version 5 collections.
High assurance Globus Connect Server version 5 mapped collections do not require a data_access consent since they have additional session requirements. For more information see the High Assurance Collections guide.
Instead, users will need to consent to the Transfer service accessing the specific mapped collection on behalf of the user. This is implemented as a dependent scope ("data_access") for the specific mapped collection, on the transfer scope. Transfer API access will need to present tokens that have the dependent scope for the specific mapped collection. These consents are not time limited, and are granted once per application, unless explicitly revoked by the user.
If you know ahead of time which mapped collections your application will be
using, you should add their data_access scopes to the
Authorization Grant’s
scope field when users authenticate to your application. The scopes will be
in the format
urn:globus:auth:scope:transfer.api.globus.org:all[*https://auth.globus.org/scopes/COLLECTION_UUID/data_access]
where COLLECTION_UUID is the uuid of the mapped collection to which the user is consenting.
If you don’t know which mapped collections your application will be using, you will need to respond reactively to ConsentRequired errors raised when calls are made to mapped collections users have not yet consented to. These error responses include the required scopes in their JSON bodies, which should then be used in a new user authentication flow to get the user’s consent and new tokens.
Globus SDK users may benefit from the Best-Effort Proactive Handling of ConsentRequired and Minimal File Transfer Script Handling ConsentRequired example scripts as examples of proactive and reactive handling of ConsentRequired errors respectively.
If your application uses a
Client Credentials Grant
to get tokens to act as itself, add the required data access scopes to the
scope filed requested in that grant. Client Identities consent to scopes
just by requesting them.
4.2. Authentication Timeouts
Unlike X509 certificates, OAuth2 consents do not expire. To maintain the same
level of security as Globus Connect Server version 4 in Globus Connect Server version 5, mapped collections are configured with
an authentication timeout, visible as the authentication_timeout_mins in the
Globus Connect Server version 5 Mapped Collection document document or Transfer Endpoint document.
Users must have authenticated with an identity from one of the mapped
collection’s required domains within that timeout in order to use that mapped
collection. Otherwise some form of Permission Denied error will be returned.
These errors will be in the
session error format
with an authorization_parameters field containing the values needed for the
user to re-authenticate. Only high assurance collections will enforce that
the authentication is in the same session as your application however.
Unlike with Globus Connect Server version 4, this requirement is only enforced at the time data access is initiated. Users will not have to reauthenticate once the transfer is submitted.
Authentication timeouts do not apply to client identities. If your application uses a Client Credentials Grant to get tokens to act as itself rather than as a user, then you do not need to handle timeouts when using those tokens.
4.3. User Impact
The main differences users will notice when using your application against Globus Connect Server version 5 endpoints will be with re-authentication. They may see a different login flow that prompts them to log in as a specific identity requested by the mapped collection. The images below show what such a flow generally looks like from a user perspective.
Users may also notice that they don’t have to reactivate long lived tasks that are against Globus Connect Server version 5 endpoints unlike Globus Connect Server version 4 which require reactivation when x509 credentials expire.
If users who are used to proactively deactivating their Globus Connect Server version 4 endpoints wish
to do the same for Globus Connect Server version 5 endpoints, they should revoke their data_access
consent for that specific endpoint. They can either do this directly on the
Globus Auth Consent page or
using the "Manage Consent" shortcut on the top right of the collection overview
page as shown below.
5. Application changes to use Globus Connect Server version 5 guest collections
Globus Connect Server version 4 shared endpoints, Globus Connect Personal shared endpoints, and Globus Connect Server version 5 guest collections all use permissions created in the Globus Transfer service to allow users access, so very little needs to change in your application to support managing permissions and accessing data on shared endpoints or guest collections. If your application also needs to create guest collections or manage roles on them, then you will need to make calls to the GCS Manager API as described in the above Using the GCS Manager API section.
As with mapped collections, the Transfer API calls for submitting tasks and other data access operations are the same between shared endpoints and guest collections.
Permission management in the Transfer API is also identical between shared endpoints and guest collections.