Globus can be used to access and share content from AWS S3. This document describes how to establish and use Globus to access and share data you have access to on AWS S3.
You will use a Globus Connect Server with the AWS S3 Connector run by your institution, which acts as the gateway between the Globus ecosystem of endpoints and the AWS S3 Service. Such servers are referred to as "GCSv5 Connector".
By following the steps in this document, you will create a Globus guest collection that can access AWS S3 files, called "AWS S3 Share", so that Globus capabilities can be used with AWS S3 storage. By default, the content on the "AWS S3 Share" collection will only be accessible to you. You can, if you choose, explicitly set permissions to grant others access via Globus to the content after you have created the "AWS S3 Share" collection.
The following is a summary of steps needed to create your AWS S3 Share:
Find your institution’s Globus Connect Server with AWS S3 enabled (GCSv5 Connector) and open the Collection Creation app. You can get the URL to the registration app from your institution, or search for it in the endpoints page. See section 1 for details.
Using the app, register your AWS S3 credentials with the GCSv5 Connector if needed. This registration is associated with one of your identities in a Globus account for security purposes, so that only you can come back to access and manage that registration. See section 2 for details.
Create one or more AWS S3 collections using your registered AWS S3 credentials. See section 3 for details.
To begin, you must find your institution’s Globus Connect Server that supports the AWS S3 connector (GCSv5 Connector). You can get that information from your institution or search for such endpoints on the Globus Endpoints page.
Choose the "Shares" tab on the endpoint and click on the "Add a Shared Endpoint".
Select which Storage Gateway that you wish to use to create the Guest
Collection. The AWS S3 Storage Gateways will have
(S3) appended to
There are two types of AWS S3 Storage Gateways: unauthenticated, which provide access to public AWS S3 Buckets, and authenticated, which provide access to the AWS S3 Buckets your credentials have access to. If the AWS S3 Storage Gateway is for unauthenticated data access or you have already completed AWS S3 Credential Registration, then Collection creation app will proceed to the Create Your AWS S3 Collection section.
The first time you use the registration app for a Globus Connect Server, a consent screen will be presented for you to allow the app to access your AWS S3 bucket(s) with the Globus Connect Server.
The registration app prompts you to provide an Access Key ID and Secret Key. These should be credentials to an IAM account which has access permissions to the AWS S3 buckets you wish to access from Globus. See here for details on the required permissions. If you’ve already completed this step, you will not be prompted to create a credential, but you may click on "manage these credentials" to delete or create a new credential.
To create the AWS S3 Guest Collection, chose the folder you want accessible via the collection. In the "Globus Endpoint Information" section, enter information that makes your Google Drive shared endpoint easy to identify and find with the Globus search features. You can choose the directory to open by default when the endpoint is accessed.
Your AWS S3 Guest Collection has now been created and you can use it to access the contents of your AWS S3 Buckets.
At this point, only you can access the contents of the AWS S3 Buckets via the new guest collection. If desired, you can share content with others via Globus by selecting "Share data on this new endpoint with others" and setting the appropriate permissions.