Globus Connect Server Administration Guides
  • Quickstart Guide
  • Installation Guide
  • Data Access Admin Guide
  • Domain Guide
  • HTTPS Access to Collections
  • Identity Mapping Admin Guide
  • Globus OIDC Installation Guide
  • v5.3 Migration Guide
  • Troubleshooting Guide
  • Command-Line Reference
    • Audit
      • Load
      • Query
      • Dump
    • Endpoint
      • Setup
      • Show
      • Update
      • Reset Advertised Owner String
      • Set Advertised Owner String
      • Set Owner
      • Set Subscription ID
      • Migrate53
      • Cleanup
      • Domain
      • Role
      • Upgrade
    • OIDC
      • Create
      • Delete
      • Register
      • Show
      • Update
    • Node
      • Create
      • Setup
      • List
      • Show
      • Update
      • Cleanup
      • Update Vhost
      • Delete
    • Login
    • Session
      • Consent
      • Show
      • Update
    • Whoami
    • Logout
    • Storage Gateway
      • Create
      • List
      • Show
      • Update
      • Delete
    • Collection
      • Create
      • List
      • Show
      • Reset Advertised Owner String
      • Set Advertised Owner String
      • Update
      • Delete
      • Domain
      • Role
    • Sharing Policy
      • Create
      • List
      • Show
      • Delete
    • Self Diagnostic
  • Globus Connect Server Manager API
    • Responses
    • Schemas
    • Authorization
    • Versioning
    • Endpoint
    • Roles
    • Nodes
    • Storage Gateways
    • Collections
    • User Credentials
    • Domains
    • Sharing Policies
    • ActiveScale
    • Azure Blob
      • Azure Blob
    • BlackPearl
    • Box
    • Ceph
    • Google Cloud Storage
    • Google Drive
    • HPSS
    • iRODS Connector
    • OAuth Credential API
    • OneDrive
    • POSIX Connector
    • POSIX Staging Connector
    • S3
  • API Access for Portals
  • Application Migration Guide
  • Change Log
Skip to main content
Globus Docs
  • APIs
    Auth Transfer Groups Search Python SDK Helper Pages
  • How To
  • Guides
    Globus Connect Server High Assurance Collections for Protected Data Command Line Interface Premium Storage Connectors Globus Automation Services Security Modern Research Data Portal
  • Support
    FAQs Mailing Lists Contact Us Check Support Tickets
  1. Home
  2. Globus Connect Server
  3. Installation Guide
  4. Globus Connect Server Manager API
  5. S3

S3

The Globus Connect Server S3 Connector provides access to S3-compatible storage via Globus Connect Server.

This document describes the data types and API routes provided by this connector.

S3 Connector Virtual Filesystem

The S3 connector provides a distributed object store, where each data object is accessed based on a bucket name and an object name.

The S3 connector attempts to make this look like a regular filesystem, by treating the bucket name as the name of a directory in the root of the storage gateway’s file system. For example, if a user has access to buckets bucket1 and bucket2, then those buckets would show up as directories when listing /.

The S3 connector also treats the / character as a delimiter in the S3 API so that it can present something that looks like like subdirectories. For example, the object object1 in bucket1 would appear as /bucket1/object1 to the S3 connector, and the object object2/object3 in bucket2 would appear as a file called object3 in the directory /bucket2/object2.

Authenticated and Anonymous Access

Each S3 Storage Gateway can be configured to perform either authenticated or unauthenticated access to S3 data. When creating an S3 Storage Gateway, you must choose which type of access to require.

authenticated

Globus users must register an S3 Credential with Globus Connect Server in order to access data on its collections. The credential must be associated with a policy that allows the IAM permissions used by the {s3_connector_type}.

unauthenticated

Globus users can only access public AWS Buckets.

S3 Storage Gateway Policies

The s3_user_credential_required, s3_buckets, and s3_endpoint properties control access to an Amazon S3 or compatible resource.

Endpoint

The s3_endpoint property is used by Globus Connect Server to contact the S3 API to access data on this storage gateway. This may be an Amazon S3 URL, a regional Amazon S3 URL, or the URL endpoint of another compatible storage system.

Access Mode

If the s3_user_credential_required property is enabled, then each user accessing collections on this storage gateway must register an S3 key_id and secret_key with the storage gateway.

Bucket Restrictions

The s3_buckets property value is the list of bucket names which are allowed access by this storage gateway.

If no buckets are configured, then any buckets accessible using the user’s registered S3 key_id and secret_key may be accessed by collections on this storage gateway. If any are configured, then they act as restrictions to which buckets are visible and accessible on collections on this storage gateway.

S3 User Credentials

S3 user credentials can either be a placeholder for unauthenticated bucket access, or can contain an S3 key_id and secret access_key value. The type of credential needed for any particular collection is based on the configuration of the storage gateway that it uses.

Storage Gateway Allowing Anonymous Bucket Access

In order to access a collection using the S3 connector configured for anonymous bucket access, users don’t have to do anything to create a credential. It will be automatically provisioned on first use. Users may disable and reenable a credential by changing the value of the invalid property.

Storage Gateway Requiring a Key

In order to access a collection using the S3 connector configured to use a S3 key, users must supply key information to the GCS Manager. This can be most easily done via the Globus web application. If a user does not supply key information, the credential will have its invalid property set to true and must be updated by the user to include key data.

For AWS, the key provided by the user must have permissions to access buckets and objects as described in the Globus AWS documentation. Consult your storage system documentation for key requirements for other S3-compatible systems supported by Globus.

Creating a Credential

When creating a credential, the caller MUST include values for the DATA_TYPE and storage_gateway_id properties. If any of these are missing, the credential can not be created. If the identity_id value is missing, the identity with the lowest-valued identity id value in the caller’s identity set which meets the storage gateway policies is used.

In order for a credential to be valid, it must have values for the DATA_TYPE, s3_key_id and s3_secret_key in the policies document. If any of these are missing, the credential is created, but its invalid property is set to true and it cannot be used until the user has updated the credential.

Updating a Credential

When updating a credential, the caller may only change the invalid property and the s3_key_id and s3_secret_key properties in the policies document. All other fields are read-only.

  • Document Schemas

  • Quickstart Guide
  • Installation Guide
  • Data Access Admin Guide
  • Domain Guide
  • HTTPS Access to Collections
  • Identity Mapping Admin Guide
  • Globus OIDC Installation Guide
  • v5.3 Migration Guide
  • Troubleshooting Guide
  • Command-Line Reference
    • Audit
      • Load
      • Query
      • Dump
    • Endpoint
      • Setup
      • Show
      • Update
      • Reset Advertised Owner String
      • Set Advertised Owner String
      • Set Owner
      • Set Subscription ID
      • Migrate53
      • Cleanup
      • Domain
      • Role
      • Upgrade
    • OIDC
      • Create
      • Delete
      • Register
      • Show
      • Update
    • Node
      • Create
      • Setup
      • List
      • Show
      • Update
      • Cleanup
      • Update Vhost
      • Delete
    • Login
    • Session
      • Consent
      • Show
      • Update
    • Whoami
    • Logout
    • Storage Gateway
      • Create
      • List
      • Show
      • Update
      • Delete
    • Collection
      • Create
      • List
      • Show
      • Reset Advertised Owner String
      • Set Advertised Owner String
      • Update
      • Delete
      • Domain
      • Role
    • Sharing Policy
      • Create
      • List
      • Show
      • Delete
    • Self Diagnostic
  • Globus Connect Server Manager API
    • Responses
    • Schemas
    • Authorization
    • Versioning
    • Endpoint
    • Roles
    • Nodes
    • Storage Gateways
    • Collections
    • User Credentials
    • Domains
    • Sharing Policies
    • ActiveScale
    • Azure Blob
      • Azure Blob
    • BlackPearl
    • Box
    • Ceph
    • Google Cloud Storage
    • Google Drive
    • HPSS
    • iRODS Connector
    • OAuth Credential API
    • OneDrive
    • POSIX Connector
    • POSIX Staging Connector
    • S3
  • API Access for Portals
  • Application Migration Guide
  • Change Log
© 2010- The University of Chicago Legal Privacy Accessibility