Dropbox Connector

The Globus Dropbox connector can be used for access and sharing of data on Dropbox. The connector is available as an add-on subscription to organizations with a Globus Standard subscription - please contact us for pricing.

This document describes how to install the Dropbox Connector and configure Dropbox Storage Gateways and Collections. After these steps are complete, any Globus user you have authorized can register a credential to access Dropbox files that they have access to and, if enabled, can create guest collections for sharing access using those credentials by following the instructions in How To Share Data Using Globus.

This document assumes that you or another administrator has already installed Globus Connect Server v5.4.70 or higher on all data transfer nodes, and that you have an administrator role on that endpoint.

The installation must be done by a system administrator, and has the following distinct set of steps:

  • Register a Dropbox Application which the connector will use to access the Dropbox APIs.

  • Create a storage gateway on the endpoint configured to use the Dropbox Connector.

  • Create a mapped collection using the Dropbox Storage Gateway to provide access to Dropbox Storage Gateway data.

Please contact us at support@globus.org if you have questions or need help with configuration and use of the Dropbox Connector.

Dropbox Connector Virtual Filesystem

Dropbox presents its data as a hierarchical list of files and folders, much like a POSIX filesystem.

The Dropbox Connector provides the user’s top-level "All files" space at the root directory of a Dropbox Connector storage gateway. For Dropbox business accounts, the user’s personal folder as well as accessible team folders will be shown as subdirectories. For Dropbox personal accounts, the user’s personal folder is the root directory.

Shared folders that have been joined will be shown in the user’s personal folder.

Registration of endpoint with Dropbox

Globus Connect Server needs to be registered as an application with Dropbox so that users can authorize the endpoint to access Dropbox on their behalf. The following steps describe how to create a Dropbox application to obtain a client id and secret for Globus Connect Server to use.

Prerequisites

It is necessary that these steps be performed on a fully functional Globus Connect Server 5.4 endpoint.

You will need a Dropbox account to complete these steps, and the registration will be stored under that account. This account is only for registration of the application and has no bearing on which user accounts will be allowed to use this endpoint to access data.

Registration Steps

  1. To register the endpoint with Dropbox, go to the Dropbox App Console

  2. Select Create app to add a new application.

    1. For 1. Choose an API, select Scoped access.

    2. For 2. Choose the type of access, select Full Dropbox.

    3. For 3. Name your app, enter a name such as Globus Connect Server - [your org]. This will be displayed to users of your collection when they are prompted to log in to Dropbox during credential registration. You will be prompted to change the name if the value entered is not unique.

    4. Select Create app to proceed to the app settings.

  3. On the Settings page, you will configure the initial app settings.

    1. For Development users, select Enable additional users.

    2. Record the values for App key and App secret. These will be used to configure the storage gateway --dropbox-client-id and --dropbox-client-secret options, respectively.

    3. For Redirect URIs add the auth callback URL for your endpoint.

      To find that value, you can run the command

      globus-connect-server endpoint show

      You’ll see output that looks something like this:

      Display Name:    Test Endpoint
ID:              669ec822-ca79-455c-89a7-cccb7aefbf8e
Subscription ID: 6e62e6d7-e368-45f4-a23d-fb41243e8005
Public:          True
GCS Manager URL: https://21542.data.globus.org
Network Use:     normal

      You can construct the auth callback URL by appending /api/v1/authcallback to the value of the GCS Manager URL. In this example case, the result is https://21542.data.globus.org/api/v1/authcallback.

    4. For Allow public clients (Implicit Grant & PKCE), select Disallow.

    5. Leave the other options empty or unchanged and select the Permissions tab.

  4. On the Permissions page, you will chose the scopes required by the connector.

    1. Under Account Info, account_info.read will be checked.

    2. Under Files and folders, check files.metadata.read, files.content.write, and files.content.read.

    3. Under OpenID Connect, check profile, openid, and email.

    4. All other permissions should remain unchecked.

    5. Click Submit, to save these permissions.

  5. If desired, select the Branding tab to configure additional login screen details.

  6. App registration is complete.

Development Status

The app will be fully functional in development status, for up to 500 users. Users will be notified of the development status when logging in to a Dropbox collection.

You can apply for production status via the Dropbox App Console. However, until the app has been used by 50 users, the application will not be reviewed by Dropbox. See the Dropbox approval documentation for more info. At this time, Globus does not have further guidance on promoting an app to production status. If you are nearing 50 users, you can contact support@globus.org for assistance with the process.

Dropbox Configuration Encryption

All configuration information, including Dropbox secrets and user credential information, is encrypted with a secret key on the node servicing the request before storing it locally and uploading it to GCS cloud services for distribution to other nodes in the endpoint. The encryption key is only available locally to the node and is secured such that only the node admin has access.

Storage Gateway

A Dropbox Storage Gateway is created with the command globus-connect-server storage-gateway create dropbox, and can be updated with the command globus-connect-server storage-gateway update dropbox.

Before looking into the policy options specific to the Dropbox Connector, please familiarize yourself with the Globus Connect Server v5 Data Access Guide which describes the steps to create and update a storage gateway, using the POSIX connector as an example. The commands to create and update a storage gateway for the Dropbox Connector are similar.

Dropbox Connector Storage Gateway Policies

The Dropbox Connector has policies to manage application credentials, and set the user api rate limit.

Application Credentials

The --dropbox-client-id and --dropbox-client-secret command-line options provide information for Globus Connect Server to authenticate with Dropbox Connector. These values must be configured in order to be able to access data on collections created with the Dropbox Connector type.

These are configured after registering the application with Dropbox as described in the Dropbox Connector configuration guide.

Example 1. Setting Dropbox Connector Application Credentials.

For our example, we’ll assume we’ve obtained credentials as described above. We’ll use the command-line options --dropbox-client-id and --dropbox-client-secret to configure these on our storage gateway.

    --dropbox-client-id CLIENT_ID \
    --dropbox-client-secret CLIENT_SECRET

User API Rate Limit

The optional --dropbox-user-api-rate-limit command-line option allows you to configure a value for the User API Rate Limit in order to try to increase performance, or avoid throttling issues when interacting with the Dropbox API. While the connector will attempt to gracefully handle retries when throttling occurs, it is better to avoid throttling. The value of the setting is a number of API operations per second per user. Dropbox does not publish explicit rate limits, but the connector default is set at 20 requests per second.

Example 2. Setting a User API Rate Limit

For our example, we’ll use the default. If we wanted to set a limit of 25 operations per second per user, we would use the following --dropbox-user-api-rate-limit option.

    --dropbox-user-api-rate-limit 25

Creating the Storage Gateway

Now that we have decided on all our policies, we’ll use the command to create the storage gateway.

% globus-connect-server storage-gateway create dropbox \
    "Dropbox Storage Gateway" \
    --domain example.org \
    --dropbox-client-id CLIENT_ID \
    --dropbox-client-secret CLIENT_SECRET

Storage Gateway Created: 7187a9a0-68e4-48ea-b3b9-7fd06630f8ab

This was successful and outputs the ID of the new storage gateway ( 7187a9a0-68e4-48ea-b3b9-7fd06630f8ab in this case) for our reference. Note that this will always be a unique value if you run the command. If you forget the id of a storage gateway, you can always use the command globus-connect-server storage-gateway list to get a list of the storage gateways on the endpoint.

You can also add other policies to configure additional identity mapping and path restriction policies as described in the Globus Connect Server v5 Data Access Guide.

Note that this creates the storage gateway, but does not yet make it accessible via Globus and HTTPS. You’ll need to follow the steps in the next section to to create a collection.

Collection

A Dropbox Collection is created with the command globus-connect-server collection create, and can be updated with the command globus-connect-server collection update.

As the Dropbox Connector does not introduce any policies beyond those used by the base collection type, you can follow the sequence in the Collections Section of the Globus Connect Server v5 Data Access Guide. Recall however, that the paths are interpreted as described above in Dropbox Connector Virtual Filesystem.

User Credential

As mentioned above, access to mapped collections on a Dropbox require users to register credentials. These credentials are created by performing an authentication flow with Dropbox. This is initiated by visiting the Credentials tab of the collection. The user is directed through credential registration when they first attempt to access a Dropbox collection.

When registering credentials, the Dropbox account username must match the mapped username on the collection (by default the Globus account username, unless identity mapping is configured). The Dropbox account username is the user’s verified email address.

Alternately, the storage-gateway --dropbox-allow-any-account command-line option can be set to allow access to any Dropbox account.

Limitations

Non-downloadable document types, such as Dropbox Paper and Google Docs, can not be transferred and are excluded from listings.

High network-use conncurrency settings may result in Dropbox errors such as too_many_write_operations, particularly when transferring a large number of smaller files. These failed operations will be retried.

Appendix A: Document Types for the Dropbox Connector

DropboxStoragePolicies Document

Connector-specific storage gateway policies for the Dropbox connector

One of the following schemas:

​

{
  "DATA_TYPE": "dropbox_storage_policies#1.0.0",
  "allow_any_account": true,
  "auth_callback": "string",
  "client_id": "string",
  "secret": "string",
  "user_api_rate_limit": 0
}

DropboxUserCredentialPolicies Document

Connector-specific user credential policies for the Dropbox connector

One of the following schemas:

​

{
  "DATA_TYPE": "dropbox_user_credential_policies#1.0.0",
  "access_token": "string",
  "email": "string",
  "refresh_token": "string",
  "root_info": {},
  "scopes": [
    "profile",
    "openid",
    "email",
    "account_info.read",
    "files.metadata.read",
    "files.content.write",
    "files.content.read"
  ],
  "sub": "string",
  "token_expiry": "2019-08-24T14:15:22Z"
}