Last Updated: June 15, 2020

1. Introduction

This installation guide provides an overview of Globus Connect Server version 5 for system administrators who will install and operate the service.

Globus Connect Server version 5 is the next evolution of the server software. It provides new capabilities and enhancements for both administrators and users, and platform features to build interesting solutions for data management.

Important

Globus Connect Server version 5 does not yet have all of the features available in version 4. Until it does, we are offering a series of point releases (e.g., version 5.4), each release adding incrementally more capabilities. At this time, there is no path to upgrade or migrate from version 4 to version 5. When version 5 has feature parity with version 4, we will provide an upgrade mechanism and instructions for its use. In the meantime, Globus Connect Server version 5 releases are in limited production and are intended for organizations that need to use specific features that aren’t available in version 4. Others should continue to use Globus Connect Server version 4 until version 5 is fully featured and ready for broad deployment.

The latest version, 5.4, supports the following features:

  • Deployments with multiple data transfer nodes

  • Guest collections (data sharing with collaborators)

  • Mapped collections (access for users with local accounts)

  • HTTPS access to data - for direct access from browsers and other HTTPS clients

  • GridFTP access to data - for reliable, bulk data transfer via the Globus transfer service

  • High assurance features for management of protected data

  • Support for the following storage systems: Google Drive, Google Cloud Storage, POSIX, Box, CEPH, S3, SpectraLogic BlackPearl

2. Globus Connect Version 5 Terminology

The Globus Connect Server architecture has evolved to support several new capabilities. This section provides an overview of the components in Globus Connect Server version 5 and how they relate to version 4 components.

  • Endpoint (changed from version 4): The endpoint is a deployment of Globus Connect Server version 5. A single endpoint may optionally include multiple Data Transfer Nodes (DTNs) or servers. Each server or DTN is referred to as Node. The endpoint provides the interface for server management and configuration.

  • Storage connector: A storage connector allows the endpoint to use a particular type of storage. (E.g., POSIX file system, Google Drive.) You may configure multiple storage connectors for a single endpoint, allowing simultaneous access to all connectors.

  • Storage gateway: Storage gateways provide the storage access policies for the endpoint’s connected storage systems. A storage gateway is a named, discoverable interface by which authorized users can create and manage collections on a connected storage system. A connected storage system may have multiple storage gateways.

  • Collection: Collections provide the data access interfaces, allowing access via HTTPS (client/server access), GridFTP (asynchronous bulk transfer), and REST API (for advanced operations). In Globus Connect Server version 5, a collection is a named set of files (or blobs), hierarchically organized in folders, associated with a specific storage gateway. Access to a collection is authenticated with Globus Auth-issued OAuth2 access tokens, with data access policies defined in the collection itself. Globus Connect Server version 5 supports two types of collections:

    • Mapped collection: Each user accessing the collection must have a local account on the storage system. Their Globus identity is mapped to their local account. In version 4, these are called “host endpoints.”

    • Guest collection: Users can access the collection without a local account on the storage system. Access is based on permissions granted by an authorized user via Globus. In version 4, these are called “shared endpoints.”

Globus Connect Server version 5

With the above architecture, Globus Connect Server version 5 supports many new features including:

  • Multiple storage types connected to the same endpoint

  • Multiple storage gateways against the same storage type

  • Clear separation between management and configuration, and data access interfaces

  • End-to-end backup and synchronizatoin of configuration data on endpoints

  • HTTPS access to the data in addition to bulk data access via GridFTP.

3. Prerequisites

Important

The prerequisites listed in this section must be met before you begin to install Globus Connect Server version 5 on your system. Contact us if you have any questions regarding the prerequisites.

3.1. Supported Linux distributions

Globus Connect Server version 5 is currently supported on the following Linux distributions:

  • CentOS 7, 8

  • Debian 9, 10

  • Fedora 30, 31, 32

  • Red Hat Enterprise Linux 7, 8

  • Ubuntu 16.04 LTS, 18.04 LTS, 19.10 and 20.04 LTS

Note

Globus Connect Server version 5 cannot be run on the same machine as Globus Connect Server version 4.

3.2. Administrator privileges

You must have administrator (root) privileges on your system to install Globus Connect Server version 5; sudo can be used to perform the installation.

3.3. System time synchronization

Your system must be running ntpd or another daemon for synchronizing with standard time servers.

3.4. Internet-accessible system

Other hosts on the Internet must be able to initiate connections to the system where you will be installing Globus Connect Server version 5. Your network administrator may be able to offer assistance if you run into problems, or contact us.

3.5. Open TCP ports

If your system is behind a firewall, several TCP ports must be opened for Globus to work. You may need to coordinate with your network or security administrator to open the ports.

The TCP ports that must be open for the default Globus Connect Server version 5 installation are as follows.

  • Ports 50000—​51000 inbound and outbound to/from ANY

    • Used for GridFTP data channel traffic.

    • The use of the default port range is strongly recommended (you can read why here).

    • Data channel traffic is sent directly between endpoints—​it is not relayed by the Globus service.

  • Port 443 inbound from ANY

    • Used by Globus Connect Server version 5 Manager Service

    • Used for GridFTP control channel traffic.

    • Used for HTTPS access to collections.

  • Port 443 outbound to ANY

    • Used to communicate with the Globus service via its REST API.

    • Used to communicate with cloud storage services.

    • Used to pull Globus Connect Server version 5 packages from the Globus repository.

4. Installation

A Globus Connect Server version 5 deployment includes installation of multiple components: (a) endpoint for servers and networking configuration, (b) storage gateways with connectors for storage system and policies information, and (c) mapped collections for users to access data.

This section covers the installation and setup of endpoint, and links to next sections for storage gateway and collection setup. As we walk through each part of this installation, links to alternate configurations and connectors (e.g., Google Drive storage gateway) will be provided. You can customize and fine-tune this configuration to your specific needs later without doing a reinstallation.

Before continuing, please confirm that the prerequisites detailed in the previous section have been met.

Important

If you’ve already configured an endpoint using Globus Connect Server version 5.3, you can use the Migration Guide to upgrade to Globus Connect Server version 5.4. This will preserve existing collections, ACLs, Pause Rules, and bookmarks.

4.1. Install Globus Connect Server version 5 software

Skip to the appropriate section for your Linux distribution and follow the instructions to install Globus Connect Server version 5 on your system.

This must be done on each system which will be acting as a Data Transfer Node for the endpoint(s) you create.

$ sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ sudo yum install http://downloads.globus.org/toolkit/gt6/stable/installers/repo/rpm/globus-toolkit-repo-latest.noarch.rpm
$ sudo yum-config-manager --enable Globus-Connect-Server-5-Stable
$ sudo yum-config-manager --enable Globus-Toolkit-6-Stable

Finally, install Globus Connect Server:

$ sudo yum install globus-connect-server54
$ sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
$ sudo yum install http://downloads.globus.org/toolkit/gt6/stable/installers/repo/rpm/globus-toolkit-repo-latest.noarch.rpm
$ sudo yum-config-manager --enable Globus-Connect-Server-5-Stable
$ sudo yum-config-manager --enable Globus-Toolkit-6-Stable

Finally, install Globus Connect Server:

$ sudo yum install globus-connect-server54
$ sudo curl -LOs http://downloads.globus.org/toolkit/gt6/stable/installers/repo/deb/globus-toolkit-repo_latest_all.deb
$ sudo dpkg -i globus-toolkit-repo_latest_all.deb
$ sudo sed -i /etc/apt/sources.list.d/globus-toolkit-6-stable*.list \
        -e 's/\^# deb /deb /'
$ sudo sed -i /etc/apt/sources.list.d/globus-connect-server-stable*.list \
        -e 's/^# deb /deb /'
$ sudo apt-key add /usr/share/globus-toolkit-repo/RPM-GPG-KEY-Globus
$ sudo apt-get update
$ sudo apt-get install globus-connect-server54
$ sudo curl -LOs http://downloads.globus.org/toolkit/gt6/stable/installers/repo/deb/globus-toolkit-repo_latest_all.deb
$ sudo dpkg -i globus-toolkit-repo_latest_all.deb
$ sudo sed -i /etc/apt/sources.list.d/globus-toolkit-6-stable*.list \
        -e 's/\^# deb /deb /'
$ sudo sed -i /etc/apt/sources.list.d/globus-connect-server-stable*.list \
        -e 's/^# deb /deb /'
$ sudo apt-key add /usr/share/globus-toolkit-repo/RPM-GPG-KEY-Globus
$ sudo apt-get update
$ sudo apt-get install globus-connect-server54

4.2. Create the endpoint

With the Globus Connect Server version 5 software installed on your server, the next step is to create the endpoint. You will register the endpoint to get credentials to access Globus services then run the globus-connect-server endpoint setup command to setup the endpoint across all of the Globus services it will need to function.

4.2.1. Create service credentials

The first step in creating your endpoint is to register it with Globus and obtain credentials for the endpoint. These credentials allow the endpoint to securely identify itself to, and interact with Globus services.

  1. Log into the Globus Developers Console, developers.globus.org.

  2. Click Register a new Globus Connect Server v5.

  3. Click Add another project and fill out the form. This project will be used to track your Globus Connect Server registrations. Keep it separate from any other projects you might have.

  4. Use the Add…​ menu to add other appropriate users in your organization as administrators of the project. Adding other administrators helps your organization avoid losing administrative control should any one administrator leave your organization.

  5. From the Add…​ menu for the project click Add a new Globus Connect Server and fill out the form. The display name will be used to identify this endpoint to users when they access it for the first time. Use the same name here that you plan to use in later steps so your users will have a consistent experience.

  6. Click Generate a New Client Secret and fill out the form.

  7. Save the Client ID and Client Secret values. You will use them soon when creating your Globus Connect Server version 5 endpoint.

Important

Each endpoint requires a new Globus Connect Server version 5 registration with its own Client ID and Client Secret. These registrations may be within the same project. It is important that the Client ID created in this step only be used to set up a single endpoint, and that it is not reused to set up additional endpoints.

4.2.2. Setup the endpoint

To finish creating the endpoint, run the globus-connect-server endpoint setup command. This command need not be run as root user, and is run only once per endpoint, on the first node (server/DTN) setup for the endpoint. Subsequent servers/nodes that are added should not use this command, and separate instructions are provided for that.

This command creates an endpoint in the Globus Transfer service, and obtains a DNS name and certificate for hosting the endpoint. In addition, it generates an encryption key used to confidentially manage configuration data which is backed up to Globus hosted service for management. The encryption key is written to the file deployment-key.json in the current directory. The data in this file is necessary to configure Data Transfer Nodes to host Globus Connect services.

Important

The deployment-key.json has encryption keys to ensure configuration data backed up is encrypted. In the case of endpoints will multiple DTNs, this key with be needed on every Data Transfer Node, and cannot be recovered by Globus.

The globus-connect-server endpoint setup command takes a number of command-line options, but the following ones are required to create an endpoint:

DISPLAY_NAME

Name for the Endpoint

--organization string

Organization operating the Endpoint

--client-id TEXT

CLIENT_ID obtained from the registration in the previous step.

--owner string

Identity username of the user who will be the administrator of this endpoint. (e.g. janedoe@example.edu). This must be an identity that can log into Globus to manage the endpoint.

Once run the command will ask for the Client Secret you obtained in the previous step, and ask you to agree to Let’s Encrypt’s terms of service for having Globus Connect Server obtain a certificate for the endpoint using Letsencrypt’s ACME protocol. The command takes a few minutes to complete as certificates are provisioned on the endpoint.

Example
globus-connect-server endpoint setup "My GCSv5.4 Endpoint" \
    --organization "Example Organization" \
    --client-id "3113dd2a-6199-4c3e-b08f-a4ac4b5ae5c3" \
    --owner admin@example.edu

The command returns information about the endpoint that may be useful for additional configuration later, including the domain name of the endpoint, a link to send to subscription managers to set the endpoint as managed, and the redirect URI needed if Google Drive or Cloud connectors will be used with this endpoint.

See globus-connect-server endpoint setup documentation for more information about all options to this command, including additional information about the endpoint such as description, and contact information.

4.3. Start the server

Run the globus-connect-server node setup command to configure and start the Globus services on the Data Transfer Node. This command must be done as the root user, as it enables and starts systemd services. The deployment-key.json file from the previous step will be used by this command.

The globus-connect-server endpoint setup command takes a number of command-line options, but the --client-id option, which takes the Client ID obtained when registering the endpoint, is the only required option.

Once run the command will ask for the Client Secret you obtained when registering the endpoint.

Example
sudo globus-connect-server node setup \
    --client-id "3113dd2a-6199-4c3e-b08f-a4ac4b5ae5c3" \

See globus-connect-server node setup for more information about all options to this command.

4.4. Add Data Transfer Nodes to the endpoint

Endpoints can have multiple Data Transfer Nodes and these are called "nodes". For each Data Transfer Node, the Globus Connect Server software must be downloaded and installed as described in install section.

Once installation of the software is complete, the node will be setup to be included in the endpoint. The setup requires the deployment-key.json file from the previous step, and the file should be copied over to each Data Transfer Node.

Run the globus-connect-server node setup command to configure and start the Globus services on each Data Transfer Node. This command must be done as the root user, as it enables and starts systemd services. This will use the deployment-key.json file from the previous step must be available on the Data Transfer Nodes, and the command must be run once on each Data Transfer Node to enable the Globus services on that node.

Example
sudo globus-connect-server node setup \
    --client-id "3113dd2a-6199-4c3e-b08f-a4ac4b5ae5c3" \

See globus-connect-server node setup for more information about all options to this command.

4.5. Log into the endpoint

For further configuration and management, tokens for authentication to the Globus Connect Server Manager service are needed. The globus-connect-server login command can be used to log into the endpoint for further configuration. This command will print out a login URL to follow to authenticate with Globus, and return an access code that needs to be pasted into the command-line tool. This will result in an authentication token that can be used to talk to the Globus Connect Server management API. As long as you are on a Data Transfer Node of an endpoint the "localhost" argument can be used to get credentials for that endpoint.

Login Command Example
globus-connect-server login localhost

See globus-connect-server login for more information about all options to this command.

4.6. View endpoint configuration

You can now use the globus-connect-server endpoint show command to display the configuration of the endpoint.

Example
globus-connect-server endpoint show

Display Name:    My GCSv5.4 Endpoint
ID:              a44ca3ef-a8b9-4b73-aa5f-546ed5ab7e66
Subscription ID: None
Public:          True
GCS Manager URL: https://09ad6.0.glob.us
Network Use:     normal
Organization:    Example Organization

4.7. Set the endpoint as managed

Endpoints that require premium functionality—​such as guest collections for data sharing and premium connectors—​must be managed under a Globus subscription.

If your organization has a subscription, and your Globus account has the subscription manager role, you may set the endpoint as managed using the globus-connect-server command as follows.

Set the Endpoint as Managed
globus-connect-server endpoint set-subscription-id DEFAULT

If you are not the subscription manager for your organization, you will have to request your organization’s subscription manager to set the endpoint as managed. They can refer to the FAQ on using the Globus web application to set that option.

5. Next Steps

At this point, you’ve installed Globus Connect Server version 5 on your Data Transfer Nodes and created an endpoint. You should follow the Data Access Guide to configure storage gateway and collections on your endpoint so users can access data via Globus.

Read the Command Line Reference for a complete description of the globus-connect-server command-line tool.

6. Globus Help Resources

6.1. Documentation Website

This website (docs.globus.org) contains a wealth of information about configuring and using the Globus service. Many common issues can be resolved quickly by browsing our frequently asked questions and reading the relevant guides and how-to’s. We recommend consulting these resources first when looking for fast resolution to any issue you are having with the Globus service.

6.2. Mailing Lists

If you use Globus, then participating in one or more of the public email lists is an excellent way to keep in touch with your peers in the Globus Community. For questions about managing your Globus deployment, e.g. installing software for a Globus endpoint, configuring your firewall, and integrating your institution’s identity system, subscribe to the admin list. For other inquiries and discussions, try the user or developer lists. For more information on mailing lists and how to subscribe, click here.

6.3. Globus Support

Questions or issues that pertain to Globus Connect Server version 5 installation or to any client or service that is used in the Globus software-as-a-service (SaaS) or platform-as-a-service (PaaS) offering can be directed to the Globus support team by submitting a ticket. Subscriptions include a guaranteed support service level.

When submitting a ticket for an issue with Globus Connect Server, please include the endpoint name, a description of your issue, and screenshot/text dumps of any errors you are seeing. Please also include the output of Globus Connect Server’s self-diagnostic command, run as root, from the server hosting the endpoint:

globus-connect-server self-diagnostic

© 2010- The University of Chicago Legal