Globus Connect Server v4 Installation Guide
- 1. Introduction
- 2. Globus Connect Server Prerequisites
- 3. Globus Connect Server Installation and Endpoint Creation
- 4. Globus Connect Server Configuration
- 5. NAT/Firewall support and configuration
- 6. Generating and Monitoring Log Files
- 7. Getting Help
- 8.
- 9.
- 10. Roles and privileges
- Appendix A: Understanding Data Channel Traffic
- Appendix B: Globus Connect Server Configuration Packages
- Appendix C: How to update a Globus Connect Server install
- Appendix D: Endpoint Identity Provider Options
- Appendix E: Setting Endpoint Network Use Options
- Appendix F: GridFTP Hostcerts, Hostkeys, and Trusted Certs
- Appendix G: Configuring an endpoint with multiple I/O nodes
- Appendix H: Uninstalling Globus Connect Server v4
- Glossary
Last Updated: September 11, 2019
1. Introduction
Globus allows you, as a resource provider, to easily offer reliable, secure, high-performance research data management capabilities to your users and their collaborators, directly from your own storage. Globus is software-as-a-service (SaaS), enabled by the cloud and designed to let users manage their research files, regardless of number and size, as effortlessly as they manage their photos with popular cloud-based services. Globus lets you provide authorized users with web, command line, and REST interfaces for listing, transferring, replicating, and sharing directories and files on your Globus-connected storage resources.
This guide is written primarily for system administrators of multi-user Linux systems who want to take full advantage of Globus capabilities and optimize the connection of their resources to the Globus service. Administrators who are installing Globus for the first time, as well as those with considerable experience, should all find helpful information in this document.
Starting with the basics, this guide covers the initial installation of Globus Connect Server and the creation of a Globus server endpoint that lets authorized users transfer files between your storage and other Globus endpoints. The Globus Connect Server installation downloads (1) Globus versions of GridFTP, MyProxy, and OAuth for MyProxy; (2) scripts for set up/configuration and integration into the Globus service; and (3) Globus-maintained dependency packages needed by the other components.
Globus file transfer is free-of-charge for non-profit research and educational use. Some premium Globus features described in this guide are only available to subscribers.
Subscriptions help ensure that Globus can continue to serve the research data management needs of non-profit users for many years to come.
2. Globus Connect Server Prerequisites
2.1. Supported Linux distributions
Globus Connect Server is currently supported on the following Linux distributions:
2.2. Administrator privileges
You must have administrator (root) privileges on your system
to install Globus Connect Server;
sudo
can be used to perform the installation.
2.3. System time synchronization
Your system must be running ntpd
or another daemon for synchronizing with standard time servers.
2.4. Globus account for your organization
You must have a Globus ID to be used for your organization account that is distinct from any personal Globus ID you may have.
2.5. Internet-accessible system
Other hosts on the Internet must be able to initiate connections to the system where you will be installing Globus Connect Server. If your system is behind a network address translation (NAT) firewall/router, you cannot use the default configuration to install Globus—please see the configuration instructions in the NAT/firewall section. Otherwise, perform the checks shown below to confirm that your system meets the default accessibility requirements. If you are installing on an Amazon EC2 instance, you can skip ahead to the Open TCP ports section.
Your network administrator may be able to offer assistance if you run into problems, or contact us.
2.5.1. Check hostname local DNS resolution
Execute this command on the system where you plan to install Globus Connect Server:
$ hostname -f
Confirm that a fully qualified domain name (FQDN) is returned (e.g., \'ep1.transfer.globus.org' ).
2.5.2. Check hostname external DNS resolution
Use a public DNS server operated by a different organization to
verify that the returned FQDN is publicly resolvable.
More concretely, you can use nslookup
to check that your server’s
FQDN resolves against one of Google’s public DNS servers:
$ nslookup 'ep1.transfer.globus.org' 8.8.4.4
If you get a message of the form '"** server can’t find ep1.transfer.globus.org: NXDOMAIN"', your system’s hostname is not resolvable via public DNS and you need to address the issue before continuing with the installation.
2.6. Open TCP ports
If your system is behind a firewall, select TCP ports must be open for Globus to work. You may need to coordinate with your network or security administrator to open the ports.
The TCP ports that must be open for the default Globus Connect Server installation, together with brief descriptions of each, are listed here:
-
Port 2811 inbound from 54.237.254.192/29
-
Used for GridFTP control channel traffic.
-
-
Ports 50000—51000 inbound and outbound to/from Any
-
Used for GridFTP data channel traffic.
-
Ports in this range will be bound and used by GridFTP only during transfers as needed.
-
The use of the default port range is strongly recommended (you can read why here).
-
Data channel traffic is sent directly between endpoints—it is not relayed by the Globus service.
-
-
Port 443 outbound to 54.237.254.192/29 and nexus.api.globusonline.org
-
Used to communicate with the Globus service via its REST API.
-
nexus.api.globusonline.org is a CNAME for an Amazon ELB; IP addresses in the ELB are subject to change.
-
-
Port 443 outbound to downloads.globus.org
-
Used to pull Globus Connect Server packages from the Globus repository.
-
downloads.globus.org is a CNAME for an Amazon CloudFront Distribution; IP addresses used by the CloudFront Distribution are subject to change.
-
-
Port 443 outbound to crl.cilogon.org
-
Used to pull CRL file updates for endpoints configured to use CILogon based activation
-
-
Port 7512 inbound from 54.237.254.192/29
-
Used for MyProxy traffic.
-
Needed if your server will run MyProxy service.
-
-
Port 443 inbound from Any
-
Used for OAuth traffic.
-
Needed if your server will run OAuth service.
-
OAuth traffic comes directly from clients using your OAuth service—it is not relayed by the Globus service.
-
3. Globus Connect Server Installation and Endpoint Creation
This section covers the installation of Globus Connect Server and the set up of a Globus server endpoint with the default configuration—the recommended starting point for new resource providers. You will be able to fine-tune this configuration later without doing a reinstall.
Before continuing, it is important to confirm that the prerequisites detailed in the previous section have been met.
3.1. Install Globus Connect Server
Skip to the appropriate section for your Linux distribution and follow the instructions to install Globus Connect Server on your system.
3.1.1. CentOS, Fedora, Red Hat Enterprise Linux, Scientific Linux
First, add the Globus Connect Server repository to your package management system:
$ sudo curl -LOs https://downloads.globus.org/toolkit/globus-connect-server/globus-connect-server-repo-latest.noarch.rpm
$ sudo rpm --import https://downloads.globus.org/toolkit/gt6/stable/repo/rpm/RPM-GPG-KEY-Globus
$ sudo yum install globus-connect-server-repo-latest.noarch.rpm
You will need the EPEL repository available and enabled on your system. If this is not already the case, you can install the EPEL repository with the instructions below.
Install EPEL repository on CentOS 6, Red Hat Enterprise Linux 6, and Scientific Linux 6:
$ sudo curl -LOs https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
$ sudo yum install epel-release-latest-6.noarch.rpm
Install EPEL repository on CentOS 7, Red Hat Enterprise Linux 7, and Scientific Linux 7:
$ sudo curl -LOs https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ sudo yum install epel-release-latest-7.noarch.rpm
Finally, install Globus Connect Server:
$ sudo yum install globus-connect-server
3.1.2. SuSE Linux Enterprise Server
First, add the Globus Connect Server repository to your package management system:
$ sudo curl -LOs https://downloads.globus.org/toolkit/globus-connect-server/globus-connect-server-repo-latest.noarch.rpm
$ sudo rpm --import https://downloads.globus.org/toolkit/gt6/stable/repo/rpm/RPM-GPG-KEY-Globus
$ sudo zypper install globus-connect-server-repo-latest.noarch.rpm
Next, retrieve and install the additional required repositories:
$ sudo zypper ar http://download.opensuse.org/repositories/Apache/SLE_12_SP2/Apache.repo
$ sudo zypper ar http://download.opensuse.org/repositories/Apache:/Modules/Apache_SLE_12_SP2/Apache:Modules.repo
$ sudo rpm --import http://download.opensuse.org/repositories/Apache/SLE_12_SP2/repodata/repomd.xml.key
$ sudo rpm --import http://download.opensuse.org/repositories/Apache:/Modules/Apache_SLE_12_SP2/repodata/repomd.xml.key
$ sudo zypper remove libapr1
Finally, install Globus Connect Server:
$ sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-Globus
$ sudo zypper install globus-connect-server
3.1.3. Debian, Ubuntu
First, add the Globus Connect Server repository to your package management system:
$ sudo curl -LOs https://downloads.globus.org/toolkit/globus-connect-server/globus-connect-server-repo_latest_all.deb
$ sudo dpkg -i globus-connect-server-repo_latest_all.deb
$ sudo apt-get update
Then, install Globus Connect Server:
$ sudo apt-get install globus-connect-server
3.2. Create Globus Endpoint
Before creating your Globus server endpoint, choose a suitable second part for your endpoint name. Then, edit the Globus Connect Server configuration file, /etc/globus-connect-server.conf, and set Name to your choice (geosciences in the example shown), and Public to True. These two changes in the [Endpoint] section of the file will allow authorized users to find and access your endpoint.
[Endpoint] Name = geosciences Public = True
After editing the configuration file, run:
$ sudo globus-connect-server-setup
When prompted, enter the Globus username and password for your Globus organization account. When the globus-connect-server-setup command completes, your Globus endpoint is ready to be accessed by users with logins on your system.
3.3. Access Globus Endpoint
You (or any user on your system who has signed up for a Globus account) should now be able to access the Globus endpoint you just created by navigating to the Globus Transfer Files page. We recommend that you confirm your endpoint is functioning properly by performing some test transfers, as described here.
3.4. Update Globus Endpoint
The set of software components that make up Globus Connect Server are improved and released on an ongoing basis. To take advantage of the latest features we encourage you to update your local installation from the Globus repository regularly. The steps for doing an update are found here. We also recommend that you go here and subscribe to the Admin mailing list to receive Globus release notes.
4. Globus Connect Server Configuration
During the initial (default) installation of Globus Connect Server,
you edited two configuration options in the
/etc/globus-connect-server.conf file,
the Name and Public options in the [Endpoint] section.
You probably noticed that there are many, many other options
that can be configured.
This section of the Globus Globus Connect Server Installation Guide briefly covers a few of the most commonly changed
options in the globus-connect-server.conf file. After updating settings in the
/etc/globus-connect-server.conf file you must run the
globus-connect-server-setup
command (as root) before the settings
will take effect on your endpoint.
4.3. [Security] section
4.3.1. IdentityMethod option
This option has three legal values: MyProxy, OAuth, and CILogon. For a graphical overview of the authentication flows each of these methods use, see here. If you wish to use MyProxy as your endpoint’s identity method, then you need to be sure to specify the Server option in the [MyProxy] section. If you wish to use OAuth as your endpoint’s identity method, then you need to be sure to specify the Server option in the [OAuth] section, and may also need to specify the Server option in the [MyProxy] section if you are using MyProxy on the server to provide authentication for the OAuth service. If you are using CILogon, then you will also need to specify the CILogonIdentityProvider option in the [Security] section.
4.3.2. CILogonIdentityProvider option
This option specifies the identity provider to use with CILogon. See https://cilogon.org/include/idplist.xml for a list of valid providers. To configure your endpoint to use a specific provider, set this option to the value of the "Organization_Name" given for that provider at the previous URL. Be sure to also set "IdentityMethod=CILogon" in the [Security] section of the config file if you intend to use CILogon for your endpoint.
4.4. [GridFTP] section
4.4.1. Server option
This option specifies the hostname of the GridFTP server. This should match the hostname of the server except, possibly, if NAT is being used. Can be left blank if you don’t want to configure a GridFTP server on this host. If a GridFTP service is to be run on this server, then the default value of "Server = %(HOSTNAME)s" is suitable to most cases, so long as the configured hostname on the server matches the public FQDN that is going to be used for the server.
4.4.2. ServerBehindNAT option
This option specifies that the server is behind a NAT firewall/router. See the NAT section for details.
4.4.3. RestrictPaths option
This option is used to both enable and/or disable specified file paths for all Globus users that are authorized to use this endpoint. By default, all paths are enabled for access.
4.4.4.
This is a boolean value that determines if sharing is enabled on this particular endpoint. This option is only used on a managed endpoint, and will be ignored otherwise.
4.4.5.
This option is used to both enable and/or disable specified file paths for all Globus users that are authorized to use this shared endpoint. By default, all paths are enabled for access. NOTE: When accessing a shared endpoint through Globus, the SharingRestrictPaths option can only set further access restrictions to the endpoint owner’s local unix file system permissions. Globus does not and can not override the local unix file system permissions. This option is only used on a managed endpoint, and will be ignored otherwise.
4.5. [MyProxy] section
4.5.1. Server option
This option specifies the hostname of the MyProxy server. If you are running the MyProxy server on this host, then this should match the hostname of this server except, possibly, if NAT is being used. If you are using a MyProxy server on a different host, then use the hostname of that host. Can be left blank if you don’t want to configure a MyProxy server at all. If the MyProxy service for the endpoint is to be run on this server, then the default value of "Server = %(HOSTNAME)s" is suitable to most cases, so long as the configured hostname on the server matches the public FQDN that is going to be used for the server. Be sure to also set "IdentityMethod=MyProxy" in the [Security] section of the config file if you intend to use MyProxy for your endpoint.
4.5.2. ServerBehindNAT option
This option specifies that the server is NATed. See the NAT section for details.
4.6. [OAuth] section
4.6.1. Server option
This option specifies the hostname of the OAuth server. If you are running the OAuth server on this host, then this should match the hostname of this server except, possibly, if NAT is being used. If you are using an OAuth server on a different host, then use the hostname of that host. Can be left blank if you don’t want to configure an OAuth server at all. If the OAuth service for the endpoint is to be run on this server, then the default value of "Server = %(HOSTNAME)s" is suitable to most cases, so long as the configured hostname on the server matches the public FQDN that is going to be used for the server. Be sure to also set "IdentityMethod=OAuth" in the [Security] section of the config file if you intend to use OAuth for your endpoint.
4.6.2. ServerBehindNAT option
This option specifies that the server is NATed. See the NAT section for details.
5. NAT/Firewall support and configuration
The Globus Connect Server package provides configuration tools for several related services to enable administrators to easily configure a Globus endpoint. The globus-connect-server.conf file controls how the services used by Globus are configured, and includes configuration options to manage firewall-related configuration of services. Each service provided by the Globus Connect Server packages may be configured separately as described below.
Note that the descriptions below include examples of Globus Connect Server service configurations only. Configuring the firewalls themselves to allow the ports and host connections is not discussed. See the Open TCP ports section for a discussions of the ports used by Globus Connect Server.
5.1. Configuring I/O Nodes
Globus Connect Server I/O nodes provide a GridFTP service to Globus. Options related to firewalls in the [GridFTP] section of the configuration file are: Server, ServerBehindNAT, IncomingPortRange, OutgoingPortRange, and DataInterface.
By default, Globus Connect Server configures the GridFTP server assuming that incoming TCP connections are allowed to port 2811, and the range 50000-51000 on the GridFTP server node.
5.1.1. Using GridFTP behind a NAT Firewall
To use a GridFTP behind a NAT firewall, set the Server option to the public name of the GridFTP server, and set the ServerBehindNAT option to True. This causes globus-connect-server-io-setup to generate GridFTP configuration for the node even if the Server name doesn’t match the node’s local hostname. This requires that the GridFTP server is visible from Globus at the address associated on the public internet with the name that is the Server value.
5.1.2. NAT Firewall Example
As an example, this configures the GridFTP server to run on the current host, using public-gridftp.example.org as its public name and listening on port 22811 instead of the default 2811. In order for this to work, the NAT firewall must allow connections to TCP port 22811 and the range 50000-51000 on the I/O node. By default, the Server name is used to construct the data interface name as well, but this behavior can be changed (see Using GridFTP with a Particular Data Interface).
[GridFTP] Server = public-gridftp.example.org:22811 ServerBehindNAT = True
5.1.3. Using GridFTP with Firewall Port Restrictions
To use a GridFTP server with a firewall with incoming and/or outgoing port restrictions, use the IncomingPortRange and OutgoingPortRange configuration options. The former restricts the TCP port range that the GridFTP server listens on for ephemeral connections to a port range. The OutgoingPortRange restricts the TCP source port range that the GridFTP server uses when creating outgoing data connection sockets. For both of these items, the syntax of the port range is startport,endport (e.g., 50000,51000).
5.1.4. Port Restrictions Example
As an example, this configures the GridFTP server to listen for TCP connections on ports from 4000 to 5000 instead of the default 50000 to 51000. This will require configuration on the firewall to allow those ports to connect directly to the I/O node.
[GridFTP] Server = public-gridftp.example.org:22811 IncomingPortRange = 4000,5000
5.1.5. Using GridFTP with a Particular Data Interface
The GridFTP server can also be configured to use a different IP address for its incoming data connections by setting the DataInterface option in the configuration file. By default, the GridFTP server will use the same IP address as that associated with the Server value. This can be altered, for example, to create a limited-use endpoint that uses a high-speed interconnect between I/O resources, but is not generally accessible from the internet.
5.2. Configuring an ID Node
The Globus Connect Server ID node provides a MyProxy service. This service generates short-lived credentials which are used to authenticate with the GridFTP server. Globus may be configured to access this service directly, or access it via a web-based OAuth interface. This is chosen by the presence or absence of an [OAuth] section in the globus-connect-server.conf file.
By default, the MyProxy service listens on TCP port 7512. It makes no outgoing TCP connections. Like the GridFTP servers on the I/O nodes, the [MyProxy] section contains Server and ServerBehindNAT configuration options, which function like the ones in the GridFTP section.
If the MyProxy service is not being used directly by the Globus service (that is, if a OAuth server is being used), then the MyProxy service need only be reachable by the Web node and, during initial configuration, by the I/O nodes. No other nodes will require access the MyProxy service in normal operation.
5.2.1. Using MyProxy behind a NAT Firewall
To use a MyProxy server behind a NAT firewall, set the Server option to the public name of the MyProxy server, and set the ServerBehindNAT option to True. This causes globus-connect-server-id-setup to generate MyProxy configuration for the node even if the Server name doesn’t match the node’s local hostname. If you are configuring an OAuth server, the Server option must be accessible from the Web node and I/O nodes (during configuration); otherwise, it must be accessible from Globus.
5.2.2. NAT Firewall Example
As an example, this configures the MyProxy server to run on the current host, using public-myproxy.example.org as its public name and listening on port 17512 instead of the default 7512. In order for this to work, the NAT firewall must allow connections to the TCP port 17512 on the ID node.
[MyProxy] Server = public-myproxy.example.org:17512 ServerBehindNAT = True
5.3. Configuring a Web Node
The Globus Connect Server Web node provides OAuth service to Globus. There are a few configuration options related to firewalls in the [OAuth] section of the configuration file. These are Server and ServerBehindNAT.
Unlike the other service nodes, the Web node is somewhat less configurable, as it relies on an external Apache server to accept TCP connections. Configuring the Apache server to listen on a different TCP port is out of scope of this note. The Server value may only contain a hostname, and the port 443 (https) is used. Globus does not support OAuth servers on alternate ports.
5.3.1. Using OAuth behind a NAT Firewall
To use an OAuth server behind a NAT firewall, set the Server option to the public name of the OAuth server, and set the ServerBehindNAT option to True. This causes globus-connect-server-web-setup to generate OAuth configuration for the node even if the Server name doesn’t match the node’s local hostname. This requires that the OAuth server is visible from Globus at the address associated on the public internet with the name that is the Server value.
5.3.2. NAT Firewall Example
As an example, this configures the OAuth server to run on the current host, using public-oauth.example.org as its public name. In order for this to work, the NAT firewall must allow connections to TCP port 433 on the web node.
[OAuth] Server = public-oauth.example.org ServerBehindNAT = True
6. Generating and Monitoring Log Files
6.1. GridFTP Server Log Files
On recent versions of Globus Connect Server, the GridFTP log is located at:
/var/log/gridftp.log
On recent versions of Globus Connect Server, the configuration settings for the GridFTP log file are found at:
/etc/gridftp.d/globus-connect-server-gridftp-logging
On older versions of Globus Connect Server, logging for the GridFTP service is not enabled by default. In order to enable logging, it is necessary to specify the appropriate options in the GridFTP configuration files. One way to do this would be to create a text file named:
/etc/gridftp.d/globus-connect-server-gridftp-logging
Next, place the following options into the file:
log_single /var/log/gridftp.log log_level ERROR,WARN
After saving the file, restart the GridFTP server with this command:
$ sudo service globus-gridftp-server restart
At this point, the GridFTP server will log all ERROR and WARN events to the
/var/log/gridftp.log
file. Additional details concerning logging for the
GridFTP server are available in the globus-gridftp-server
man page here.
6.2. MyProxy Server Log Files
By default, the MyProxy server logs events to the LOG_DAEMON facility. This means that, by default, MyProxy events will be found in the following locations:
CentOS, Fedora, Red Hat Enterprise Linux, Scientific Linux:
/var/log/messages
Debian/Ubuntu:
/var/log/syslog
SuSE Linux Enterprise Server:
/var/log/messages
Please note that if the logging location for the LOG_DAEMON facility has been changed from the default in your syslogd config, then MyProxy events may be found in a different location.
6.3. MyProxy Delegation Service (myproxy-oauth) Log Files
Events for myproxy-oauth will be logged to the apache log file directory. By default, this will be found at the following locations:
CentOS, Fedora, Red Hat Enterprise Linux, Scientific Linux:
/var/log/httpd/
Debian/Ubuntu:
/var/log/apache2/
SuSE Linux Enterprise Server:
/var/log/apache2/
For further info on the MyProxy Delegation Service see the README.
7. Getting Help
7.1. Troubleshooting Common Problems
This section describes some basic tests you can run when you experience problems with a transfer or an endpoint. These tests can help you narrow down the potential causes of the issue and simplify troubleshooting.
7.1.1. Test Basic Endpoint Functionality
An important verification of endpoint health is to confirm that the endpoint is able to successfully participate in transfers from and to other endpoints.
Globus maintains two test endpoints, Globus Tutorial Endpoint 1 and Globus Tutorial Endpoint 2, that are always available for users to access when checking the functionality of their own endpoints.
First, attempt to transfer the contents of the /share/godata/
directory on the Globus Tutorial Endpoint 1 endpoint to your own endpoint.
After that, attempt to transfer those same files to the /~/
directory on the Globus Tutorial Endpoint 2 endpoint.
If these tests both succeed, then your endpoint is functional and able to serve as the destination and the source of transfers.
For more detailed instructions on how to use the Globus service to transfer files, see here.
7.1.2. Verify globus-gridftp-server Service
Another important check on servers hosting a Globus endpoint is to verify that the
globus-gridftp-server service has properly started and is running.
To do this, first use the ps
command to see if there is an instance of globus-gridftp-server running:
$ ps aux | grep globus-gridftp-server
root 604 0.0 0.7 97924 7312 ? Ss 14:18 0:00 /usr/sbin/globus-gridftp-server -c /etc/gridftp.conf -C /etc/gridftp.d -pidfile /var/run/globus-gridftp-server.pid -no-detach -config-base-path /
If you do not see an instance of globus-gridftp-server running, then the service has not started. You can try to start it by executing the globus-connect-server-setup
command and then checking to see if an instance of globus-gridftp-server appears in the ps
output. If you still don’t see an instance of globus-gridftp-server running after issuing the globus-connect-server-setup
command, you can take a look in the logs for clues as to what might be wrong.
If there is an instance of globus-gridftp-server running, you can then check to see if you can connect to it locally via the telnet
command:
$ telnet 127.0.0.1 2811
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
220 ip-172-31-31-13.us-west-2.compute.internal GridFTP Server 7.26 (gcc64, 1433516164-85) [Globus Toolkit 6.0.1428430525 GCS-4.0.18] ready.
If there is an instance of globus-gridftp-server running, but you can’t connect locally, then there is probably a local firewall rule on the server that is interfering with your attempt to connect.
If you are successful with the local telnet connection, then try telnet again using the public FQDN associated with your server and see if that works. If you’re not able to connect when using the public FQDN, then you may have a firewall issue or a name resolution issue that is preventing you from being able to connect properly.
7.1.3. Verify myproxy-server Service
Another important check on servers hosting a Globus endpoint that use MyProxy for endpoint authentication is to verify that the
myproxy-server service has properly started and is running.
Note that not all endpoints use the MyProxy service, so this check is only useful if your endpoint uses MyProxy for endpoint authentication.
To run this check, first use the ps
command to see if there is an instance of myproxy-server running:
$ ps aux | grep myproxy-server
root 602 0.0 0.1 110812 1112 ? S 14:18 0:00 /usr/sbin/myproxy-server -s /var/lib/myproxy -c /var/lib/globus-connect-server/myproxy-server.conf -s /var/lib/globus-connect-server/myproxy-ca/store
If you do not see an instance of myproxy-server running, then the service has not started. You can try to start it by executing the globus-connect-server-setup
command and then checking to see if an instance of myproxy-server appears in the ps
output. If you still don’t see an instance of myproxy-server running after using the globus-connect-server-setup
command, you can take a look in the logs for clues as to what might be wrong.
If there is an instance of myproxy-server running, you can then check to see if you can connect to it locally via the telnet
command:
$ telnet 127.0.0.1 7512
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
type something here and hit enter
VERSION=MYPROXYv2
RESPONSE=1
ERROR=authentication failed
Connection closed by foreign host.
If there is an instance of myproxy-server running, but you can’t connect locally, then there is probably a local firewall rule on the server that is interfering with your attempt to connect.
If you are successful with the local telnet connection, then try telnet again using the public FQDN associated with your server and see if that works. If you’re not able to connect when using the public FQDN, then you may have a firewall issue or a name resolution issue that is preventing you from being able to connect properly.
7.2. Globus Help Resources
7.2.1. Documentation Website
This website (docs.globus.org) contains a wealth of information about configuring and using the Globus service. Many common issues can be resolved quickly by browsing our frequently asked questions and reading the relevant guides and how-tos. We recommend consulting these resources first when looking for fast resolution to any issue you are having with the Globus service.
7.2.2. Mailing Lists
If you use Globus, then participating in one or more of the public email lists is an excellent way to keep in touch with your peers in the Globus Community. For questions about managing your Globus deployment, e.g. installing software for a Globus endpoint, configuring your firewall, and integrating your institution’s identity system, subscribe to the admin list. For other inquiries and discussions, try the user or developer lists. For more information on mailing lists and how to subscribe, click here.
7.2.3. Globus Support
Questions or issues that pertain to Globus Connect Server installation or to any client or service that is used in the Globus software-as-a-service (SaaS) or platform-as-a-service (PaaS) offering can be directed to the Globus support team by submitting a ticket. Subscriptions include a guaranteed support service level.
When submitting a ticket for an issue with Globus Connect Server, please include the endpoint name, a description of your issue, and screenshot/text dumps of any errors you are seeing. Please also include the output of the following commands, run as root, from the server hosting the GCS endpoint:
uname -a
ifconfig
ping $(hostname -f)
cat /etc/os-release; cat /etc/redhat-release
cat /etc/gridftp.d/*
cat /etc/gridftp.conf
globus-gridftp-server --version
grep -v "^$\|^;" /etc/globus-connect-server.conf
8.
8.1. Description
Sharing allows an endpoint’s users to grant and control access to their data stored on the endpoint. A more detailed description of the benefits and features of sharing can be found here. A discussion of the security flow for share access/creation can be found here.
To illustrate let’s consider a case in which Adam, who is employed by ABC University and is the admin of the abcu#ep1 endpoint, wants to enable sharing on his endpoint for some of his users - such as Bob, Sue, etc. - so as to allow them to share with their collaborators (e.g. Ann) who do not have local accounts. In this case, Adam has decided that he wants to allow users to share out of the "globus" directory in their home directory and nowhere else.
8.2. Requirements
Sharing can only be enabled on a managed endpoint. Only organizations with a subscription are able to create managed endpoints.
For Adam to be able to enable sharing on his abcu#ep1 endpoint he will first need to be sure that ABC University has a subscription with Globus.
8.3. Make an endpoint managed
You can see how to make your endpoint managed in our FAQ.
8.4. How to enable and disable sharing in Globus Connect Server
To enable sharing on a managed endpoint, "Sharing = True" must be set in the [GridFTP] section of the /etc/globus-connect-server.conf file. Once this option is set, and the globus-connect-server-setup command is run to effect the configuration change, a managed endpoint will have sharing enabled. Similarly, setting "Sharing = False" will disable sharing on the endpoint. Note: This configuration option will have no effect on an endpoint that is not managed. Looking back to our use case with Adam, he will next need to log in (as root) to the server hosting the abcu#ep1 endpoint. He will then edit the /etc/globus-connect-server.conf file so as to set "Sharing = True" in the [GridFTP] section.
8.5. How to control sharing in Globus Connect Server
8.5.1. Controlling what can be shared
It is possible to control which file system paths on the endpoint are allowed for sharing via the "SharingRestrictPaths" option, which is also in the [GridFTP] section of the /etc/globus-connect-server.conf file. Remember, after making changes to the /etc/globus-connect-server.conf file it is necessary to run the 'globus-connect-server-setup' command before those changes will take effect. Additional details about these configuration options can be found here and also in the globus-connect-server.conf file itself.
In our scenario, Adam wants to allow his users to create shares that point to the "globus" directory in their home directory. Adam has also decided that he wants to allow his users to create shares that can both be read and written to, so he will specify the SharingRestrictPaths option in his endpoint’s globus-connect-server.conf file like so:
SharingRestrictPaths = RW~/globus
8.5.2. Controlling who can share
Recall that our use case with Adam required that sharing be enabled for only certain users. At this point sharing is enabled for all users on the server hosting the abcu#ep1 endpoint. By editing the SharingUsersAllow, SharingGroupsAllow, SharingUsersDeny, and SharingGroupsDeny options in the [GridFTP] section of the /etc/globus-connect-server.conf file, Adam can restrict which local users on the server are allowed to create shares on the abcu#ep1 endpoint. By default, when no value is set for any of these options, GCS will allow sharing for all local user accounts. Any users or groups listed in the SharingGroupsDeny or SharingUsersDeny options will not be allowed to share. If any value is set for SharingUsersAllow or SharingGroupsAllow, then only users and groups explicitly listed in these options will be allowed to share.
Since Adam only wants to allow certain users to be able to share, he has decided to create a local unix group on his system and adds the users he wants to allow to share to that unix group. Adam thus creates the globus_sharing_allow unix group and then adds Bob’s local user account to that group. Adam then edits his /etc/globus-connect-server.conf file and sets "SharingGroupsAllow = globus_sharing_allow". Adam next runs the globus-connect-server-setup command so that his configuration changes will take effect. With this change made, now all users who are members of the globus_sharing_allow group - such as Bob - will be able to share. Sue, who Adam has not yet added to the globus_sharing_allow group, is still unable to share. To address this, Adam adds Sue’s local user account to the globus_sharing_allow group and Sue is then able to share.
8.6. How to create and manage shares
Instructions for how to create and manage access to a share on a managed endpoint can be found here.
8.7. Details on effective share permissions
The share permissions you grant to a Globus user are just one part of the puzzle in determining that user’s actual level of access to a shared endpoint. To accurately determine the effective permissions that a user will have to a share we must also consider local file system permissions. In the case of a shared endpoint, all users accessing the share have the same local file system permissions as the local user that was used to activate on the endpoint to create the share. When considering the effective permissions that a user has to a share, it is important to remember that the most restrictive of the Globus permissions and the local file system permissions will determine access.
8.7.1. Local user creating a shared endpoint
To illustrate with an example, let’s consider Bob who wants to create a share on endpoint abcu#ep1. Bob has a local account on the server hosting abcu#ep1 with username of bob_local. Bob’s Globus account is bob_globus. When Bob wants to create his share hosted by abcu#ep1, he will log into the Globus Web App using his bob_globus credentials. He will then activate on the abcu#ep1 endpoint using his bob_local credentials. At that point, Bob can create his share. Let’s say Bob creates a share named bob_globus#myshare, which points to the /home/bob_local/globus directory. Now, Bob can assign share permissions to other users that he wants to have access to his share. He can give Globus users read or write permissions to the share as he desires.
8.7.2. Local user sharing with another local user
Let’s say that Bob gives Sue - who has a Globus account of sue_globus and a local account of sue_local - read and write access to the share. Now, if Sue wants to access the bob_local#myshare share, she will log into the Globus Web App using her sue_globus credentials. At this point, Sue’s ability to access the bob_local#myshare share depends on both the share permissions that Bob granted the sue_globus Globus user to the share, as well as the file system permissions that the local bob_local account has to the file system that the share points to. Even if Sue has read and write access at the share level, she still won’t - for example - be able to write to the share if the bob_local local user doesn’t have write permissions at the file system level in the file system space that the share points to. Notice that the permissions (or lack thereof) for the sue_local account played no role in determining Sue’s access to the share.
8.7.3. Local user sharing with a non local user
Along the same lines let’s consider what happens when Bob grants share access to Ann, who has a Globus account of ann_globus - and no local account on the server. For this example let’s say that Bob grants the ann_globus account read and write access to the bob_local#myshare share - which are the same permissions that he granted Sue. As sue_globus and ann_globus have identical share permissions, they now have identical levels of access to the share.
8.7.4. Summary
To sum up, both the share permissions granted to a Globus user, as well as the file system rights of the local user that the Globus user (who owns the share) activated on the endpoint with when the share was created, must be taken into account when determining effective permissions to a share.
9.
The management console, available on managed endpoints, provides a graphical web interface that can be used to monitor endpoint activity and to identify and troubleshoot faults that may indicate underlying infrastructure issues. An Administrator for an endpoint decides who has access to the Management Console for an endpoint via the assignment of the Activity Manager or Activity Monitor role to users, as appropriate. Instructions on how to manage and assign roles for an endpoint can be found here.
You can read about the details and benefits of the management console here.
10. Roles and privileges
Users (or groups) can be granted various roles on any managed endpoint, with each role granting the user (or group) different privileges with respect to that endpoint. All roles can be managed via the Transfer API or the Roles tab on the Endpoints page on the Globus Web App.
The following roles are supported on managed endpoints.
-
Administrator
-
Has full control over the endpoint definition of the endpoint.
-
Can delete endpoint definition
-
Can see endpoint definition even if set to private
-
Can manage roles for endpoint
-
An administrator for an S3 endpoint or a share also has all of the abilities of an Access Manager
-
Can be granted by other administrators. The creator of the endpoint is granted Administrator role by default
-
Has Access Manager and Activity Manager capabilities inherited
-
-
Activity Manager
-
Has full access to the Management Console for the endpoint and Activity Monitor capabilities
-
Can see endpoint definition even if set to private
-
Can be granted by any user who has Administrator role on the endpoint
-
-
Activity Monitor
-
Has read only access to the Management Console for the endpoint
-
Can see endpoint definition even if set to private
-
Can be granted by any user who has Administrator role on the endpoint
-
-
Access Manager
-
Can manage permissions on any endpoints that supports sharing (S3 or shared endpoints)
-
Has read/write access to folders and files on the endpoint
-
If the endpoint is set to private (in the case of S3 endpoint), cannot see the endpoint.
-
Can be granted by any user who has Administrator role on the endpoint
-
Does not have Activity Manager and/or Activity Monitor capabilities without being granted such explicitly
-
Appendix A: Understanding Data Channel Traffic
The data channel is where Globus Connect Server actually transmits the data that is being moved between endpoints. The default port range used for data channel connections is TCP 50000 to 51000. We strongly recommend that all endpoints be configured to use the default data port range, as this will provide maximum compatibility with other endpoints that are also configured to use the default data port range and have their firewall rules configured to allow traffic in this range. If your endpoint uses a non-default data port range, then you are - in effect - requiring other sites to potentially have to create additional firewall rules in order to be able to communicate properly with your endpoint. Many sites will not want to do this, which will thus limit the ability of your endpoint to interoperate with the majority of endpoints which are configured to use the default port range.
If two endpoints (ep1 and ep2) are to be able to successfully conduct transfers, then those endpoints must each be able to connect to each other in their configured data port ranges. For example, consider the following:
Globus Connect Server ep1 uses data port range 40000 to 41000
Globus Connect Server ep2 uses data port range 50000 to 51000
When two Globus Connect Server endpoints attempt to conduct a transfer, the endpoint that will be the recipient in that transfer picks out a port (or ports) in its configured data port range that it will listen on to receive the transfer from the sender endpoint. This port value gets communicated back from the receiver endpoint to the sender endpoint via GridFTP control channel data mediated by the Globus service, which both the sender and recipient are listening to on port 2811. Once the sender endpoint receives the data port range info for the recipient endpoint, it then initiates an outbound connection to the recipient to that port (or ports) on the recipient to conduct the actual data transfer. The recipient endpoint will only listen on the selected data port(s) for the duration of the transfer. Once the transfer is over, the recipient endpoint will no longer bind the chosen data port nor listen on it.
To illustrate, consider the case of ep1 and ep2 mentioned above. If ep1 wanted to send ep2 a file, then ep2 would pick out a port (or ports) in its configured data port range of 50000 to 51000. For the sake of example let’s say that port 50021 has been chosen. This value would then get communicated from ep2 to ep1, via the Globus service through the GridFTP control channel that both ep1 and ep2 are listening to. At that point, ep1 would then initiate a connection out to port 50021 on ep2.
To further illustrate, consider again the case of ep1 and ep2 mentioned above. If ep2 wanted to send ep1 a file, then ep1 would pick out a port (or ports) in its configured data port range of 40000 to 41000. For the sake of example let’s say that port 40331 has been chosen. This value would then get communicated from ep1 to ep2, via the Globus service through the GridFTP control channel that both ep1 and ep2 are listening to. At that point, ep2 would then initiate a connection out to port 40331 on ep1.
It is also important to consider what happens in cases where one endpoint is a Globus Connect Server endpoint and the other endpoint is a Globus Connect Personal endpoint. In such cases, the Globus Connect Personal endpoint will always initiate the connection to the Globus Connect Server endpoint for the transfer. Thus, it will always be the Globus Connect Server endpoint that picks the port (or ports) on which it will listen for that connection. This is the case irrespective of which endpoint is the sender or the recipient. As discussed previously, this information gets communicated from the Globus Connect Server endpoint to the Globus Connect Personal endpoint via the Globus service.
After looking at the example given we can see that, in terms of firewall rules, the outbound rules for ep1 must allow it to connect outbound to ep2 on ep2’s configured data port range if ep1 is to be able to send files to ep2. In terms of inbound rules, the firewall rules for ep1 must be configured to allow it to accept inbound connections on its own configured data port range for it to be able to receive files from other endpoints. The firewall rules for the data port range of any endpoint will be similar, and must allow outbound connections to the configured data port range of a remote endpoint for the local endpoint to be able to send files to the remote endpoint, and must allow inbound connections to the configure data port range of the local endpoint for that endpoint to be able to receive files from other endpoints.
As illustrated, an endpoint must be able to receive inbound connections on its own configured data port range, as well as be able to make outbound connections to the data port range of any endpoint it wishes to communicate with. If all Globus Connect Server admins pick their own custom port ranges, then this quickly leads to a situation in which site firewall policies become littered with custom rules for these various port ranges and endpoints. However, if everyone uses the default data port range, then firewall rules are much more predictable and manageable. It is for this reason that we recommend that everyone use the default data port range for their endpoint. Those who use a custom data port range may find that they have problems with their endpoint being able to communicate with other endpoints, for the reasons detailed above. Those using custom data port ranges may also find that the admins of other sites and endpoints may not be willing to set up custom firewall rules to accommodate custom data port range choices.
Appendix B: Globus Connect Server Configuration Packages
Globus Connect Server is delivered as a set of packages that may be used to configure and update services for use on a Globus endpoint. The most commonly used package is globus-connect-server. It is used to configure all services for a Globus endpoint on a server and creates configuration files for the various services based on a common Globus endpoint configuration file. Using the default configuration file will configure and enable all services needed to create a single-server endpoint (see configuration file example here).
The configuration file is parsed by globus-connect-server-setup (man page here), which is run when an endpoint is initially created or its configuration is updated. This file is also parsed by globus-connect-server-cleanup (man page here), which is run when it is desired to clean up the old endpoint configuration on the server. Both of these commands work by calling various component scripts that are sub-packages of the globus-connect-server package.
In addition to the consolidated globus-connect-server package, there are sub-packages, each with their own configuration programs which operate on the Globus Connect Server configuration file. Each sub-package operates on one of the services as described below, and depends on the packages needed to configure that service. The sub-packages are:
-
globus-connect-server-io may be used to install a Globus GridFTP server that implements a file transfer service.
-
globus-connect-server-id may be used to install a Globus MyProxy server that implements an identity provider service.
-
globus-connect-server-web may be used to install a MyProxy OAuth server that integrates the MyProxy service with a branded web interface.
Installing one of these sub-packages does not configure the services. The administrator must run the setup program associated with that package in order to actually do the configuration. Likewise, when one of the sub-packages is removed, it does not disable the service that it was used to configure; this is done by running the cleanup program associated with the sub-package.
Globus Connect Server I/O
The globus-connect-server-io package configures a GridFTP server. The main actions executed by this package during setup and cleanup are described below. This package depends on the authorization callouts, MyProxy (client side) programs, and the GridFTP server program.
globus-connect-server-io-setup (link to man page here)
-
Fetch a certificate from the Globus Connect CA and write GridFTP configuration to use it
-
Write GridFTP configuration to enable sharing (note: sharing may only be enabled on managed endpoints with a valid subscription)
-
Fetch MyProxy trust roots (if configured to use a MyProxy server)
-
Write GridFTP configuration for authorization callouts
-
If using CILogon, install CILogon CA and CRLs in the globus-connect-server certificate directory and add a cron job to refresh the CRL
-
If using a remote MyProxy server, fetch the MyProxy service certificate and trust roots and install them into the globus-connect-server certificate directory
-
-
(Re)start the GridFTP server
-
Enable the GridFTP server to start on reboots
-
Bind the GridFTP server to a Globus endpoint
globus-connect-server-io-cleanup (link to man page here)
-
Remove Globus Connect CA certificate if used
-
Remove GridFTP service configuration
-
Remove the Globus endpoint binding
-
Remove CILogon CRL cron job
-
Stop the GridFTP service
-
Disable the GridFTP service
Please see the command line tools for managing an I/O node configuration for more information.
Globus Connect Server Identity
The globus-connect-server-id package is used to configure a MyProxy identity service. This service can be configured as an identity provider using system passwords or as a certificate store for certificates generated elsewhere. The default configuration method for Globus Connect Server is as an identity provider. As an identity provider it will generate short-lived certificates for users if they are able to authenticate with their login password. As a certificate store, certificates are generated by some process outside of MyProxy and may be added to the store by the normal MyProxy Commands. This package depends on the MyProxy server and globus-simple-ca.
globus-connect-server-id-setup (link to man page here)
-
Fetch a certificate from the Globus Connect CA and write MyProxy server configuration to use it
-
Set up the MyProxy CA if acting as an identity provider
-
(Re)start the MyProxy server
-
Enable the MyProxy server to start on reboots
globus-connect-server-id-cleanup (link to man page here)
-
Stop the MyProxy CA service
-
Disable the MyProxy CA service
Globus Connect Server Web
The globus-connect-server-web package is used to configure a MyProxyOAuth identity service. This service provides a web interface to a MyProxy service, which may be running on the same node or elsewhere. This web interface may be customized by adding site-specific style sheets and images to make it conform to the look of the organization running the service.
globus-connect-server-web-setup (link to man page here)
-
Enable mod_ssl and the default SSL site if needed
-
Copy the OAuth site configuration to /etc/httpd/conf.d (rpm) or /etc/apache2/conf.d (deb)
-
Restart the web server
-
Enable the web server to start on reboots
globus-connect-server-web-cleanup (link to man page here)
-
Disable mod_ssl and the default SSL site if we enabled it during setup
-
Remove the OAuth site configuration file from /etc/httpd/conf.d or /etc/apache2/conf.d
-
Restart the web server
Appendix C: How to update a Globus Connect Server install
If your endpoint has multiple nodes, it is important to understand that all nodes need to be kept at the same version of the Globus software.
If you are planning a major version upgrade of the operating system where Globus Connect Server is installed, please contact Globus Support for additional guidance and instructions to ensure the endpoint state is maintained, and remains operational after the upgrade.
If your endpoint is a Managed Endpoint, then we recommend you use the Management Console to create a pause rule for your endpoint prior to performing the update. This will allow users to be automatically notified that the endpoint is undergoing maintenance and will hold user’s transfer tasks in a paused state during the update process instead of allowing the tasks to generate errors. Once the update is complete, the pause rule can be removed and normal operations on the endpoint can resume.
Follow these instructions to update your install:
$ sudo yum update \*globus\* \*myproxy\* \*gsi-openssh\*
$ sudo zypper refresh
$ sudo zypper up \*globus\* \*myproxy\* \*gsi-openssh\*
$ sudo apt-get update
$ sudo apt-get install --only-upgrade ".*globus.*" ".*myproxy.*" ".*gsi-openssh.*"
After updating your packages, be sure to restart the services and ensure that the update takes full effect by running:
$ sudo globus-connect-server-setup
Appendix D: Endpoint Identity Provider Options
Globus Connect Server can be configured to use one of three different Identity Provider options. The configuration details for each option are explained below. For a graphical overview of the authentication flows each of these methods use, see here.
MyProxy identity provider
To configure an endpoint to use the MyProxy identity provider, set "IdentityMethod = MyProxy" in the [Security] section of the config file and set the Server option in the [MyProxy] section to the hostname of the MyProxy server to be used for the endpoint. The default settings in the /etc/globus-connect-server.conf file will configure an endpoint to use MyProxy as the identity provider. If the servers on your endpoint are using NAT, then be sure to reference the NAT section for additional configuration details.
OAuth identity provider
To configure an endpoint to use the OAuth identity provider, set "IdentityMethod = OAuth" in the [Security] section of the config file and set the Server option in the [OAuth] section to the hostname of the OAuth server to be used for the endpoint. If the servers on your endpoint are using NAT, then be sure to reference the NAT section for additional configuration details. If you wish to use OAuth then you may wish to consider these additional options as well:
-
You may wish to use a CA issued ssl/tls server certificate for your web server. The exact details for setting this up will vary depending on the format that your CA provides your certificate to you in, your apache web server configuration, as well as the OS and version you are using. In all cases you’ll need to edit your apache configuration to set the "SSLCertificateFile" directive to point at the cert file you were issued by your CA. You may also need to set the "SSLCertificateKeyFile" and "SSLCertificateChainFile" directives as well, depending on the format of your certificate and the instructions you received from your CA concerning how to install the cert.
-
You may wish to use a custom logo or custom CSS to give your OAuth page a look and feel more consistent with your organization. You can configure a custom CSS by setting the Stylesheet option in the [OAuth] section of the /etc/globus-connect-server.conf file to the file containing the style sheet you wish to use. You can set a custom logo by setting the Logo option in the [OAuth] section of the /etc/globus-connect-server.conf file to the image file of the logo you wish to use.
CILogon identity provider
To configure an endpoint to use the CILogon identity provider, set "IdentityMethod = CILogon" in the [Security] section of the config file and set the CILogonIdentityProvider option in the [Security] section to the name of the CILogon provider to be used with the endpoint. See https://cilogon.org/ for a list of valid providers.
An endpoint configured in this way will attempt to map the CILogon credential to a local account on the server hosting the endpoint by matching it with a local account with the same account name as provided in the InCommon eduPersonPrincipalName (ePPN) given with the CILogon credential. Organizations participating in CILogon must make certain that their Shibboleth server releases the ePPN attribute to CILogon for this to work. To check if an identity provider is supplying the ePPN check here.
Appendix E: Setting Endpoint Network Use Options
Globus transfer uses configured network use levels and location of an endpoint to determine performance parameters to set on transfers against the endpoint. Administrators of endpoint may override the default values to best suit their deployment and needs. The configuration settings from source and destination endpoints are used to determine the concurrency and parallelism options used for a given transfer, thus leveraging the available transfer capacity, without overwhelming smaller capacity endpoints during transfers with larger capacity endpoints.
The location parameter is used to determine the distance and hence expected latency between the two endpoints, and is used in the automatic tuning of the transfers. By default the value of location parameter is automatically determined by Globus, but can be set by the endpoint administrator to explicit coordinates (in decimal degrees). This parameter cannot be set for S3 endpoints or shared endpoints.
Network use is set to "Normal" level by default. An administrator of a managed endpoint can set the network use levels for transfers against their endpoint. Endpoints that have multiple physical servers, and good end to end connectivity (network and storage) can set higher network use to ensure that Globus uses the bandwidth available, while smaller deployments can set this to lower levels.
Three preset options are provided for the endpoint administrator, which have the following values:
Option | Value |
---|---|
Minimal |
MaxConcurrency = 1 PreferredConcurrency = 1 MaxParallelism = 1 PreferredParallelism = 1 |
Normal (Default) |
MaxConcurrency = number of servers * 4 PreferredConcurrency = number of servers * 2 MaxParallelism = 8 PreferredParallelism = 4 |
Aggressive |
MaxConcurrency = number of servers * 8 PreferredConcurrency = number of servers * 4 MaxParallelism = 16 PreferredParallelism = 4 |
Note: S3 endpoints do not support parallelism options, only concurrency.
In addition to above, an administrator can choose the "Custom" option that lets them set absolute values for both concurrency and parallelism. All these options have a limit of 64 for MaximumConcurrency and MaximumParallelism. These values can be modified by using the —network-use option on endpoint-modify command in the Globus CLI.
For a given transfer, the concurrency is calculated as the smallest value across the MaximumConcurrency values of both endpoints, and the maximum of the PreferredConcurrency of both endpoints. Parallelism is also calculated similarly, with an additional consideration for transfers with high latency (trans-oceanic transfers) where the parallelism is set to minimum of the Maximum Parallelism value set for both endpoints.
Appendix F: GridFTP Hostcerts, Hostkeys, and Trusted Certs
This material is intended mainly for those interested in using a custom host certificate - i.e. a hostcert not generated by globus-connect-server-setup. However, anyone interested in having a more in-depth understanding of how GCS handles/uses certificates may find the details below to be of use.
hostcert basics
A host certificate is used to provide a method to determine the identity of a computer system. The Globus service must be able to verify that the hostcert in use by a GridFTP server belongs to the server listed in the URI for that server in the endpoint definition. For this to happen, one of the following must be true:
-
The hostcert used by the GridFTP server on each of the i/o nodes must have a cert subject identical to the Subject DN for the Server entry for the i/o node in the endpoint definition. The value in the Subject DN field of the endpoint definition for the server entry must match the cert subject of the hostcert in use on the GridFTP server, unless the Subject DN field is left empty in the endpoint definition.
-
The hostcert used by the GridFTP server on each of the i/o nodes must have a subjectAltName extension of type dNSName (see RFC 2818) that matches the hostname in the URI for the server entry for the i/o node in the endpoint definition. Leave the Subject DN field for the server in the endpoint definition blank to validate server identity in this way. If a hostcert contains a subjectAltName extension of type dNSName then this method of server identification will always be used in preference to the method discussed below in C, unless the Subject DN field for the i/o node in the endpoint definition is set, in which case method A above will be used.
-
The hostcert used by the GridFTP server on each of the i/o nodes must have a Subject CN that matches the GridFTP hostname in the URI for the server entry for the i/o node in the endpoint definition. Leave the Subject DN field for the server in the endpoint definition blank to validate server identity in this way.
Additionally, the hostcert used must be issued by a CA that the Globus service trusts.
By default, GCS will automatically generate a hostcert (and associated hostkey) issued by the Globus CA for a GridFTP server when the 'globus-connect-server-setup' command is run. This hostcert will be issued with a cert subject CN set to a random UUID. The properties for this auto-issued cert cannot be altered or modified. By default, when run on an i/o node, 'globus-connect-server-setup' will automatically update the endpoint definition so as to set the subject DN for the server entry that corresponds to the i/o node in the endpoint definition to be equal to the value of the auto-generated hostcert subject.
Those sites that must use a custom hostcert, rather than the Globus CA issued hostcert auto-generated by GCS as discussed above, will need to request their hostcert from an IGTF CA, a list of which can be found here.
Naturally, the hostcert and hostkey used by the GridFTP server must be part of the same key pair. To ensure that the hostcert and hostkey belong to the same key pair, one can check the hash of the modulus of the hostcert and hostkey to ensure that they match like so:
if [ "$(openssl x509 -modulus -noout -in hostcert.pem | sha256sum)" == "$(openssl rsa -modulus -noout -in hostkey.pem | sha256sum)" ]; then echo "hostcert and hostkey have matching modulus"; else echo "hostcert and hostkey modulus do not match"; fi
globus-connect-server.conf options
The following describes the portions of the globus-connect-server.conf file that are relevant to the hostcert, hostkey and trusted certs directory. As always, changes to the globus-connect-server.conf file do not take effect on the endpoint until the 'globus-connect-server-setup' command is run.
hostcert
By default, GCS will look for the hostcert at:
/var/lib/globus-connect-server/grid-security/hostcert.pem
The path where GCS looks for the hostcert can be changed from the default by setting the "CertificateFile" value in the [Security] section of the globus-connect-server.conf file. By default, GCS will auto-generate a new hostcert every time the 'globus-connect-server-setup' command is run. To prevent this see below.
hostkey
By default, GCS will look for the hostkey at:
/var/lib/globus-connect-server/grid-security/hostkey.pem
The path where GCS looks for the hostkey can be changed from the default by setting the "KeyFile" value in the [Security] section of the globus-connect-server.conf file. By default, GCS will auto-generate a new hostkey every time the 'globus-connect-server-setup' command is run. To prevent this see below.
Trusted certs directory
By default, GCS looks for its trusted certs here:
/var/lib/globus-connect-server/grid-security/certificates/
The path where GCS looks for its trusted certs can be changed from the default by setting the "TrustedCertificateDirectory" value in the [Security] section of the globus-connect-server.conf file.
If you are using GCS, and are using an auto-generated Globus CA signed hostcert, then GCS will fully manage the contents of this directory for you and it is unlikely that you will ever need to manage the contents of your trusted certs directory manually. If, however, you are using a custom hostcert and/or are managing this directory manually, then the below information should help you to understand what files should be in this directory.
Trusted certs directory contents
The trusted certs directory must contain a copy of the CA cert that signed the hostcert. If the CA that signed the hostcert was not a root CA, then the trusted certs directory must contain a copy of all of the intermediate CA certs needed to chain back to the root CA, and must also contain a copy of the root CA cert.
The certs in the trusted certs directory must be named according to this pattern: "cert_subject_hash.0". The corresponding .signing_policy file for each cert must be named according to this pattern: "cert_subject_hash.signing_policy". The following command can be used to find the subject hash value for a cert:
openssl x509 -subject_hash -noout -in cert.pem
Configuring trusted certs for endpoint using MyProxy based activation
If the endpoint is configured for MyProxy (or OAuth for MyProxy) based activation, then the trusted certs directory must contain a copy of the CA cert for the MyProxy server, as well as any additional certs needed to chain that cert back to a root CA as described above. It is also required that the corresponding .signing_policy files for the cert(s) be placed into the trusted certs directory as well. By default, when using the MyProxy service that ships with GCS, the trusted certs directory will be automatically configured with the needed MyProxy CA certs.
Configuring trusted certs for endpoint using CILogon based activation
If the endpoint is configured for CILogon based activation, then the trusted certs directory must contain a copy of the CILogon certs as found here:
In addition to these certs described above, the trusted certs directory must contain a copy of the .signing_policy file for each of the certs. By default, GCS will automatically configure the trusted certs directory with the needed CILogon CA certs when GCS is configured to use CILogon based activation.
Appendix G: Configuring an endpoint with multiple I/O nodes
This appendix will explain how to add additional I/O nodes to an existing, functional endpoint.
Any data you intend to make available to other Globus users should be located on a distributed file system (DFS) that spans the I/O nodes. Failure to locate the data on a properly configured DFS will result in inconsistencies in the ways that data is stored, retrieved, and presented to Globus users. Details concerning the proper configuration of a DFS are beyond the scope of this document, but the following links should prove helpful for those looking for more information:
Prerequisites
To set up an endpoint with multiple I/O nodes you must first deploy GCS to a single node, which is a subject covered at length in the rest of the GCS Install Guide. It is also important that you have the distributed file system you intend to use for data storage for this endpoint mounted on this node. You must also then test to ensure that the endpoint is working properly at this stage, prior to adding any additional I/O nodes. These are all prerequisites for the remaining steps.
Adding Additional I/O Nodes
-
When adding an additional I/O node to the endpoint after the first, begin by installing the GCS software onto the new node.
-
Next, configure the globus-connect-server.conf file on the new node such that the value of the [Endpoint].Name option is the same as what is shown for that option’s value in the globus-connect-server.conf file of the first node.
-
The new node must now be configured to use the same Identity Provider that the original node was configured to use. The exact steps will depend upon which type of Identity Provider the endpoint has been configured to use.
-
If the endpoint has been configured to use the CILogon Identity Provider, then configure the value of the [Security].IdentityMethod option in the globus-connect-server.conf file on the new node to be CILogon and also configure the value of the [Security].CILogonIdentityProvider option to have the same value as shown for that option in the original node’s globus-connect-server.conf file.
-
If the endpoint has been configured to use the MyProxy Identity Provider, then configure the value of the [Security].IdentityMethod option in the globus-connect-server.conf file on the new node to be MyProxy. Next, look at the value of the [MyProxy].Server option in the original node’s globus-connect-server.conf file. If this value is set to %(HOSTNAME)s, then configure the [MyProxy].Server option in the globus-connect-server.conf file of the new node to be the DNS resolvable hostname of the first node. Otherwise, configure the [MyProxy].Server option in the globus-connect-server.conf file of the new node to be the same as what is shown for this value in the globus-connect-server.conf file of the first node. Finally, set the [MyProxy].CaSubjectDN option in the globus-connect-server.conf file of the new node to be equal to the certificate subject of the MyProxy CA certificate being used by the MyProxy server. If this option isn’t present in your globus-connect-server.conf file then you will need to add it. Assuming a standard GCS install with the first node being configured to run the MyProxy service, the certificate subject of the MyProxy CA certificate could be found by running the following command on the first node:
openssl x509 -subject -noout -in /var/lib/globus-connect-server/myproxy-ca/cacert.pem
-
If the endpoint has been configured to use the OAuth Identity Provider, then configure the value of the [Security].IdentityMethod option in the globus-connect-server.conf file on the new node to be OAuth. Next, look at the value of the [OAuth].Server option in the original node’s globus-connect-server.conf file. If this value is set to %(HOSTNAME)s, then configure the [OAuth].Server option in the globus-connect-server.conf file of the new node to be the DNS resolvable hostname of the first node. Otherwise, configure the [OAuth].Server option in the globus-connect-server.conf file of the new node to be the same as what is shown for this value in the globus-connect-server.conf file of the first node. Next, configure the [MyProxy].Server and [MyProxy].CaSubjectDN options in the globus-connect-server.conf file on the new node to use values appropriate for the MyProxy service that the node running the OAuth service has been configured to use. These values can be determined by examining the configuration of the node running the MyProxy service, as is discussed in the previous paragraph.
-
-
Finally, make whatever additional edits are needed to the globus-connect-server.conf file on the new node to complete configuration - enable sharing if needed, configure NAT for GridFTP if needed, etc. Once the configuration of the globus-connect-server.conf file is finished, run the globus-connect-server-setup command on the new node to add it to the endpoint.
Your endpoint will now be configured with an additional I/O node. To add additional I/O nodes to your endpoint, simply repeat steps 1-4 on each new I/O node.
Appendix H: Uninstalling Globus Connect Server v4
To uninstall Globus Connect Server v4, perform the following steps:
Remove Globus Packages
After cleaning up your old endpoint definition remove the old Globus packages like so:
$ sudo yum remove \*globus\*
$ sudo yum remove \*myproxy\*
$ sudo zypper remove \*globus\*
$ sudo zypper remove \*myproxy\*
$ sudo apt-get purge ".*globus.*"
$ sudo apt-get purge ".*myproxy.*"
Remove any Globus configuration files
Finally, ensure that you remove old config that might still be left behind after removing the packages:
$ sudo rm /etc/globus-connect-server.conf
$ sudo rm -r /etc/grid-security
$ sudo rm -r /var/lib/globus-connect-server
$ sudo rm /etc/gridftp.conf
Glossary
- Access Manager
-
The access manager role grants the ability to control read and/or write access permissions for other Globus users on a shared endpoint. You can read a more in-depth discussion here.
- Endpoint
-
This is general term used to refer to a specific service that Globus can use to perform file transfers and other functions. For example:
-
I transferred files to the endpoint
-
I created a share for you on the endpoint
-
The endpoint’s file system is fast
-
- Endpoint Definition
-
This term refers to the metadata about the endpoint, stored as an object in the Globus.org database, used to simplify using and referring to the endpoint for users. Examples of such metadata include the hostname, port, OAuth server, default directory, etc… Much of the information in the endpoint definition is sent to Globus when the globus-connect-server-setup command is run.
- Endpoint Name
-
The name of your endpoint uniquely identifies it among all Globus endpoints and allows others to search for and find your endpoint via the Globus service. A full endpoint name takes the form of: ORGANIZATION_ACCOUNT_NAME#UNIQUE_SHORT_NAME. As indicated, the first part of an endpoint’s full name is the name of the organization account that was used to create the endpoint. The first part of the endpoint name will be the same for all endpoints in your organization. The second part of the endpoint name will be unique within your organization. It is a good idea choose something that describes the purpose of the endpoint as the second part of the endpoint name; doing so makes it easier for users to find the particular endpoint they are looking for. For example, if ABC University (with an organization account name of "abcu") had a geosciences-related endpoint, they might name it abcu#geosciences.
- GridFTP
-
GridFTP is an extension of the standard File Transfer Protocol (FTP) for high-speed, reliable, and secure data transfer.
- id node
-
A physical/virtual server providing MyProxy service to an endpoint
- i/o node
-
A physical/virtual server providing GridFTP service to an endpoint
- Managed Endpoint
-
A managed endpoint is an endpoint that is covered under a subscription and allows advanced features (like sharing) to be enabled. To convert an existing endpoint into a managed endpoint see this writeup.
- MyProxy
-
MyProxy is open source software for managing X.509 Public Key Infrastructure (PKI) security credentials (certificates and private keys). See the MyProxy website for more information.
- OAuth for MyProxy
-
OAuth for MyProxy provides an OAuth-compliant REST web interface to the MyProxy service for providing user certificates to Globus. See the 'OAuth' section of the MyProxy webpage for more information.
- Organization Account
-
This is a regular Globus ID, but it will be used for a specific purpose for your organization. It will be used to create and manage endpoints for your organization. It is a good idea to pick an account name that is easily recognized as being associated with your endpoints, as the name of your Organization account will be the first part of the endpoint name for every endpoint created for your organization. For example, ABC University might create the organization account of "abc" or "abc", which would allow them to create the endpoint abc#myhpcmachine in the organization account. Please do not choose account names that correspond to copyrighted or trademarked terms unless your organization has rights to those terms.
- Shared Endpoint
-
A shared endpoint enables a particular folder to be shared with other Globus users. Only authorized users of a managed endpoint may create shared endpoints. One or more shared endpoints may be created on a managed endpoint. The shared endpoint owner can grant read/write permissions and assign the access manager role for the shared endpoint.
- web node
-
A physical/virtual server providing OAuth service to an endpoint