Globus Connect Personal Troubleshooting Guide

1. Introduction

This document will discuss methods to troubleshoot common issues for Globus Connect Personal (GCP) based endpoints. We’ll use the term workstation in this doc to refer to any laptop, desktop, or server hosting a GCP endpoint.

2. Troubleshooting Firewall Issues

For a Globus file transfer to work, a number of network connections must be able to be established. Thus, one of the most common issue is with campus or other firewalls that prevent these connections from being established. The most common issues of this nature can be divided into these categories:

  • Connections between your workstation and the Globus services

  • Connections between your workstation and other Globus endpoint(s)

2.1. Troubleshooting issues connecting to the Globus Services

If the Globus Connect Personal software is not able to connect to the Globus service, then it cannot work correctly.

2.1.1. Common Errors

You will most likely become aware of problems connecting to the Globus service when first setting your endpoint up or after changes have been made to your network - including moving your workstation to a different network.

Installing GCP fails producing a "relaytool setup failed", "Could not complete collection registration", or "Connection timed out" error
error_1_image

The actual text of the error may vary somewhat from the sample image above. These sort of errors indicate that the workstation where you are attempting to install the Globus Connect Personal Software cannot connect to the Globus service.

The Globus Connect Personal software shows a 'Not Connected' message

If GCP is unable to connect to the Globus service, then the Globus icon (circle with a 'g' in it) in your system tray or menu bar will appear grayed out and will have an exclamation point in it:

error_2_image

Clicking on the Globus icon will then show you a brief description of what is wrong:

error_3_image

If you see the 'Not Connected' error after clicking on the Globus icon, then the GCP software is not able to connect to the Globus service.

2.1.2. Troubleshooting Steps

If you suspect that you’re having problems connecting to the Globus service from your workstation, there are a couple of steps you can take to attempt to troubleshoot the issue further.

Test Your Internet Connection

You’ll first want to verify that your workstation has a functional Internet connection. You can do this by simply opening your web browser and navigating to the Globus website at the URL below:

If you’re not able to browse to the Globus website on the system where your endpoint is located, then you’ll need to work with your organization’s IT staff or your ISP to resolve your Internet connectivity issues before you’ll be able to use the GCP software or access your endpoint.

Test Connectivity to the Globus Service

If your issue persists even after verifying that you have a functional Internet connection, you’ll next want to test connectivity to the relay.globusonline.org host from your system.

The GCP software must be able to connect to port 2223 on the relay.globusonline.org server in order to be able to function. A simple way to test if such connections can be made is to use the 'telnet' command to attempt to make such a connection like so:

telnet relay.globusonline.org 2223

If attempting to run the 'telnet' command produces something like a 'command not found' error, then you don’t have telnet installed on your system and will need to install it to perform the test.

If the connection is successful, you should see output like this:

Trying 184.73.255.160...
Connected to relay.globusonline.org.
Escape character is \'^]'.
SSH-2.0-OpenSSH_5.9p1-hpn13v11 GSI_GSSAPI_GPT_5.4 GSI

If the command produces different output than the above, then the connection attempt was not successful. In such a case, we have some basic Troubleshooting Steps that you’ll want to try next.

2.2. Troubleshooting Issues Connecting to Other Endpoints

All GCP endpoints must be able to connect to other endpoints via Data channel connections. During a transfer data is moved between endpoints using Data Channel connections. If there are problems establishing these Data Channel connections between endpoints, then transfers will not work correctly.

2.2.1. Common Errors

You will most likely become aware of Data Channel issues with your endpoint after you notice that transfers to or from your endpoint appear to fail. For transfers that have failed, you can look for errors in the transfer log like so:

  • Go to the Activity page on the Globus website

  • Find the failed job

  • Click the "Event Log" tab

  • Find the "fault events"

We’ll discuss some of the more common errors you might find below.

Example Data Channel Error A - No route to host
Error (transfer)
Endpoint: YOUR_PERSONAL_ENDPOINT_NAME (YOUR_PERSONAL_ENDPOINT_UUID)
Server: Globus Connect
File: VARIES
Command: VARIES
Message: Fatal FTP response
---
Details: 500-Command failed. : globus_xio: The GSI XIO driver failed to
establish a connection via the underlying protocol.\r\n500-globus_xio:
Unable to connect to 198.51.100.10:50263\r\n500-globus_xio: System error
in connect: No route to host\r\n500-globus_xio: A system call failed:
No route to host\r\n500 End.\r\n
Example Data Channel Error B - Connection timed out
Error (transfer)
Endpoint: YOUR_PERSONAL_ENDPOINT_NAME (YOUR_PERSONAL_ENDPOINT_UUID)
Server: Globus Connect
File: VARIES
Command: VARIES
Message: Fatal FTP response
---
Details: 500-Command failed. : globus_xio: The GSI XIO driver failed to
establish a connection via the underlying protocol.\r\n500-globus_xio:
Unable to connect to 198.51.100.10:50329\r\n500-globus_xio: System error
in connect: Connection timed out\r\n500-globus_xio: A system call failed:
Connection timed out\r\n500 End.\r\n
Example Data Channel Error C - Connection refused
Error (transfer)
Endpoint: YOUR_PERSONAL_ENDPOINT_NAME (YOUR_PERSONAL_ENDPOINT_UUID)
Server: Globus Connect
File: VARIES
Command: VARIES
Message: Fatal FTP response
---
Details: 500-Command failed. : globus_xio: The GSI XIO driver failed to
establish a connection via the underlying protocol.\r\n500-globus_xio:
Unable to connect to 198.51.100.10:50611\r\n500-globus_xio: System error
in connect: Connection refused\r\n500-globus_xio: A system call failed:
Connection refused\r\n500 End.\r\n

All of the above are examples of fault messages you might see if there is a firewall or other networking issue preventing data channel connections from your personal endpoint to a remote endpoint. You’ll notice that these faults are generated by your personal endpoint. You’ll also notice in all cases that the error message contains Unable to connect to IP_ADDRESS:PORT. This is telling us that the personal endpoint is not able to establish a data channel connection to the remote endpoint at that particular IP and port. The IP address and ports given in the example errors above are just samples for illustration purposes.

Example Data Channel Error D - Could Not Verify Credential
Error (transfer)
Endpoint: YOUR_PERSONAL_ENDPOINT_NAME (YOUR_PERSONAL_ENDPOINT_UUID)
Server: Globus Connect
File: VARIES
Command: VARIES
Message: Data channel authentication failed
---
Details: 500-Command failed. : an authentication operation failed
\r\n500-globus_xio_gsi: gss_init_sec_context failed.\r\n500-GSS
failure: \r\n500-GSS Major Status: Authentication Failed\r\n
500-GSS Minor Status Error Chain:\r\n500-globus_gsi_gssapi: SSL
handshake problems\r\n500-OpenSSL Error: ssl/statem/statem_clnt.c:1914:
in library: SSL routines, function tls_process_server_certificate:
certificate verify failed\r\n500-globus_gsi_callback_module:
Could not verify credential\r\n500-globus_gsi_callback_module:
Can't get the local trusted CA certificate: Untrusted self-signed certificate in chain
with hash d4c3b2a1\r\n500-\r\n500 End.\r\n

This sort of error tells us that the endpoint doesn’t trust the cert being offered for the data channel connection. This generally only happens if there is something interfering with the establishment of the data channel session between the two endpoints involved in the transfer. Data channel traffic looks similar to https traffic in some ways, so firewall or network policy designed to limit or monitor such traffic can interfere with the establishment of data channel sessions between endpoints. We sometimes see these sorts of errors for endpoints located behind https intercept proxies or similar devices. Globus data channel traffic cannot be proxied in this way, so sites that do operate with policy designed to intercept https/ssl traffic will need to configure exceptions for Globus data channel traffic for endpoints operating on their network.

2.2.2. Troubleshooting Steps

If you are seeing errors such as this for your job, you’ll want to take steps to understand the scope of the issue so as to best determine where the actual problem is. We offer a set of Troubleshooting Steps in this doc to help you do this.

2.3. Troubleshooting ICE Negotiation Issues

ICE Negotiation issues are a special category of error that can occur when attempting to transfer data between 2 Globus Connect Personal endpoints when at least one of those endpoints is located on a network that doesn’t support the needed STUN and ICE protocols.

2.3.1. Description and Explanation

If transfers with your personal endpoint produce errors such as "Details: 500 globus_xio: ICE negotiation failed" then you are experiencing ICE negotiation issues. You will most likely become aware of ICE negotiation issues by seeing related error events in the transfer log for your job on the Activity page on the Globus website.

Example ICE Negotiation Error
Error (session setup)
Endpoint: YOUR_PERSONAL_ENDPOINT_NAME (YOUR_PERSONAL_ENDPOINT_UUID)
Server: Globus Connect
Command: SITE UPRT  8JLg tb4UM/hvXl+OAV1E6KLWiw 1,2013266431,192.168.129.1,52324,
host 2,2013266431,192.168.168.1,52320,host 3,2013266431,198.51.100.10,48542,host 4,
1677721855,198.51.100.10,52324,srflx
Message: Fatal FTP response
---
Details: 500 globus_xio: ICE negotiation failed.\r\n

"ICE negotiation failed" refers to Globus Connect Personal’s use of the STUN and ICE protocols to do "NAT hole punching". These are protocols for traversing a NAT device, typically something like a home router, and establishing peer-to-peer connections between two Globus Connect Personal endpoints.

Some networks are incompatible with these protocols. In particular, if both of the endpoints are behind symmetric NAT, ICE cannot be used to establish connections. If you’re getting errors related to this, it strongly suggests that your network topology does not allow ICE to create a connection between the Globus Connect Personal endpoints you’re using.

Your endpoints should function correctly when used with any Globus Connect Server, including Globus Tutorial Endpoint 1 and Globus Tutorial Endpoint 2. That’s because Globus Connect Personal will simply make outbound connections to Globus Connect Server and doesn’t need to try to use ICE.

In order for two Globus Connect Personal Endpoints to connect to one another using ICE, they need to be able to communicate with one another using UDP on ephemeral ports. You can see a detailed document on the ports which are needed by Globus Connect Personal here.

The requirement for Outbound UDP 32768-65535 refers to the use of ephemeral ports.

Transfers between two Globus Connect Personal endpoints require the use of ICE and STUN to be able to work. If one or both of the Globus Connect Personal endpoints involved in such an attempted transfer are on networks that don’t support these protocols, then such transfers will not be possible.

2.3.2. Potential Solutions

  • Contact your local IT staff to see if they can help you to address the STUN and ICE related issues.

  • Relocate one or both of the endpoints to networks that support STUN and ICE.

  • Avoid STUN and ICE related issues by putting your data through an extra hop using a Globus Connect Server endpoint. First transfer from the source personal endpoint to the server endpoint, then from the server endpoint to the destination personal endpoint. If you do this, wait for the first transfer to complete before submitting the second one, or you won’t get a complete data transfer.

2.4. GCP Network Troubleshooting

If you believe you are having firewall or network related issues with your personal endpoint, the following series of steps can help you to better determine the nature and scope of the problem.

2.4.1. Transfer with the Globus Tutorial Endpoint

Test Process

If you were able to successfully complete the set up process for your endpoint, then start here. If not, then skip to the next step immediately.

You’ll first want to attempt to transfer files in both directions between your personal endpoint and the Globus Tutorial Endpoint. If you are new to the Globus service, this tutorial will explain how to perform transfers.

  • Try to transfer a few small files from your personal endpoint to the Globus Tutorial Endpoint

  • Try to transfer a few small files from the Globus Tutorial Endpoint to your personal endpoint

Users have a 10MB quota on the Tutorial Endpoint, so you’ll want to keep that in mind when doing your test transfers.

Test Succeeds

If you find that transfers to and from your personal endpoint and the Globus Tutorial Endpoint SUCCEED, but you still encounter problems when transferring with a different particular endpoint, then it is likely that the problem is related to that other endpoint. You’ll want to reach out to the support staff for that remote endpoint and ask for their help in resolving the issue. If the admin of that endpoint should require further support, then they can open a ticket with us so that we can work directly with them to resolve the issue.

Test Fails

If either of the above test transfers FAIL, then the issue is likely on the side of your personal endpoint. Go ahead and proceed with the next step.

2.4.2. Temporarily Disable the Host Firewall

Test Process

You’ll next want to try disabling the host firewall on your workstation.

  • If you were unable to setup/install the Globus Connect Personal software previously when your host firewall was enabled, go ahead and try again when your host firewall is disabled.

  • If you were unable to conduct transfers with the Globus Tutorial Endpoint previously when your host firewall was enabled, go ahead and try again when your host firewall is disabled.

Test Succeeds

If your attempts SUCCEED when your host firewall is disabled, then you’ll need to configure your host firewall to be consistent with our Globus Connect Personal Firewall Requirements Documentation. You’ll want to reach out to your local IT staff if you need assistance in configuring your host firewall.

Test Fails

If your attempts still FAIL then go ahead and proceed with the next step.

2.4.3. Attempt to Create a New Endpoint on a Different Workstation on the Same Network

Test Process

If possible, go ahead and attempt to setup a new personal endpoint on a different workstation at the same location/network as the workstation you are having trouble with is located. If the setup attempt succeeds, then go ahead and attempt to transfer with the Globus Tutorial Endpoint as well.

Test Succeeds

If you find that you now SUCCEED in setting up the Globus Connect Personal software and conducting transfers when using the personal endpoint on the alternate workstation, then the issue is very likely with the original workstation.

Test Fails

If you find that you still FAIL to be able to setup the Globus Connect Personal software or conduct transfers when using the personal endpoint on the alternate workstation, then go ahead and proceed with the next step.

2.4.4. Attempt to Create a New Endpoint on a Different Workstation on a Different Network

Test Process

If possible, go ahead and attempt to setup a new personal endpoint on a different workstation at a different location/network than the workstation you are having trouble with is located. For example, if the original workstation is located at work, try using a different workstation located at home. If the setup attempt succeeds, then go ahead and attempt to transfer with the Globus Tutorial Endpoint as well.

Test Succeeds

If you find that you now SUCCEED in setting up the Globus Connect Personal software and conducting transfers when using the personal endpoint on the alternate workstation, then the issue is very likely with the original workstation or the network where it is located. You’ll want to reach out to the IT staff at the original location and work with them to get the Globus Connect Personal software working at that location. You’ll want to be sure to mention the Globus Connect Personal Firewall Requirements Documentation to your IT staff when you contact them so they can ensure that policy for any campus firewalls will meet those requirements.

Test Fails

If you find that you still FAIL to be able to setup the Globus Connect Personal software or conduct transfers when using the personal endpoint on the alternate workstation, then go ahead and open a support ticket with us so we can help you look into things further.