Ceph Connector

Last Updated: April 4, 2017

The Ceph connector enables use of a Globus data access interface on an Ceph storage system, via the Ceph Object Gateway. This requires the installation of Globus Connect Server and an additional package that is specific to the Ceph storage system called the Ceph DSI.

The Ceph connector is a premium feature available only to Globus subscribers, and is thus only available for Managed Endpoints.

For adding the Ceph connector to Globus Connect Server v5 endpoint, please refer to Ceph Connector for GCSv5

Prerequisites

A functional Globus Connect Server installation is required for installation and use of the Ceph connector. The Globus Connect Server Installation Guide provides detailed documentation on the steps for installing and configuring a server endpoint.

The Ceph connector is available for all distributions supported by Globus Connect Server.

Supported Ceph versions

The Ceph connector been tested against Jewel and Luminous.

Supported Globus Connect Server versions

The Ceph DSI should be used with the latest version of Globus Connect Server 4.x.

Installation

Install the package globus-gridftp-server-ceph from the Globus repository.

For RedHat-based systems:

$ yum install globus-gridftp-server-ceph

For Debian-based systems:

$ apt-get install globus-gridftp-server-ceph

For SLES 11-based systems:

$ zypper install globus-gridftp-server-ceph

Configuration

The Ceph DSI requires the following steps for configuration:

  • Create a RADOS Gateway User with users:read capabilities

  • Configure the Ceph DSI

  • Enable the Ceph DSI

  • Restart the GridFTP server

Ceph Admin User

The {connector_type} requires a RADOS Gateway User with the users:read capability in order to map Globus users to Ceph keys.

Create a RADOS Gateway User with users:read capabilities

This identity is used by the {connector_type} to look up keys associated with the Ceph user_id that the GridFTP session is authorized to run as.

This command must be run on a host with access to the ceph client.admin keyring in order to create the globus Ceph user_id:

$ radosgw-admin user create \
    --uid=globus \
    --display-name "Globus Ceph Connector" \
    --caps="users=read"

Note in the output for this command the access_key and secret_access_key fields of the keys object, as those will be needed in the next step. If you forget to record those, you can use the following command to retrieve the same information:

$ radosgw-admin user info --uid=globus

Configure the Ceph DSI

The package contains an example configuration file in /etc/globus/globus-gridftp-server-ceph.conf

The format of the file is very simple:

  • Comments begin with #

  • Configuration values are set by a line of the form name = value

There is no special quoting syntax, and whitespace is ignored between tokens.

At the very minimum, the configuration values "host_name", "ceph_rg_admin_access_key_id", and "ceph_rgw_admin_secret_access_key" must be set. There are comments in the file describing all available configuration options.

Note
This file contains the keys for the gridftp Ceph user which can read all Ceph user’s keys---do not change the permissions of this file to make it readable by anyone besides root.

Create a service user account

Since ceph users need not have user accounts on the local endpoint, ceph transfers will be configured to run under a local service user account. Create a user named globus-ceph. This account name will be used below as the value of the process_user configuration option.

Globus Connect Server configuration that refers to $HOME, such as SharingStateDir, will be using the home directory of this account. Ensure that these files are only accessible by the globus-ceph account.

Enable the Ceph DSI

Create the file /etc/gridftp.d/gridftp-ceph containing these lines:

 threads 2
 load_dsi_module ceph
 process_user globus-ceph

Restart the GridFTP Server 

$ service globus-gridftp-server restart

Troubleshooting

To enable a debugging log for the ceph dsi, set the environment variable GLOBUS_S3_DEBUG "1023,/tmp/s3.log" to enable a highly verbose log of the DSI. This can be easily done for a gridftp configuration by creating a file /etc/gridftp.d/ceph-debug with the contents

 $GLOBUS_S3_DEBUG "1023,/tmp/s3.log"

Basic Endpoint Functionality Test

After completing the installation, you should do some basic transfer tests with your endpoint to ensure that it is working. We document a process for basic endpoint functionality testing here.