SpectraLogic BlackPearl Connector

Globus Connect Server version 5.3 is no longer supported. Please install version 5.4. If you need to upgrade an existing Globus Connect Server version 5.3, please read the Migration Guide and use the v5.3 migration tool.

Last Updated: September 4, 2019

Note: Globus is now certified with BlackPearl 5.x

The BlackPearl Connector enables use of a Globus data access interface on a BlackPearl storage system. This requires the installation of Globus Connect Server. The connector is available as an add-on subscription to organizations with a Globus Standard subscription - please contact us for pricing.

This document describes how to install and configure the BlackPearl Connector as well as create a BlackPearl Storage Gateway. After the installation is complete, any authorized user can establish a connection to the BlackPearl buckets that they have access to by following the steps in this How To in order to create a guest collection using a configured BlackPearl Storage Gateway on the endpoint. The system administrator can also create a mapped collection using a configured high assurance BlackPearl Storage Gateway, by following the instructions in the mapped collection section of the high assurance Globus Connect Server version 5 guide.

The installation must be done by a system administrator, and has the following distinct set of steps:

  • Installation of the packages needed for Globus Connect Server version 5 endpoint as well as creation of the endpoint itself.

  • Create a storage gateway on the endpoint configured to use the BlackPearl Connector.

Please contact us at support@globus.org if you have questions or need help with installation and use of the BlackPearl Connector.

Endpoint Installation

The BlackPearl Connector requires a functional Globus Connect Server 5 endpoint in order to be used. Instructions for installing and configuring and endpoint using Globus Connect Server 5 can be found here. The rest of this document assumes that a functional Globus Connect Server 5 endpoint is being used when attempting to configure the BlackPearl Connector.

SpectraLogic BlackPearl Account Access

When a user accesses a collection, Globus Connect Server will map the Globus Account to an account on the BlackPearl storage device. This process is handled with two steps:

  1. The mapping configured on a storage gateway maps a Globus User to local account name.

  2. The BlackPearl Connector then maps the local account name to a BlackPearl access id and secret using an access id file as described below.

SpectraLogic BlackPearl Access ID File Format

The access id file contains mappings, one per line, that will provide a mapping from the local account name to valid BlackPearl account credentials. This text file must reside in a location that is accessible to the gcsweb local account. There are three whitespace delimited colums per line:

  • The local username which is the result of the Storage Gateway user mapping

  • The SpectraLogic BlackPearl Access Id

  • The SpectraLogic BlackPearl Secret Key

Example 1. Example Access ID File
bob  amFzb25hbHQ= K9YxrEaj
jane bm90amFzb24= JPBkGzQJ

In this example, there are entries for the local usernames bob and jane. Any other local username is denied access to the BlackPearl Storage Gateway.

User bob will access the SpectraLogic BlackPearl system with the Access ID amFzb25hbHQ= and the secret key K9YxrEaj.

Note the path of the access id file; it will be used when creating the storage gateway.

Important
This file contains access keys for the BlackPearl service. Make sure to limit the file’s permissions to the gcsweb user created during GCS installation.

Creating a Storage Gateway using the BlackPearl Connector

To create a BlackPearl Storage Gateway on an endpoint, use the `globus-connect-server-config storage-gateway create' command. For example:

$ sudo globus-connect-server-config storage-gateway create \
    --connector "BlackPearl" \
    --display-name "BlackPearl Storage Gateway" \
    --root "/" \
    --domain example.edu \
    --s3-endpoint http://10.85.21.23:8000 \
    --bp-access-id-file /etc/blackpearl/access-id-file

Storage Gateway Created: 99d351bc-cdb4-4cee-be86-3bb01e4b1022

Note that the ID of the new storage gateway is given in the output.

This would create a storage gateway on the endpoint that:

  1. Uses the "BlackPearl" storage connector and is called "BlackPearl Storage Gateway".

  2. Causes new collections to be rooted at '/' on the http://10.85.21.23:8000 endpoint.

  3. Uses the access id file path, /etc/blackpearl/access-id-file, from the previous configuration step to map users to BlackPearl credentials.

  4. Allows Globus users with an identity from the example.edu Identity Provider to create collections, if they have a BlackPearl account mapped to the username of their example.edu domain identity.

The globus-connect-server-config storage-gateway create command supports the following options for storage gateways configured to use the BlackPearl Connector, in addition to the common options supported for all storage connectors:

--s3-endpoint ENDPOINT

Full URL of S3 compatible storage location. May include a port. e.g. http://10.85.21.23:8000. If you are using TLS with BlackPearl, you must specify a fully qualified domain name (fqdn), for example https://bp.example.com, and the fqdn must match the BlackPearl’s TLS certificate DN. Do not use a trailing / on the EndPoint value; doing so will cause requests to BlackPearl to fail. This is a required parameter for this connector.

--bp-access-id-file ACCESS_ID_FILE

Absolute path to the text file described above that maps Globus users to BlackPearl access ID and secret. This is a required parameter for this connector.

Creating a collection on a BlackPearl Storage Gateway

Once a BlackPearl Storage Gateway has been configured on the endpoint, permitted users can then create mapped collections using the storage gateway. These collections allow permitted Globus users access to the Ceph service using their Ceph username. The process of creating a new mapped collection using a storage gateway configured to use the BlackPearl Connector is found here. The system administrator can also create a mapped collection using a configured high assurance Ceph Storage Gateway, by following the instructions in the mapped collection section of the Globus Connect Server version 5 guide. Please also refer to the Globus Connect Server install document for the various options available in the tool to manage storage gateways.