SpectraLogic BlackPearl Connector

Note: Globus is now certified with BlackPearl 5.x

The SpectraLogic BlackPearl Connector enables use of a Globus data access interface on a BlackPearl storage system. This requires the installation of Globus Connect Server. The connector is available as an add-on subscription to organizations with a Globus Standard subscription - please contact us for pricing.

This document describes how to install and configure the SpectraLogic BlackPearl Connector as well as create a BlackPearl Storage Gateway. After the installation is complete, any authorized user can establish a connection to the BlackPearl buckets that they have access to by following the steps in this How To in order to create a guest collection using a configured BlackPearl Storage Gateway on the endpoint. The system administrator can also create a mapped collection using a configured high assurance BlackPearl Storage Gateway, by following the instructions in the mapped collection section of the high assurance Globus Connect Server version 5 guide.

The installation must be done by a system administrator, and has the following distinct set of steps:

  • Create a storage gateway on the endpoint configured to use the SpectraLogic BlackPearl Connector.

  • Create a SpectraLogic BlackPearl mapped collection.

Please contact us at support@globus.org if you have questions or need help with installation and use of the SpectraLogic BlackPearl Connector.

SpectraLogic BlackPearl Connector Virtual Filesystem

SpectraLogic BlackPearl Connector provides a distributed object store, where each data object is accessed based on a bucket name and an object name.

The SpectraLogic BlackPearl Connector attempts to make this look like a regular filesystem, by treating the bucket name as the name of a directory in the root of the storage gateway’s file system. For example, if a user has access to buckets bucket1 and bucket2, then those buckets would show up as directories when listing /.

The SpectraLogic BlackPearl Connector also treats the / character as a delimiter in the SpectraLogic BlackPearl API so that it can present something that looks like like subdirectories. For example, the object object1 in bucket1 would appear as /bucket1/object1 to the SpectraLogic BlackPearl connector, and the object object2/object3 in bucket2 would appear as a file called object3 in the directory /bucket2/object2.

SpectraLogic BlackPearl Account Access

When a user accesses a collection, Globus Connect Server will map the Globus Account to an account on the BlackPearl storage device. This process is handled with two steps:

  1. The mapping configured on a storage gateway maps a Globus User to local account name.

  2. The SpectraLogic BlackPearl Connector then maps the local account name to a BlackPearl access id and secret using an access id file as described below.

SpectraLogic BlackPearl Access ID File Format

The access id file contains mappings, one per line, that will provide a mapping from the local account name to valid BlackPearl account credentials. This text file must reside in a location that is accessible to the gcsweb local account. There are three whitespace delimited columns per line:

  • The local username which is the result of the Storage Gateway user mapping

  • The SpectraLogic BlackPearl Access Id

  • The SpectraLogic BlackPearl Secret Key

Example 1. Example Access ID File
bob  amFzb25hbHQ= K9YxrEaj
jane bm90amFzb24= JPBkGzQJ

In this example, there are entries for the local usernames bob and jane. Any other local username is denied access to the BlackPearl Storage Gateway.

User bob will access the SpectraLogic BlackPearl system with the Access ID amFzb25hbHQ= and the secret key K9YxrEaj.

Note the path of the access id file; it will be used when creating the storage gateway.

Important
This file contains access keys for the BlackPearl service. Make sure to limit the file’s permissions to the gcsweb user created during GCS installation.

The SpectraLogic BlackPearl Connector requires a functional Globus Connect Server 5 endpoint in order to be used. Instructions for installing and configuring and endpoint using Globus Connect Server 5 can be found here. The rest of this document assumes that a functional Globus Connect Server 5 endpoint is being used when attempting to configure the SpectraLogic BlackPearl Connector.

Storage Gateway

A SpectraLogic BlackPearl Connector Storage Gateway is created with the command globus-connect-server storage-gateway create blackpearl, and can be updated with the command globus-connect-server storage-gateway update blackpearl.

Before looking into the policy options specific to the SpectraLogic BlackPearl Connector, please familiarize yourself with the Globus Connect Server v5 Data Access Guide which describes the steps to create and update a storage gateway, using the POSIX connector as an example. The commands to create and update a storage gateway for the SpectraLogic BlackPearl Connector are similar.

SpectraLogic BlackPearl Connector Storage Gateway Policies

The SpectraLogic BlackPearl Connector has policies to configure the URL of the S3-compatible API endpoint providing access to the SpectraLogic BlackPearl system, and to specify the path to a configuration file which maps account names to SpectraLogic BlackPearl credentials.

Endpoint

The --s3-endpoint command-line option is used by Globus Connect Server to contact the S3-compatible API to access data on a SpectraLogic BlackPearl system.

Example 2. Selecting an S3 API Endpoint

For our example, we’ll use one running on blackpearl.example.org (you must of course use the URL of the SpectraLogic BlackPearl that is run by your organization).

--s3-endpoint blackpearl.example.org

Access ID File

The --blackpearl-access-id-file command-line option is used to allow Globus Connect Server to find a configuration file on the Data Transfer Node which contains information about SpectraLogic BlackPearl names and keys.

The format of the file is described in SpectraLogic BlackPearl Account Access section of this document.

Example 3. Setting the path to the --blackpearl-access-id-file.

For our example, we’ll instruct Globus Connect Server to use the file located at /etc/blackpearl/access-id-file. This file must be identical on each Data Transfer Node in order for file access to work reliably.

--blackpearl-access-id-file /etc/blackpearl/access-id-file

Creating the Storage Gateway

Now that we have decided on all our policies, we’ll use the command to create the storage gateway.

% globus-connect-server storage-gateway create blackpearl \
    "BlackPearl Storage Gateway" \
    --domain example.org \
    --s3-endpoint blackpearl.example.org \
    --blackpearl-access-id-file /etc/blackpearl/access-id-file

Storage Gateway Created: 7187a9a0-68e4-48ea-b3b9-7fd06630f8ab

This was successful and outputs the ID of the new storage gateway ( 7187a9a0-68e4-48ea-b3b9-7fd06630f8ab in this case) for our reference. Note that this will always be a unique value if you run the command. If you forget the id of a storage gateway, you can always use the command globus-connect-server storage-gateway list to get a list of the storage gateways on the endpoint.

You can also add other policies to configure additional identity mapping and path restriction policies as described in the Globus Connect Server v5 Data Access Guide.

Note that this creates the storage gateway, but does not yet make it accessible via Globus and HTTPS. You’ll need to follow the steps in the next section.

Collection

A SpectraLogic BlackPearl Collection is created with the command globus-connect-server collection create, and can be updated with the command globus-connect-server collection update.

As the SpectraLogic BlackPearl Connector does not introduce any policies beyond those used by the base collection type, you can follow the sequence in the Collections Section of the Globus Connect Server v5 Data Access Guide. Recall however, that the paths are interpreted as described above in SpectraLogic BlackPearl Connector Virtual Filesystem.

Appendix A: Document Types for the SpectraLogic BlackPearl Connector

BlackPearlStoragePolicies Document

Connector-specific storage gateway policies for the Blackpearl connector

One of the following schemas:

​

{
  "DATA_TYPE": "blackpearl_storage_policies#1.0.0",
  "bp_access_id_file": "string",
  "s3_endpoint": "string"
}