SpectraLogic BlackPearl Connector
Last Updated: June 18, 2020
Note: Globus is now certified with BlackPearl 5.x
The SpectraLogic BlackPearl Connector enables use of a Globus data access interface on a BlackPearl storage system. This requires the installation of Globus Connect Server. The connector is available as an add-on subscription to organizations with a Globus Standard subscription - please contact us for pricing.
This document describes how to install and configure the SpectraLogic BlackPearl Connector as well as create a BlackPearl Storage Gateway. After the installation is complete, any authorized user can establish a connection to the BlackPearl buckets that they have access to by following the steps in this How To in order to create a guest collection using a configured BlackPearl Storage Gateway on the endpoint. The system administrator can also create a mapped collection using a configured high assurance BlackPearl Storage Gateway, by following the instructions in the mapped collection section of the high assurance Globus Connect Server version 5 guide.
The installation must be done by a system administrator, and has the following distinct set of steps:
Create a storage gateway on the endpoint configured to use the SpectraLogic BlackPearl Connector.
Create a SpectraLogic BlackPearl mapped collection.
Please contact us at email@example.com if you have questions or need help with installation and use of the SpectraLogic BlackPearl Connector.
- SpectraLogic BlackPearl Connector Virtual Filesystem
- SpectraLogic BlackPearl Account Access
- Storage Gateway
- Appendix A: Document Types for the SpectraLogic BlackPearl Connector
SpectraLogic BlackPearl Connector provides a distributed object store, where each data object is accessed based on a bucket name and an object name.
The SpectraLogic BlackPearl Connector attempts to make this look like a regular
filesystem, by treating the bucket name as the name of a directory in
the root of the storage gateway’s file system. For example, if a user
has access to buckets
bucket2, then those buckets would
show up as directories when listing
The SpectraLogic BlackPearl Connector also treats the
/ character as a delimiter in
the SpectraLogic BlackPearl API so that it can present something that looks like
like subdirectories. For example, the object
would appear as
/bucket1/object1 to The SpectraLogic BlackPearl connector, and
bucket2 would appear as a file called
object3 in the directory
When a user accesses a collection, Globus Connect Server will map the Globus Account to an account on the BlackPearl storage device. This process is handled with two steps:
The mapping configured on a storage gateway maps a Globus User to local account name.
The SpectraLogic BlackPearl Connector then maps the local account name to a BlackPearl access id and secret using an access id file as described below.
The access id file contains mappings, one per line, that will provide a
mapping from the local account name to valid BlackPearl account credentials.
This text file must reside in a location that is accessible to the
local account. There are three whitespace delimited colums per line:
The local username which is the result of the Storage Gateway user mapping
The SpectraLogic BlackPearl Access Id
The SpectraLogic BlackPearl Secret Key
bob amFzb25hbHQ= K9YxrEaj jane bm90amFzb24= JPBkGzQJ
In this example, there are entries for the local usernames bob and jane. Any other local username is denied access to the BlackPearl Storage Gateway.
User bob will access the SpectraLogic BlackPearl system with the Access ID amFzb25hbHQ= and the secret key K9YxrEaj.
Note the path of the access id file; it will be used when creating the storage gateway.
The SpectraLogic BlackPearl Connector requires a functional Globus Connect Server 5 endpoint in order to be used. Instructions for installing and configuring and endpoint using Globus Connect Server 5 can be found here. The rest of this document assumes that a functional Globus Connect Server 5 endpoint is being used when attempting to configure the SpectraLogic BlackPearl Connector.
A SpectraLogic BlackPearl Connector Storage Gateway is created with the command globus-connect-server storage-gateway create blackpearl, and can be updated with the command globus-connect-server storage-gateway update blackpearl.
Before looking into the policy options specific to the SpectraLogic BlackPearl Connector, please familiarize yourself with the Globus Connect Server v5 Data Access Guide which describes the steps to create and update a storage gateway, using the POSIX connector as an example. The commands to create and update a storage gateway for the SpectraLogic BlackPearl Connector are similar.
The SpectraLogic BlackPearl Connector has policies to configure the URL of the S3-compatible API endpoint providing access to the SpectraLogic BlackPearl system, and to specify the path to a configuration file which maps account names to SpectraLogic BlackPearl credentials.
The --s3-endpoint command-line option is used by Globus Connect Server to contact the S3-compatible API to access data on a SpectraLogic BlackPearl system.
For our example, we’ll use one running on
blackpearl.example.org (you must of course use the URL of the
SpectraLogic BlackPearl that is run by your organization).
The --blackpearl-access-id-file command-line option is used to allow Globus Connect Server to find a configuration file on the Data Transfer Node which contains information about SpectraLogic BlackPearl names and keys.
The format of the file is described in SpectraLogic BlackPearl Account Access section of this document.
For our example, we’ll instruct Globus Connect Server to use the file located at
/etc/blackpearl/access-id-file. This file must be identical on each Data
Transfer Node in order for file access to work reliably.
Now that we have decided on all our policies, we’ll use the command to create the storage gateway.
% globus-connect-server storage-gateway create blackpearl \ "BlackPearl Storage Gateway" \ --domain
/etc/blackpearl/access-id-fileStorage Gateway Created:
This was successful and the output the ID of the new storage gateway (
in this case) for our reference. Note that this will always
be a unique value if you run the command. If you forget the id of a storage
gateway, you can always use the command
list to get a list of the storage gateways on the endpoint.
You can also add other policies to configure additional identity mapping and path restriction policies as described in the Globus Connect Server v5 Data Access Guide.
Note that this creates the storage gateway, but does not yet make it accessible via Globus and HTTPS. You’ll need to follow the steps in the next section.
As the SpectraLogic BlackPearl Connector does not introduce any policies beyond those used by the base collection type, you can follow the sequence in the Collections Section of the Globus Connect Server v5 Data Access Guide. Recall however, that the paths are interpreted as described above in SpectraLogic BlackPearl Connector Virtual Filesystem.
The BlackPearlStoragePolicies document describes Black Pearl-specific policies. This document contains policies to locate the Black Pearl S3 endpoint and map account names to API keys.
Type of this document
The URL of the S3 endpoint of the BlackPearl appliance to use to access collections on this Storage Gateway.
Path to the file which provides mappings from usernames within the configured identity domain to the ID and secret associated with the user’s BlackPearl account