Description

The Ingest API is used to send metadata into Globus Search.

URL

/v1/index/<index_name>/ingest

Method

POST

HTTP Headers

Authorization: Bearer <Globus Auth token> 1
Content-Type: must be "application/json"

Query Parameters

None

Request Body

a GIngest document

Response Body

{
  "success": true,
  "num_documents_ingested": Integer,
  "as_identity": IdentityID
}

Error Responses

HTTP Code 403: Forbidden if the identity associated with the bearer token is not permitted to use this API.

HTTP Code 400: Input document is not formatted as JSON or is not a properly formatted GIngest Document

HTTP Code 409: Conflict if the index is Locked for writing.

The body will contain a GError document

1 The token must have the urn:globus:scopes:search.api.globus.org:all or urn:globus:scopes:search.api.globus.org:ingest scope, and must belong to a user with admin or privileged_user permissions against <index_name>

GIngest Document & Subdocuments

The body of an Ingest API call is a GIngest document. This is a simple container for GMetaList and GMetaEntry documents.

Some of the most important details about how to correctly format an Ingest API call are contained in the documentation for those documents, particularly GMetaEntry.

Please read those docs (and the examples there) in addition to the examples below.

Examples

cURL Example 1

Ingesting a single entry

  • in the index 4de0e89e-a395-11e7-bc54-8c705ad34f60

  • with a subject of https://example.com/foo/bar

  • with a null entry_id

  • public visibility

curl -XPOST 'https://search.api.globus.org/v1/index/4de0e89e-a395-11e7-bc54-8c705ad34f60/ingest' \ # <1>
    --data '{
      "ingest_type": "GMetaEntry", # <2>
      "ingest_data": {
        "subject": "https://example.com/foo/bar",
        "visible_to": ["public"],
        "content": { # <3>
          "foo/bar": "some val"
        }
      }
    }'
  1. The Index ID is provided in the URL

  2. The datatype of the ingest_data document in GMetaEntry for a single entry

  3. content is an arbitrary JSON body

cURL Example 2

Ingesting a list of entries

  • in the index 4de0e89e-a395-11e7-bc54-8c705ad34f60

  • with subject values of https://example.com/foo/bar and https://example.com/foo/bar/baz

  • with entry_id values of null, "alpha", and "beta"

  • public visibility and visibility only to the user globus@globus.org

    • The ID of globus@globus.org is 46bd0f56-e24f-11e5-a510-131bef46955c, so this is the value which will be used

curl -XPOST 'https://search.api.globus.org/v1/index/4de0e89e-a395-11e7-bc54-8c705ad34f60/ingest' \
    --data '{
      "ingest_type": "GMetaList", # <1>
      "ingest_data": {
        "gmeta": [ # <2>
          { # <3>
            "subject": "https://example.com/foo/bar",
            "visible_to": ["public"],
            "content": {
              "foo/bar": "some val"
          },
          {
            "subject": "https://example.com/foo/bar",
            "id": "alpha",
            "visible_to": [
              "urn:globus:auth:identity:46bd0f56-e24f-11e5-a510-131bef46955c" # <4>
            ],
            "content": {
              "foo/bar": "some otherval"
            }
          },
          {
            "subject": "https://example.com/foo/bar/baz",
            "id": "alpha",
            "visible_to": [
              "urn:globus:auth:identity:46bd0f56-e24f-11e5-a510-131bef46955c"
            ],
            "content": {
              "foo/bar/baz": "some val"
            }
          },
          {
            "subject": "https://example.com/foo/bar/baz",
            "id": "beta",
            "visible_to": ["public"],
            "content": {
              "foo/bar/baz": "some otherval"
            }
          }
        ]
      }
    }'
  1. This time, the ingest_data is of type GMetaList

  2. GMetaList.gmeta is an array of GMetaEntry documents

  3. This entry does not specify an id, so its entry_id is null

  4. This notation is a Principal URN

cURL Example 3

Ingesting a list of entries

  • in the index 4de0e89e-a395-11e7-bc54-8c705ad34f60

  • with subject values of https://example.com/foo/

  • with entry_id values of "alpha", and "beta"

  • public visibility and visibility only to the Group with ID 0a4dea26-44cd-11e8-847f-0e6e723ad808

curl -XPOST 'https://search.api.globus.org/v1/index/4de0e89e-a395-11e7-bc54-8c705ad34f60/ingest' \
    --data '{
      "ingest_type": "GMetaList",
      "ingest_data": {
        "gmeta": [
          {
            "subject": "https://example.com/foo",
            "id": "alpha",
            "visible_to": [
              "urn:globus:group:id:0a4dea26-44cd-11e8-847f-0e6e723ad808" # <1>
            ],
            "content": {
              "foo/bar": "some val"
            }
          },
          {
            "subject": "https://example.com/foo",
            "id": "beta",
            "visible_to": ["public"],
            "content": {
              "foo/bar/baz": "some otherval"
            }
          }
        ]
      }
    }'
  1. This notation is a Principal URN for a Group


© 2010- The University of Chicago Legal