Delete by Query

Delete by query provides a powerful method for removing a large number of documents in a single operation.

The operation removes an entire subject where there is a match on the query. That is, if even a single entry for a subject matches the query, then all entries, as well as the subject itself, will be removed from the index.

Warning

Delete by query should be used with care, as it may not always be obvious which documents it will delete.

This is similar to the result of performing a query: the set of subjects deleted will exactly match the set of subjects returned by the query used for delete by query.

You may want to test delete by query operations by first executing the query as a search.

Due to the broad capability of delete by query to change the state of the index, it can only be executed by a user with 'owner' or 'admin' permissions on the index.

Delete By Query is submitted as an asynchronous Task, which can then be monitored using the Get Task API. Once your task is complete, the data will be removed from the search index and will no longer appear in query results.

Method	POST
URL	/v1/index/<index_id>/delete_by_query
Authentication required?	Yes
Required Roles	You must have `owner`, `admin`, or `writer` access
Request Body	A GSearchRequest See note below.
Response Body	A DeleteByQueryResponse

Note

Delete by query uses the same input structure as a normal query, GSearchRequest, but only makes use of the q, advanced_query, and filters fields.

All other fields, including those related to pagination, do not change the full set of subjects which match a query, and are therefore ignored by the delete by query operation.

This allows the same GSearchRequest to be used with delete by query as may have been used for a testing or regular query request without need for alteration.

Authentication & Authorization

Tokens for this call must have one of these scopes.

urn:globus:scopes:search.api.globus.org:all
urn:globus:scopes:search.api.globus.org:ingest

Request Schemas

GSearchRequest

This is the main document type for encoding a complex Search query.

Field Name	Type	Description
q	String	User-supplied query, conforming to the query syntax. Required if there are no filters.
advanced	Boolean	Defaults to False When true, interpret q with the advanced query syntax
filters	Array	Optional. An array of GFilter Documents. Filters to apply to the search

Field Name

Type

Description

String

User-supplied query, conforming to the query syntax. Required if there are no filters.

advanced

Boolean

Defaults to False

When true, interpret q with the advanced query syntax

filters

Array

Optional. An array of GFilter Documents. Filters to apply to the search

Examples

{
  "q": "the quick brown fox jumps"
}

{
  "q": "a search with filtering",
  "filters": [
    {
      "type": "range",
      "field_name": "path.to.date",
      "values": [
        {
          "from": "*",
          "to": "2014-11-07"
        }
      ]
    }
  ]
}

{
  "q": "(queries can be fancy AND cool) OR (NOT extravagant)",
  "advanced": true
}

GFilter

A GFilter document is one of several document types which encode a filter. The type of filter is identified by the type field. See the table below for the various filter types.

Type	Schema
`match_all`	GFilterMatch
`match_any`	GFilterMatch
`range`	GFilterRange
`geo_bounding_box`	GFilterGeoBoundingBox
`exists`	GFilterExists
`not`	GFilterNot

GFilterMatch

A matching filter for finding results which match some set of text terms.

"match_any" and "match_all" refer to the different possible behaviors of the filter values. As their names imply, if "match_any" is specified, the filter will match results for which any of filter values match, while "match_all" requires that all of the values match on every result.

Field Name Type Description

Field Name	Type	Description
type	String	One of `{"match_any", "match_all"}`
field_name	String	The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.
values	Array of Strings or Booleans	The values to evaluate against the field_name. If the field is a boolean field, this must be an array of booleans only. For string fields, it may be a mixture of strings or booleans.

type

String

One of {"match_any", "match_all"}

field_name

String

The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.

values

Array of Strings or Booleans

The values to evaluate against the field_name.

If the field is a boolean field, this must be an array of booleans only. For string fields, it may be a mixture of strings or booleans.

Note

"match_any" and "match_all" are the same when there’s only one value as far as filtering is concerned, but they may have different impact on the way that facets are interpreted.

Examples

{
  "type": "match_any",
  "field_name": "globus_metadata.resource_type",
  "values": [
    "Globus Endpoint"
  ]
}

{
  "type": "match_all",
  "field_name": "globus_metadata.keywords",
  "values": [
    "hpc",
    "internet2",
    "uchicago"
  ]
}

{
  "type": "match_any",
  "field_name": "globus_metadata.snorkels",
  "values": [
    "few",
    "many",
    true
  ]
}

Note

This filter is only valid if globus_metadata.snorkels is a string field because string fields can contain boolean values.

If it is a boolean field (which cannot contain string values), the query will fail with an error regarding the improper mapping of "few" and "many" onto a boolean field.

GFilterRange

A range filter for finding results which have numeric or date values within a specified range.

Field Name Type Description

Field Name	Type	Description
type	String	Must have the value `"range"`
field_name	String	The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.
values	Array of Objects	The values to evaluate against the field_name. This must be an Array of Objects each with the fields from and to.

type

String

Must have the value "range"

field_name

String

The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.

values

Array of Objects

The values to evaluate against the field_name.

This must be an Array of Objects each with the fields from and to.

Note

values.from and values.to may be the special string "*" indicating that the range is unbounded on this end. An example is given below.

Examples

{
  "type": "range",
  "field_name": "path.to.date",
  "values": [
    {
      "from": "1970-01-01",
      "to": "2015-01-01"
    }
  ]
}

{
  "type": "range",
  "field_name": "cardinality_of_foobar",
  "values": [
    {
      "from": "10",
      "to": "50"
    }
  ]
}

This example filter has multiple clauses. The combination is implicitly joined with "or" semantics. This means that we allow values from 0 to 5, and greater than or equal to 10.

{
  "type": "range",
  "field_name": "cardinality_of_foobar",
  "values": [
    {
      "from": "0",
      "to": "5"
    },
    {
      "from": "10",
      "to": "*"
    }
  ]
}

GFilterGeoBoundingBox

A bounding box filter for finding geo_shape and geo_point values which intersect with a specified bounding box.

Field Name	Type	Description
type	String	Must have the value `"geo_bounding_box"`
field_name	String	The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.
top_left	Object	An object describing a coordinate pair. It must contain the keys `lat` and `lon`, each of which must have a numeric value.
bottom_right	Object	An object describing a coordinate pair. It must contain the keys `lat` and `lon`, each of which must have a numeric value.

Note

top_left is required to be northwest of bottom_right.

Examples

{
  "type": "geo_bounding_box",
  "field_name": "country.center",
  "top_left": {
    "lat": 49.1,
    "lon": -124.9
  },
  "bottom_right": {
    "lat": 24.9,
    "lon": -67.1
  }
}

GFilterExists

An "existence" filter which checks if a field is present in a document with a non-null value. Note that a field being present but with a value of null is considered the same, under exists filters, as the field being absent from the document.

Field Name	Type	Description
type	String	Must have the value `"exists"`
field_name	String	The field to which the filter refers. Any dots (".") must be escaped with a preceding backslash ("\") character.

Examples

The following filter finds documents where the field foo exists:

{
  "type": "exists",
  "field_name": "foo"
}

GFilterNot

A "not" filter for inverting any other valid filter.

Field Name	Type	Description
type	String	Must have the value `"not"`
filter	Object	Any valid GFilter object, except for another "not" filter.

Examples

The following filter finds documents where the field foo does not exist:

{
  "type": "not",
  "filter": {
    "type": "exists",
    "field_name": "foo"
  }
}

Response Schemas

DeleteByQueryResponse

Field Name	Type	Description
task_id	UUID	the ID of the submitted task
num_subjects_deleted	Integer	This is a legacy field kept for compatibility. It will always be `0`.

Example

{
  "task_id": "450538fb-cf9c-48fc-bd6f-08abc5e86da9",
  "num_subjects_deleted": 0
}