API Change History

Public release notes for all versions of Globus Search.

1.37.4 (2025-05-22)

  • Internal robustness enhancements

1.37.3 (2025-05-08)

  • Internal robustness enhancements

1.37.2 (2025-05-07)

  • Internal robustness enhancements

1.37.1 (2025-05-02)

  • Globus Search now behaves better when it is unable to serve a query request.

    In particular, users whose queries time out should no longer see very slow hanging requests.

1.37.0 (2025-04-30)

  • The Index List API now accepts a filter_roles query parameter which takes a comma-delimited list of role names, allowing users to request a subset of indices on which they have specific roles.

1.36.0 (2025-04-09)

  • Internal robustness enhancements

1.35.0 (2025-04-02)

  • Added a new field, q_settings to all v1 POST-style queries, including query#1.0.0, scroll#1.0.0, and delete_by_query#1.0.0.

    q_settings expects a dictionary as input with the keys mode and default_operator.

1.34.1 (2025-02-26)

  • Internal robustness enhancements

1.34.0 (2025-02-17)

  • Internal robustness enhancements

1.33.0 (2025-01-31)

  • Internal robustness enhancements

1.32.3 (2025-01-17)

  • Internal robustness enhancements

1.32.2 (2025-01-10)

  • Internal robustness enhancements

1.32.1 (2024-12-09)

  • Internal robustness enhancements

1.32.0 (2024-12-06)

  • Internal robustness and performance enhancements

1.31.0 (2024-11-19)

Enhancements

  • Internal robustness enhancements

1.30.0 (2024-11-13)

Enhancements

  • Add a new delete_by_query#1.0.0 document format for Delete By Query requests.

    The new format does not allow the use of fields, such as sort, which were previously allowed by the API but which had no effect on the deletion.

    Delete By Query requests that do not specify that the @version is delete_by_query#1.0.0 will continue to function without modification.

1.29.0 (2024-11-06)

Enhancements

  • When getting information about a specific index, the response document now includes the permissions of the principal that made the request.

1.28.1 (2024-10-23)

Enhancements

  • Queries with version query#1.0.0 are now rejected if they define any facets with identical names.

  • Queries with version query#1.0.0 now support facet-specific additional_filters fields.

    • An aggregate limit of 96 has been applied to all filters, post_facet_filters, and additional_filters in a single query.

  • Added a new scroll document version, scroll#1.0.0, which handles filters akin to the new query#1.0.0 definition.

1.27.0 (2024-10-07)

Enhancements

  • Internal robustness enhancements

1.26.0 (2024-09-27)

Bugfixes

  • Improved the behavior of authentication failures due to upstream errors.

Enhancements

  • Added a new query document version, query#1.0.0, which:

    • Replaces the keyword post_filter with post_facet_filters.

    • Removes the keywords @datatype and @version from sort, boost, facet, and filter type sub-documents.

    • Removes support for "auto-listifying nested fields" in sort, boost, facet, and filter sub-documents. Nested fields must always be supplied as a list, never singleton objects.

  • Globus Search now supports a new filter type, GFilterLike, which implements basic wildcard query semantics. A "like" filter takes the form {"type": "like", "field_name": "my_field", "value": "filter-expression"}.

    • Filter expressions may contain *, matching any number of characters, and ?, matching any one character.

    • The fields specified by "like" filters must be text fields.

  • Internal robustness and performance enhancements

1.25.0 (2024-08-28)

Enhancements

  • Added two new query filter types: and and or to support composing and unioning query criteria together. For more information on how to use these filters, see GFilterAnd and GFilterOr.

  • Added four new keywords to range filters: gte, lte, gt, lt to allow for more precise range filtering. For more information on how to use these keywords, see POST Queries and Filters.

Changes

  • Removed the deprecated result_format_version parameter from the POST Query API.

1.24.0 (2024-08-21)

Changes

  • Removed the deprecated result_format_version parameter from the following three APIs: GET Query, Get By Entry ID, Get By Subject.

1.23.0 (2024-07-24)

Enhancements

  • Internal enhancements

1.22.0 (2024-07-10)

Enhancements

  • Added BurstQueryCapacity rate limiting.

    • Search queries which take longer than 2 seconds to execute are now tracked in a 30-second expiring rate limit window (associated with the caller).

    • Any caller accumulating more than 90 of these "BurstSeconds" (query execution seconds beyond the 2-second allotment) will now receive a 429 on subsequent post-search, get-search, and scroll operations until their 30-second rate limit window expires.

1.21.0 (2024-05-31)

Enhancements

  • Globus Search now supports a new type of filter for searching geospatial datatypes. A geo_shape filter tests the relationship between data in the index and a provided GeoJSON Polygon.

    • The new filter can be requested using "type": "geo_shape" in a POST query filter.

    • geo_shape filters must contain a shape field containing a GeoJSON Polygon. Other geometries may be added in the future.

    • geo_shape filters support a field, relation, which determines which relationship between shapes is being tested. relation can be one of intersects or within and defaults to intersects.

    • intersects filters can be used on geo_shape and geo_point data, but within filters can only be used on geo_shape data.

    • GeoJSON Polygons are defined by RFC 7946, the GeoJSON format specification.

  • Globus Search now supports post_filter as a parameter on filters, controlling whether they are applied before or after facets are calculated. The default behavior for post_filter, when omitted, matches Globus Search’s previous behavior for facet and filter interactions.

  • Internal robustness and performance enhancements

1.20.0 (2023-10-18)

Enhancements

  • A new filter type has been added, "type": "exists". These filters take no additional parameters beyond the type and the field name, and match documents where a desired field is present with a non-null value.

  • Add support for filters of type "not", which negate their contained filter. A "Not Filter" contains another filter inside the "filter" field and inverts the meaning of that filter. For example, a filter of the form

    {
  "type": "not",
  "filter": {
    "type": "match_all",
    "field_name": "foo",
    "values": [
      "bar"
    ]
  }
}

    will match any document in which the field foo does not contain the string bar. For more information on how to use "not", see the reference documentation on POST Queries and Filters.

  • Internal robustness improvements

1.19.1 (2023-08-02)

Bugfixes

  • Fixed an issue in which specifying a negative boost factor on a query would result in opaque errors. These are now caught and rejected with a more specific error message.

1.19.0 (2023-06-28)

Enhancements

  • Globus Search will no longer represent boolean values internally as strings; they will be represented as booleans. This results in an unavoidable backwards-incompatible API change.

    • This fixes a bug in which ingesting a document with the strings "True" or "False" would return documents on query containing boolean true or false.

    • For existing indices with mappings already defined, documents will continue to ingest successfully, as boolean values are allowed for string fields.

    • For any existing documents prior to this change containing the boolean values true or false, Globus Search responses will now contain "True" and "False". This reflects the data transformation which happened in the past and affected existing documents, but which is no longer re-transformed on output.

    • In certain cases, some sequences of documents which would have previously ingested successfully will now fail. In particular, it used to be valid to first index a boolean value (true or false) for a field and later store string values in that field. This is no longer allowed, as the boolean type of the field is now correctly assigned and enforced.

    • For assistance updating existing indices to translate data, or to help assess the impact of this change on applications, please contact support@globus.org

  • The match_any and match_all filter clauses now accept boolean values. In JSON, these are true and false. These will only match on boolean field types, not on data which was translated to "True" or "False" in a past version of Globus Search.

    • For users wishing to filter on boolean fields which predate the v1.19.0 release, a filter of the following form can be used to match booleans:

      {
  "field_name": "foo",
  "type": "match_any",
  "values": [
    "True",
    true
  ]
}

    • For users wishing to filter on boolean fields created after the v1.19.0 release, match_any and match_all filters cannot take string values. Note that this only holds for newly defined fields. For any such new fields, the match_any and match_all filters should only contain the true boolean values true and false, as in:

      {
  "field_name": "foo",
  "type": "match_any",
  "values": [
    true,
    false
  ]
}

  • Index management APIs are now available under the /v1/ namespace:

    • Index Create: POST /v1/index

    • Index Delete: DELETE /v1/index/<index_id>

    • Index Reopen: POST /v1/index/<index_id>/reopen

The beta versions of the index management APIs are still available, but officially deprecated. They are aliases of the /v1/ routes.

1.18.10 (2023-04-27)

Enhancements

  • Internal robustness enhancements

1.18.9 (2023-04-19)

Enhancements

  • Added support for batch document deletion via new batch-delete-by-subject route

  • A task requesting the deletion of a non-existent document is now marked as "SUCCESS"

1.18.8 (2023-03-30)

Bugfixes

  • Fix task cleanup; tasks older than 30 days will now be deleted as expected

Enhancements

  • Include the Search version in a version field for GET /

  • When insufficient scopes are used for an API request, the code field is now set to Forbidden.InsufficientScopes so that error summary info will more easily include the cause

  • Improve the preparation of pending tasks, increasing performance

1.18.7

Enhancements

  • Internal robustness enhancements

1.18.6 (2023-02-15)

Changes

  • Added the field task_type to response objects in the GetTask and TaskList APIs

Bugfixes

  • Fixed error messaging on incorrectly typed fields whose name contains a .

1.18.5 (2022-12-15)

Changes

  • Increase query execution timeout from 10 to 30 seconds

1.18.4 (2022-10-26)

Enhancements

  • Support for all_authenticated_users as a document visibility value

1.18.3

Enhancements

  • Internal robustness enhancements

1.18.2

Enhancements

  • Internal robustness enhancements

1.18.1

Enhancements

  • Internal robustness enhancements

1.18.0

Changes

  • Range filters are now always inclusive on their upper bounds. They previously would be exclusive unless the upper and lower bounds matched exactly, which resulted in surprising semantics when date ranges were used.

Bugfixes

  • Get by Entry ID now correctly handles the case of the null Entry ID. It previously returned all entries instead of the entry with a null ID.

  • Invalid facet names containing '[', ']', '>' are now detected and rejected with a specific error.

Enhancements

  • Delete by Subject is now allowed for users with the writer role on an index. This previously required admin or owner

1.17.2

Bugfixes

  • A malformed Authorization header will always be treated as a 401 Unauthorized error

Enhancements

  • Internal robustness improvements

1.17.1

Enhancements

  • Internal robustness improvements

1.17.0

Bugfixes

  • Queries with unbounded or large numbers of buckets in aggregations may now trigger a 400-class error (a bad query error) instead of a 500. These are triggered by users sending a type of query which exceeds internal limits

  • The role_id shown in a NoSuchRole error is now rendered correctly

Enhancements

  • Beta support for geo_bounding_box filters and geo_point and geo_shape datatypes. These are now documented in our openapi and redoc documentation but are not yet meant for general use

  • Improvements to robustness to reduce the frequency of failed queries

  • Ingest errors may now contain a field in additional_details named caused_by which shows a detailed message about the cause of the failure

1.16.1

Enhancements

  • Internal robustness improvements

1.16.0

Bugfixes

  • Some request failures due to internal timeouts have been fixed

Enhancements

  • A new (beta) API for index deletion is now available to index owners and admins, DELETE /beta/index/<index_id>

    • A deleted index enters a delete-pending state prior to deletion. For trial indices, deletion will take place within a few minutes. For non-trial indices, the index will be unusable but will only be deleted after 30 days

    • Owners of a non-trial index in the delete-pending state can use the reopen API, POST /beta/index/<index_id>/reopen to change the index status back to open from delete-pending

    • When an index is marked delete-pending, PENDING tasks on that index are immediately stopped and set to a state of CANCELLED. Tasks which are already in PROGRESS are allowed to complete

  • A new field is added to Index List and Index Show APIs, status. status has a value of either "open" or "delete-pending", depending on the index state

1.15.4

Changes

  • @context expansion, which is no longer used by any indices, has been removed

Bugfixes

  • Fixed an issue in which rate limiting could rarely cause requests to fail incorrectly

1.15.3

Changes

  • The limit on trial indices per user has been raised from 1 to 3

1.15.2

Changes

  • The @context expansion feature on Ingest is now considered deprecated and will be disabled on all new indices

    • When context-expansion is disabled, the content field is left unmodified prior to data being indexed, and @context may be a valid field name

Bugfixes

  • Delete By Query operations with filter clauses were not handled correctly, resulting in failing tasks

Enhancements

  • The is_trial and subscription_id attributes are now included in the Index Show and Index List APIs

1.15.1

Bugfixes

  • Fix the enforcement of scope requirements on Index Create (POST /beta/index)

1.15.0

Enhancements

  • A new (beta) index creation API is now available to all Globus users

    • The user who creates an index will be given an owner role on it

    • Users are still only allowed to be owner on one trial index. In effect, this means that each user can only create one index

    • Available at POST /beta/index, the call accepts a display_name and description

    • Indices created in this way are marked as trials and may be automatically deleted after a period of time. Contact support@globus.org to associate a trial index with your subscription and convert it to non-trial status

  • Globus Search has a new rate limit of 5 requests/second

  • Internal performance improvements

1.14.1

Enhancements

  • Role documents now include the creation date of the role

1.14.0

Changes

  • There is now a limit on the number of roles per index. No more than 50 roles may be created.

  • The subject field is now limited to 500 bytes (not characters)

Bugfixes

  • Allow the use of a bare wildcard for an advanced query string, or as the right-hand side of a field expression. These expressions were removed in 1.13.0 and are being reintroduced here.

    • may only appear on its own, either as the full query or the right-hand-side of a field, as in or foo:*

Enhancements

  • A new role, owner, has been added to Search with all of the permissions of admin

    • Multiple users may have owner on an index. There is no limit on the number of owners other than the limit on total number of roles.

  • A new role deletion API is now available for index admins and owners for removing permissions for users (DELETE /v1/index/<index_id>/role/<role_id>)

    • Roles may be deleted by role ID, retrieved via role listing

    • The last owner role on an index may not be deleted. Attempting to remove it will result in an error

    • admin users can otherwise delete owner roles and owner users can delete admin roles

    • An admin or owner user is permitted to remove their own permissions on an index

  • A new role creation API is now available for index admins and owners for adding permissions for users (POST /v1/index/<index_id>/role)

    • Roles may be assigned to any valid principal URN, meaning any identity or group

    • Only users with owner permissions can create owner roles

    • Duplicate roles are not allowed

    • A trial index may only have one owner assigned. owner role creation is not allowed on these

  • Role responses from creation and role listing now include the role ID (for use in role delete)

  • Internal enhancements and robustness improvements

1.13.0

Changes

  • Remove support for API operations by index_name (deprecated behavior). All APIs now require the use of the index_id instead

  • Remove support for "query templates" (unused feature)

  • The request payload limit has been changed to 10MB

Bugfixes

  • A bad Authorization header will now always be rejected with an error, even when making an API call which does not require authentication.

Enhancements

  • Major improvements to advanced query parsing. New errors should help indicate which part of a large query was malformed.

    • The slash (/) character no longer needs to be escaped in advanced queries. This means that https\://foo:bar is a valid search, equivalent to what previously had to be expressed as https\:\/\/foo:bar

  • Add new /v1/index/<index_id>/role_list API accessible to index admins

    • Index writers cannot view the role list

    • Like index_list, this API requires the urn:globus:auth:scope:search.api.globus.org:all scope

  • Add a new /v1/index_list API which lists indices on which you have writer or admin permissions

    • Calls to /v1/index_list require the urn:globus:auth:scope:search.api.globus.org:all scope

  • The set of date formats accepted and autodetected are now stricter to avoid incorrect date detection. See the doc on datatype mapping for details

  • Numerous internal enhancements!

1.12.1

Enhancements

  • Internal stability and performance improvements

1.12.0

Enhancements

  • Autogenerated documentation for Globus Search is now available. This documentation is subject to change as the API changes.

  • A new API is added alongside /search called /scroll

    • Scrolling is not limited to 10,000 results, and can therefore be used for deep pagination

    • Scrolling does not support sorting, faceting, or boosting

    • This API uses marker-style pagination in which an opaque marker is returned on each request and can be used to request the next page of results

  • Sorted queries with will now always return results with a stable ordering, even when the sort criteria do not guarantee a stable sort

1.11.4

Enhancements

  • Improved validation for facets and filters in a number of cases. Facets of one type (e.g. terms) may not contain fields specific to another facet type (e.g. the histogram field for numeric histograms)

  • Improve performance on very large query results

1.11.3

Enhancements

  • Internal improvements to task processing speeds

1.11.2

Bugfixes

  • Bad authentication data on queries now correctly produces a 401 Unauthenticated response

Enhancements

  • New role based filtering of results using the principal_sets field

    • Entry documents may now include a mapping, principal_sets, which lists collections of principal URNs

    • You can filter queries with a new parameter, filter_principal_sets, which matches your identities and groups against the principal_sets of various entries

    • This mechanism can be used for cases like role-based filtering, e.g. listing admins of an object and asking for filter_principal_sets=admins

  • bypass_visible_to is now supported on Get Entry and Get Subject

  • Type mismatches on ingest now produce more detailed error messages

    • If the type of an ingested field does not match the data in the index, Search will now attempt to produce a more useful error message in the failed task document

    • New data should be found in the additional_details field of a failing task

Changes

  • The internal /unstable/ API has been renamed to /beta/

  • Internal stability enhancements

1.11.1

Bugfixes

  • Deleting the last entry of a subject now correctly deletes the subject rather than leaving the subject in place but empty

1.11.0

Enhancements

  • Internal performance, robustness, and security improvements

  • Convert synchronous write operations into asynchronous tasks

    • delete-by-subject, delete-by-entry-id, and delete-by-query operations now all run asynchronously as tasks, which can be monitored just like ingest tasks

    • These operations have had their responses changed to return task_id. The standard successful response for a newly submitted task is {"acknowledged": true, "task_id": …​}

    • The delete-by-subject and delete-by-entry-id results are encoded simply in the status of their relevant tasks as SUCCESS or FAILED

    • The delete-by-query result, which shows the number of documents deleted, is now exposed through the additional_details field of the resulting task document on SUCCESS

    • delete-by-subject and delete-by-entry-id return the field {"removed": true} in all cases for backwards compatibility. This field is not meaningful and will be removed in a future release

    • delete-by-query returns the field {"num_subjects_deleted": 0} in all cases for backwards compatibility. This field is not meaningful and will be removed in a future release

  • Task records now expire after 90 days — attempting to lookup a task more than 90 days after its completion date will return a 404

1.10.0

Bugfixes

  • Fix an issue in which date histogram facets returned an extra, empty facet named "meta"

Enhancements

  • Become strict on inputs and error on unknown fields and unknown parameters

    • This applies to query parameters and POST/PUT method bodies

    • Unexpected fields and unexpected parameters will cause validation errors, with an HTTP status of 400

  • This provides a safer development environment in which typos cannot lead to parameters being unintentionally ignored. For example, /search?q=foo&limitt=1 will cause an error, rather than simply ignoring the malformed parameter, limitt

    • Added support for sum and avg metric facets.

  • These are requested by setting the facet’s type to sum or avg, and return numeric results for value

  • For sum and avg facets, the additional field missing is supported, to define a value to be used for documents without the relevant field name. By default, missing values are ignored and do not count towards sum or avg

1.9.1

Bugfixes:

  • Fix the enforcement of visible_to policies on facets when data is partially visible to the user

    • If a single subject contained at least two entries, one visible to the user and one non-visible, facets returned results for the non-visible entry

    • Subjects which do not contain entries visible to a user were not visible by this method

    • The facet did not need to be on a field in the visible entry, so this method could have been used to discover the presence non-visible field names via a guess-and-check approach

1.9.0

Bugfixes

  • Queries using filters without a query string will no longer produce errors

Enhancements

  • Improvements to input validation, ensuring that various inputs produce more consistent and better error messages

    • As a consequence of this change, some error messages have changed in format slightly

    • The following values for the code field have been removed: BadRequest.DocumentValidationError, BadRequest.UnsupportedDocumentVersion, BadRequest.UnexpectedDatatype, BadRequest.IncompleteDocument. Instead, any errors related to input validation will use BadRequest.ValidationError

    • No HTTP status codes have changed. Validation errors are still HTTP 400s

  • Search will no longer accept connections using old SSL versions. TLSv1.2 or later is required, as a security measure

  • Internal security and stability improvements

1.8.2

Enhancements

  • Internal stability improvements

1.8.1

Enhancements

  • Internal security improvements

1.8.0

Bugfixes

  • Requests which have both invalid parameters and improper Authorization headers will more consistently get Unauthorized errors (HTTP 401), not Bad Request errors (HTTP 400)

  • Empty GMetaList ingest documents are now rejected with a validation error, including a message describing the cause

Enhancements

  • Backwards Incompatible: The default result_format_version has been changed in all contexts to 2019-08-27

  • Search results now include a has_next_page field for easier pagination

1.7.2

Bugfixes

  • Using non-RFC UUID variants in <index_id> fields will now be rejected with validation errors (HTTP 400)

1.7.1

Bugfixes

  • Remove behavior which drops null and other nullary values. These will now propagate through correctly so that documents can contain null.

1.7.0

Enhancements

  • Performance optimizations for repeated searches in a short window

  • Admins of an index may now bypass visible_to restrictions by setting bypass_visible_to=True on their Searches

    • In this mode, index admins may see all documents in an index, regardless of visible_to restrictions

    • pass bypass_visible_to=True in a GET search or set "bypass_visible_to": true in a POST search

    • this capability is not available on non-query APIs (e.g. get-by-subject)

  • Queries may now set result_format_version=2019-08-27 in order to get results in a new format

    • This parameter changes the format of GMetaResult documents. Details on the two format versions are provided in the GMetaResult documentation

    • The result_format_version parameter is also available on Get-By-Subject and Get-By-EntryID operations

    • In a future release, this behavior will become the default, but you will be able to ensure you continue to get results in the current format if you start passing result_format_version=2017-09-01

    • In the long term, the 2017-09-01 result format will be deprecated and removed

  • Remove deprecated resource_type parameter

1.6.0

Enhancements

  • Significant improvements to internal handling of Ingest Tasks

    • Overall Task throughput has been improved

    • Tasks will no longer take a long time to fail when they mismatch the underlying schema

    • Task processing order has been improved to treat Indices more fairly

1.5.0

Bugfixes

  • Ingesting fields over 32 KB in size will be rejected with an HTTP 400, rather than queuing as tasks which will ultimately fail

Enhancements

  • New JSONSchema validation can be set (by a service administrator) on an index

    • Schema validation is applied during ingest request processing and accepts or rejects documents at that time

    • Documents passing schema validation may later fail due to type mismatches if the detected underlying field type does not match the schema specification

1.4.3

Enhancements

  • Ingest document sizes over 10MB are now caught as errors immediately and trigger and HTTP 400 response instead of queuing as tasks which will ultimately fail

1.4.2

Bugfixes

  • Administrative index updates are more robust

1.4.1

Bugfixes

  • null in a match_any or match_all filter causes a validation error (HTTP 400)

Enhancements

  • Accelerate handling of ingest payloads smaller than 5MB

1.4.0

Enhancements

  • GMetaResult documents include a new field, entry_ids, containing the array of entry IDs for the elements of the content array

    • For any index i, the value at entry_ids[i] is the entry ID for content[i]

    • This "unzipped" representation of entry_ids was chosen to avoid making backwards-incompatible changes in the response format.

1.3.3

Bugfixes

  • Sending unicode in a subject for subject operations no longer errors

1.3.2

Bugfixes

  • Sorting on nonexistent fields gives accurate error

  • Fix errors on certain malformed range queries

1.3.1

Bugfixes

  • Fix @context expansion on ingest tasks to restore correct behavior

1.3.0

Bugfixes

  • Timeout errors were incorrectly classified as 502. This is corrected and they are now returned as HTTP 504 errors

  • Certain range filters were mishandled and resulted in errors

  • Histogram bucketing facets on text-type fields now produces a better error

Enhancements

  • Major documentation refresh

  • Add the new Task API which supports getting a task and a list of tasks

    • /v1/task/<task_id> gets a current Task document. To make this call, you must have permissions to ingest data into the index which is referred to by the task

    • /v1/task_list/<index_id> gets a list of the 1000 most recent Tasks which were started against an index

  • The Ingest API no longer attempts to synchronously write data to an index, but returns a task_id handle for the asynchronous background task

    • Clients which depended on the num_documents_ingested field on ingest will now find that it always has a value of 0

    • A successful return from an ingest API call no longer guarantees that data was written to the index successfully — callers must now check the status of the generated Task using the new Task API

1.2.6

Bugfixes

  • NaN, Infinity, and -Infinity are now rejected with an informative 400-class error, as they cannot be stored in Globus Search as Float values

Enhancements

  • Error documents now include the request_id for better troubleshooting. Please include the request_id when reporting unexpected errors

1.2.5

Enhancements

  • Internal stability improvements

1.2.4

Enhancements

  • Internal stability improvements

1.2.3

Bugfixes

  • Numeric histograms with bad ranges correctly result in 400 errors

  • Handle malformed dates which failed parsing

1.2.0

Bugfixes

  • Fix Group membership checks to include full identity set

  • Unauthed use of restricted APIs no longer produces (incorrect) 403s, but 401s instead

Enhancements

  • Index Locking. When data is being reindexed by Search (e.g. because of a requested update to schema information), the index will now enter a Locked state.

    • Previously, writes were allowed but had a risk of being discarded by the reindex operation.

    • While an index is Locked, attempts to write to it will return 409 Conflict Errors. Any impacted operations have had their documentation pages updated.

  • Unicode-aware Sorting. Sorts on text fields containing non-ascii characters should now better reflect typical character orderings.

    • This supports all unicode text fields.

    • There are known cases where language-agnostic unicode sorting differs from typical phonebook ordering for a specific language. These are not bugs in Search.

1.1.1

Bugfixes

  • In certain situations, Index IDs were not being treated equivalently to Index Names for permissions checks. Using Index IDs should now work correctly for APIs which incorrectly returned 403 errors.

1.1.0

Enhancements

  • Add support for Delete-by-Query operations

    • Delete-by-Query requires admin privileges

    • This operation can delete documents which are not visible to the caller, so it should be used with great care. Please read the documentation carefully before using

  • All indices are now reachable using their Index ID value in place of the <index_name> — this is now the preferred method of reaching an index

    • All operations will continue to work normally with the current index_name values

    • Simply GET /v1/index/<index_name> and look at the resulting Index ID value to get the ID of an index for which you know the name. At the bottom of these notes, we provide an example of doing this

    • In a future version, we plan to remove the uniqueness constraint on index_name and make it an editable display name field for indices. At that time it will cease to be possible to resolve an index_name to an index_id value

  • A new namespace has been added to Globus Search under /unstable/ (as opposed to /v1/). Such APIs will not be included in our documentation or release notes, and may change without warning. They are for internal use only.

1.0.2

Enhancements

  • Improvements to search processing speed

1.0.1

Bugfixes

  • Fix the advanced query parsing’s handling of spaces in parenthesized expressions. The bugfix in v1.0.0 introduced failures on expressions like foo ( bar) and (foo ) bar. These should now parse correctly

1.0.0

Bugfixes

  • Fix the advanced query parsing’s handling of operators applied to parenthesized expressions. Expressions like foo (bar baz)^2 should now parse correctly

0.4.1

Bugfixes

  • Fix the handling of operators inside of field values in advanced query parsing. Expressions like foo:(bar && baz) should now parse correctly

Enhancements

  • Enforce size limits on indices — each index is now limited to a total size

    • If your quota is too low, please contact us about raising it

    • All current indices should be well within their quotas

    • If you fill your quota, you may notice that your usage may mildly exceed your stated quota. This is not a bug. Search can’t accurately evaluate the storage cost of documents until they are stored, so quota enforcement is done by checking if you have exceeded your quota.

  • Add an API endpoint for getting data about indices, including their current quota and usage

    • This is a simple GET call which returns a GSearchIndex document

0.4.0

Breaking Changes

  • GMetaEntry.visible_to is now required to only contain "public" or Principal URNs

    • Other values will cause 400 BadRequest errors due to a malformed document

  • Users are no longer prevented from writing/deleting one another’s GMetaEntry documents

    • If two users, Bob and Alice, both ingest GMetaEntry documents with the same subject and id values (missing or null values included), the last write will win! Only one document will be created. Previously, one document would be created for each user

  • GMetaEntry.id values may no longer contain the character !. We have plans to give this character special meaning in the future, and are considering it a reserved character in this specific context henceforth

0.3.0

Breaking Changes

  • No longer works with datasearch-client versions 0.1.0 and lower. Please ensure that if you are using the search-client, you are using v0.2.0 of that library.

    • The search-client was renamed in v0.2.0 from datasearch-client, which is why both names are written here.

Note
The versions of search-client do not, and are not meant to, match the versions of the Search API. Any similarities in version numbers are purely incidental.

  • Drop support for calls to /v1/<index_name>/*. Calls to index-oriented APIs (Query, Ingest, Entry Operations) must always be written as /v1/index/<index_name>/*. This was the preferred convention referred to in v0.2.0

  • Support for the urn:globus:auth:scope:datasearch.api.globus.org:all scope, deprecated in v0.2.0, is being dropped.

Enhancements

  • Sorting and Faceting on long field contents now works

    • We have repaired a bug regarding support for long field values. This applies to the contents of single fields, not the depth or length of their field names. i.e. In the document {"a": "b"}, this refers to "b".

    • Values longer than 10000 characters will not be included in Facet results. They are also considered "last" by Sorting, regardless of their "true" sort ordering.

  • New Preferred Syntax for visible_to. Values in visible_to other than "public" should now be qualified in the Principal URN format.

    • Existing visible_to entries have been upgraded, so you do not need to reindex your data.

  • Begin renaming resource_type to query_template. We feel this name more accurately represents the meaning/usage of this parameter. The current changes are carefully kept backwards compatible, but we recommend transitioning all usage to the newer forms.

    • Add new parameter, query_template which is a rename of resource_type to GET /v1/<index_name>/search. We recommend starting to use query_template for GET-style searches, as resource_type will be dropped in an upcoming release

    • Add a new @version for GSearchRequest documents, 2017-09-01. We intend to drop support for the 2016-11-09 versioned documents in a future release

    • GSearchRequest version 2017-09-01 takes query_template as an attribute, in lieu of resource_type

    • Unversioned GSearchRequest documents currently are treated as version 2016-11-09. We will change this in a future release

  • Add a new APIs for fetching Query Templates These are unauthenticated; query template contents are considered publicly readable.

    • GET /index/<index_name>/query_template/<template_name> can be used to lookup an existing query template.

    • GET /index/<index_name>/query_template can be used to get a list of query templates associated with an index.

  • Numerous enhancements to error output for debugging purposes

  • Internal robustness and stability improvements

0.2.0

Breaking Changes

  • Drop support for calls to /v1/search and /v1/ingest. Calls now must always be qualified with a relevant index name. The preferred new form is /v1/index/<index_name>/search and /v1/index/<index_name>/ingest.

  • A number of conditions which caused malformed advanced-mode queries to trigger 500-class errors now produce (more correct) 400-class errors

  • Advanced mode queries now require that the following characters be escaped with a preceding \ character in fieldnames: .:+-()*\/ Additionally, spaces must be backslash-escaped

  • Any documents specifying a fieldname for use must escape any dots contained in the fieldname. No other special characters should be escaped, ONLY .. This applies to GBoost, GFacet, GFilter, GSort

Enhancements

  • Advanced mode queries support the use of . as a path separator. Given the document

{
  "a": {
    "b": 1
  },
  "a.b": 2
}

we take a.b to refer to the value 1, and a\.b to refer to the value 2.

  • Documents specifying fieldnames can use . as a path separator, with the same meaning as in advanced queries

  • Add sorting functionality to structured queries, using the new GSort document type within the GSearchRequest document.

  • Add new Entry Operations APIs and Subject Operations APIs

    • Includes support for deletion of metadata, requiring only that you are allowed to index data — no new permissions control these operations yet

  • Fieldnames are no longer required to be URIs. Any JSON document is now a valid GMetaContent document

  • Add new documentation pages, which we will be updating regularly

  • Add support for new scope names, urn:globus:auth:scope:search.api.globus.org:all, urn:globus:auth:scope:search.api.globus.org:search, urn:globus:auth:scope:search.api.globus.org:ingest

    • The …​:search scope can only be used for Searches — not for Ingesting data or the new entry and subject operations

    • The …​:ingest scope can only be used for ingesting and the new entry and subject operations, not for Searches

    • urn:globus:auth:scope:datasearch.api.globus.org:all is still supported, but considered deprecated. It will be supported at least until version 0.3.0, but you should begin moving off of it

  • Internal stability and security fixes

  • Internal changes and bugfixes to data storage layer

  • Simplify several error messages and make them clearer