Search API
  • Globus Search
  • Overview
  • API Usage & Basics
  • Ingest
  • Query
  • Types, Type Detection, and Schemas
  • Error Handling
  • API Reference
    • Batch Delete by Subject
    • Create or Update Entry
    • Delete by Query
    • Delete by Subject
    • Entry Delete
    • Entry Show
    • Index Create
    • Index Delete
    • Index List
    • Index Reopen
    • Index Show
    • Index Update
    • Ingest
    • Query - GET
    • Query - POST
    • Role Create
    • Role Delete
    • Role List
    • Scroll Query
    • Subject Show
    • Task List
    • Task Show
  • Guides
    • Geospatial Search
    • Role Based Filtering
    • Searchable Files
  • Globus Search Limits
  • API Change History
Skip to main content
Globus Docs
  • Getting Started
    Getting Started

    Getting Started and Tutorial docs cover how to perform some activity or provide an introduction to a feature. They are not comprehensive, but help you get started with Globus or with new Globus features.

    • Users
    • Admins
    • Developers
  • Reference
    Reference
    • Service
      • Auth
      • Groups
      • Transfer
      • Timers
      • Flows
      • Compute
      • Search
    • Agents
      • Globus Connect Server
      • GCS CLI
      • Globus Connect Personal
      • Globus Compute
    • SDK
      • Python
      • JS
    • Clients
      • CLI
    • Security and Compliance
      • Product Security
      • Privacy
      • Solutions for Sensitive Data
      • FAQs
  • Solutions & Guides
    Solutions & Guides

    Find practical approaches for leveraging Globus in research environments, integrating with platforms, and building science gateways. Access hands-on guides, integration instructions, and real-world scenarios for advanced usage.

    • Portals/Science Gateways
    • Guides
  • Support
    Support

    Find answers to frequently asked questions, connect with the community by joining our mailing lists, or reach out directly to Globus support.

    • FAQs
    • Mailing Lists
    • Contact Us
    • Check Support Tickets
  • Site Search
  1. Home
  2. Globus Services
  3. Globus Search
  4. Types, Type Detection, and Schemas

Types, Type Detection, and Schemas

Table of Contents
  • 1. Type Detection on Ingest
    • 1.1. Automatic Widening Conversions
  • 2. Date Formatted Strings
  • 3. Changing the Type of Indexed Data

Globus Search uses dynamic schemas for your data in an index.

Whenever a new field is seen in a document, it is added to the schema for the index. However, that does not mean that Globus Search does not have any schemas for data at all. Rather, it maps input data to types, and then, once those types have been detected, it enforces them strictly.

1. Type Detection on Ingest

When a document is added to an index via the Ingest API, any fields which do not exist in the index have their types detected and added to the index.

For example, if "foo": 1 is added to the index, then "foo" is identified as being an integer. Attempting to store the data "foo": "bar" in the index at a later date — even in a separate document — will be rejected because "bar" is not numeric data.

Type detection understands the following types and maps them as follows:

Input Type Example Values Detected Type

Boolean

true, false

Boolean

Float

2.5, 1.0

Float

Integer

1, 1024

Integer

Date String

"2020-03-16"

Date

String

"foo", "some text"

String

1.1. Automatic Widening Conversions

In the above example, "foo": 1 causes a later entry of "foo": "bar" to be rejected. However, if the data is indexed in the opposite order, the results will be different.

First, "foo": "bar" would be detected as a string. However, "foo": 1 can be put through a widening conversion to "foo": "1" and stored, and Search will do this automatically.

Similarly, integers can be converted to floats and booleans to strings.

Search will not automatically perform any lossy conversions, such as truncating floats to integers.

2. Date Formatted Strings

Globus Search understands a subset of ISO 8601 timestamps as dates, and additionally implements the common variants of a space (rather than T) as the separator.

The following string formats are valid dates:

yyyy-MM-dd
yyyy-MM-dd HH:mm:ss
yyyy/MM/dd HH:mm:ss Z
yyyy-MM-dd'T'HH:mm:ssZ
yyyy-MM-dd'T'HH:mm:ss.SSSZ

So, for example, each of the following would parse as a valid date:

2020-07-01
1970-01-01 00:00:00
1986-08-11T22:01:15+00:00

3. Changing the Type of Indexed Data

Changing the type of a field requires a full reindex of all data in the index. It therefore requires intervention of the Globus Search team.

If you have indexed data in Globus Search and the type is incorrect, please file a support ticket with support@globus.org.

  • Globus Search
  • Overview
  • API Usage & Basics
  • Ingest
  • Query
  • Types, Type Detection, and Schemas
  • Error Handling
  • API Reference
    • Batch Delete by Subject
    • Create or Update Entry
    • Delete by Query
    • Delete by Subject
    • Entry Delete
    • Entry Show
    • Index Create
    • Index Delete
    • Index List
    • Index Reopen
    • Index Show
    • Index Update
    • Ingest
    • Query - GET
    • Query - POST
    • Role Create
    • Role Delete
    • Role List
    • Scroll Query
    • Subject Show
    • Task List
    • Task Show
  • Guides
    • Geospatial Search
    • Role Based Filtering
    • Searchable Files
  • Globus Search Limits
  • API Change History
© 2010- The University of Chicago Legal Privacy Accessibility