How to format a date to insert in Elasticsearch

I’m coding a Java app to insert data in Elasticsearch 7.5.1. When creating the index the property was set like this:

"datetime":{
   "type":"date"            
}

Now when inserting the date I’m getting this error:

org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [datetime] of type [date] in document with id '195'. Preview of field's value: '2018-11-23 10:38:00'

I’m currently doing it like this:

DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
String v = dateFormat.format(date);

And checking a working index I can see it’s formatted like this, example: 2019-10-09T11:11:38.879-04:00

What is the mask to create that format?

Answer

According to the docs, you can specify multiple date formats for your field “datetime”.

The datetime from the error message above, 2018-11-23 10:38:00, needs to be mapped with yyyy-MM-dd HH:mm:ss in order to get indexed.

Please consider setting a timezone or even better use ISO 8601 format as elastic internally stores all dates as UTC. If you don’t provide a Timezone, the datetime you are indexing will be assumed as UTC but your application may run in a different Timezone. This can cause trouble 😉

Back to the solution. Define the mapping like this for the datetime field:

PUT my_index
{
  "mappings": {
    "properties": {
      "datetime": {
        "type":   "date",
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd'T'HH:mm:ss.SSSZ||epoch_millis"
      }
    }
  }
}

And it will process datetime values like

• 2018-11-23 10:38:00 (your current format)

• 2001-07-04T12:08:56.235-0700 (ISO 8601)

• 1095379198 (unix time)

If you want more background information on date & time handling in java, have a look at this. Old, but gold.

Cheers!