Elasticsearch date format parsing error - date

Question

I am trying to specify a date format for an Elasticsearch field according to ISO 8601 as:
YYYY-MM-DDThh:mm:ssTZD
I put the field mapping like so:
"properties": {
"startDate": {
"type": "date",
"format": "YYYY-MM-DD'T'hh:mm:ss'TZD'"
}
}
When I try to insert a document with this field's value as: "startDate": "2018-01-10T07:07:07+01:00", I get this error:
"type": "mapper_parsing_exception",
"reason": "failed to parse [afield.startDate]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Invalid format: \"2018-01-10T07:07:07+01:00\" is malformed at \"+01:00\""
}
Is there something wrong in the date I am inserting? I'm following the example given in this W3C guide (https://www.w3.org/TR/NOTE-datetime):
Complete date plus hours, minutes and seconds:
YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00)

Answer 1

Elasticsearch datetime formats can be found here: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html
You want a datetime with no milliseconds. To do that, use the following:
PUT my_index
{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"type1" : {
"properties": {
"startDate": {
"type": "basic_date_time_no_millis",
"format": "yyyyMMdd'T'HHmmssZ"
}
}
}
}
}
Timezone should be handled in the timestamp, and can be pulled from a query like:
GET /my_index/_search
{
"query": {
"range" : {
"timestamp" : {
"gte": "2018-01-10 07:07:07",
"lte": "now",
"time_zone": "+01:00"
}
}
}
}

Answer 2

For custom date formats in Elasticsearch you have to check the Joda-time documentation.
In your case you can try with yyyy-MM-dd'T'HH:mm:ssZ:
"properties": {
"startDate": {
"type": "date",
"format": "yyyy-MM-dd'T'HH:mm:ssZ"
}
}
With this you should be able to insert dates and make searches using the dates like the one you used in your example: 2018-01-10T07:07:07+01:00.

Answer 3

For all the people out there that have a date in the format :
2016-08-14T00:00:00+02:00
or :
yyyy-MM-ddTHH:mm:ss+HH:MM
You might consider using this format in elasticsearch :
format: "yyyy-MM-dd'T'HH:mm:ss+02:00"