How get date from elasticsearch with one format? - date

Question

I have create an index like that:
PUT twitter
PUT twitter/_mapping/myType
{
"myType" : {
"properties" : {
"message" : {"type" : "date",
"date_detection": true,
"store" : true }
}
}
}
Then I put several documents:
POST twitter/myType
{
"message":123456
}
I have this document and other with message values: "123456",-123456,"2014-01-01","-123456" (Note string and numeric difference here). Only document with value "12#3454" failed to put.
So now I execute:
GET twitter/myType/_search?pretty=true&q=*:*
And results are:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 1,
"hits": [
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5JvFsHvhUOO_5MdfCv",
"_score": 1,
"_source": {
"message": -123456
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5Ju6aOvhUOO_5MdfCs",
"_score": 1,
"_source": {
"message": "123456"
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5Ju0KOvhUOO_5MdfCq",
"_score": 1,
"_source": {
"message": "2014-01-01"
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5JvDiGvhUOO_5MdfCu",
"_score": 1,
"_source": {
"message": "-123456"
}
}
]
}
}
Why I get these value in date fields instead of string value - ISODateTimeFormat.dateOptionalTimeParser? Is there a way to get all date with one format (e.g. string or millis)?
Elasticsearch version is 1.4.3

Answer 1

That's the _source you are seeing, meaning the exact JSON you indexed, no formatting, nothing.
If you want to see what actually ES indexed (meaning the date in milliseconds), you can use fielddata_fields:
GET /twitter/myType/_search
{
"query": {
"match_all": {}
},
"fielddata_fields": [
"message"
]
}
And the answer to your question is that is not actually available out-of-the-box. You need to use script_fields:
GET /twitter/myType/_search
{
"query": {
"match_all": {}
},
"fielddata_fields": [
"message"
],
"_source": "*",
"script_fields": {
"my_script": {
"script": "new Date(doc[\"message\"].value)"
}
}
}
Also, your mapping is wrong: date_detection should be put in the type not in the field:
PUT twitter
{
"mappings": {
"myType": {
"date_detection": true,
"properties": {
"message": {
"type": "date",
"store": true
}
}
}
}
}
And from the output below you'll see how ES treats those numbers you put in there:
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5J93Q-I7tQJ10g6jk5",
"_score": 1,
"_source": {
"message": "123456"
},
"fields": {
"message": [
3833727840000000
],
"my_script": [
"123456-01-01T00:00:00.000Z"
]
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5J93Q-I7tQJ10g6jk4",
"_score": 1,
"_source": {
"message": 123456
},
"fields": {
"message": [
123456
],
"my_script": [
"1970-01-01T00:02:03.456Z"
]
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5J93Q-I7tQJ10g6jk8",
"_score": 1,
"_source": {
"message": "-123456"
},
"fields": {
"message": [
-3958062278400000
],
"my_script": [
"-123456-01-01T00:00:00.000Z"
]
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5J93Q-I7tQJ10g6jk7",
"_score": 1,
"_source": {
"message": "2014-01-01"
},
"fields": {
"message": [
1388534400000
],
"my_script": [
"2014-01-01T00:00:00.000Z"
]
}
},
{
"_index": "twitter",
"_type": "myType",
"_id": "AU5J93Q-I7tQJ10g6jk6",
"_score": 1,
"_source": {
"message": -123456
},
"fields": {
"message": [
-123456
],
"my_script": [
"1969-12-31T23:57:56.544Z"
]
}
}
]