I'm implement query in Elastic4s library. But I don't know how to implement a following Json query for Elasticsearch.
{
"bool": {
"must": [
{
"match_all": {}
},
{
"keywordQuery": "hogehoge"
}
]
}
}
I don't know how to implement this part of Json query.
{
"keywordQuery": "hogehoge"
}
This is a code I implemented halfway.
boolQuery().must(Seq(matchAllQuery(), query("{keywordQuery: hogehoge}")))
and this is an output of an above code.
{
"bool": {
"must": [
{
"match_all": {}
},
{ "queryString": {
"query": "{keywordQuery": "hogehoge}"
}
}
]
}
}
I expect
{
"keywordQuery": "hogehoge"
}
but actually
{ "queryString": {
"query": "{keywordQuery": "hogehoge}"
}
}
Would you help me please?
I can't find a reference to keywordQuery in the ElasticSearch DSL documentation at https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl.html or https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl.html - maybe you need a Term query?
(on e.g. Logstash indices 'text' fields have a non-analysed subfield called '.keyword' so if I do a "keyword query" I normally do termQuery("field.keyword","value))
I don't think you need to include matchAllQuery() as it's kinda implied that you start off with the full set of results, so you could drop the bool and simplify the query to:
{
"query": {
"term": {
"field.keyword": "value"
}
}
}
In Elastic4s this would be:
client.execute {
termQuery("field.keyword", "value")
}
Related
My document is the following
{
"name":"Name1",
"status":"active",
"points":[
{
"lag":"final"
},
{
"lag":"final"
}
]
},
{
"name":"Name2",
"status":"active",
"points":[
{
"lag":"final"
},
{
"lag":""
}
]
}
I need to get all the documents that have some value in the lag field, for this example should get two document,
I tried with this query, but it only returns me when all points have full lag
{ "points.lag":{$not:{ $eq:"" }},status:{$in:['active']}}
Play
You need to use elemMatch to check whether atleast one element matches the condition.
db.collection.find({
"points": {
"$elemMatch": {
"lag": {
$ne: null
}
}
}
})
Let's say I have a Mongo database that contains objects such as :
[
{
"test": {
"123123": {
"someField": null
}
}
},
{
"test": {
"323143": {
"someField": "lalala"
},
"121434": {
"someField": null
}
}
},
{
"test": {
"4238023": {
"someField": "afafa"
}
}
},
]
As you can see, the keys right under "test" can vary.
I want to find all documents that have at least one someField that is not null.
Something like find : "test.*.someField": { $ne: null } ( * represents any value here)
How can i do this in mongoose ? I'm thinking an aggregation pipeline will be needed here but not exactly sure how.
Constraints :
I don't have much control over the db schema in this scenario.
Ideally i don't want to have to do this logic in nodeJS, I would like to query directly via the db.
The trickiest part here is that you cannot search keys that match a pattern. Luckily there is a workaround. Yes, you do need an aggregation pipeline.
Let's look at an individual document:
{
"test": {
"4238023": {
"someField": "afafa"
}
}
}
We need to query someField, but to get to it, we need to somehow circumvent 4238023 because it varies with each document. What if we could break that test object down and look at it presented like so:
{
"k": "4238023",
"v": {
"someField": "afafa"
}
}
Suddenly, it get a heck of a lot easier to query it. Well, mongodb aggreation offers a function called $objectToArray which does exactly that.
So what we are going to do is:
Convert the test object into an array for each document.
Match only documents where AT LEAST ONE v.someField is not null.
Put it back together to look as your original documents, minus the ones that do not match the null criterion.
So, here is the pipeline you need:
db.collection.aggregate([
{
"$project": {
"arr": {
"$objectToArray": "$$ROOT.test"
}
}
},
{
"$match": {
arr: {
$elemMatch: {
"v.someField": {
$ne: null
}
}
}
}
},
{
"$project": {
"_id": 1,
"test": {
$arrayToObject: "$arr"
}
}
}
])
Playground: https://mongoplayground.net/p/b_VNuOLgUb2
Note that in mongoose you will run this aggregation the same way you would do it in a terminal... well plus the .then.
YourCollection.aggregate([
...
...
])
.then(result => console.log(result))
I have expiration_date date field in my Elasticsearch and request from user to be able to "full-text" search in this field and I have only one input for this.
So my initial mapping is:
PUT my_index
{
"mappings": {
"_doc": {
"properties": {
"expiration_date": {
"type": "date"
}
}
}
}
}
and as a value I have for example: 2021-08-27T10:48:00.293Z.
User would like be able to search this by 2021, 2021-08, 2021-08-27, 27-08-2021 and 08-2021. For all this search terms I have only one input field which is used to search in other fields as well (fields like title, description etc.).
My idea to achieve this was to introduce some multi-fields to the base field. So something like:
PUT my_index
{
"mappings": {
"_doc": {
"properties": {
"expiration_date": {
"type": "date",
"fields": {
"yyyy-mm-dd" : {
//what to do here?
},
"yyyy-mm" : {
//what to do here?
},
"yyyy" : {
//what to do here?
},
"mm-yyyy" : {
//what to do here?
},
"dd-mm-yyyy" : {
//what to do here?
}
}
}
}
}
}
}
But I'm wondering if this is doable this way? Is something similar doable in any way only at the Elasticsearch side? Or I should rather prepare something similar on my application side, send it to ES and just use it there?
Probably, the best solution would be to use the custom formats for date field in Elasticsearch:
PUT my_index
{
"mappings": {
"_doc": {
"properties": {
"expiration_date": {
"type": "date",
"format": "year||year_month||year_month_day||dd-MM-yyyy||MM-yyyy||strict_date_optional_time||epoch_millis"
}
}
}
}
}
Then you can range query the field:
{
"query": {
"bool": {
"must": [{
"range": {
"expiration_date": {
"gte": "27-01-2001"
}
}
}]
}
}
}
Where you can use any format specified in mapping.
This solution would be the most scalable: you can just add more formats (available here or you can construct one) and reindex the data to support any new formats.
We're using elastic4s for ElasticSearch 2.2.0. A number of queries is stored as JSON on disk and used as rawQuery via the elastic4s driver. The score in the result differs between the query being submitted via command line or the elastic4s driver. The elastic4s driver always returns score of 1 for all results, while the command line execution yields two different scores (for different data types).
The code for elastic4s:
val searchResult = client.execute {
search in indexName types(product, company, orga, "User", "Workplace") rawQuery preparedQuery sourceInclude(preparedSourceField:_*) sort {sortDefintions:_*} start start limit limit
}.await
Note that I removed anything but rawQuery preparedQuery and it didn't change the score 1. The full query via the command line is quite long:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "${search}",
"fields": [
"name",
"abbreviation",
"articleNumberManufacturer",
"productLine",
"productTitle^10",
"productSubtitle",
"productDescription",
"manufacturerRef.name",
"props"
]
}
}
],
"filter": [
{
"or": [
{
"bool": {
"must": [
{
"type": {
"value": "Product"
}
},
{
"term": {
"publishState": "published"
}
}
],
"must_not": [
{
"term": {
"productType": "MASTER"
}
},
{
"term": {
"deleted": true
}
}
]
}
}
]
}
]
}
}
}
Note that this is almost preparedQuery but for the replacement of $search with the search query. The elastic search REST client returns a score of 3.075806 for the matches.
elastic4s rawQuery will wrap your rawQuery-JSON in another query object.
it's like you would query for
{ "query": { "query": {
"bool": {
"must": [
{
"multi_match": {
"query": "${search}",
...
just remove your wrapping "query" from you JSON and the response will show varying scores.
Alternatively you can try to use extraSource instead of rawQuery, like described in elastic4s docu. although it didn't work for me at all:
ErrorMessage:
value extraSource is not a member of com.sksamuel.elastic4s.SearchDefinition
I am using laravel 4.2, mongodb and elasticsearch. Below is a working code, I am trying to convert this advanced where queries to elasticsearch queries:
$products = Product::where(function ($query) {
$query->where (function($subquery1){
$subquery1->where('status', '=', 'discontinued')->where('inventory', '>', 0);
});
$query->orWhere (function($subquery2){
$subquery2->where('status', '<>', 'discontinued');
});
})->get();
All I can get so far is just returning discontinued products, the code below works but it is not what I need:
$must = [
['bool' =>
['should' =>
['term' =>
['status' => 'discontinued']
]
]
]
];
Can you show me how can I achieve the same query I first described above but in elasticsearch? I want to return discontinued products with inventory, then also return products that are not equal to discontinued.
The WHERE query you've described can be expressed in SQL like this
... WHERE (status = discontinued AND inventory > 0)
OR status <> discontinued
In Elasticsearch Query DSL, this can be expressed like this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"status": "discontinued"
}
},
{
"range": {
"inventory": {
"gt": 0
}
}
}
]
}
},
{
"bool": {
"must_not": [
{
"term": {
"status": "discontinued"
}
}
]
}
}
]
}
}
}
}
}
Translating this query into PHP should now be straightforward. Give it a try.