I have a document in MongoDB 3.2 with the following structure:
"_id" : ObjectId("5759815b94db5928bea3c3a5"),
"source" : "pons1",
"libraries" : [
{
"archive" : “deko1”,
"last_access" : ISODate("2016-06-09T14:45:04.644+0000"),
"books" : [
{
"title": "American Gods",
"author": "Neil Gaiman"
},
{
"title": "A Little Life",
"author": "Hanya Yanagihara"
}
]
},
{
"archive" : “deko90”,
"last_access" : ISODate("2016-06-10T12:45:03.624+0000"),
"books" : [
{
"title": "Sociology of News",
"author": "Michael Schudson"
},
{
"title": "City of God",
"author": "Augustine of Hippo"
}
]
}
]
There is an array (“books”) inside of another array (“libraries”).
Since the book titles are indexed as "text," I want to be able to conduct a free text search, and return only the relevant array elements.
For instance, if I search for the term “Gods,” I would like to see the following result:
"_id" : ObjectId("5759815b94db5928bea3c3a5"),
"source" : "pons1",
"libraries" : [
{
"archive" : “deko1”,
"last_access" : ISODate("2016-06-09T14:45:04.644+0000"),
"books" : [
{
"title": "American Gods",
"author": "Neil Gaiman"
}
]
},
{
"archive" : “deko90”,
"last_access" : ISODate("2016-06-10T12:45:03.624+0000"),
"books" : [
{
"title": "City of God",
"author": "Augustine of Hippo"
}
]
}
]
In MongoDB 3.2, you can filter the elements of an array using “$filter” (https://docs.mongodb.com/manual/reference/operator/aggregation/filter/).
The problem is that you cannot use text search ($text) as a condition for $filter.
$text can only be used in the first stage of the aggregation pipeline ($match) (https://docs.mongodb.com/manual/tutorial/text-search-in-aggregation/).
There is one obvious workaround: give up the power of MongoDB’s text search and maybe work with regex.
That does not seem a good option for me. I’d rather not lose the diacritic insensitivity and other interesting functionalities of MongoDB’s text search.
Is there a way of reconciling $filter and $text in the same MongoDB query?
Related
I have documents like this:
{
"_id" : ObjectId("5d302a6da9b1d6ae926c16a3"),
"location" : "car",
"playlist" : [
{
"playlist_number" : "1",
"number_of_songs" : "100",
"drive" : [
{
"drive_number" : "13",
"drive_name" : "Ooms",
"songs" : [
"59 Volvo",
"Yield Not To Temptation - Single Version",
"Johnny Angel",
"The Madison Time",
"Little Brown Jug",
"Only In My Dreams"
]
},
{
"drive_number" : "Z3",
"drive_name" : "codes",
"songs" : [
"59 Volvo",
"Perfect",
"Hands clap",
"I Ran",
"Falling down",
"Sounds of middle earth"
]
}
]
}
]
}
There can be many other location as well that may contain variable number of playlist objects (which in turn can contain n number of playlist_number, you get the idea).
When I search for a song by name for eg: 59 Volvo I should get back location, playlist_number, drive_number and drive_name. I tried with nested $elemMatch but it gives the whole document.
I have a large DB with various inconsistencies. One of the items I would like to clear up is changing the country status based on the population.
A Sample of the data is:
{ "_id" : "D", "name" : "Deutschland", "pop" : 70000000, "country" : "Large Western" }
{ "_id" : "E", "name" : "Eire", "pop" : 4500000, "country" : "Small Western" }
{ "_id" : "G", "name" : "Greenland", "pop" : 30000, "country" : "Dependency" }
{ "_id" : "M", "name" : "Mauritius", "pop" : 1200000, "country" : "Small island"}
{ "_id" : "L", "name" : "Luxembourg", "pop" : 500000, "country" : "Small Principality" }
Obviously I would like to change the country field go something more uniform, based on population size.
I've tried this approach, but obviously missing some way of tying into an update of the country field.
db.country.updateMany( { case : { $lt : ["$pop" : 20000000] }, then : "Small country" }, { case : { $gte : ["$pop" : 20000000] }, then : "Large country" }
Edit: Posted before I was finished writing.
I was thinking to use $cond functionality, to basically return if true, do X, if false, do y, while using the updateMany.
Is this possible, or is there a workaround?
You really want want bulkWrite() using two "updateMany" statements within it instead. Aggregation expressions cannot be used to do "alternate selection" in any form of update statement.
db.country.bulkWrite([
{ "updateMany": {
"filter": { "pop": { "$lt": 20000000 } },
"update": { "$set": { "country": "Small Country" } }
}},
{ "updateMany": {
"filter": { "pop": { "$gt": 20000000 } },
"update": { "$set": { "country": "Large Country" } }
}}
])
There is still an outstanding "feature request" on SERVER-6566 for "conditional syntax", but this is not yet resolved. The "bulk" API was actually introduced after this request was raised, and really can be adapted as shown to do more or less the same thing.
Also using $out in an aggregation statement as was otherwise suggested is not an option to "update" and can only write to a "new collection" at present. The slated change from MongoDB 4.2 onwards would allow $out to actually "update" an existing collection, however this would only be where the collection to be updated is different from any other collection used within the gathering of data from the aggregation pipeline. So it is not possible to use an aggregation pipeline to update the same collection as what you are reading from.
In short, use bulkWrite().
I am using an aggregation pipeline with the MongoDB Java driver version 3.6. If I have documents that look something like:
doc1 --
{
"CAR": {
"VIN": "ASDF1234",
"YEAR": "2018",
"MAKE": "Honda",
"MODEL": "Accord"
},
"FEATURES": [
{
"AUDIO": "MP3",
"TIRES": "All Season",
"BRAKES": "ABS"
}
]
}
doc2 --
{
"CAR": {
"VIN": "ASDF1234",
"AVAILABILITY": "In Stock"
}
}
And if I submit a query like:
collection.aggregate(
Arrays.asList(
Aggregates.match(
and(
in("CAR.VIN", vinList),
or(
eq("CAR.MAKE", carMake),
eq("CAR.AVAILABILITY", carAvailability),
)
)
)
)
)
Let us assume that there are exactly two different records for which the "CAR.VIN" criteria match for every VIN, and I am going to get two results. Rather than deal with two results each time, I would like to merge the documents so that the result looks like this:
{
"CAR": {
"VIN": "ASDF1234",
"YEAR": "2018",
"MAKE": "Honda",
"MODEL": "Accord",
"AVAILABILITY": "In Stock"
},
"FEATURES": [
{
"AUDIO": "MP3",
"TIRES": "All Season",
"BRAKES": "ABS"
}
]
}
The example where I have two and only two results trivializes my need for this. Imagine that vinList is a list of 10000 values, and it might return 2 x 10000 documents. When I return an AggregateIterable to the client that is calling my code, I do not want to impose the requirement that they have to group or collate the results in any way, but that they will receive one document for each result that has all of the information that they will want to parse, cleanly and easily.
Of course, people will suggest that the data is simply combined into one document with all of the data in the MongoDB collection. For reasons that I cannot control, there are two separate documents corresponding to each VIN in the same collection, and that is something that I am unable to change. There is a value in our system that makes this more reasonable than it might seem, so please don't focus on this apparent problem with the data.
I am trying, with not much luck, to utilize the Aggretes.group() operation to merge the fields in my aggregation pipeline. Accumulators.push seems to be the closest operation to what I need, but I do not want to complicate the document structure with extra arrays, etc. Is there a straightforward approach that I am not seeing?
you can try $mergeObjects added in mongo v3.6
db.cc.aggregate(
[
{
$group: {
_id : "$CAR.VIN",
CAR : {$mergeObjects : "$CAR"},
FEATURES : {$mergeObjects : {$arrayElemAt : ["$FEATURES", 0 ]}}
}
}
]
).pretty()
result
{
"_id" : "ASDF1234",
"CAR" : {
"VIN" : "ASDF1234",
"YEAR" : "2018",
"MAKE" : "Honda",
"MODEL" : "Accord",
"AVAILABILITY" : "In Stock"
},
"FEATURES" : {
"AUDIO" : "MP3",
"TIRES" : "All Season",
"BRAKES" : "ABS"
}
}
>
to get features as array
db.cc.aggregate(
[
{
$group: {
_id : "$CAR.VIN",
CAR : {$mergeObjects : "$CAR"},
FEATURES : {$push : {$arrayElemAt : ["$FEATURES", 0 ]}}
}
}
]
).pretty()
result
{
"_id" : "ASDF1234",
"CAR" : {
"VIN" : "ASDF1234",
"YEAR" : "2018",
"MAKE" : "Honda",
"MODEL" : "Accord",
"AVAILABILITY" : "In Stock"
},
"FEATURES" : [
{
"AUDIO" : "MP3",
"TIRES" : "All Season",
"BRAKES" : "ABS"
},
null
]
}
>
I have my document structure as below:
{
"codeId" : 8.7628945723895E13, // long numeric value stored in scientific notation by Mongodb
"problemName" : "Hardware Problem",
"problemErrorCode" : "97695686856",
"status" : "active",
"problemDescription" : "ghdsojgnhsdjgh sdojghsdjoghdghd i0dhgjodshgddsgsdsdfghsdfg",
"subProblems" : [
{
"codeId" : 8.76289457238896E14,
"problemName" : "Some problem",
"problemErrorCode" : "57790389503490249640",
"problemDescription" : "This is edited",
"status" : "active",
"_id" : ObjectId("589476eeae39b20b1c15535b")
},
...
]
}
I have a search field which should search by codeId which basically serves as parentCodeID in search fields as shown below
Now, along with parentIdCode I want to search for codeId, problemCode, problemName and problemDescription as well.
How do I query the submodules with a regex search and at same time tag some parent field with "$or" clause etc. to achieve this ?
You can try something like this.
query = {
'$or': [{
"codeId":somevalue
}, {
"subProblems.codeId": {
"$regex": searchValue,
"$options": "i"
}
}, {
//rest of sub modules fields
}]
};
I have a document structure like follows:
{
"_id": ...,
"name": "Document name",
"properties": {
"prop1": "something",
"2ndprop": "other_prop",
"other3": ["tag1", "tag2"],
}
}
I can't know the actual field names in properties subdocument (they are given by the application user), so I can't create indexes like properties.prop1. Neither can I know the structure of the field values, they can be single value, embedded document or array.
Is there any practical way to do performant queries to the collection with this kind of schema design?
One option that came to my mind is to add a new field to the document, index it and set used field names per document into this field.
{
"_id": ...,
"name": "Document name",
"properties": {
"prop1": "something",
"2ndprop": "other_prop",
"other3": ["tag1", "tag2"],
},
"property_fields": ["prop1", "2ndprop", "other3"]
}
Now I could first run query against property_fields field and after that let MongoDB scan through the found documents to see whether properties.prop1 contains the required value. This is definitely slower, but could be viable.
One way of dealing with this is to use schema like below.
{
"name" : "Document name",
"properties" : [
{
"k" : "prop1",
"v" : "something"
},
{
"k" : "2ndprop",
"v" : "other_prop"
},
{
"k" : "other3",
"v" : "tag1"
},
{
"k" : "other3",
"v" : "tag2"
}
]
}
Then you can index "properties.k" and "properties.v" for example like this:
db.foo.ensureIndex({"properties.k": 1, "properties.v": 1})