Get items of array by index in MongoDB - mongodb

So I have a data structure in a Mongo collection (v. 4.0.18) that looks something like this…
{
"_id": ObjectId("242kl4j2lk23423"),
"name": "Doug",
"kids": [
{
"name": "Alice",
"age": 15,
},
{
"name": "James",
"age": 13,
},
{
"name": "Michael",
"age": 10,
},
{
"name": "Sharon",
"age": 8,
}
]
}
In Mongo, how would I get back a projection of this object with only the first two kids? I want the output to look like this:
{
"_id": ObjectId("242kl4j2lk23423"),
"name": "Doug",
"kids": [
{
"name": "Alice",
"age": 15,
},
{
"name": "James",
"age": 13,
}
]
}
It seems like I should easily be able to get them by index, but I'm not seeing anything in the docs about how to do that. The real-world problem I'm trying to solve has nothing to do with kids, and the array could be quite lengthy. I'm trying to break it up and process it in batches without having to load the whole thing into memory in my application.
EDIT (non-sequential indexes):
I noticed that since I asked about item 1 & 2 that $slice would suffice…however, what if I wanted items 1 & 3? Is there a way I can specify specific array indexes to return?
Any ideas or pointers for how to accomplish that?
Thanks!

You are looking for the $slice projection operator if the desired selection are near each other.
https://docs.mongodb.com/manual/reference/operator/projection/slice/
This would return the first 2
client.db.collection.find({"name":"Doug"}, { "kids": { "$slice": 2 } })
returns
{'_id': ObjectId('5f85f682a45e15af3a907f51'), 'name': 'Doug', 'kids': [{'name': 'Alice', 'age': 15}, {'name': 'James', 'age': 13}]}
this would skip the first kid and return the next two (second and third)
client.db.collection.find({"name":"Doug"}, { "kids": { "$slice": [1, 2] } })
returns
{'_id': ObjectId('5f85f682a45e15af3a907f51'), 'name': 'Doug', 'kids': [{'name': 'James', 'age': 13}, {'name': 'Michael', 'age': 10}]}
Edit:
Arbitrary selections 1 and 3 probably need to route through an aggregation pipeline rather than a simple query. The performance shouldn't be too much different assuming you have an index on the $match field.
Steps of your pipeline should be pretty obvious and you should be able to take it from here.
Hate to point to RTFM, but that's going to be super helpful here to at least be acquainted with the pipeline operations.
https://docs.mongodb.com/manual/reference/operator/aggregation/
Your pipeline should:
$match on your desired query
$set some new field kid_selection to element 1 (second element) and element 3 (4th element) since counting starts at 0. Notice the prefixed $ on the "kids" key name in the kid_selection setter. When referencing a key in the document you're working on, you need to prefix with $
project the whole document, minus the original kids field that we've selected from
client.db.collection.aggregate([
{"$match":{"name":"Doug"}},
{"$set": {"kid_selection": [
{ "$arrayElemAt": [ "$kids", 1 ] },
{ "$arrayElemAt": [ "$kids", 3 ] }
]}},
{ "$project": { "kids": 0 } }
])
returns
{
'_id': ObjectId('5f86038635649a988cdd2ade'),
'name': 'Doug',
'kid_selection': [
{'name': 'James', 'age': 13},
{'name': 'Sharon', 'age': 8}
]
}

Related

Put properties with different name in one field in MongoDB

I am getting requests from different devices as Json. Some of them show temperature as "T", some other as "temp" and it can be different in other devices. is that possible to define in MongoDB to put all of these values in single field "temperature"?
Doesn't matter if it is "temp" or "T" or "tempC", just put all of them in "temperature" field.
Here is an example of my data:
[
{ "ip": "12:3B:6A:1A:E6:8B", "type": 0, "t": 37},
{ "ip": "22:33:66:1A:E6:8B", "type": 1, "temperature": 40},
{ "ip": "1A:3C:6A:1A:E6:8B", "type": 1, "temp": 30}
]
I want to put temp, t and temperature in Temperature field in my collection.
You can use $ifNull operator to control which value should be transferred into your output, like below:
db.col.aggregate([
{
$addFields: { Temperature: { $ifNull: [ { $ifNull: [ "$t", "$temperature"] }, "$temp" ] } }
},
{
$project: {
t: 0,
temperature: 0,
temp: 0
}
}
])
This will merge that three fields into one Temperature taking first not empty value. Additionally if you want to update your collection, you can add $out as a last aggregation stage like { $out: col } but keep in mind that it will entirely replace your source collection.
I think mongodb supports regular expression but they are meant to search datas, not to insert them based on fieldname matches.
I am quite sure you shall use some kind of facade in front of your database to achieve that.

Query on the last element of an array in MongoDB when the array size is stored in a variable

I have a dataset in MongoDB and this is an example of a line of my data:
{ "conversionDate": "2016-08-01",
"timeLagInDaysHistogram": 0,
"pathLengthInInteractionsHistogram": 4,
"campaignPath": [
{"campaignName": "name1", "source": "sr1", "medium": "md1", "click": "0"},
{"campaignName": "name2", "source": "sr1", "medium": "md1", "click": "0"},
{"campaignName": "name1", "source": "sr2", "medium": "md2", "click": "1"},
{"campaignName": "name3", "source": "sr1", "medium": "md3", "click": "1"}
],
"totalTransactions": 1,
"totalValue": 37.0,
"avgCartValue": 37.0
}
(The length of campaignPath is not constant, so each line can have a different amount of elements.
And I want to find elements that matches "source = sr1" in the last element of campaignPath.
I know I can't do a query with something like
db.paths.find(
{
'campaignPath.-1.source': "sr1"
}
)
But, since I have "pathLengthInInteractionsHistogram" stored which is equal to the length of campaignPath lenght, can't I do something like:
db.paths.find(
{
'campaignPath.$pathLengthInInteractionsHistogram.source': "sr1"
}
)
Starting with MongoDB 3.2, you can do this with aggregate which provides the $arrayElemAt operator which accepts a -1 index to access the last element.
db.paths.aggregate([
// Project the original doc along with the last campaignPath element
{$project: {
doc: '$$ROOT',
lastCampaign: {$arrayElemAt: ['$campaignPath', -1]}
}},
// Filter on the last campaign's source
{$match: {'lastCampaign.source': 'sr1'}},
// Remove the added lastCampaign field
{$project: {doc: 1}}
])
In earlier releases, you're stuck using $where. This will work but has poor performance:
db.paths.find({
$where: 'this.campaignPath[this.pathLengthInInteractionsHistogram-1].source === "sr1"'
})
which you could also do without using pathLengthInInteractionsHistogram:
db.paths.find({$where: 'this.campaignPath[this.campaignPath.length-1].source === "sr1"'})

MongoDB updating wrong subdocument in an array

my gamefamilies collection looks like this
{
"_id": ObjectId('54cc3ee7894ae60c1c9d6c74'),
"game_ref_id": "REF123",
..
"yearwise_details": [
{
"year": 1,
...
"other_details": [
{
"type": "cash",
"openingstock": 988
..
},
{
"type": "FLU",
"openingstock": 555
..
},
..other items
]
},
{
"year": 2,
...
"other_details": [
{
"type": "cash",
"openingstock": 3000,
....
},
...
{
"type": "ghee",
"openingstock": 3000,
...
},
..
]
}
]
}
My update query
db.gamefamilies.update({"game_ref_id": "REF123", "teamname": "manisha","yearwise_details.year": 2, "yearwise_details.other_details.type": "ghee"}, {"$set": {"yearwise_details.0.other_details.$.openingstock": 555} });
Document is getting picked up correctly. I expect to update year 2's item type="ghee" but instead year 1's 2nd item (type FLU) gets updated. What am I doing wrong ?
Any help would be greatly appreciated.
regards
Manisha
Unfortunately, there is not yet support for nested $ positional operator updates.
So you can hardcode the update with
db.gamefamilies.update({"game_ref_id": "REF123",
"teamname": "manisha",
"yearwise_details.year": 2,
"yearwise_details.other_details.type": "ghee"},
{"$set":
{"yearwise_details.1.other_details.$.openingstock": 555}});
But notice that the yearwise_details.1.other_details is hardcoding that you want the second value of the array (it is 0-indexed, so the 1 is referencing the second element). I am assuming you found the command you have in your question because it worked for the first element of the array. But it will only ever work on the first element and the command above will only ever work on the second element.

MongoDB $elemMatch of $elemMatch and good practice

H,
I'm trying to update the version field in this object but I'm not able to make a query with 2 nested $match. So what I would like to do is get the record with file id 12 and version 1.
I would ask also if is it a good practice have more the one nested array in mongoDB (like this object)...
Query:
db.collection.find({"my_uuid":"434343"},{"item":{$elemMatch:{"file_id":12,"changes":{$elemMatch:{"version":1}}}}}).pretty()
Object:
{
"my_uuid": "434343",
"item": [
{
"file_id": 12,
"no_of_versions" : 1,
"changes": [
{
"version": 1,
"commentIds": [
4,
5,
7
]
},
{
"version": 2,
"commentIds": [
10,
11,
15
]
}
]
},
{
"file_id": 234,
"unseen_comments": 3,
"no_of_versions" : 2,
"changes": [
{
"version": 1,
"commentIds": [
100,
110,
150
]
}
]
}
]
}
Thank you
If you want the entire documents that satisfy the criteria returned in the result, then I think it's fine. But if you want to limit the array contents of item and changes to just the matching elements, then it could be a problem. That's because, you'll have to use the $ positional operator in the projection to limit the contents of the array and only one such operator can appear in the projection. So, you'll not be able to limit the contents of multiple arrays within the document.

Is there a way to upsert a list with a single query?

I know this question has been asked before, but that's a different scenario.
I'd like to have a collection like this:
{
"_id" : ObjectId("4c28f62cbf8544c60506f11d"),
"pk": 1,
"forums": [{
"pk": 1,
"thread_count": 10,
"post_count": 20,
}, {
"pk": 2,
"thread_count": 5,
"post_count": 24,
}]
}
What I want to do is to upsert a "forum" item, incrementing counters or adding an item if it does not exist.
For example to do something like this (I hope it makes sense):
db.mycollection.update({
"pk": 3,
"forums.pk": 2
}, {
"$inc": {"forums.$.thread_count": 1},
"$inc": {"forums.$.post_count": 1},
}, true)
and have:
{
"_id" : ObjectId("4c28f62cbf8544c60506f11d"),
"pk": 1,
"forums": [{
"pk": 1,
"thread_count": 10,
"post_count": 20,
}, {
"pk": 2,
"thread_count": 5,
"post_count": 24,
}]
},
{
"_id" : ObjectId("4c28f62cbf8544c60506f11e"),
"pk": 3,
"forums": [{
"pk": 2,
"thread_count": 1,
"post_count": 1,
}]
}
I can surely make it in three steps:
Upsert the whole collection with a new item
addToSet the forum item to the list
increment forum item counters with positional operator
That's to say:
db.mycollection.update({pk:3}, {pk:3}, true)
db.mycollection.update({pk:3}, {$addToSet: {forums: {pk:2}}})
db.mycollection.update({pk:3, 'forums.pk': 2}, {$inc: {'forums.$.thread_counter': 1, {'forums.$.post_counter': 1}})
Are you aware of a more efficient way to do it?
TIA, Germano
As you may have discovered, the positional operator cannot be used in upserts:
The positional operator cannot be combined with an upsert since it requires a matching array element. If your update results in an insert then the "$" will literally be used as the field name.
So you won't be able to achieve the desired result in a single query.
You have to separate the creation of the document from the counter update. Your own solution is on the right track. It can be condensed into the following two queries:
// optionally create the document, including the array
db.mycollection.update({pk:3}, {$addToSet: {forums: {pk:2}}}, true)
// update the counters in the array item
db.mycollection.update({pk:3, 'forums.pk': 2}, {$inc: {'forums.$.thread_counter': 1, 'forums.$.post_counter': 1}})