How to return index of array item in Mongodb? - mongodb

The document is like below.
{
"title": "Book1",
"dailyactiviescores":[
{
"date": 2013-06-05,
"score": 10,
},
{
"date": 2013-06-06,
"score": 21,
},
]
}
The daily active score is intended to increase once the book is opened by a reader. The first solution comes to mind is use "$" to find whether target date has a score or not, and deal with it.
err = bookCollection.Update(
{"title":"Book1", "dailyactivescore.date": 2013-06-06},
{"$inc":{"dailyactivescore.$.score": 1}})
if err == ErrNotFound {
bookCollection.Update({"title":"Book1"}, {"$push":...})
}
But I cannot help to think is there any way to return the index of an item inside array? If so, I could use one query to do the job rather than two. Like this.
index = bookCollection.Find(
{"title":"Book1", "dailyactivescore.date": 2013-06-06}).Select({"$index"})
if index != -1 {
incTarget = FormatString("dailyactivescore.%d.score", index)
bookCollection.Update(..., {"$inc": {incTarget: 1}})
} else {
//push here
}

Incrementing a field that's not present isn't the issue as doing $inc:1 on it will just create it and set it to 1 post-increment. The issue is when you don't have an array item corresponding to the date you want to increment.
There are several possible solutions here (that don't involve multiple steps to increment).
One is to pre-create all the dates in the array elements with scores:0 like so:
{
"title": "Book1",
"dailyactiviescores":[
{
"date": 2013-06-01,
"score": 0,
},
{
"date": 2013-06-02,
"score": 0,
},
{
"date": 2013-06-03,
"score": 0,
},
{
"date": 2013-06-04,
"score": 0,
},
{
"date": 2013-06-05,
"score": 0,
},
{
"date": 2013-06-06,
"score": 0
}, { etc ... }
]
}
But how far into the future to go? So one option here is to "bucket" - for example, have an activities document "per month" and before the start of a month have a job that creates the new documents for next month. Slightly yucky. But it'll work.
Other options involve slight changes in schema.
You can use a collection with book, date, activity_scores. Then you can use a simple upsert to increment a score:
db.books.update({title:"Book1", date:"2013-06-02", {$inc:{score:1}}, {upsert:true})
This will increment the score or insert new record with score:1 for this book and date and your collection will look like this:
{
"title": "Book1",
"date": 2013-06-01,
"score": 10,
},
{
"title": "Book1",
"date": 2013-06-02,
"score": 1,
}, ...
Depending on how much you simplified your example from your real use case, this might work well.
Another option is to stick with the array but switch to using the date string as a key that you increment:
Schema:
{
"title": "Book1",
"dailyactiviescores":{
{ "2013-06-01":10},
{ "2013-06-02":8}
}
}
Note it's now a subdocument and not an array and you can do:
db.books.update({title:"Book1"}, {"dailyactivityscores.2013-06-03":{$inc:1}})
and it will add a new date into the subdocument and increment it resulting in:
{
"title": "Book1",
"dailyactiviescores":{
{ "2013-06-01":10},
{ "2013-06-02":8},
{ "2013-06-03":1}
}
}
Note it's now harder to "add-up" the scores for the book so you can atomically also update a "subtotal" in the same update statement whether it's for all time or just for the month.
But here it's once again problematic to keep adding days to this subdocument - what happens when you're still around in a few years and these book documents grow hugely?
I suspect that unless you will only be keeping activity scores for the last N days (which you can do with capped array feature in 2.4) it will be simpler to have a separate collection for book-activity-score tracking where each book-day is a separate document than to embed the scores for each day into the book in a collection of books.

According to the docs:
The $inc operator increments a value of a field by a specified amount.
If the field does not exist, $inc sets the field to the specified
amount.
So, if there won't be a score field in the array item, $inc will set it to 1 in your case, like this:
{
"title": "Book1",
"dailyactiviescores":[
{
"date": 2013-06-05,
"score": 10,
},
{
"date": 2013-06-06,
},
]
}
bookCollection.Update(
{"title":"Book1", "dailyactivescore.date": 2013-06-06},
{"$inc":{"dailyactivescore.$.score": 1}})
will result into:
{
"title": "Book1",
"dailyactiviescores":[
{
"date": 2013-06-05,
"score": 10,
},
{
"date": 2013-06-06,
"score": 1
},
]
}
Hope that helps.

Related

MongoDB document setup and aggregation

I'm pretty new to MongoDB and while preparing data to be consumed I got into Aggregation... what a powerful little thing this database has! I got really excited and started to test some things :)
I'm saving time entries for a companyId and employeeId ... that can have many entries... those are normally sorted by date, but one date can have several entries (multiple registrations in the same day)
I'm trying to come up with a good schema so I could easily get my data exactly how I need and as a newbie, I would rather ask for guidance and check if I'm in the right path
my output should be as
[{
"company": "474A5D39-C87F-440C-BE99-D441371BF88C",
"employee": "BA75621E-5D46-4487-8C9F-C0CE0B2A7DE2",
"name": "Bruno Alexandre":
"registrations": [{
"id": 1448364,
"spanned": false,
"spannedDay": 0,
"date": "2019-01-17",
"timeStart": "09:00:00",
"timeEnd": "12:00:00",
"amount": {
"days": 0.4,
"hours": 2,
"km": null,
"unit": "days and hours",
"normHours": 5
},
"dateDetails": {
"week": 3,
"weekDay": 4,
"weekDayEnglish": "Thursday",
"holiday": false
},
"jobCode": {
"id": null,
"isPayroll": true,
"isFlex": false
},
"payroll": {
"guid": null
},
"type": "Sick",
"subType": "Sick",
"status": "APP",
"reason": "IS",
"group": "LeaveAndAbsence",
"note": null,
"createdTimeStamp": "2019-01-17T15:53:55.423Z"
}, /* more date entries */ ]
}, /* other employees */ ]
what is the best way to add the data into a collection?
Is it more efficient if I create a document per company/employee and add all registration entries inside that document (it could get really big as time passes)... or is it better to have one document per company/employee/date and add all daily events in that document instead?
regarding aggregation, I'm still new to all this, but I'm imagining I could simply call
RegistrationsModel.aggregate([
{
$match: {
date: { $gte: new Date('2019-01-01'), $lte: new Date('2019-01-31') },
company: '474A5D39-C87F-440C-BE99-D441371BF88C'
}
},
{
$group: {
_id: '$employee',
name: { '$first': '$name' }
}
},
{
// ... get all registrations as an Array ...
},
{
$sort: {
'registrations.date': -1
}
}
]);
P.S. I'm taken the Aggregation course to start familiarized with all of it
Is it more efficient if I create a document per company/employee and
add all registration entries inside that document (it could get really
big as time passes)... or is it better to have one document per
company/employee/date and add all daily events in that document
instead?
From what I understand of document oriented databases, I would say the aim is to have all the data you need, in a specific context, grouped inside one document.
So what you need to do is identify what data you're going to need (getting close to the features you want to implement) and build your data structure according to that. Be sure to identify future features, cause the more you prepare your data structure to it, the less it will be tricky to scale your database to your needs.
Your aggregation query looks ok !

Put properties with different name in one field in MongoDB

I am getting requests from different devices as Json. Some of them show temperature as "T", some other as "temp" and it can be different in other devices. is that possible to define in MongoDB to put all of these values in single field "temperature"?
Doesn't matter if it is "temp" or "T" or "tempC", just put all of them in "temperature" field.
Here is an example of my data:
[
{ "ip": "12:3B:6A:1A:E6:8B", "type": 0, "t": 37},
{ "ip": "22:33:66:1A:E6:8B", "type": 1, "temperature": 40},
{ "ip": "1A:3C:6A:1A:E6:8B", "type": 1, "temp": 30}
]
I want to put temp, t and temperature in Temperature field in my collection.
You can use $ifNull operator to control which value should be transferred into your output, like below:
db.col.aggregate([
{
$addFields: { Temperature: { $ifNull: [ { $ifNull: [ "$t", "$temperature"] }, "$temp" ] } }
},
{
$project: {
t: 0,
temperature: 0,
temp: 0
}
}
])
This will merge that three fields into one Temperature taking first not empty value. Additionally if you want to update your collection, you can add $out as a last aggregation stage like { $out: col } but keep in mind that it will entirely replace your source collection.
I think mongodb supports regular expression but they are meant to search datas, not to insert them based on fieldname matches.
I am quite sure you shall use some kind of facade in front of your database to achieve that.

Adding Fields to an existing Array

I have another problem to solve here. Thinking in arrays sometimes could be very challenging. Here is what I am lined up with. This is what my data looks like: -
{
"_id": { "Firm": "ABC", "year": 2014 },
"Headings": [
{
"costHead": "MNF",
"amount": 500000
},
{
"costHead": "SLS",
"amount": 25000
},
{
"costHead": "OVRHD",
"amount": 100
}
]
}
{
"_id": { "Firm": "CDF", "year": 2015 },
"Headings": [
{
"costHead": "MNF",
"amount": 15000
},
{
"costHead": "SLS",
"amount": 100500
},
{
"costHead": "MNTNC",
"amount": 7500
}
]
}
As you can see, I have a list that has a whole bunch of sub-documents.
Here is what I want to do .. I need to add more elements to this "Headings" list which should be : -
{
"costHead": "FxdCost",
"amount": "$Headings.amount (for costhead MFC) + $Headings.amount (for costhead OVRHD)"
}
I am unsure as to how to produce the above. Here are some challenges: -
I can addToSet the new subdocument I wish to add but the problem is addToSet can only be used in the group stage - which would be expensive (unless of course there is no other way).
Even if I use addToSet, I always have to use the $ operator to refer to elements that I read from my JSON file. Now the element I am trying to add here (costHead: FxdCost) is not present in my JSON file and hence I cannot use the $ operator.
Does anyone have any advice on how to go about this. This is after all basic ETL.

How to multi-sort MongoDB entry with dynamic keys, on two suboptions?

I'm trying to sort this in MongoDB with mongojs on a find():
{
"songs": {
"bNppHOYIgRE": {
"id": "bNppHOYIgRE",
"title": "Kygo - ID (Ultra Music Festival Anthem)",
"votes": 1,
"added": 1428514707,
"guids": [
"MzM3NTUx"
]
},
"izJzdDPH9yw": {
"id": "izJzdDPH9yw",
"title": "Benjamin Francis Leftwich - Atlas Hands (Samuraii Edit)",
"votes": 1,
"added": 1428514740,
"guids": [
"MzM3NTUx"
]
},
"Yifz3X_i-F8": {
"id": "Yifz3X_i-F8",
"title": "M83 - Wait (Kygo Remix)",
"votes": 0,
"added": 1428494338,
"guids": []
},
"nDopn_p2wk4": {
"id": "nDopn_p2wk4",
"title": "Syn Cole - Miami 82 (Kygo Remix)",
"votes": 0,
"added": 1428494993,
"guids": []
}
}
}
and I want to sort the keys in the songs on votes ascending and added descending.
I have tried
db.collection(coll).find().sort({votes:1}, function(err, docs) {});
but that doesn't work.
If this is an operation that you're going to be doing often, I would strongly consider changing your schema. If you make songs an array instead of a map, then you can perform this query using aggregation.
db.coll.aggregate([{ "$unwind": "$songs" }, { "$sort": { "songs.votes": 1, "songs.added": -1 }}]);
And if you put each of these songs in a separate songs collection, then you could perform the query with a simple find() and sort().
db.songs.find().sort({ "votes": 1, "added": -1 });
With your current schema, however, all of this logic would need to be in your application and it would get messy. A possible solution would be to get all of the documents and while iterating through the cursor, for each document, iterate through the keys, adding them to an array. Once you have all of the subdocuments in the array, sorting the array according to votes and added.
It is possible, but unnecessarily complex. And, of course, you wouldn't be able to take advantage of indexes, which would have an impact on your performance.
You already include the key inside the subdocument, so I would really recommend you reconsider your schema.

MongoDB update all fields of array error

Im tring to set 0 the items.qty of a document obtains by a id query.
db.warehouses.update(
// query
{
_id:ObjectId('5322f07e139cdd7e31178b78')
},
// update
{
$set:{"items.$.qty":0}
},
// options
{
"multi" : true, // update only one document
"upsert" : true // insert a new document, if no existing document match the query
}
);
Return:
Cannot apply the positional operator without a corresponding query field containing an array.
This is the document that i want to set all items.qty to 0
{
"_id": { "$oid" : "5322f07e139cdd7e31178b78" },
"items": [
{
"_id": { "$oid" : "531ed4cae604d3d30df8e2ca" },
"brand": "BJFE",
"color": "GDRNCCD",
"hand": 1,
"model": 0,
"price": 500,
"qty": 0,
"type": 0
},
{
"brand": "BJFE",
"color": "GDRNCCD",
"hand": 1,
"id": "23",
"model": 0,
"price": 500,
"qty": 4,
"type": 0
},
{
"brand": "BJFE",
"color": "GDRNCCD",
"hand": 1,
"id": "3344",
"model": 0,
"price": 500,
"qty": 6,
"type": 0
}
],
"name": "a"
}
EDIT
The detail missing from the question was that the required field to update was actually in a sub-document. This changes the answer considerably:
This is a constraint of what you can possibly do with updating array elements. And this is clearly explained in the documentation. Mostly in this paragraph:
The positional $ operator acts as a placeholder for the first element that matches the query document
So here is the thing. Trying to update all of the array elements in a single statement like this will not work. In order to do this you must to the following.
db.warehouses.find({ "items.qty": { "$gt": 0 } }).forEach(function(doc) {
doc.items.forEach(function(item) {
item.qty = 0;
});
db.warehouses.update({ "_id": doc._id }, doc );
})
Which is basically the way to update every array element.
The multi setting in .update() means across multiple "documents". It cannot be applied to multiple elements of an array. So presently the best option is to replace the whole thing. Or in this case we may just as well replace the whole document since we need to do that anyway.
For real bulk data, use db.eval(). But please read the documentation first:
db.eval(function() {
db.warehouses.find({ "items.qty": { "$gt": 0 } }).forEach(function(doc) {
doc.items.forEach(function(item) {
item.qty = 0;
});
db.warehouses.update({ "_id": doc._id }, doc );
});
})
Updating all the elements in an array across the whole collection is not simple.
Original
Pretty much exactly what the error says. In order to use a positional operator you need to match something first. As in:
db.warehouses.update(
// query
{
_id:ObjectId('5322f07e139cdd7e31178b78'),
"items.qty": { "$gt": 0 }
},
// update
{
$set:{"items.$.qty":0}
},
// options
{
"multi" : true,
"upsert" : true
}
);
So where the match condition fins the position of the items that are less than 0 then that index is passed to the positional operator.
P.S : When muti is true it means it updates every document. Leave it false if you only mean one. Which is the default.
You can use the $ positional operator only when you specify an array in the first argument (i.e., the query part used to identify the document you want to update).
The positional $ operator identifies an element in an array field to update without explicitly specifying the position of the element in the array.