MongoDB View vs Function to abstract query and variable/parameter passed - mongodb

I hate to risk asking a duplicate question, but perhaps this is different from Passing Variables to a MongoDB View which didn't have any clear solution.
Below is a query to find the country for IP Address 16778237. (Outside the scope of this query, there is a formula that turns an IPV4 address into a number.)
I was wondering if we could abstract away this query out of NodeJS code, and make a view, so the view could be called from NodeJS. But the fields ipFrom and ipTo are indexed to get the query to run fast against millions of documents in the collection, so we can't return all the rows to NodeJS and filter there.
In MSSQL maybe this would have to be a stored procedure, instead of a view. Just trying to learn what is possible in MongoDB. I know there are functions, which are written in JavaScript. Is that where I need to look?
db['ip2Locations'].aggregate(
{
$match:
{
$and: [
{
"ipFrom": {
$lte: 16778237
}
},
{
"ipTo": {
$gte: 16778237
}
},
{
"active": true
}
],
$comment: "where 16778237 between startIPRange and stopIPRange and the row is 'Active',sort by createdDateTime, limit to the top 1 row, and return the country"
}
},
{
$sort:
{
'createdDateTime': - 1
}
},
{
$project:
{
'countryCode': 1
}
},
{
$limit: 1
}
)
Part 2 - after more research and experimenting, I found this is possible and runs with success, but then see trying to make a view below this query.
var ipaddr = 16778237
db['ip2Locations'].aggregate(
{
$match:
{
$and: [
{
"ipFrom": {
$lte: ipaddr
}
},
{
"ipTo": {
$gte: ipaddr
}
},
{
"active": true
}
],
$comment: "where 16778237 between startIPRange and stopIPRange and the row is 'Active',sort by createdDateTime, limit to the top 1 row, and return the country"
}
},
{
$sort:
{
'createdDateTime': - 1
}
},
{
$project:
{
'countryCode': 1
}
},
{
$limit: 1
}
)
If I try to create a view with a "var" in it, like this;
db.createView("ip2Locations_vw-lookupcountryfromip","ip2Locations",[
var ipaddr = 16778237
db['ip2Locations'].aggregate(
I get error:
[Error] SyntaxError: expected expression, got keyword 'var'
In the link I provided above, I think the guy was trying to figure how the $$user-variables work (no example here: https://docs.mongodb.com/manual/reference/aggregation-variables/). That page refers to $let, but never shows how the two work together. I found one example here: https://www.tutorialspoint.com/mongodb-query-to-set-user-defined-variable-into-query on variables, but not $$variables. I'm
db.createView("ip2Locations_vw-lookupcountryfromip","ip2Locations",[
db['ip2Locations'].aggregate(
...etc...
"ipFrom": {
$lte: $$ipaddr
}
I tried ipaddr, $ipaddr, and $$ipaddr, and they all give a variation of this error:
[Error] ReferenceError: $ipaddr is not defined
In a perfect world, one would be able to do something like:
get['ip2Locations_vw-lookupcountryfromip'].find({$let: {'ipaddr': 16778237})
or similar.
I'm getting that it's possible with Javascript stored in MongoDB (How to use variables in MongoDB query?), but I'll have to re-read that; seems like some blogs were warning against it.
I have yet to find a working example using $$user-variables, still looking.

Interpretation
You want to query a view from some server side code, passing a variable to it.
Context
Can we use an external variable to recompute a View? Take the following pipeline:
var pipeline = [{ $group:{ _id:null, useless:{ $push:"$$NOW" } } }]
We can pass system variables using $$. We can define user variables too, but the user defined variables are made out of:
Collection Data
System Variables.
Also, respect to your Part2:
A variable var variable="what" will be computed only once. Redefine variable="whatever" makes no difference in the view, it uses "what".
Conclusion
Views can only be re-computed with system variables, or user variables dependant on those system variables or collection data.
Added an answer to the post you link too.

Related

MongoDB performance: $and vs. single object with multiple keys

I have an internal service that does some operations on an order, it has a built-in required filter, and the service users can pass additional filter.
Two approaches to achieve the same thing:
A) Using $and:
async function getOrders ({ optionalFilter = {} }) {
const baseFilter = { amount: { $gt: 10 } };
const mergedFilter = { $and: [baseFilter, optionalFilter] };
return await Order.find(mergedFilter);
}
B) Merging all in the same object
async function getOrders ({ optionalFilter = {} }) {
const baseFilter = { amount: { $gt: 10 } };
const mergedFilter = { ...baseFilter, ...optionalFilter };
return await Order.find(mergedFilter);
}
I prefer the first approach because it allows me to do the following without overwriting the $gt: 10 rule while the second would break the code by overwriting the internal rule.
getOrders({ optionalFilter: { amount: { $lt: 50 } } });
My question is, is there any advantage (performance or otherwise) of choosing one over the other?
My question is, is there any advantage (performance or otherwise) of choosing one over the other?
Short answer: No. If you are not using (compound) indexes
Long Answer: Okay, let's test this up, right?
So let's take a collection of ~100k documents, and un-indexed array field:
{asset_class: "COMMDTY"}
with {$and: [{item_subclass: "Herb", asset_class: "COMMDTY"}]}
without {item_subclass: "Herb", asset_class: "COMMDTY"}
Okay, there is 114ms difference between using $and and without. But does it means to stop you using $and and avoid it? Of course, not. But as much field have added, the more slower Mongo will become. But this whole picture changes, when I add an already indexed field to a query.
{expansion: "BFA", asset_class: "COMMDTY"}
So, if you add more fields via your mergedFilter then make sure that your index (if you are using them) can be used, because as for compound index field order is very meaningful. So the query: {asset_class: "COMMDTY", expansion: "BFA" } will goes already in:

mongodb need to populate a new field with an old fields value, without destroying other data

I have a situation where a model changed at some point in time and I am faced with (for argument sake) half my data liks like this
{
_id: OID,
things: [{
_id:OID,
arm: string,
body: string
}],
other: string
}
and the other half of my data look like this
{
_id: OID,
things: [{
_id:OID,
upper_appendage: string,
body: string
}],
other: string
}
I would like to 'correct' half of the data - so that I DON'T have to accommodate both names for 'arm' in my application code.
I have tried a couple different things:
The first errors
db.getCollection('x')
.find({things:{$exists:true}})
.forEach(function (record) {
record.things.arm = record.things.upper_appendage;
db.users.save(record);
});
and this - which destroys all the other data in
db.getCollection('x')
.find({things:{$exists:true}})
.forEach(function (record) {
record.things = {
upper_appendage.arm = record.things.upper_appendage
};
db.users.save(record);
});
Keeping in mind that there is other data I want to maintain...
How can I do this???
the $rename operator should have worked for this job but unfortunately it doesn't seem to support nested array fields (as of mongodb server 4.2). instead you'd need a forEach like the following:
db.items.find({
things: {
$elemMatch: {
arm: {
$exists: true
}
}
}
}).forEach(function(item) {
for (i = 0; i != item.things.length; ++i)
{
item.things[i].upper_appendage = item.things[i].arm;
delete item.things[i].arm; ;
}
db.items.update({
_id: item._id
}, item);
})
note: i've assumed you want to make all records have upper_appendageand get rid of 'arm' field. if it's the other way you want, just switch things around.

Updating a value that is dependent on a newly updated document key

My goal is to add a comment to my CommentFeed and while doing that I want to push that comment into my topComments field and also update the 'numOfComments' . I want to limit the topComments to only 3 comments (How would I even set that up?). And how do I take the previous value of numOfComments and add one to it?
CommentFeed.findOneAndUpdate(
{ _id: commentId },
{
$push: {
comments: {
text: req.body.text
},
$push: topComments:{text: req.body.text}, <--- Limit this somehow to only allow an array length of 3?
$set: numOfComments: ? , <---What kind of logic is used here?
}
},
{ new: true }
)
CommentFeed Schema
const CommentFeedSchema = new Schema({
topComments:[{text:{type:String}}],
numOfComments:{type:Number},
comments: [
text: { type: String, required: true }
]});
For the first issue (limiting the topComments array size) you can use the $slice operator. This has already been answered in other questions. But you might consider computing topComments from comments using the$slice operator in the projection argument:
CommentFeed.find( {}, { comments: { $slice: -3 } } )
For the second issue (updating a document using existing fields from that document), it is not something you can do in a simple findOneAndUpdate call. This was also discussed in other questions.
But you might consider computing numOfComments instead of updating it every time. You can do that with the $size operator of the aggregation framework:
CommentFeed.aggregate({$project: { numOfComments: { $size:"$comments" }}})

Issue in Updating an Array Object in mongoDb

I am trying to update an Array object based on a condition. Following is my scenario :-
I want to update status from current to archive.
I have tried many things for hours but still no luck. Like this :-
db.user.update({
'injury._id': ObjectId("5374cb4d1e0386c02800006a"),
'injury.injurydata.locationaddressinjury': {
$elemMatch: {
'status': 'current'
}
}
}, {
$set: {
'injury.injurydata.locationaddressinjury.status': 'archive'
}
})
The picture made it hard to read the structure of your data. But I guess the update you are looking for would be something like this:
db.user.update({
'injury._id': ObjectId("5374cb4d1e0386c02800006a"),
'injury.injurydata.locationaddressinjury': {
$elemMatch: {
'status': 'current'
}
}
}, {
$set: {
'injury.injurydata.locationaddressinjury.$.status': 'archive'
}
});
The $ would refer to the element you found. While if you are looking for a way to update all element at one time. I'm afraid $elemMatch would just match the first element that satisfies your condition.

Does Moongoose 3.8.8 support $position operator?

Does Moongoose 3.8.8 (the lastest version) support $position (http://docs.mongodb.org/manual/reference/operator/update/position/) operator from MongoDB 2.6.0?
In the following code example the new elements is inserted in the end of the array userActivity.activities:
model:
var userActivity = new schema({
userId: {type:String, required:true, unique:true},
activities: [activity]
});
var activity = new schema({
act: {type: Number, required:true},
});
query:
var activity = { act: 1 };
model.userActivity.update(
{ _id: dbact._id },
{ $push: { activities: {
$each: [ activity ],
$position: 0
}
}
},
function (err, numAffected) {
if (!err) {
// do something
}
});
This actually doesn't matter and never matters for any "framework" implementation and I do not mind explaining why.
Every single "framework" ( such as Mongoose, Mongoid, Doctrine, MongoEngine, etc, etc, etc ) are all basically built upon a basic "driver" implementation that has in most cases been developedby the MongoDB staff themselves. So the basic functionality is always ther even if you need to "delve" down to a level in order to use those "native" methods.
So here would be the native usage example in this case:
List.collection.update(
{},
{ "$push": {
"list": {
"$each": [ 1, 2, 3 ],
"$position": 0 }
}
},function(err,NumAffected) {
console.log("done");
});
Note the "collection" method used from the model, which is getting the "raw" collection details from the driver. So you are using it's method and not some "wrapped" method that may be doing additional processing.
The next and most basic reason is if you cannot find the method and application of the operators that you need the here is a simple fact.
Every single operation as used by the methods in every framework and basic driver method is essentially a call to the "runCommand" method in the basic API. So since that basic call is available everywhere ( in some form or another, because it has to be ), then you can do everything that you find advertised on the MongoDB site with every language implementation on any framework.
But the short call to your particular request is, since this is not actually a method call but is simply part of the BSON arguments as passed in, then of course there is no restriction by a particular language driver to actually use this.
So you can use these new argument without of course updating to the most recent version. But you probably will get some nice methods to do so if you actually do.
Yes, you should be able to use it directly as Mongoose will pass through the update clause:
Model.update(
query, /* match the document */
{ $push:
{ yourArrayField:
{
$each: [ 1, 2, 3 ],
$position: 0
}
}
}, function (err, res) { /* callback */ });
The above would insert the values 1, 2, 3 at the front of the array named yourArrayField.
As it's just a pass-through, you'll need to make sure it works with the server version that you're connecting the client to.