I am attempting to calculate the total amount of money spent being tracked inside of our database. Each order document contains a field "total_price"
I am attempting to use the following code:
db.orders.aggregate({
$group: {
_id: null,
total: {$sum: "$total_price"}
}
})
Unfortunately, the only output I get is: { "result" : [ { "_id" : null, "total" : 0 } ], "ok" : 1 }
But to verifiy there is actually numerical data stored, and just not being totaled: db.orders.find()[0].total_price this results in 8.99
Any help would be greatly appreciated. I have very little experience using MongoDB. I've only covered the basics at this point.
Thank you in advance.
$sum only works with ints, longs and floats. Right now, there is no operator to parse a string into a number, although that would be very useful. You can do this yourself as is described in Mongo convert all numeric fields that are stored as string but that would be slow.
I would suggest you make sure that your application stores numbers as int/long/float, and that you write a script that iterators over all your documents and updates the value. I would also suggest that you add a feature request at https://jira.mongodb.org/browse/SERVER to add an operator that converts a string to a number.
Related
we have a collection with big amount of documents, lets say around 100k. We now want to count the number of documents which has the key x set.
If I try it with Collection.countDocuments({ x: { $exists: true }}) I get the result, but it creates instantly a warning in the console: Query Targeting: Scanned Objects / Returned has gone above 1000.
So, is there a better way to count the documents? There is a Index on the field, is it possible to get the length of the index?
Thanks
Theres no real way of viewing the index trees in Mongo, what other people have linked you just returns the size of the tree, I'm not sure how useful that information is in this context.
Now to your question is this the best way to count?.
The answer is Yes ... -ish.
countDocuments is a wrapper function, it just simulates the following pipeline:
db.collection.aggregate([
{ $match: <query> },
{ $group: { _id: null, n: { $sum: 1 } } } )
])
This pipeline is the most efficient way to go, but the difference between running this aggregation and using the wrapper function is about 100-200 milliseconds, depending on your machine spec.
Meaning if you're looking for "way" better performance you're not going to find it.
With that said this warning is stupid, it just means you have more than 1000 documents with that field. The true purpose of it is to alert you in the case you're trying to query 1-20 documents without a proper index.
You can use the indexSizes field returned by the stats() method.
The stats() method "Returns statistics about the collection".
See example here :
https://docs.mongodb.com/manual/reference/method/db.collection.stats/#basic-stats-lookup
{
...,
"indexSizes" : {
"_id_" : 237568,
"cuisine_1" : 143360,
"borough_1_cuisine_1" : 151552,
"borough_1_address.zipcode_1" : 151552
},
...
}
indexSize key return size as in space used in storing not count
Check With Explain if index getting used or not . (Update in question Also)
can use hint option to check the performance after specifying index
Or precalculate count by $inc operator might good option if possible in you use case
try cursor.count if its faster countDocument should been faster but no harm in checking
https://docs.mongodb.com/manual/reference/method/cursor.count/
I tried to update a field in a document which was long integer. But it was updated to the value '14818435007969200' instead of '14818435007969199'.
db.getCollection('title').updateMany({},
{$set:{'skillId':[NumberLong(14818435007969199)]}})
db.getCollection('title').find({})
{
"_id" : ObjectId("5853351c0274072315da2426"),
"skillId" : [
NumberLong(14818435007969200)
]
}
Is there any solution? I am using robomongo 0.9.0.
The mongo shell treats all numbers as floating point values. So while using the NumberLong() wrapper pass the long value as string or risk the loss for precision or conversion mismatches.
This should work as expected.
db.getCollection('title').updateMany({},
{$set:{'skillId':[NumberLong("14818435007969199")]}})
Just to demonstrate for example.
So when converting 14818435007969199 to binary base 2 value you get 110100101001010100100111000010110001101111011110110000 which when converted back to base 10 is 14818435007969200
You can checkout the floating point arithmetic for more details.
here is an example with where condition in the query
db.CustomerRatibs.update(
{ custRatibId:'8b19bfdbac7b468b9c3edafc37ad5409' },
{ $set:
{
uAt : NumberLong(1536581726000)
}
},
{
multi:false
}
)
Is there a way to match a value with every array and sub document inside the document in mongodb collection and return the document
{
"_id" : "2000001956",
"trimline1" : "abc",
"trimline2" : "xyz",
"subtitle" : "www",
"image" : {
"large" : 0,
"small" : 0,
"tiled" : 0,
"cropped" : false
},
"Kytrr" : {
"count" : 0,
"assigned" : 0
}
}
for eg if in the above document I am searching for xyz or "ab" or "xy" or "z" or "0" this document should be returned.
I actually have to achieve this at the back end using C# driver but a mongo query would also help greatly.
Please advice.
Thanks
You could probably do this using '$where'
db.mycollection({$where:"JSON.stringify(this).indexOf('xyz')!=-1"})
I'm converting the whole record to a big string and then searching to see if your element is in the resulting string. Probably won't work if your xyz is in the fieldnames!
You can make it iterate through the fields to make a big string and then search it though.
This isn't the most elegant way and will involve a full tablescan. It will be faster if you look through the individual fields!
While Malcolm's answer above would work, when your collection gets large or you have high traffic, you'll see this fall over pretty quickly. This is because of 2 things. First, dropping down to javascript is a big deal and second, this will always be a full table scan because $where can't use an index.
MongoDB 2.6 introduced text indexing which is on by default (it was in beta in 2.4). With it, you can have a full text index on all the fields in the document. The documentation gives the following example where a text index is created for every field and names the index "TextIndex".
db.collection.ensureIndex(
{ "$**": "text" },
{ name: "TextIndex" }
)
I feel like I must be missing something obvious. Here is the aggregation, as one would post it in the shell:
db.documents.aggregate(
{ $project: { title: 1, "date.year": 1, decade:
{ $subtract: ['$date.year', { $mod: ['$date.year', 10]}]}
}})
This is supposed to take a list of documents, each with a date.year field, and add a decade field indicating which decade the document is in (1900, 1910, etc.) I'm planning on further transforming the data after I get that added.
The problem is that when I run the aggregation, I get:
{
"errmsg" : "exception: $subtract resulted in a non-numeric type",
"code" : 16413,
"ok" : 0
}
If I change $subtract to $add, it works fine (but doesn't give me the right result, of course.) So what's going on with the subtraction? Why am I getting a non-numeric type when I subtract but a number when I add?
Thanks in advance!
This looks like a bug in the aggregation framework - it's not handling subtraction correctly when the fields you are operating for are not set in the documents going through the pipeline.
It's been fixed in 2.3.2 (I can't reproduce this - it projects "null" when "date" is not set) but one way you can work around this limitation is by adding a $match condition to your pipeline, i.e. prefix {$project} with:
{$match: {"date.year":{$exists:true}}}
When I have two MongoDB documents like this...
db.test.insert( {"value" : "10123"} );
db.test.insert( {"value" : "160"} );
The result of a query like:
db.test.find({"value" :{$gt : "12"} });
is..
{ "_id" : ObjectId("4c6d1b92304326161b678b89"), "value" : "160" }
It's obvious, that a string comparison is made, so that my first value is not returned.
Is there any way to cast within the query?
Something like:
db.test.find({ (int) "value" :{$gt : 12} });
would be great. A query like
db.test.find({"value" :{$gt : 12} }); // without the quotes around "12"
returns nothing.
You can use the following JavaScript expression:
db.test.find("this.value > 12")
This uses JavaScript's automatic conversion from string to number.
I have a similar workaround, i find that if you can use the mongo shell, you can write an statement to do this in javascript, but capable of using indexes.
var myItems = []
var it = db.test.find({},{value:1})
while (it.hasNext()){
var item = it.next();
if(parseInt(item.value) > 12)
myItems.push(item);
}
If you want this to run faster than previus solution, you have to ensure the index on the value field.
Type casting in MongoDB is available after version >= 4.0. Check MongoDB's aggregation operator $convert and similar operators. Since you wanted to convert string to int you can use $toInt:
db.collection.find({ $expr: { $gt: [ { $toInt: "$value" }, 12 ] } })
Test : mongoplayground
Note :
Here we're converting value which is a string field to int on the fly & Since it got converted to int - we're comparing it to input of type int. Your output documents will still have original type for value field which is string (we're not actually changing type in response docs, if needed use aggregation & it's stage $project to see int values for field value in response docs).
Since we're using aggregation operators in .find() we need to wrap everything in $expr.
Even though this is pretty common nowadays & is easy to do, but remember we're casting string to int on every Read, rather if you can take care of this on Writes or Updates it would be easy & more efficient.
To convert String into int use this
db.test.find({'year': {$type: 2}}).forEach(
function (x) {
x.value=new NumberInt(x.value);
db.test.save(x)}
)
And after that you can directly query like :
db.test.find({"value" :{$gt : 12} });
$gt wasn't set for this use case. You would have to use regex for strings. Probably easier to just create a new field with the number as a number.