I am trying to perform an aggregation query on a collection. I want to divide 2 of the fields and check if this is less than a certain value.
The way this db was made, it has all the fields as strings, and this isn't something I can really change right now. I wondered if there was a way to cast the values as numbers, maybe using $let or something?
As it is I currently get the error
exception: $divide only supports numeric types, not String and String
This is an example of the sort of query I am trying to run:
db.myCollection.aggregate([{
"$project": {
"mins": {
"$divide": ["$ALOWOVRLTOTL", "$FEESCALLCOST"]}
}
},
"$match": {
"mins": {
"$lte": 40
}
}
}])
You won't be able to perform the divide operation on non-numerical values. A possible solution to this would be to run an update and parse the String values to a numerical value.
Have a look at this question:
how to convert string to numerical values in mongodb
Related
I have a document (inside aggregation, after $group stage) which have an object (but I could form array, if I needed it to) with number values.
MongoPlayground example with data and my aggregate query available here.
And I want to make a new _id field during next $project stage, consisted of this three number values, like:
item_id | unix time | pointer
_id: 453435-41464556645#1829
The problem is, that when I am trying to use $concat, the query returns me an error like:
$concat only supports strings, not int
So here is my question: is it possible to achieve such results? I have seen the relevant question MongoDB concatenate strings from two fields into a third field, but it didn't cover my case.
The $concat only concatenate strings, these fields $_id.item_id contains int value and $_id.last_modified double value,
The $toString converts a value to a string,
_id: {
$concat: [
{
$toString: "$_id.item_id"
},
" - ",
{
$toString: "$_id.last_modified"
}
]
}
Playground: https://mongoplayground.net/p/SSlXW4gIs_X
I cant spot what the issue here is. This returns nothing
db.mycoll.aggregate([
{$project:{CreatedAt:"$CreatedAt",Now:"$$NOW",DateFloor:{$add:["$$NOW",-24*60*60000]}}},
{$match:{CreatedAt:{$gte:"$DateFloor"}}}
])
But this returns results - substituting DateFloor with actual value
db.mycoll.aggregate([
{$project:{CreatedAt:"$CreatedAt",Now:"$$NOW",DateFloor:{$add:["$$NOW",-24*60*60000]}}},
{$match:{CreatedAt:{$gte: ISODate("2020-04-28T23:17:56.547Z")}}}
])
Issue with your query is when you're doing :
{$match:{CreatedAt:{$gte:"$DateFloor"}}}
You're actually checking for documents where CreatedAt field's value to be greater than input string value "$DateFloor", So $match is not considering "$DateFloor" as another field in the same document rather it's considering it as a string value. So to compare two fields from same document you need to use $gte with $expr (which will let you use aggregation expressions within the query language).
{
$match: {
{
$expr: {
$gte: ["$CreatedAt", "$DateFloor"];
}
}
}
}
So you might get confused when I say aggregation expressions & why $gte needs to be wrapped inside $expr - In your actual query $gte refers to comparison operator in MongoDB but in this above query $gte refers to aggregation pipeline operator where both technically does the same but which is what needed to compare two fields from same document.
I am using mongo server version 3.4, so my question pertains to the functionality of that version. I cannot upgrade anytime soon, so please keep that in mind. If have a field in some documents in a MongoDB collection that may contain a string but also have trailing characters, how might I find them when submitting multiple "startsWith" strings to be evaluated in the same query? I may have some difficulty explaining this, so let me show some examples. Let's say that I have a field called "description" in all of my documents. This description might be encoded so that the text is not completely straightforward. Some values might be:
green:A-4_ABC
yellow:C-12_456
red:A-431_ZXCVQ
yellow_green:C-12_999
brown:B-3_R
gray:EN-44_195
EDIT: I think I made a mistake with using words in my keys. The keys are a randomized string of numbers, letters, and underscores, followed by a colon, then one to three letters, followed by a dash, then a couple of numbers, then an underscore, and lastly followed by several alphanumeric characters:
LKEF543SLI54EH2J897FQ_HF234EWOH:ZX-82_FR2
I realize that this sounds arbitrary and stupid, but it is an encoding of information that is intended to result in a unique key. It is in data that I receive, so I cannot change it, unfortunately.
Now, I want to find all of the documents with descriptions that start with any of the following values, and all of these values must be submitted in the same query. I might have hundreds of submitted values, and I need to get all matching documents at once. Here is a short list of what might be submitted in a single query:
green:A-4
red:A-431
gray:EN-44
yellow_green:C-12
Note that it was not accidental that the text is everything prior to the last underscore. And, as with one of the examples, there might be more than one underscore. With my use case, I cannot create a query that hard-codes these strings in the javascript regex format. And the $in filter does not work with "startsWith" functionality, particularly when you pass in a list of strings (though I am familiar with supplying a list of hard-coded javascript regexes). Is there any way to use the $in operator where I can take a list of strings that are passed in from the user who wants to run a query like this? Or is there something equivalent? The cherry on the top of all of this would be to find a way to project the matching document with the string that it matched (either from the query, or by some substring magic that I cannot seem to figure out).
EDIT: Specifically, when I find each document, I want to be able to project everything from they key up until the LAST underscore, like:
LKEF543SLI54EH2J897FQ_HF234EWOH:ZX-82
(along with its value)
Thanks in advance for any nudges in the right direction.
We use $objectToArray to get {k:field_name, v:field_value} array. Then we split by _ token all values and convert to object with $arrayToObject operator.
Next step we apply $match operator to filter documents and exclude data with $unset.
Note: If your document contains array or subdocuments, we may use $filter before we convert $objectToArray.
db.collection.aggregate([
{
$addFields: {
data: {
$arrayToObject: {
$map: {
input: {
$objectToArray: "$$ROOT"
},
in: {
k: "$$this.k",
v: {
$arrayElemAt: [
{
$split: [
{
$toString: "$$this.v"
},
"_"
]
},
0
]
}
}
}
}
}
}
},
{
$match: {
"data.green": "A-4",
"data.red": "A-431",
"data.gray": "EN-44",
"data.yellow_green": "C-12"
}
},
{
$unset: "data"
}
])
MongoPlayground
Let's say I have four documents in my collection:
{u'a': {u'time': 3}}
{u'a': {u'time': 5}}
{u'b': {u'time': 4}}
{u'b': {u'time': 2}}
Is it possible to sort them by the field 'time' which is common in both 'a' and 'b' documents?
Thank you
No, you should put your data into a common format so you can sort it on a common field. It can still be nested if you want but it would need to have the same path.
You can use use aggregation and the following code has been tested.
db.test.aggregate({
$project: {
time: {
"$cond": [{
"$gt": ["$a.time", null]
}, "$a.time", "$b.time"]
}
}
}, {
$sort: {
time: -1
}
});
Or if you also want the original fields returned back: gist
Alternatively you can sort once you get the result back, using a customized compare function ( not tested,for illustration purpose only)
db.eval(function() {
return db.mycollection.find().toArray().sort( function(doc1, doc2) {
var time1 = doc1.a? doc1.a.time:doc1.b.time,
time2 = doc2.a?doc2.a.time:doc2.b.time;
return time1 -time2;
})
});
You can, using the aggregation framework.
The trick here is to $project a common field to all the documents so that the $sort stage can use the value in that field to sort the documents.
The $ifNull operator can be used to check if a.time exists, it
does, then the record will be sorted by that value else, by b.time.
code:
db.t.aggregate([
{$project:{"a":1,"b":1,
"sortBy":{$ifNull:["$a.time","$b.time"]}}},
{$sort:{"sortBy":-1}},
{$project:{"a":1,"b":1}}
])
consequences of this approach:
The aggregation pipeline won't be covered by any of the index you
create.
The performance will be very poor for very large data sets.
What you could ideally do is to ask the source system that is sending you the data to standardize its format, something like:
{"a":1,"time":5}
{"b":1,"time":4}
That way your query can make use of the index if you create one on the time field.
db.t.ensureIndex({"time":-1});
code:
db.t.find({}).sort({"time":-1});
In my documents, the decimal values are stored as String ( Because of my BigDecimal in Java converted to String). This is perfectly OK and required when I am looking one document at a time. However, I require them to be treated as Double during aggregation purpose (for e.g., $sum).
Is there a way so that I can covert the String to Double during $project operation (or otherwise) during my aggregation pipeline. So that my successive pipelines will do $sum on the Double field.
I am not considering the option of storing them as Double, as I would need to preserve the precision per document basis, but I am OKif the precision is lost during my aggregation operation.
There is no operator to do conversions in aggregation pipeline. Is it possible to store two different values one as BigDecimal / String and another as a double / float value? That can be one of quick and dirty option if you are ok to ignore the flakiness due to the floating point values.
You can use $toDouble, which is essentially a short hand for $convert with to: "double"
Assume we have a document;
{
"str": "3.66"
}
and to convert the string value to double;
db.collection.aggregate([
{
$addFields: {
double: {
$toDouble: "$str"
}
}
}
])
will result in;
{
"str": "3.66",
"double": 3.66
}
Where it can be used by other stages of aggregation pipeline.
Check on Mongo Playground
Mongodb version 4.0 introduced the $toInt operator in the aggregation pipeline. You can use that to convert easily from string to number.
Link to documentation: https://docs.mongodb.com/manual/reference/operator/aggregation/toInt/