Azure cosmos throwing error for $round in MongoDB query pipeline - mongodb

I am using Azure Cosmos MongoDB and storing API Request logs. I wanted to know how many request were made per second. So I am grouping them on basis of timestamp after dividing the timestamp by 1000 then rounding off with 0 decimal places
1593096483234 to 1593096483.234 to 1593096483
and taking count of documents furnished by above grouping then sorting on basis of count in decreasing order.
Query
db.server_api_logs.aggregate([
{
"$group": {
"_id": "$timestamp"
}
},
{
"$project": {
"summation": {
"$round": {
"$toDouble": {
"$divide": ["$_id",1000]
}
}
}
}
},
{
"$group": {
"_id": "$summation",
"summer": {"$sum": 1}
}
},
{
"$sort": {"summer": -1}
}
]
Getting this error when try to run above query
Unable to execute the selected commands
Mongo Server error (MongoCommandException): Command failed with error 168 (168): 'unknown operator: $round' on server <server address>
The full response is:
{
"ok" : 0.0,
"errmsg" : "unknown operator: $round",
"code" : NumberInt(168),
"codeName" : "168"
}
It is working totally fine on local using compass but not on Cosmos using Robo3T. Am I doing something wrong?

As stated in MongoDB documentation, $round operator was introduced in version 4.2 of MongoDB.
As stated in Azure Cosmos DB documentation, for this moment, Azure Cosmos DB's API for MongoDB is compatible with MongoDB server version 3.6.
This is the reason, why your query works on local MongoDB instance (I believe 4.2 or newer) but fails for Azure Cosmos DB.

Related

I can sort on ISODate attributes, but the "$lt" and "$gt" queries are failing in my MongoDB Collections

I am centralizing data on my Atlas Cluster from bare-metal servers running community MongoDB databases locally. However I have an issue with my ISODates() attributes that are converted to "$date" dictionaries, after which the sort() query still works but the "$lt" and "$gt" queries are failing.
Attribute in local db
"createdTime": ISODate("2023-01-20T11:20:50.268Z")
Attribute in the Atlas Cluster'db :
"createdTime" : { '$date': '2023-01-20T11:20:50.268Z' }
In my Atlas Cluster's collection, I am still able to sort on the time attribute :
db.MyCollection.find().sort({createdTime:-1}).map(x => x.createdTime)
[
{ '$date': '2023-01-20T11:20:50.268Z' },
{ '$date': '2023-01-20T11:20:26.587Z' },
{ '$date': '2023-01-20T11:20:04.108Z' },
]
But when I try to use the operators "$gt" and "$lt" then mongo doesn't know what to do :
db.MyCollection.countDocuments({ createdTime: { $lt: ISODate("2023-02-01T00:00:00.000Z")} })
0
// even with "$date" as a key
db.MyCollection.countDocuments({ "createdTime.$date": { $lt: ISODate("2023-02-01T00:00:00.000Z")}})
0
Eventually, I figured I could make it work with a Javascript function to convert the "$date" objects back to ISODate, but I want to be able to use the "$lt" and "$gt" in my Python API, relying on pymongo.
Pymongo code :
res = db["MyCollection"].count_documents({"createdTime": {"$lt": datetime.now() }})
# res = 0
Any idea how I could solve what I assume is a date formatting issue?
Firstly I don't understand why the date is inside another object with a private field (starting with a $ sign). You can directly store the IDODate as you have mentioned in your question.
"createdTime": ISODate("2023-01-20T11:20:50.268Z")
without putting it inside an object.
If you did not do it in purpose, this could be a migration issue between when using mongoexport with a datetime filter using the --query option. This behaviour is documented here.
Your first query:
db.MyCollection.countDocuments({ createdTime: { $lt: ISODate("2023-02-01T00:00:00.000Z")} })
returns nothing, because createdTime is an object.
Your second query:
db.MyCollection.countDocuments({ "createdTime.$date": { $lt: ISODate("2023-02-01T00:00:00.000Z")}})
returns nothing because createdTime.$date cannot be directly be accessed in the query. Fields with $ prefixed are not openly accesible in Mongo. What you have to do to make this work is to use an expression with a getField, that would enable you to access this.
For example, this query should give you the expected result.
db.MyCollection.countDocuments({$expr: {$lt: [{$getField: {$literal: "createdTime.$date"}}, ISODate("2023-02-01T00:00:00.000Z")]}})
But a better solution would be to keep createTime itself as an ISODate.

AWS DocDB $project Requires Outputs, doesnt match mongo API

I am using the $project stage in an aggregation pipeline using Pymongo on AWS DocumentDB, like so:
{ '$project': { 'foo': 0 } }
This results in the following error:
$projection requires at least one output field
As far as I can tell, the mongo docs state that as of Mongo version 3.4 you can use $project to specify exclusions of fields (see: https://www.mongodb.com/docs/v4.0/reference/operator/aggregation/project/). The example they provide looks like:
db.books.aggregate( [ { $project : { "lastModified": 0 } } ] )
Does docdb not match this part of the API spec? or am I missing something?
EDIT: I'm also unable to use the _id filter specifically, which is also highlighted in mongo docs.
{'$project': {'_id': 0}}
Excluding just _id is not currently supported, i.e. { '$project': { '_id': 0 } }
But, projecting a non existing field can enforce exclusion: { '$project': { '_id': 0, 'foo_field': 0 } }

How to replace $$ROOT for group in AWS DocumentDB - Feature not supported: $$ROOT

I have a Node app interacting with a Mongo database. When running on the old setup or locally on MongoDB 4.4.0
When I deploy to AWS Document DB 3.6.0 and try to run a query with the below $group clause, I get an error that Feature not supported: $$ROOT
{
$group: {
"_id": "$user_id",
"total_paid_amount": {
$sum: "$amount"
},
"data": {
$push: "$$ROOT"
}
}
}
According to this doc https://docs.aws.amazon.com/documentdb/latest/developerguide/developerguide.pdf, $$ROOT is not supported in DocumentDB
How can I work around this issue, or perform the same query without pushing $$ROOT?
Thanks in advance!
Ended up just having to push the items I actually needed/wanted instead of all via $$ROOT
$push: {
fieldOne: "$fieldOne",
fieldTwo: "$fieldTwo"
}
Support for $$ROOT was added to Amazon DocumentDb in October 2021.

Mongo shell aggregation query issue

I am fairly new to MongoDB and I am facing a strange issue.
I've been able to build an aggregation pipeline using a compass.
The query produces the expected result in the GUI but it returns nothing in mongo shell.
Here is the query :
db.searchQueryCollection.aggregate([{
$match: {
time_stamp: {
$gte: ISODate("2018-08-13 17:33:20.000"),
$lt: ISODate("2018-08-13 19:33:20.000")
}
}
}, {
$group: {
_id: "$purpose",
groupedPurpose: {$push: "$purpose"}
}
}, {$project: {number_of_results: {$size: "$groupedPurpose"}}}])
Here is the output in compass :
Any idea on what's going wrong?
In mongo shell, you should use as below:
db.getCollection("YOUR_COLLECTION_NAME").aggregate([{
$match: {
time_stamp: {
$gte: ISODate("2018-08-13 17:33:20.000"),
$lt: ISODate("2018-08-13 19:33:20.000")
}
}
}, {
$group: {
_id: "$purpose",
groupedPurpose: {$push: "$purpose"}
}
}, {$project: {number_of_results: {$size: "$groupedPurpose"}}}])
If none of your aggregation queries return a result, you may be connected to the wrong database.
If you're connecting using a database driver, check your connection string. In the following example, the database name is at the end of the connection string.
MONGODB_URI=mongodb+srv://username:password#cluster0.abcdefg.mongodb.net/databasename
If you're connecting using MongoDB shell, run the following two commands:
show dbs;
admin 288.00 KiB
config 328.00 KiB
local 62.66 MiB
mydatabase 1.24 MiB
use mydatabase;

how to run aggregate query in mongodb client on RockMongo or mViewer

I've just started using mongo db, I'm using rockmongo client and my ubuntu terminal as another client. I' ve realized the query using group aggregation like this:
db.archiveImpl.group(
{
key: {accountID: true, phone: true },
cond: {'userId': "oiuoifas2309u"},
reduce: function(object, prev){ },
initial:{}
}
)
And the above query is not getting run in the rockmongo , I initially thought that the query was wrong. and I ran in on the terminal which pulls me the data. So I downloaded and installed mViewer and it didn't work either so I believe there is a way to run these kind of queries in mongo web clients
thanks for reading this .
You can run aggregate query in "tools->command" panel in your homepage of rockmongo.
You may enter query in the input panel like this:
{
aggregate : "ns",
pipeline : [
...
]
}
That will be ok! I have tried, and it works!
Full Example:
{ aggregate : "COLLECTION_NAME",
pipeline : [
{ $unwind : "$SUB_ARRAY" },
{ $match: {"SUB_ARRAY.field": "value"}},
]
}