I want to make a mongodb view called "orders4H" that always returns documents from our "orders" collection that have a createdAt field value that is <= NOW-4hours. Then, any tool that is using the "orders4H" view as a collection source will always get the most current order data for the last 4 hours. The view creation requires an aggregation pipeline. However, I cannot figure out how to make a pipeline that will $match on a relative date.
In mongodb 4.2 they added an aggregation system variable called NOW which seems like it would be the thing to use, if only I could get it to work in a query document. I can use it to make new fields, for example, if I put timeNow: "$$NOW" in a $addFields stage, I get a new field with the current datetime. Now I want to use this "$$NOW" in a query evaluation.
Short version of question - how can I write a $match pipeline stage that will perform a server-time-relative comparison with a datetime field from the document?
MongoDB tickets imply that this support has been added ... but I cannot figure out how to use the $$NOW in a query. Anyone out there know how to do this?
Here are the relevant tickets that I found:
https://jira.mongodb.org/browse/SERVER-37713
https://jira.mongodb.org/browse/SERVER-23656
With help from mongodb tech support, I was able to get a $match stage that works with mongodb 4.2 to make a time-relative query:
[
{ $match:
{ $expr:
{ $gt: [
"$_created_at",
{ $subtract: [ "$$NOW", 4 * 60 * 60 * 1000] } ]
}
}
}
]
Related
I have a field called file_date, and in the start of the query I've set a new field called start_date which is 10 days back using:
{ $set: { start_date:{ $subtract: [ "$$NOW", 10*24*60*60*1000 ] } } }.
I want to use $match to find all the results where file_date is bigger then start_date.
I've tried using $gte but I can't seem to get the right syntax.
I'm using mongo 4.2 so I cant use SubstructDate
Thanks for any help
Changing the dates to strings using $toString.
using $gte
select records using aggregate:
db.getCollection('stock_records').aggregate(
[
{
"$project": {
"info.created_date": 1,
"info.store_id": 1,
"info.store_name": 1,
"_id": 1
}
},
{
"$match": {
"$and": [
{
"info.store_id": "563dcf3465512285781608802a"
},
{
"info.created_date": {
$gt: ISODate("2021-07-18T21:07:42.313+00:00")
}
}
]
}
}
])
select records using find:
db.getCollection('stock_records').find(
{
'info.store_id':'563dcf3465512285781608802a',
'info.created_date':{ $gt:ISODate('2021-07-18T21:07:42.313+00:00')}
})
What is difference between these queries and which is best for select by id and date condition?
I think your question should be rephrased to "what's the difference between find and aggregate".
Before I dive into that I will say that both commands are similar and will perform generally the same at scale. If you want specific differences is that you did not add a project option to your find query so it will return the full document.
Regarding which is better, generally speaking unless you need a specific aggregation operator it's best to use find instead, it performs better
Now why is the aggregation framework performance "worse"? it's simple. it just does "more".
Any pipeline stage needs aggregation to fetch the BSON for the document then convert them to internal objects in the pipeline for processing - then at the end of the pipeline they are converted back to BSON and sent to the client.
This, especially for large queries has a very significant overhead compared to a find where the BSON is just sent back to the client.
Because of this, if you could execute your aggregation as a find query, you should.
Aggregation is slower than find.
In your example, Aggregation
In the first stage, you are returning all the documents with projected fields
For example, if your collection has 1000 documents, you are returning all 1000 documents each having specified projection fields. This will impact the performance of your query.
Now in the second stage, You are filtering the documents that match the query filter.
For example, out of 1000 documents from the stage 1 you select only few documents
In your example, find
First, you are filtering the documents that match the query filter.
For example, if your collection has 1000 documents, you are returning only the documents that match the query condition.
Here You did not specify the fields to return in the documents that match the query filter. Therefore the returned documents will have all fields.
You can use projection in find, instead of using aggregation
db.getCollection('stock_records').find(
{
'info.store_id': '563dcf3465512285781608802a',
'info.created_date': {
$gt: ISODate('2021-07-18T21:07:42.313+00:00')
}
},
{
"info.created_date": 1,
"info.store_id": 1,
"info.store_name": 1,
"_id": 1
}
)
The $match expression on mongodb's stitch application does not work properly.
I am trying to set up a simple update trigger that will only work on one field in a collection.
The trigger setup provides a $match aggregation which seems simple enough to set up.
For example if I want the trigger to only fire when the field "online" in a specified collection gets set to "true" I would do:
{"updateDescription.updatedFields":{"online":"true"}}
which for a stitch trigger is the same as:
{$match:{{updateDescription.updatedFields:{online:"true"}}}
The problem is when i try to match an update on a field that is an object.(for example hours:{online:40,offline:120}
For some reason $exists or $in does not work
So doing:
{"updateDescription.updatedFields":{"hours":{"$exists":true}}
does not work,neither does something like:
{"updateDescription.updatedFields":{"hours.online":{"$exists":true}}
The $match for the trigger is supposed to work exactly like a normal mongo $match.
They just provide one example :
{
"updateDescription.updatedFields": {
"status": "blocked"
}
}
The example is from here:
https://docs.mongodb.com/stitch/triggers/database-triggers/
I tried 100's of variations but i can't seem to get it
The trigger is working fine if the match is a specific value like:
{"updateDescription.updatedFields":{"hours.online":{"$numberInt\":"20"}}
and then i set the hours.online to 20 in the database.
I was able to have it match items by using an explicit $expr operator or declare it as a single field not an embedded object. ie. "updateDescription.updatedFields.statue": "blocked"
I struggled with this myself, trying to get a trigger to fire when a certain nested field was updated to any value (rather than just one specific one).
The issue appears to have to do with how change streams report updated fields.
With credit and thanks to MongoDB support, I can finally offer this as a potential solution, at least for simpler cases:
{
"$expr": {
"$not": {
"$cmp": [{
"$let": {
"vars": { "updated": { "$objectToArray": "$updateDescription.updatedFields" } },
"in": { "$arrayElemAt": [ "$$updated.k", 0 ] }
}},
"example.subdocument.nested_field"
]
}
}
}
Of course replace example.subdocument.nested_field with your own dot-notation field path.
I have been trying for a while to extract the insertion date of a mongodb document and add it as a new field to the same document.
I'm trying to do it using the mongo and mongo shell aggregation framework without getting good results.
Here is my query
db.getCollection('my_collection').aggregate(
[
{
$match: {
MY QUERY CRITERIA
}
},
{
$addFields: { "insertTime": "$_id.getTimestamp()" }
}
]
)
I am trying to extracr insertion time from _id using the function getTimestamp() but for sure there is somtehing about aggregation framework syntax that I am missing because I can not do what I am trying to do in my query.
This works perfect:
ObjectId("5c34f746ccb26800019edd53").getTimestamp()
ISODate("2019-01-08T19:17:26Z")
But this does not work at all:
"$_id.getTimestamp()"
What I am missing?
Thanks in advance
Say I have an object with field state, I want to update this field, while keeping the previous value of state in previous_state field. First, I have tried to make an update with unset-rename-set:
collection.update(query, {$unset: {previous_state: ""}, $rename: {state: "previous_state"}, $set: {state: value}})
no surprise it did not work. After reading:
Update MongoDB field using value of another field
MongoDB update: Generate new field based on existing field, or update in place
Update field with another field's value in the document
I am nearly convinced that I do not have a solution to perform this in a single query. So the question is what is the best practice to do it?
There are various ways to do it, depending on the version of MongoDB, and they are described in this answer from another thread: https://stackoverflow.com/a/37280419/5538923 .
For MongoDB 3.4+, for example, there is this query that can be put in MongoSH:
db.collection.aggregate(
[
{ "$addFields": {
"previous_state": { "$concat": [ "$state" ] },
"state": { "$concat": [ "$state", " customly modified" ] }
}},
{ "$out": "collection" }
])
Also note that this query works only when the MongoDB instance is not sharded. When sharded (e.g., often the case in Microsoft Azure CosmosDB), the method described in that answer for MongoDB 3.2+ works, or alternatively put a new collection as destination (in the $out close), and then import the data in the original collection, after removing all the data there.
One solution (if you've got onlty one writer) could be to trigger your update in two steps:
> var previousvalue = collection.findOne(query).state;
> collection.update(query, {$set: {"previous_state": previousvalue, "state": newstatevalue}});