I have been trying for a while to extract the insertion date of a mongodb document and add it as a new field to the same document.
I'm trying to do it using the mongo and mongo shell aggregation framework without getting good results.
Here is my query
db.getCollection('my_collection').aggregate(
[
{
$match: {
MY QUERY CRITERIA
}
},
{
$addFields: { "insertTime": "$_id.getTimestamp()" }
}
]
)
I am trying to extracr insertion time from _id using the function getTimestamp() but for sure there is somtehing about aggregation framework syntax that I am missing because I can not do what I am trying to do in my query.
This works perfect:
ObjectId("5c34f746ccb26800019edd53").getTimestamp()
ISODate("2019-01-08T19:17:26Z")
But this does not work at all:
"$_id.getTimestamp()"
What I am missing?
Thanks in advance
Related
I have a field called file_date, and in the start of the query I've set a new field called start_date which is 10 days back using:
{ $set: { start_date:{ $subtract: [ "$$NOW", 10*24*60*60*1000 ] } } }.
I want to use $match to find all the results where file_date is bigger then start_date.
I've tried using $gte but I can't seem to get the right syntax.
I'm using mongo 4.2 so I cant use SubstructDate
Thanks for any help
Changing the dates to strings using $toString.
using $gte
I´m new with WebFlux and MongoDB. I´m trying to use aggregation in a capped collection with tailable cursor, but I´m nothing getting sucessful.
I´d like to execute this mongoDB query:
db.structures.aggregate(
[
{
$match: {
id: { $in: [8244, 8052]}
}
},
{ $sort: { id: 1, lastUpdate: 1} },
{
$group:
{
_id: {id: "$id"},
lastUpdate: { $last: "$lastUpdate" }
}
}
]
)
ReactiveMongoOperations gives me option to "tail" or "aggregation".
I´m able to execute aggregation:
MatchOperation match = new MatchOperation(Criteria.where("id").in(8244, 8052));
GroupOperation group = Aggregation.group("id", "$id").last("$lastUpdate").as("lastUpdate");
Aggregation aggregate = Aggregation.newAggregation(match, group);
Flux<Structure> result = mongoOperation.aggregate(aggregate,
"structures", Structure.class);
Or tail cursor
Query query = new Query();
query.addCriteria(Criteria.where("id").in(8244, 8052));
Flux<Structure> result = mongoOperation.tail(query, Structure.class);
Is it possible? Tail and Aggregation together?
Using aggregation was the way that I found to get only the last inserted document for each id.
Without aggregation I get:
query without aggregation
With aggregation:
query with aggregation
Tks in advance
The tailable cursor query creates a Flux that never completes (never emits onComplete event) and that Flux emits records as they are inserted in the database. Because of that fact I would think aggregations are not allowed by the database engine with the tailable cursor.
So the aggregation doesn't make sense in a way because on every newly inserted record the aggregation would need to be recomputed. Technically you can do a running aggregation where for every returned record you compute the wanted aggregate record and send it downstream.
One possible solution would be to do the aggregations programmatically on the returned "infinite" Flux:
mongoOperation.tail(query, Structure.class)
.groupBy(Structure::id) // create independent Fluxes based on id
.flatMap(groupedFlux ->
groupedFlux.scan((result, nextStructure) -> { // scan is like reduce but emits intermediate results
log.info("intermediate result is: {}", result);
if (result.getLastUpdate() > nextStructure.getLastUpdate()) {
return result;
} else {
result.setLastUpdate(nextStructure.getLastUpdate());
return result;
}
}));
On the other hand you should probably revisit your use case and what you need to accomplish here and see if something other than capped collection should be used or maybe the aggregation part is redundant (i.e. if newly inserted records always have the lastUpdate property larger then the previous record).
I'm using parse-server version 2.7.4 with parse (the JavaScript interface) version 1.11.1. I'm trying to execute an aggregate query using match on a date column. Here's what my pipeline looks like:
[
{
match: {
'_created_at': new Date('2018-04-01T00:00:00.000Z')
}
}
]
I most definitely have records in the database greater than that date, yet this query returns me zero records in JavaScript. However, if I run it in the Mongo shell, I get documents returned. Here is what I execute in the Mongo shell:
[
{
$match: {
'_created_at': ISODate('2018-04-01T00:00:00.000Z')
}
}
]
Am I doing something wrong with the parse JavaScript API or does it not support match aggregation on date columns? Thanks for any help.
I'm using MongoDB's Aggregate Framework. I have an existing field in each of my documents:
time: Date
And I wish create a new field timeBlock based on a simple funcion:
var dateToTimeBlock = function(dateString, timeBlock){
return new Date(dateString).valueOf() / timeBlock
}
I understand that $group can add fields, but the functions used to calculate those fields seems to be built into mongo, eg, $avg, $add, etc. Is it possible to generate a computed value based on an arbitrary field?
You can compute fields using aggregation framework. If you want to use native JavaScript functions (like valueOf on Date object) you will have to use Map-Reduce.
Although, aggregation framework is not as flexible as Map-Reduce, in most cases it will be significantly faster. If performance is critical I would precalculate those values and use a simple query instead.
If you want to use aggregation, you can simplify it by converting the Date into an integer or add a new field in the document that's an integer. Then you can than do your calculations easily with $divide.
db.coll.aggregate([
{ $project : { dateToTime: { $divide : [ "$timeInt", timeBlock ] }}}
]);
Or if timeBlock is a field in the same document you can do it like this:
db.coll.aggregate([
{ $project : { dateToTime: { $divide : [ "$timeInt", "$timeBlock" ] }}}
]);
I want to save the aggregation framework result to a new collection.
I know that's impossible with the framework at the moment with the command itself.
Is there a workaround in the shell?
Starting with Mongo 2.6.0 you can do this natively without any additional manipulation.
db.<collection>.aggregate( [
{ <operation> },
{ <operation> },
...,
{ $out : "<output-collection>" }
] )
Check the new aggregation operator $out for more detailed example.
P.S. using this way you are not limited to 16Mb size.
Update: See Salvador's answer for a more efficient way to do this in MongoDB 2.6+.
Save the aggregation result in a variable and then insert its result property into a new collection:
var result = db.foo.aggregate(...);
db.bar.insert(result.result);
result.result is no longer working, try toArray().
var result = db.coll.aggregate(...);
db.bar.insert(result.toArray());