OneCollection.find({}, {sort: {time1: -1, time2: -1}});
This one, time1 is prior to time2.
I want to sort based on two fields together.
If time1 does not exists, using time2.
Or just use equal priority.
Either way will do. thanks
I've managed to handle similiar situation using aggregation. You can try something like this:
OneCollection.aggregate([
{
$addFields: {
time: {
$cond: {
if: {
$and: [
{ $ifNull: ['$time1', false] },
{ $gt: ['$time2', '$time1'] }
]
},
then: '$time1',
else: '$time2' } }
}
},
{ $sort: { time: -1 } },
{ $project: { time: false } }
]);
It adds temporally field time which stores time1 value if it exists and is greater than time2 value or time2 value otherwise and uses it for sorting. Then it removes time field from result document after sort operation is complete.
3 suggestions:
Sort clientside - that way you can just use the arr.sort([compareFunction])
Transform the publications and add the field https://www.eventedmind.com/items/meteor-transforming-collection-documents
Add the sortfield to the data model (update existing data), and when saving/editing objects in the future also add/edit the sortfield. Don't include the field in publications, but use it to sort when necessary.
EDIT: I would go with 3 - that way the publications would be more efficient
On the server-side, you could try the meteorhacks aggregate package, and perform the comparison in your publish method (you may want to cache the result). Note that this is untested:
if (Meteor.server) {
Meteor.publish("OneCollection", function () {
if (!this.userId) { // kick them out if they're not logged in
this.stop();
return;
}
var pipeline = [
$project: {
time: { $max: [ "time1", "time2" ] }
}
];
return OneCollection.aggregate(pipeline);
}
}
Related
I have a schema that looks like this:
var Counter = new Schema ({
_id: ObjectId,
date: Date,
counter: Number
})
On the request, I send date of a day and expect the date to be added and the counter to increase. Now when I add, a counter gets incremented ie 1,2,3 etc and a date gets added.
Now here is a problem: I want a counter to reset to 0 when a different date is given,(such to say on every new day, the counter should start at 0) and then start a counter increment again etc...
This is my code on how I have tried:
Counter.findOneAndUpdate(
{
$set:{
"date: thedate",
},
$inc: {
"counter: counter+1"
}
)
How do I achieve this ?
UPDATE FOR MORE CLARIFIATION
Take this example of two documents
{
"_id": ObjectId("1111"),
"date": "2020-04-13",
"counter": 0,
}
{
"_id": ObjectId("2222"),
"date": "2020-04-29",
"counter": 0,
}
My collection has more than one document. I want to update the document based on its id. For this case i want to update the 1st document of id 1111
Now if give an input date, say 2020-04-13 and id of '1111' which matches the first document, it should increment the couter to 1. If I give an the same date again (with same id of 111) it should increment the counter to 2.
If again I give an input date of 2020-04-14(which is another date) on the same first document of id 1111 it should reset the counter to 0.
Now How do I achieve this?
As you can execute update-with-an-aggregation-pipeline in .update() operations starting MongoDB version >= 4.2, try below querie :
Counter.findOneAndUpdate(
{ _id: ObjectId("............") }, // filter query - Input of type ObjectId()
/** Update date field with input & re-create 'counter' field based on condition */
[{
$addFields: {
date: inputDate, // Input type of date, In general Date is saved in MongoDB as ISODate("2020-05-06T00:00:00.000Z")
counter: { $cond: [ { $eq: [ "$date", inputDate ] }, { $add: [ "$counter", 1 ] }, 0 ] }
}
}]
)
Test : Test aggregation pipeline here : mongoplayground
I'm still not clear what you want to achieve but you can try this method by breaking the find and update
Counter.findOne({}) //match inside {} condition
.then(counter => {
if(counter.date){ //apply date condition here
counter.counter+=1;
} else {
counter.counter = 0;
}
counter.save()
.then(updatedCounter => {
// further action you want to take
})
})
.catch(err => {
//handle err here
})
You can use $inc operator in MongoDB here.
Example:
export async function incrementCounter(db, formId) {
return db.collection('posts').updateOne(
{ _id: postId },
{
$inc: {
comments: 1 // Increments by 1. Similarly, -2 will decrement by 2.
}
}
)
}
For the reset functionality, you can use MongoDB Atlas Triggers. You can also use a third-party library like mongodb-cron.
I have a document which is structured like this:
{
'item_id': '12345'
'total_score': 100,
'user_scores': {
'ABC': 40,
'DEF': 60
}
}
I'm using PyMongo, but documentation of MongoDB seems easily translatable across different distributions. With PyMongo, I could update user scores with:
collection.update_one(
{ 'item_id': '12345' },
{ '$set': { 'user_scores.GHI': 20 } },
upsert=True
)
Which results in this:
{
'item_id': '12345'
'total_score': 100,
'user_scores': {
'ABC': 40,
'DEF': 60,
'GHI': 20
}
}
The issue is of course that the total_score is now incorrect. I want that total score to update, so that in a future query, I can quickly ascertain the score of each result, and even sort by score.
One solution could be to find an existing document using find_one({'item_id: '12345'}), (create if it doesn't exist), then update with new scores, and update total score. The problem there is that I want to run thousands of these at the same time, and it's far more efficient to call bulk_write on a series of requests.
So, a better solution would be to do two sequential update requests:
request1 = UpdateOne(
{ 'item_id' : '12345' },
{ '$set': { 'user_scores.GHI': 20 } },
upsert = True
)
request2 = UpdateOne(
{ 'item_id' : '12345' },
{ '$set': { 'total_score': { '$sum': { '$values': 'user_scores' } } } },
upsert = True
)
The first request updates the user scores, same as before. The second request, there are two concepts going on. The syntax for this isn't correct, but here's what I'm trying to do:
I need to get the values from the user_scores dictionary. { '$values': 'user_scores' } is how I've tried to convey this.
That gives me an array of values. I know these are all numeric, so I now need to sum those, conveyed with { '$sum': { '$values': 'user_scores' } }.
I can run these batch updates consecutively, so there's no risk of summing the wrong thing. The danger with having a total_score field will always be that it isn't updated and thus doesn't contain the correct number. I'd imagine this is a common case with document-based models?
If you're using Mongo version 4.2+ they introduced a new feature: pipelined updates, Meaning now you can do what you want in one go:
db.collection.updateOne({ 'item_id' : '12345' },
[
{ '$set': { 'user_scores.GHI': 20 } },
{ '$set': { 'total_score': { '$sum': [ "$user_scores.GHI", "$user_scores.ABC", "$user_scores.GHI"] } } },,
]);
Unfortunately this is not possible for lesser Mongo versions hence if that is the case you'll have to keep using your solution which is splitting this into 2 actions.
EDIT:
For dynamic update we can use $map and $objectToArray like so:
db.collection.updateOne(
{'item_id': '12345'},
[
{'$set': {'user_scores.GHI': 20}},
{
'$set':
{
'total_score': {
'$sum': {
'$map': {
'input': {'$objectToArray': '$user_scores'},
'as': 'score',
'in': '$$score.v'
}
}
}
}
}
]);
I would like to group my search results by date. The results are then also counted.
var fromDate = new Date(req.query.fromDate);
var toDate = new Date(req.query.toDate);
global.databaseStatistic.collection(global.ARTICLEPRESS).aggregate([
{
$match: {
article: { $in: articleIdArray },
date: { $gte: fromDate, $lte: toDate }
}
},
{
$group: {
"_id": {
"date": {
$dateToString: { format: "%d.%m.%Y", date: "$date" }
}
},
"count": { $sum: 1 }
}
}
])
This works perfectly, but I would also like to show all days that have a count of zero. It should be so displayed every day within an interval, also having a count of zero. How can I do that?
as it looks very trivial - it is not.
There is no way to have a reference sequence to compare with, even $lookup cannot help as this a kind of inner join type.
The way you could have this done is a kind of post process of result-set returned to mongoose.
The steps I have in mind could be:
create array of dates from req.query.fromDate to req.query.toDate formated in the same way as in query
remove entries in array which we have in result set
merge our array with count:0 and date
sort results (if needed)
any comments welcome!
For speed, I'd like to limit a query to 10 results
db.collection.find( ... ).limit(10)
However, I'd also like to know the total count, so to say "there were 124 but I only have 10". Is there a good efficient way to do this?
By default, count() ignores limit() and counts the results in the entire query.
So when you for example do this, var a = db.collection.find(...).limit(10);
running a.count() will give you the total count of your query.
Doing count(1) includes limit and skip.
The accepted answer by #johnnycrab is for the mongo CLI.
If you have to write the same code in Node.js and Express.js, you will have to use it like this to be able to use the "count" function along with the toArray's "result".
var curFind = db.collection('tasks').find({query});
Then you can run two functions after it like this (one nested in the other)
curFind.count(function (e, count) {
// Use count here
curFind.skip(0).limit(10).toArray(function(err, result) {
// Use result here and count here
});
});
cursor.count() should ignore cursor.skip() and cursor.limit() by default.
Source: http://docs.mongodb.org/manual/reference/method/cursor.count/#cursor.count
You can use a $facet stage which processes multiple aggregation pipelines within a single stage on the same set of input documents:
// { item: "a" }
// { item: "b" }
// { item: "c" }
db.collection.aggregate([
{ $facet: {
limit: [{ $limit: 2 }],
total: [{ $count: "count" }]
}},
{ $set: { total: { $first: "$total.count" } } }
])
// { limit: [{ item: "a" }, { item: "b" }], total: 3 }
This way, within the same query, you can get both some documents (limit: [{ $limit: 2 }]) and the total count of documents ({ $count: "count" }).
The final $set stage is an optional clean-up step, just there to project the result of the $count stage, such that "total" : [ { "count" : 3 } ] becomes total: 3.
There is a solution using push and slice: https://stackoverflow.com/a/39784851/4752635
I prefe
First for filtering and then grouping by ID to get number of filtered elements. Do not filter here, it is unnecessary.
Second query which filters, sorts and paginates.
Solution with pushing $$ROOT and using $slice runs into document memory limitation of 16MB for large collections. Also, for large collections two queries together seem to run faster than the one with $$ROOT pushing. You can run them in parallel as well, so you are limited only by the slower of the two queries (probably the one which sorts).
I have settled with this solution using 2 queries and aggregation framework (note - I use node.js in this example, but idea is the same):
var aggregation = [
{
// If you can match fields at the begining, match as many as early as possible.
$match: {...}
},
{
// Projection.
$project: {...}
},
{
// Some things you can match only after projection or grouping, so do it now.
$match: {...}
}
];
// Copy filtering elements from the pipeline - this is the same for both counting number of fileter elements and for pagination queries.
var aggregationPaginated = aggregation.slice(0);
// Count filtered elements.
aggregation.push(
{
$group: {
_id: null,
count: { $sum: 1 }
}
}
);
// Sort in pagination query.
aggregationPaginated.push(
{
$sort: sorting
}
);
// Paginate.
aggregationPaginated.push(
{
$limit: skip + length
},
{
$skip: skip
}
);
// I use mongoose.
// Get total count.
model.count(function(errCount, totalCount) {
// Count filtered.
model.aggregate(aggregation)
.allowDiskUse(true)
.exec(
function(errFind, documents) {
if (errFind) {
// Errors.
res.status(503);
return res.json({
'success': false,
'response': 'err_counting'
});
}
else {
// Number of filtered elements.
var numFiltered = documents[0].count;
// Filter, sort and pagiante.
model.request.aggregate(aggregationPaginated)
.allowDiskUse(true)
.exec(
function(errFindP, documentsP) {
if (errFindP) {
// Errors.
res.status(503);
return res.json({
'success': false,
'response': 'err_pagination'
});
}
else {
return res.json({
'success': true,
'recordsTotal': totalCount,
'recordsFiltered': numFiltered,
'response': documentsP
});
}
});
}
});
});
How to implement somethings similar to db.collection.find().limit(10) but while updating documents?
Now I'm using something really crappy like getting documents with db.collection.find().limit() and then updating them.
In general I wanna to return given number of records and change one field in each of them.
Thanks.
You can use:
db.collection.find().limit(NUMBER_OF_ITEMS_YOU_WANT_TO_UPDATE).forEach(
function (e) {
e.fieldToChange = "blah";
....
db.collection.save(e);
}
);
(Credits for forEach code: MongoDB: Updating documents using data from the same document)
What this will do is only change the number of entries you specify. So if you want to add a field called "newField" with value 1 to only half of your entries inside "collection", for example, you can put in
db.collection.find().limit(db.collection.count() / 2).forEach(
function (e) {
e.newField = 1;
db.collection.save(e);
}
);
If you then want to make the other half also have "newField" but with value 2, you can do an update with the condition that newField doesn't exist:
db.collection.update( { newField : { $exists : false } }, { $set : { newField : 2 } }, {multi : true} );
Using forEach to individually update each document is slow. You can update the documents in bulk using
ids = db.collection.find(<condition>).limit(<limit>).map(
function(doc) {
return doc._id;
}
);
db.collection.updateMany({_id: {$in: ids}}, <update>})
The solutions that iterate over all objects then update them individually are very slow.
Retrieving them all then updating simultaneously using $in is more efficient.
ids = People.where(firstname: 'Pablo').limit(10000).only(:_id).to_a.map(&:id)
People.in(_id: ids).update_all(lastname: 'Cantero')
The query is written using Mongoid, but can be easily rewritten in Mongo Shell as well.
Unfortunately the workaround you have is the only way to do it AFAIK. There is a boolean flag multi which will either update all the matches (when true) or update the 1st match (when false).
As the answer states there is still no way to limit the number of documents to update (or delete) to a value > 1. A workaround to use something like:
db.collection.find(<condition>).limit(<limit>).forEach(function(doc){db.collection.update({_id:doc._id},{<your update>})})
If your id is a sequence number and not an ObjectId you can do this in a for loop:
let batchSize= 10;
for (let i = 0; i <= 1000000; i += batchSize) {
db.collection.update({$and :[{"_id": {$lte: i+batchSize}}, {"_id": {$gt: i}}]}),{<your update>})
}
let fetchStandby = await db.model.distinct("key",{});
fetchStandby = fetchStandby.slice(0, no_of_docs_to_be_updated)
let fetch = await db.model.updateMany({
key: { $in: fetchStandby }
}, {
$set:{"qc.status": "pending"}
})
I also recently wanted something like this. I think querying for a long list of _id just to update in an $in is perhaps slow too, so I tried to use an aggregation+merge
while (true) {
const record = db.records.findOne({ isArchived: false }, {_id: 1})
if (!record) {
print("No more records")
break
}
db.records.aggregate([
{ $match: { isArchived: false } },
{ $limit: 100 },
{
$project: {
_id: 1,
isArchived: {
$literal: true
},
updatedAt: {
$literal: new Date()
}
}
},
{
$merge: {
into: "records",
on: "_id",
whenMatched: "merge"
}
}
])
print("Done update")
}
But feel free to comment if this is better or worse that a bulk update with $in.