I am making a migration script where I need to update a status value on all documents where the items in an array has the same value
Data structure
[
{
_id: 'asdasd',
status: 'active',
approvers: [
{status: 'approved'},
{status: 'approved'}
]
},
{
_id: 'fghfgh',
status: 'active',
approvers: [
{status: 'approved'},
{status: 'awaiting_approval'},
]
}
]
So, in this case in want to update all documents to have status 'completed' where all approvers has status 'approved'
I haven't found a good way how to create a filter like this.
What I've currently tried to do is:
db.getCollection("assignmentRequest").aggregate([
{
$match: {
'approver.0.status': {$exists:true}
}
},
{
$project: {
_id: 0,
approver: 1,
status: 1,
noOfApprovers: { $cond: { if: { $isArray: "$approvers" }, then: { $size: "$approvers" }, else: 0}},
noOfApproversThatHasApproved: {
$size: {$filter: {
'input': '$approvers',
'as': 'approver',
'cond': {
'$and': [
{
$eq: ['$$approver.status', 'approved']
}
]
}
}}
},
},
},
{
$match: {$expr: { $eq: ["$noOfApprovers", "$noOfApproversThatHasApproved"] } }
},
{
$set: {'status': 'completed'}
},
{
$project: {_id:1, status:1 }
},
{
$merge: {into: 'assignmentRequests_copy', on: '_id', whenMatched: "replace" }
}])
The filter works, but I can't get the status to update. I'm sure there are plenty of things worng with my query, but I feel like I am going down the wrong path and that there must be a simpler way of achieving this. Any pointers or help would be highly appreciated.
Edit:
After the update I want the documents to look like this:
{
_id: 'asdasd',
status: 'completed',
approvers: [
{status: 'approved'},
{status: 'approved'}
]
},
{
_id: 'fghfgh',
status: 'active',
approvers: [
{status: 'approved'},
{status: 'awaiting_approval'},
]
}
]
Here's one way you could do it by using a pipeline in the update.
db.collection.update({
"approvers.status": "approved"
},
[
{
"$set": {
"status": {
"$cond": [
{
"$reduce": {
"input": "$approvers",
"initialValue": true,
"in": {
"$and": [
"$$value",
{"$eq": ["$$this.status", "approved"]}
]
}
}
},
"completed",
"$status"
]
}
}
}
],
{"multi": true}
)
Try it on mongoplayground.net.
Related
I need the first part of $or (or equivalent query) to be resolved first and to make sure that the first query is always part of the result.
Must use query, not aggregation.
[
{ "docId": "x1" },
{ "docId": "x2" },
{ "docId": "x3" },
{ "docId": "x4" },
{ "docId": "x5" },
...
{ "docId": "xn" },
]
Query:
{
'$or': [ { docId: 'x434' },{} ],
}
I need x434 to be part of the query result, regardless of all other results.
Expected result:
[
{ docId: 'x434' },
{ docId: 'x12' },
{ docId: 'x1' },
...
]
Return:
[
{ docId: 'xn' },
{ docId: 'xn' },
{ docId: 'xn' },
...
]
Results by x434 is not always returned
I tried $or and $and queries but nothing worked. I tried regex too
{
'$or': [ { docId: 'x434' },{} ],
}
So the solution can only be an aggregation:
$match: {
'$or': [ { docId: 'x434' },{} ],
},
$addFields: {
order: {
$cond: [
{
$in: [
"$docId",
['x434']
]
},
0,
1
]
}
},
$sort: {
order: 1
},
$limit: 20
Result:
{ docId: 'x434' },
{ docId: 'x12' },
{ docId: 'x1' },
...
]```
A straightforward solution can use $facet:
db.collection.aggregate([
{$facet: {
allOther: [{$match: {docId: {$ne: "x434"}}}, {$limit: 19}],
wanted: [{$match: {docId: "x434"}}]
}},
{$project: {data: {$concatArrays: ["$wanted", "$allOther"]}}},
{$unwind: "$data"},
{$replaceRoot: {newRoot: "$data"}}
])
See how it works on the playground example
You can use $unionWith. It's behaviour is similar to UNION ALL in SQL so you can persist x434 at the start. Remember to exclude x434 in the $unionWith pipeline to avoid duplicate if needed
db.collection.aggregate([
{
$match: {
"docId": "x434"
}
},
{
"$unionWith": {
"coll": "collection",
"pipeline": [
// exclude x434 to avoid duplicate
{
$match: {
"docId": {
$ne: "x434"
}
}
}// put your other queries here
]
}
}
])
Mongo Playground
In below example, looking for new partner suggestions for user abc. abc has already sent a request to 123 so that can be ignored. rrr has sent request to abc but rrr is in the fromUser field so rrr is still a valid row to be shown as suggestion to abc
I have two collections:
User collection
[
{
_id: "abc",
name: "abc",
group: 1
},
{
_id: "xyz",
name: "xyyy",
group: 1
},
{
_id: "123",
name: "yyy",
group: 1
},
{
_id: "rrr",
name: "tttt",
group: 1
},
{
_id: "eee",
name: "uuu",
group: 1
}
]
Partnership collection (if users have already partnered)
[
{
_id: "abc_123",
fromUser: "abc",
toUser: "123"
},
{
_id: "rrr_abc",
fromUser: "rrr",
toUser: "abc"
},
{
_id: "xyz_rrr",
fromUser: "xyz",
toUser: "rrr"
}
]
My query below excludes the user rrr but it should not because its not listed in toUser field in the partnership collection corresponding to the user abc.
How to modify this query to include user rrr in this case?
db.users.aggregate([
{
$match: {
group: 1,
_id: {
$ne: "abc"
}
}
},
{
$lookup: {
from: "partnership",
let: {
userId: "$_id"
},
as: "prob",
pipeline: [
{
$set: {
users: [
"$fromUser",
"$toUser"
],
u: "$$userId"
}
},
{
$match: {
$expr: {
$and: [
{
$in: [
"$$userId",
"$users"
]
},
{
$in: [
"abc",
"$users"
]
}
]
}
}
}
]
}
},
{
$match: {
"prob.0": {
$exists: false
}
}
},
{
$sample: {
size: 1
}
},
{
$unset: "prob"
}
])
https://mongoplayground.net/p/utGMeHFRGmt
Your current query does not allow creating an existing connection regardless of the connection direction. If the order of the connection is important use:
db.users.aggregate([
{$match: {
group: 1,
_id: {$ne: "abc"}
}
},
{$lookup: {
from: "partnership",
let: { userId: {$concat: ["abc", "_", "$_id"]}},
as: "prob",
pipeline: [{$match: {$expr: {$eq: ["$_id", "$$userId"]}}}]
}
},
{$match: {"prob.0": {$exists: false}}},
{$sample: {size: 1}},
{$unset: "prob"}
])
See how it works on the playground example
For MongoDB 5 and later, I'd propose the following aggregation pipeline:
db.users.aggregate([
{
$match: {
group: 1,
_id: {
$ne: "abc"
}
}
},
{
$lookup: {
from: "partnership",
as: "prob",
localField: "_id",
foreignField: "toUser",
pipeline: [
{
$match: {
fromUser: "abc",
}
}
]
}
},
{
$match: {
"prob.0": {
$exists: false
}
}
},
{
$unset: "prob"
}
])
The following documents are returned (full result without the $sample stage):
[
{
"_id": "eee",
"group": 1,
"name": "uuu"
},
{
"_id": "rrr",
"group": 1,
"name": "tttt"
},
{
"_id": "xyz",
"group": 1,
"name": "xyyy"
}
]
The main difference is that the lookup connects the collections by the toUser field (see localField, foreignField) and uses a minimal pipeline to restrict the results further to only retrieve the requests from the current user document to "abc".
See this playground to test.
When using MongoDB < 5, you cannot use localField and foreignField to run the pipeline only on a subset of the documents in the * from*
collection. To overcome this, you can use this aggregation pipeline:
db.users.aggregate([
{
$match: {
group: 1,
_id: {
$ne: "abc"
}
}
},
{
$lookup: {
from: "partnership",
as: "prob",
let: {
userId: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$fromUser",
"abc"
]
},
{
$eq: [
"$toUser",
"$$userId"
]
}
]
}
}
}
]
}
},
{
$match: {
"prob.0": {
$exists: false
}
}
},
{
$unset: "prob"
}
])
The results are the same as for the upper pipeline.
See this playground to test.
For another, another way, this query starts from the partnership collection, finds which users to exclude, and then does a "$lookup" for everybody else. The remainder is just output formatting, although it looks like you may want to add a "$sample" stage at the end.
db.partnership.aggregate([
{
"$match": {
"fromUser": "abc"
}
},
{
"$group": {
"_id": null,
"exclude": {"$push": "$toUser" }
}
},
{
"$lookup": {
"from": "users",
"let": {
"exclude": {"$concatArrays": [["abc"], "$exclude"]
}
},
"pipeline": [
{
"$match": {
"$expr": {
"$not": {"$in": ["$_id", "$$exclude"]}
}
}
}
],
"as": "output"
}
},
{
"$project": {
"_id": 0,
"output": 1
}
},
{"$unwind": "$output"},
{"$replaceWith": "$output"}
])
Try it on mongoplayground.net.
Given documents such as
{
_id: 'abcd',
userId: '12345',
activities: [
{ status: 'login', timestamp: '10000001' },
{ status: 'logout', timestamp: '10000002' },
{ status: 'login', timestamp: '10000003' },
{ status: 'logout', timestamp: '10000004' },
]
}
I am trying to create a pipeline such as all users that have their latest login/logout activities recorded between two timestamps will be returned. For example, if the two timestamp values are between 10000002 and 10000003, the expected document should be
{
_id: 'abcd',
userId: '12345',
login: '10000003',
logout: '10000002'
}
Of if the two timestamp values are between -1 and 10000001, the expected document should be :
{
_id: 'abcd',
userId: '12345',
login: '10000001',
logout: null
}
Etc.
I know it has to do with aggregations, and I need to $unwind, etc., but I'm not sure about the rest, namely evaluating two fields from the same document array
You can try below aggregation:
db.col.aggregate([
{
$unwind: "$activities"
},
{
$match: {
$and: [
{ "activities.timestamp": { $gte: "10000001" } },
{ "activities.timestamp": { $lte: "10000002" } }
]
}
},
{
$sort: {
"activities.timestamp": -1
}
},
{
$group: {
_id: "$_id",
userId: { $first: "$userId" },
activities: { $push: "$activities" }
}
},
{
$addFields: {
login: { $arrayElemAt: [ { $filter: { input: "$activities", as: "a", cond: { $eq: [ "$$a.status", "login" ] } } } , 0 ] },
logout: { $arrayElemAt: [ { $filter: { input: "$activities", as: "a", cond: { $eq: [ "$$a.status", "logout" ] } } } , 0 ] }
}
},
{
$project: {
_id: 1,
userId: 1,
login: { $ifNull: [ "$login.timestamp", null ] },
logout: { $ifNull: [ "$logout.timestamp", null ] }
}
}
])
We need to use $unwind + $sort + $group to make sure that our activities will be sorted by timestamp. After $unwind you can use $match to apply filtering condition. Then you can use $filter with $arrayElemAt to get first (latest) value of filtered array. In the last $project you can explicitly use $ifNull (otherwise JSON key will be skipped if there's no value)
You can use below aggregation
Instead of $unwind use $lte and $gte with the $fitler aggregation.
db.collection.aggregate([
{ "$project": {
"userId": 1,
"login": {
"$max": {
"$filter": {
"input": "$activities",
"cond": {
"$and": [
{ "$gte": ["$$this.timestamp", "10000001"] },
{ "$lte": ["$$this.timestamp", "10000004"] },
{ "$lte": ["$$this.status", "login"] }
]
}
}
}
},
"logout": {
"$max": {
"$filter": {
"input": "$activities",
"cond": {
"$and": [
{ "$gte": ["$$this.timestamp", "10000001"] },
{ "$lte": ["$$this.timestamp", "10000004"] },
{ "$lte": ["$$this.status", "logout"] }
]
}
}
}
}
}}
])
I am trying to aggregate a batch of documents. There are two fields in the documents I would like to $push. However, lets say they are "_id" and "A" fields, I only want $push "_id" and "A" if "A" is $gt 0.
I tried two approaches.
First one.
db.collection.aggregate([{
"$group":{
"field": {
"$push": {
"$cond":[
{"$gt":["$A", 0]},
{"id": "$_id", "A":"$A"},
null
]
}
},
"secondField":{"$push":"$B"}
}])
But this will push a null value to "field" and I don't want it.
Second one.
db.collection.aggregate([{
"$group":
"field": {
"$cond":[
{"$gt",["$A", 0]},
{"$push": {"id":"$_id", "A":"$A"}},
null
]
},
"secondField":{"$push":"$B"}
}])
The second one simply doesn't work...
Is there a way to skip the $push in else case?
ADDED:
Expected documents:
{
"_id":objectid(1),
"A":2,
"B":"One"
},
{
"_id":objectid(2),
"A":3,
"B":"Two"
},
{
"_id":objectid(3),
"B":"Three"
}
Expected Output:
{
"field":[
{
"A":"2",
"_id":objectid(1)
},
{
"A":"3",
"_id":objectid(2)
},
],
"secondField":["One", "Two", "Three"]
}
You can use "$$REMOVE":
This system variable was added in version 3.6 (mongodb docs)
db.collection.aggregate([{
$group:{
field: {
$push: {
$cond:[
{ $gt: ["$A", 0] },
{ id: "$_id", A:"$A" },
"$$REMOVE"
]
}
},
secondField:{ $push: "$B" }
}
])
In this way you don't have to filter nulls.
This is my answer to the question after reading the post suggested by #Veeram
db.collection.aggregate([{
"$group":{
"field": {
"$push": {
"$cond":[
{"$gt":["$A", 0]},
{"id": "$_id", "A":"$A"},
null
]
}
},
"secondField":{"$push":"$B"}
},
{
"$project": {
"A":{"$setDifference":["$A", [null]]},
"B":"$B"
}
}])
One more option is to use $filter operator:
db.collection.aggregate([
{
$group : {
_id: null,
field: { $push: { id: "$_id", A : "$A"}},
secondField:{ $push: "$B" }
}
},
{
$project: {
field: {
$filter: {
input: "$field",
as: "item",
cond: { $gt: [ "$$item.A", 0 ] }
}
},
secondField: "$secondField"
}
}])
On first step you combine your array and filter them on second step
$group: {
_id: '$_id',
tasks: {
$addToSet: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.id', ''],
},
'',
],
},
then: '$$REMOVE',
else: {
id: '$tasks.id',
description: '$tasks.description',
assignee: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.assignee._id', ''],
},
'',
],
},
then: undefined,
else: {
id: '$tasks.assignee._id',
name: '$tasks.assignee.name',
thumbnail: '$tasks.assignee.thumbnail',
status: '$tasks.assignee.status',
},
},
},
},
},
},
},
}
I've been trying every method I found on SO with no success. Trying
to accomplish a seemingly simple task (very easy with json/lodash for example) in MongoDB..
I have a collection:
db.users >
[
{
_id: 'userid',
profile: {
username: 'abc',
tests: [
{
_id: 'testid',
meta: {
category: 'math',
date: '9/2/2017',
...
}
questions: [
{
type: 'add',
correct: true,
},
{
type: 'subtract',
correct: true,
},
{
type: 'add',
correct: false,
},
{
type: 'multiply',
correct: false,
},
]
},
...
]
}
},
...
]
I want to end up with an array grouped by question type:
[
{
type: 'add',
correct: 5,
wrong: 3,
},
{
type: 'subtract',
correct: 4,
wrong: 9
}
...
]
I've tried different variations of aggregate, last one is:
db.users.aggregate([
{ $match: { 'profile.tests.meta.category': 'math' }},
{
$project: {
tests: {
$filter: {
input: "$profile.tests",
as: "test",
cond: { $eq: ['$$test.meta.category', 'math'] }
}
}
}
},
{
$project: {
question: "$tests.questions"
}
},
{ $unwind: "$questions"},
])
Also tried adding $group at the end of the pipeline:
{
$group:
{
_id: '$questions.type',
res: {
$addToSet: { correct: {$eq:['$questions.chosenAnswer', '$questions.answers.correct'] }
}
}
}
No variation gave me what I'm looking for, I'm sure I'm missing a core concept, I've looked over the documentation and couldn't figure it out.. what I'm basically looking for is a flatMap to extract away all the questions of all users and group them by type.
If anyone can lead me in the right direction, I'll greatly appreciate it :) thx. (Also, I'm using Meteor, so any query has to work in Meteor mongo)
You can try below aggregation in 3.4.
$filter to filter math categories with $map to project questions array in each matching category followed by $reduce and $concatArrays to get all questions into single array for all matching categories.
$unwind questions array and $group by type and $sum to compute correct and wrong count.
db.users.aggregate([
{
"$match": {
"profile.tests.meta.category": "math"
}
},
{
"$project": {
"questions": {
"$reduce": {
"input": {
"$map": {
"input": {
"$filter": {
"input": "$profile.tests",
"as": "testf",
"cond": {
"$eq": [
"$$testf.meta.category",
"math"
]
}
}
},
"as": "testm",
"in": "$$testm.questions"
}
},
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this"
]
}
}
}
}
},
{
"$unwind": "$questions"
},
{
"$group": {
"_id": "$questions.type",
"correct": {
"$sum": {
"$cond": [
{
"$eq": [
"$questions.correct",
true
]
},
1,
0
]
}
},
"wrong": {
"$sum": {
"$cond": [
{
"$eq": [
"$questions.correct",
false
]
},
1,
0
]
}
}
}
}
])