MongoDB Spring Data gives No property '$$value' found on class exception - mongodb

//Sample Collection
{
//fields
"roleList" : [ROLE_1, ROLE_2, ROLE_3],
"siteList" : [
{
"role" : ROLE_1,
//fields
},
{
"role" : ROLE_2,
//fields
},
]
}
//Expected Output
{
//fields
"roleDiff":[ROLE_3] // roleList subtracted by set of roles present in siteList
}
//Script that works with Studio 3t
db.getCollection("SAMPLE_COLLECTION").aggregate(
// Pipeline
[
// Stage 1
{
$project: {
"roleList":1,
"siteRoles":{$ifNull: [{$reduce:{
input:"$siteList",
initialValue:[],
in:{$setUnion:["$$value", {
"$split": ["$$this.role", " "]}]}
}
}, []]}
}
},
// Stage 2
{
$project: {
"roleDiff":{ $setDifference:["$roleList", "$siteRoles"]
}
}
},
],
// Options
{
}
// Created with Studio 3T, the IDE for MongoDB - https://studio3t.com/
);
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("siteList").withInitialValue(Collections.EMPTY_SET)
.reduce(SetOperators.SetUnion.arrayAsSet(StringOperators.valueOf("$$this.role").split(" ")).union(ArrayOperators.Reduce.Variable.VALUE.getTarget()));
ProjectionOperation projectionOperationOne = Aggregation.project().andInclude(/*Some fields,*/ "roleList").and(ConditionalOperators.ifNull(reduce).then(Collections.EMPTY_LIST)).as("siteRoles");
ProjectionOperation projectionOperationTwo = Aggregation.project().andInclude(/*Some fields*/).and(SetOperators.SetDifference.arrayAsSet("roleList").differenceTo("siteRoles")).as("roleDiff");
Aggregation aggregation = Aggregation.newAggregation(projectionOperationOne, projectionOperationTwo);
AggregationResults<SiteDiff> siteDiff = mongoTemplate.aggregate(aggregation, SampleCollection.class, SiteDiff.class);
The Java code above thows exception
org.springframework.data.mapping.context.InvalidPersistentPropertyPath: No property '$$value' found on class Did you mean: ?
The query works fine with Studio3T. My intention here is to get the difference between 2 String arrays "roleList" and "siteRoles" along with some other fields in the record.
"siteRoles" has to be derived from "siteList" which is an array of object. Since $addToSet works only with $group operation, I'm finding it difficult to extract the role from "siteList". I used reduce here combining it with setUnion. There problem was role is a string and I had to convert it to array. Only way I could find was to use $split and use " " as delimiter as I'm sure role will not have space.
Finally, the script worked in studio3t but the java version is not getting executed.

If you refer to a field in an embedded array, it should give you the entire array of those values.
The aggregation should only need 1 stage:
{$project: {roleDiff: { $setDifference: ["$roleList","$siteList.role"]}}}

Related

Pymongo - Query mongdb for first array elemnet by query of list of values

Given collection:
{
"_id" : "1.1000038",
"recomendation" : [
"1.6739718"
]
}
/* 2 */
{
"_id" : "1.1000069",
"recomendation" : [
"1.9185509",
"1.9051998",
"1.9034279",
"1.8288046",
"1.8152670",
"1.858775",
"1.6224229",
"1.4591674",
"1.3862464",
"1.3427739",
"1.3080062",
"1.3003608",
"1.1694619",
"1.1634683",
"1.1590664",
"1.1524146",
"1.754599",
"1.700837",
"1.763617"
]
}
I need to query the MongoDB for a list of values and get the first element of the list of values
here is the query by mongo syntax
db.getCollection('similar_articles').find({"_id":{$in:["1.1000069","1.1000038"]}})
I don't want to filter it on the python side because it's can be too big.
I didn't find any documentation on it
desire output:
Pandas DataFrame
_id recom
1.1000038 1.6739718
1.1000069 1.9185509
I don't know pymongo so well, but you need this query:
First $match by _ids into the arreay (this is like the find you have).
And later use $project to create the field recom (you can use "recomendation" to overwrite the existing field) and set the value as the first into the array.
db.collection.aggregate([
{
"$match": { "_id": { "$in": [ "1.1000069", "1.1000038" ] } }
},
{
"$project": { "recom": { "$arrayElemAt": [ "$recomendation", 0 ] } }
}
])
Example here
Looking the doumentation it seems you only need to copy and paste this query.

How to apply group by on nested document in MongoDB using MongoTemplate?

db.students.aggregate([
{ $unwind: "$details" },
{
$group: {
_id: {
sid: "$details.student._id",
statuscode: "$details.studentStatus.statusCode"
},
total: { $sum: 1 }
}
}
]);
The query is working fine and need to convert into mongo template.
Sample document:
{
"_id" : 59,
"details" : [
{
"student" : {
"_id" : "5d3145a8523a2e602e5e0200"
},
"studentStatus" : {
"statusCode" : 1
}
}
]
}
The Spring Data MongoTemplate code for the given aggregation is as follows.
Note that I have added a project stage before the group. This project is required; if the nested fields ("details.student._id" and "details.studentStatus.statusCode") are used directly within the group stage there are errors "FieldPath field names may not contain '.'." and could not be resolved (and this only happens when you use more than one field in the grouping).
The result is same as that of the aggregation you have provided. I have used the latest of Spring and MongoDB drivers with Java 8.
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "spr_test");
Aggregation agg = newAggregation(
unwind("details"),
project("_id")
.and("details.student._id").as("sid")
.and("details.studentStatus.statusCode").as("statuscode"),
group("sid", "statuscode")
.count().as("total")
);
AggregationResults<Document> aggResults = mongoOps.aggregate(agg, "students", Document.class);
aggResults.forEach(System.out::println);

Replace all occurences of a substring in a field in all documents in MongoDB

I have a collection Brands, I have to replace the path ./assets/ to D:/data/db/images , I have multiple occurrences of this in multiple documents. How do I acheive this using a query ?
Sample of each document
{"name":"Dell","images":["./assets/dell1.jpg","./assets/dell2.jpeg","./assets/dell3.jpg"],
"captions":["Yours is here","Easy as dell","Uniquely you"],
"logo":"./assets/dell4.png",
"details":{"headOffice":"Bangalore","address":"No.12/1, Divyashree Green, Koramangala Inner Ring Rd, Domlur, Bengaluru - 560071",
"phoneNumber":"(080) 28077000 ","website":"www.dell.com"}
}
You can use Aggregation Framework's $out operator to redirect the output of your aggregation to particular collection. If you specify the same collection name then it will replace existing collection.
To overwrite existing field you can use $addFields operator. Then you just have to remove the length of ./assets/ using $substr and concatenate that with your new prefix using $concat
db.Brands.aggregate([
{
$addFields: {
images: {
$map: {
input: "$images",
as: "image",
in: {
$concat: [ "D:/data/db/images", { $substr: [ "$$image", 8, { $strLenBytes: "$$image" } ] } ]
}
}
}
}
},
{ $out: "Brands" } //replaces existing collection
])
In MongoDB 3.2 you can run following script:
db.Brands.find().forEach(function(doc){
doc.images = doc.images.map(function(image){ return image.replace("./assets/","D:/data/db/images/") })
db.Brands.save(doc);
})

In mongodb know index of array element matched with $in operator?

I am using aggregation with mongoDB now i am facing a problem here, i am trying to match my documents which are present in my input array by using $in operator. Now i want to know the index of the lement from the input array now can anyone please tell me how can i do that.
My code
var coupon_ids = ["58455a5c1f65d363bd5d2600", "58455a5c1f65d363bd5d2601","58455a5c1f65d363bd5d2602"]
couponmodel.aggregate(
{ $match : { '_id': { $in : coupons_ids }} },
/* Here i want to know index of coupon_ids element that is matched because i want to perform some operation in below code */
function(err, docs) {
if (err) {
} else {
}
});
Couponmodel Schema
var CouponSchema = new Schema({
category: {type: String},
coupon_name: {type: String}, // this is a string
});
UPDATE-
As suggested by user3124885 that aggregation is not better in performance, can anyone please tell me the performance difference between aggregation and normal query in mongodb. And which one is better ??
Update-
I read this question on SO mongodb-aggregation-match-vs-find-speed. Here the user himself commented that both take same time, also by seeing vlad-z answer i think aggregation is better. Please if anyone of you have worked on mongodb Then please tell me what are your opinion about this.
UPDATE-
I used sample json data containing 30,000 rows and tried match with aggregation v/s find query aggregation got executed in 180 ms where find query took 220ms. ALso i ran $lookup it is also taking not much than 500ms so think aggregation is bit faster than normal query. Please correct me guys if any one of you have tried using aggregation and if not then why ??
UPDATE-
I read this post where user uses below code as a replacement of $zip SERVER-20163 but i am not getting how can i solve my problem using below code. So can anybody please tell me how can i use below code to solve my issue.
{$map: {
input: {
elt1: "$array1",
elt2: "$array2"
},
in: ["$elt1", "$elt2"]
}
Now can anyone please help me, it would be really be a great favor for me.
So say we have the following in the database collection:
> db.couponmodel.find()
{ "_id" : "a" }
{ "_id" : "b" }
{ "_id" : "c" }
{ "_id" : "d" }
and we wish to search for the following ids in the collections
var coupons_ids = ["c", "a" ,"z"];
We'll then have to build up a dynamic projection state so that we can project the correct indexes, so we'll have to map each id to its corresponding index
var conditions = coupons_ids.map(function(value, index){
return { $cond: { if: { $eq: ['$_id', value] }, then: index, else: -1 } };
});
Then we can then inject this in to our aggregation pipeline
db.couponmodel.aggregate([
{ $match : { '_id' : { $in : coupons_ids } } },
{ $project: { indexes : conditions } },
{ $project: {
index : {
$filter: {
input: "$indexes", as: "indexes", cond: { $ne: [ "$$indexes", -1 ] }
}
}
}
},
{ $unwind: '$index' }
]);
Running the above will now output each _id and it's corresponding index within the coupons_ids array
{ "_id" : "a", "index" : 1 }
{ "_id" : "c", "index" : 0 }
However we can also add more items in to the pipeline at the end and reference $index to get the current matched index.
I think you could do it in a faster way simply retrieving the array and search manually. Remember that aggregation don't give you performance.
//$match,$in,$and
$match:{
$and:[
{"uniqueID":{$in:["CONV0001"]}},
{"parentID":{$in:["null"]}},
]
}
}])

Group by array of document in Spring Mongo Db

How can I group by tagValue in Spring and MongoDb?
MongoDB Query :
db.feed.aggregate([
{ $group: { _id: "$feedTag.tagValue", number: { $sum : 1 } } },
{ $sort: { _id : 1 } }
])
How can I do the same thing in Spring MongoDB, may be using Aggregation method?
Sample document of feed collections:
{
"_id" : ObjectId("556846dd1df42d5d579362fd"),
"feedTag" : [
{
"tagName" : "sentiment",
"tagValue" : "neutral",
"modelName" : "sentiment"
}
],
"createdDate" : "2015-05-28"
}
To group by tagValue, since this is an array field, you need to apply the $unwind pipeline step before the group to split the array so that you can get the actual count:
db.feed.aggregate([
{
"$unwind": "$feedTag"
}
{
"$group": {
"_id": "$feedTag.tagValue",
"number": { "$sum" : 1 }
}
},
{ "$sort": { "_id" : 1 } }
])
The following is the equivalent example in Spring Data MongoDB:
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation agg = newAggregation(
unwind("feedTag"),
group("feedTag.tagValue").count().as("number"),
sort(ASC, "_id")
);
// Convert the aggregation result into a List
AggregationResults<Feed> results = mongoTemplate.aggregate(agg, "feed", Feed.class);
List<Feed> feedCount = results.getMappedResults();
From the above, a new aggregation object is created via the newAggregation static factory method which is passed a list of aggregation operations that define the aggregation pipeline of your Aggregation.
The firt step uses the unwind operation to generate a new document for each tag within the "feedTag" array.
In the second step the group operation defines a group for each embedded "feedTag.tagValue"-value for which the occurrence count is aggregated via the count aggregation operator.
As the third step, sort the resulting list of feedTag by their tagValue in ascending order via the sort operation.
Finally call the aggregate Method on the MongoTemplate to let MongoDB perform the actual aggregation operation with the created Aggregation as an argument.
Note that the input collection is explicitly specified as the "feed" parameter to the aggregate Method. If the name of the input collection is not specified explicitly, it is derived from the input-class passed as first parameter to the newAggreation Method.