I'm using Spring Data MongoDB and applying $lookup on a field with ObjectId value,
what I'm getting is the referenced document like I want but also the fields of that document are also populated, $lookup is applied again on the fields of the referenced document which causes the query to take a very long time to execute (around 5mins).
The query works fine using a raw query from a mongo client without spring data MongoDB, the lookup is applied only once and I get a document that has ObjectId value in the fields instead of those fields being populated too.
This is the raw query
{$match : {_id : new ObjectId("63282356bb1b311dd0e63f67")}},
{$lookup : {
"from":"paper",
"let":{
"userId":"$_id"
},
"pipeline":[
{$facet:{
sent:[
{$match:{
"$expr":{"$eq":["$to", "$$userId"]}
}}
],
}}
],
"as":"papers"
}}
])
And I'm translating this to spring data mongodb like this
Aggregation aggregation = Aggregation.newAggregation(
matchStage,
aggregationOperationContext -> {
Document lookupPipeline = new Document("$lookup",
new Document("from", "paper")
.append("let", new Document("userId", "$_id"))
.append("pipeline", Arrays.<Object> asList(
new Document("$facet",
new Document("sent",Arrays.<Object>asList(
new Document("$match", new Document("$expr",
new Document("$eq", Arrays.<Object>asList("$to", "$$userId"))
))
)
)
)
))
.append("as", "papers")
);
return aggregationOperationContext.getMappedObject(lookupPipeline);
}
);
I'm using the custom aggregation because using $lookup with the pipeline field is not supported in spring data MongoDB as of now
Also I'm using #DocumentReference
public class User{
#ReadOnlyProperty
#DocumentReference
private List<Paper> papers;
}
Related
We have an aggregation pipeline setup in Spring Data Mongo like so:
MatchOperation matchStage = ...
Fields groupingFields = Fields.fields();
groupingFields.and(name1, target1);
groupingFields.and(name2, target2);
...
GroupOperation groupStage = Aggregation.group(groupingFields);
List<AggregationOperation> aggStages = new ArrayList<>();
aggStages.add(matchStage);
aggStages.add(groupStage);
Aggregation aggregation = Aggregation.newAggregation(aggStages);
Now, we want to be able to use aggregation over dates using Date operators in mongodb. This is fairly straightforward in mongodb, example below:
db.getCollection('Collection').aggregate([
{"$match": {"state": "XY"}},
{"$group": {
"_id": {
"city": "$city",
"dayOfYear": {"$dayOfYear": "$date"}
},
"totalProfit": {"$sum": "$profit"}
}}
])
My question is, how can I use the $dayOfYear operator in the Spring pipeline we have. Spring has support for DateOperators like DateOperators.DayOfWeek etc. but I am unable to incorporate it into the pipeline as we have it. How do I modify the groupStage so I can group by various date related parts as required?
For some operation I used to follow Bson document styles
#Autowired
private MongoTemplate mongoTemplate;
public List<Object> test() {
Aggregation aggregation = Aggregation.newAggregation(
match(Criteria.where("state").is("XY"))
p-> new Document("$group",
new Document("_id",
new Document("city","$city")
.append("dayOfYear",
new Document("$dayOfYear", "$date")
)
).append("totalProfit",
new Document("$sum","$$profit")
)
)
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
}
This should work, if the above aggregation you posted is working. You can refer Trick to convert.
I haven't tried DateOperator yet. But I'm curious how does it work. I will update if I get to know about it. Until then you have no blockers
I tried creating a Custom AggregationOperation based on https://github.com/krishnaiitd/learningJava/tree/master/spring-boot-sample-data-mongodb
When I used a custom aggregation in my aggregation for a lookup, it threw an exception saying the "as" field is not found on the entity.
If anybody has tried using custom AggregationOperation please share your code or let me know where I am going wrong.
Below is my code,
String lookup = "{ $lookup : { from: 'ITEM', localField : 'item_id', foreignField : '_id', as : 'item' } }";
TypedAggregation<Order> aggregation = Aggregation.newAggregation(Order.class,
new CustomAggregationOperation(lookup),
unwind("item", false));
The exception:
org.springframework.data.mapping.PropertyReferenceException: No property item found for type Order!
A TypedAggregation is a special Aggregation that holds information of the input aggregation type.
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/aggregation/TypedAggregation.html
That means that Spring will verify after each stage that your documents have not changed the structure.
Since you are trying transform original document, you need to use a standard Aggregation.
Aggregation aggregation = Aggregation.newAggregation(
new CustomAggregationOperation(lookup),
unwind("item", false)
);
List<Order> result = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Order.class), Order.class).getMappedResults();
I am trying to create an Aggregation in Spring data mongo. I am successfully able to create using OOB AggretionOperation implementations such as Lookup, Unwind etc.
However, when I try to create one using a custom AggregationOperation it gives PropertyReferenceException on lookup - "as" property item doesnt exist on the document type.
AggregationOperation lookup = new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aoc) {
return new Document("$lookup",new Document().parse("{ 'from' : 'items', 'localField' : 'item_id', 'foreignField' : '_id', 'as' : 'item'} "));
}
};
Aggregation aggregation = Aggregation.newAggregation(match(createCriteriaForRetrievingTimeSheets(queryParams)),
lookup,
....
return aggregation;
I have been trying for a while to extract the insertion date of a mongodb document and add it as a new field to the same document.
I'm trying to do it using the mongo and mongo shell aggregation framework without getting good results.
Here is my query
db.getCollection('my_collection').aggregate(
[
{
$match: {
MY QUERY CRITERIA
}
},
{
$addFields: { "insertTime": "$_id.getTimestamp()" }
}
]
)
I am trying to extracr insertion time from _id using the function getTimestamp() but for sure there is somtehing about aggregation framework syntax that I am missing because I can not do what I am trying to do in my query.
This works perfect:
ObjectId("5c34f746ccb26800019edd53").getTimestamp()
ISODate("2019-01-08T19:17:26Z")
But this does not work at all:
"$_id.getTimestamp()"
What I am missing?
Thanks in advance
I tried to implement an aggregation with the simple sum operation using spring-data-mongodb -> 1.10.11.RELEASE
I tried the following query
db.transaction.aggregate([{
$group: {
_id:"null",
netBalance: {
$sum: "$netBalance"
},
referalBalance: {
$sum: "$referalBalance"
}
}
}])
And the output on my terminal is
{ "_id" : "null", "netBalance" : 587432, "referalBalance" : 2940 }
When I tried same query, using spring mongodata
Aggregation aggregation =
Aggregation.newAggregation(
Aggregation.group("netBalance")
.sum("netBalance")
.as("netBalance"));
The result is not same as the terminal output, also I am not able to add a second field on the query. How can I modify the spring mongodata call to have the same query?
with spring you're grouping by netBalance instead of null in shell. To reproduce, just leave group param empty (null will throw an error)
Then you can apply sum on second field .
Try this code (not tested, but must work):
Aggregation aggregation =
Aggregation.newAggregation(
Aggregation.group()
.sum("netBalance").as("netBalance")
.sum("referalBalance").as("referalBalance"));