I am trying to create an Aggregation in Spring data mongo. I am successfully able to create using OOB AggretionOperation implementations such as Lookup, Unwind etc.
However, when I try to create one using a custom AggregationOperation it gives PropertyReferenceException on lookup - "as" property item doesnt exist on the document type.
AggregationOperation lookup = new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aoc) {
return new Document("$lookup",new Document().parse("{ 'from' : 'items', 'localField' : 'item_id', 'foreignField' : '_id', 'as' : 'item'} "));
}
};
Aggregation aggregation = Aggregation.newAggregation(match(createCriteriaForRetrievingTimeSheets(queryParams)),
lookup,
....
return aggregation;
Related
I'm using Spring Data MongoDB and applying $lookup on a field with ObjectId value,
what I'm getting is the referenced document like I want but also the fields of that document are also populated, $lookup is applied again on the fields of the referenced document which causes the query to take a very long time to execute (around 5mins).
The query works fine using a raw query from a mongo client without spring data MongoDB, the lookup is applied only once and I get a document that has ObjectId value in the fields instead of those fields being populated too.
This is the raw query
{$match : {_id : new ObjectId("63282356bb1b311dd0e63f67")}},
{$lookup : {
"from":"paper",
"let":{
"userId":"$_id"
},
"pipeline":[
{$facet:{
sent:[
{$match:{
"$expr":{"$eq":["$to", "$$userId"]}
}}
],
}}
],
"as":"papers"
}}
])
And I'm translating this to spring data mongodb like this
Aggregation aggregation = Aggregation.newAggregation(
matchStage,
aggregationOperationContext -> {
Document lookupPipeline = new Document("$lookup",
new Document("from", "paper")
.append("let", new Document("userId", "$_id"))
.append("pipeline", Arrays.<Object> asList(
new Document("$facet",
new Document("sent",Arrays.<Object>asList(
new Document("$match", new Document("$expr",
new Document("$eq", Arrays.<Object>asList("$to", "$$userId"))
))
)
)
)
))
.append("as", "papers")
);
return aggregationOperationContext.getMappedObject(lookupPipeline);
}
);
I'm using the custom aggregation because using $lookup with the pipeline field is not supported in spring data MongoDB as of now
Also I'm using #DocumentReference
public class User{
#ReadOnlyProperty
#DocumentReference
private List<Paper> papers;
}
We have an aggregation pipeline setup in Spring Data Mongo like so:
MatchOperation matchStage = ...
Fields groupingFields = Fields.fields();
groupingFields.and(name1, target1);
groupingFields.and(name2, target2);
...
GroupOperation groupStage = Aggregation.group(groupingFields);
List<AggregationOperation> aggStages = new ArrayList<>();
aggStages.add(matchStage);
aggStages.add(groupStage);
Aggregation aggregation = Aggregation.newAggregation(aggStages);
Now, we want to be able to use aggregation over dates using Date operators in mongodb. This is fairly straightforward in mongodb, example below:
db.getCollection('Collection').aggregate([
{"$match": {"state": "XY"}},
{"$group": {
"_id": {
"city": "$city",
"dayOfYear": {"$dayOfYear": "$date"}
},
"totalProfit": {"$sum": "$profit"}
}}
])
My question is, how can I use the $dayOfYear operator in the Spring pipeline we have. Spring has support for DateOperators like DateOperators.DayOfWeek etc. but I am unable to incorporate it into the pipeline as we have it. How do I modify the groupStage so I can group by various date related parts as required?
For some operation I used to follow Bson document styles
#Autowired
private MongoTemplate mongoTemplate;
public List<Object> test() {
Aggregation aggregation = Aggregation.newAggregation(
match(Criteria.where("state").is("XY"))
p-> new Document("$group",
new Document("_id",
new Document("city","$city")
.append("dayOfYear",
new Document("$dayOfYear", "$date")
)
).append("totalProfit",
new Document("$sum","$$profit")
)
)
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
}
This should work, if the above aggregation you posted is working. You can refer Trick to convert.
I haven't tried DateOperator yet. But I'm curious how does it work. I will update if I get to know about it. Until then you have no blockers
I tried creating a Custom AggregationOperation based on https://github.com/krishnaiitd/learningJava/tree/master/spring-boot-sample-data-mongodb
When I used a custom aggregation in my aggregation for a lookup, it threw an exception saying the "as" field is not found on the entity.
If anybody has tried using custom AggregationOperation please share your code or let me know where I am going wrong.
Below is my code,
String lookup = "{ $lookup : { from: 'ITEM', localField : 'item_id', foreignField : '_id', as : 'item' } }";
TypedAggregation<Order> aggregation = Aggregation.newAggregation(Order.class,
new CustomAggregationOperation(lookup),
unwind("item", false));
The exception:
org.springframework.data.mapping.PropertyReferenceException: No property item found for type Order!
A TypedAggregation is a special Aggregation that holds information of the input aggregation type.
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/aggregation/TypedAggregation.html
That means that Spring will verify after each stage that your documents have not changed the structure.
Since you are trying transform original document, you need to use a standard Aggregation.
Aggregation aggregation = Aggregation.newAggregation(
new CustomAggregationOperation(lookup),
unwind("item", false)
);
List<Order> result = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Order.class), Order.class).getMappedResults();
I have a following query for my mongodb. How to translate it to the equivalent code in spring data mongodb:
db.getCollection('account').find({
colorList: {$elemMatch: {
$eq:"577b"
}
}
})
one of the account collection is shown as below:
{
"_id" : ObjectId("133b6ca05e7c058819ab6e6c"),
"fleetList" : [
"577b",
"123b"
]
}
Instead of using $elemMatch and $eq, you can use $in for your query too. This query makes exactly that your query does:
db.account.find({ "colorList": { $in: ["577b"] } });
And for spring-data-mongodb method for this query is:
List<Account> findByColorListIn(List<String> colorIds); //In your case colorIds list has one element only.
If you want to stick with your query:
#Query("{'colorList': {\$elemMatch: {\$eq: ?0}}}")
List<Account> findByColorList(String colorId)
I started to use MongoDB database in my application and for data access I have chosen Spring Data for MongoDB.
I skimmed API reference and documentation and I can see that there is map-reduce integration but what about aggregation framework? I can see that it supports group by operation, which would indicate that it supports $group operator judging from this: http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/, but what about other operators, are that not supported for now?
I am asking this question because I wanted to know what kind of integration with MongoDB Sping Data provides so I know what to expect, so to speak.
Spring Data 1.3.0.RC1 is available and it does support the aggregation framework.
For example:
The shell aggregation comand:
db.eft_transactions.aggregate(
{$match:
{
service:"EFT",
source:"MARKUP",
}
},
{$group:
{
_id:"$card_acceptor_id",
tran_count:{$sum:1},
amount_sum:{$sum:"$amount"}
}
}
)
is run like this from java:
AggregationOperation match = Aggregation.match(Criteria.where("service").is("EFT").and("source").is("MARKUP"));
AggregationOperation group = Aggregation.group("card_acceptor").and("amount_sum").sum("amount").and("tran_count").count();
Aggregation aggregation = newAggregation(match, group);
AggregationResults<StoreSummary> result = this.mongoTemplate.aggregate(aggregation, "eft_transactions", StoreSummary.class);
The documentation is here
NOTE: We recently had to switch to using the BUILD-SNAPSHOT build of version 1.3.0. This change necessitated the change to 2 of the above lines which have changed to:
AggregationOperation group = Aggregation.group("card_acceptor").sum("amount").as("amount_sum").count().as("tran_count");
Aggregation aggregation = Aggregation.newAggregation(match, group);
The Spring Data MongoOperations.group() method is mapped to db.collection.group() MongoDB command and not the $group aggregation function. Currently there is no support in Spring Data MongoDB for aggregation framework. Map reduce, as you have mentioned, is supported though
Aggregation aggregation = newAggregation(
match(Criteria.where("salesyear").is(year)),
group("brand","salesyear").sum("numberOfCars").as("total"),
sort(Sort.Direction.ASC, previousOperation(), "brand")
);
Here is how to get the sum of a particular field.
private Map<String, Long> getTotalMap(){
/*
db.pDSSummaryModel.aggregate([{
$group: {
_id: null,
total: {
$sum: '$totalUniqueCustomerCount'
}
}
}])
*/
Aggregation aggregations = newAggregation(
group("null").sum("totalUniqueUserCount").as("userTotal")
.sum("totalUniqueCustomerCount").as("customerTotal"),
project("customerTotal", "userTotal")
);
AggregationResults<DBObject> results = mongoTemplate.aggregate(aggregations, "pDSSummaryModel", DBObject.class);
List<DBObject> fieldList = results.getMappedResults();
Map<String, Long> map = new HashMap<>();
if(fieldList != null && !fieldList.isEmpty()) {
for(DBObject db: fieldList){
map.put("userTotal", parseLong(db.get("userTotal").toString()));
map.put("customerTotal", parseLong(db.get("customerTotal").toString()));
}
}
return map;
}