I tried creating a Custom AggregationOperation based on https://github.com/krishnaiitd/learningJava/tree/master/spring-boot-sample-data-mongodb
When I used a custom aggregation in my aggregation for a lookup, it threw an exception saying the "as" field is not found on the entity.
If anybody has tried using custom AggregationOperation please share your code or let me know where I am going wrong.
Below is my code,
String lookup = "{ $lookup : { from: 'ITEM', localField : 'item_id', foreignField : '_id', as : 'item' } }";
TypedAggregation<Order> aggregation = Aggregation.newAggregation(Order.class,
new CustomAggregationOperation(lookup),
unwind("item", false));
The exception:
org.springframework.data.mapping.PropertyReferenceException: No property item found for type Order!
A TypedAggregation is a special Aggregation that holds information of the input aggregation type.
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/aggregation/TypedAggregation.html
That means that Spring will verify after each stage that your documents have not changed the structure.
Since you are trying transform original document, you need to use a standard Aggregation.
Aggregation aggregation = Aggregation.newAggregation(
new CustomAggregationOperation(lookup),
unwind("item", false)
);
List<Order> result = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Order.class), Order.class).getMappedResults();
Related
I'm using Spring Data MongoDB and applying $lookup on a field with ObjectId value,
what I'm getting is the referenced document like I want but also the fields of that document are also populated, $lookup is applied again on the fields of the referenced document which causes the query to take a very long time to execute (around 5mins).
The query works fine using a raw query from a mongo client without spring data MongoDB, the lookup is applied only once and I get a document that has ObjectId value in the fields instead of those fields being populated too.
This is the raw query
{$match : {_id : new ObjectId("63282356bb1b311dd0e63f67")}},
{$lookup : {
"from":"paper",
"let":{
"userId":"$_id"
},
"pipeline":[
{$facet:{
sent:[
{$match:{
"$expr":{"$eq":["$to", "$$userId"]}
}}
],
}}
],
"as":"papers"
}}
])
And I'm translating this to spring data mongodb like this
Aggregation aggregation = Aggregation.newAggregation(
matchStage,
aggregationOperationContext -> {
Document lookupPipeline = new Document("$lookup",
new Document("from", "paper")
.append("let", new Document("userId", "$_id"))
.append("pipeline", Arrays.<Object> asList(
new Document("$facet",
new Document("sent",Arrays.<Object>asList(
new Document("$match", new Document("$expr",
new Document("$eq", Arrays.<Object>asList("$to", "$$userId"))
))
)
)
)
))
.append("as", "papers")
);
return aggregationOperationContext.getMappedObject(lookupPipeline);
}
);
I'm using the custom aggregation because using $lookup with the pipeline field is not supported in spring data MongoDB as of now
Also I'm using #DocumentReference
public class User{
#ReadOnlyProperty
#DocumentReference
private List<Paper> papers;
}
I'm using Mongo Aggregation with Spring Data to retrieve data from 4 collections.
My problem is that I want to add a filter based on the createdAt on 3 of these collections.
Here the examples:
Animal
{"_id":123,"name":"Lucky","age":3,createdAt:"2022-02-01 10:00:00.000Z"}
{"_id":456,"name":"Joe","age":5,createdAt:"2022-02-10 20:03:00.000Z"}
Cat
{"idAnimal":123,"toy":6,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"toy":2,createdAt:"2022-02-10 20:05:00.000Z"}
Pet
{"idAnimal":123,"meal":3,"medicine":false,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"meal":4,"medicine":true,createdAt:"2022-02-10 20:05:00.000Z"}
What I mean to do is to get all the Animals with a Pet collection created gte(2022-03-01). The expected result would be the first cat, Lucky.
In my code I tried this
final List<Criteria> criteria = new ArrayList<>();
criteria.add(new Criteria().orOperator(
Criteria.where("createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Cat.createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Pet.createdAt").gte("2022-03-01").lte("2022-03-02")
));
and my Aggregation set up is:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.lookup("cat", "_id", "idAnimal", "Cat"),
Aggregation.lookup("pet", "_id", "idAnimal", "Pet"),
Aggregation.unwind("$_id"),
Aggregation.match(new Criteria().andOperator(criteria.toArray(new Criteria[0]))
)
);
I tried to swap the criteria, querying the Pet first, but didn't go as expected. When I run this, I get no result.
Do you have any tips? Is it possibile to execute an aggregation on Dates over multiple collections?
Thank you in advance!
I am trying to create an Aggregation in Spring data mongo. I am successfully able to create using OOB AggretionOperation implementations such as Lookup, Unwind etc.
However, when I try to create one using a custom AggregationOperation it gives PropertyReferenceException on lookup - "as" property item doesnt exist on the document type.
AggregationOperation lookup = new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aoc) {
return new Document("$lookup",new Document().parse("{ 'from' : 'items', 'localField' : 'item_id', 'foreignField' : '_id', 'as' : 'item'} "));
}
};
Aggregation aggregation = Aggregation.newAggregation(match(createCriteriaForRetrievingTimeSheets(queryParams)),
lookup,
....
return aggregation;
I need a little help with an aggregation pipeline in morphia. This is what I have in mongo:
db.transacciones.aggregate([
{$match:
{"$and":[
{"usuario._id": "xxx#gmail.com"},
{"status":{"$ne":"ERROR"}}]}},
{$sort:{"date": -1}},
{$group:{"ordenId", "transaction" : {$first:"$$ROOT"}}},
{$replaceRoot:{newRoot:"$transaction"}}])
Ok, this is what I have in morphia:
Query<TransaccionP2P> match = ds.getQueryFactory().createQuery(ds);
match.and(match.criteria("usuario._id").equal(usuarioId),
match.criteria("status").notEqual(Status.ST_ERROR));
Sort sort = Sort.descending("date");
Group group = grouping("transaction", Accumulator.accumulator("$first", (Object)"$$ROOT"));
Projection[] projections = {
Projection.projection("_id", "$transaction._id"),
...
};
AggregationPipeline pipeline = ds.createAggregation(TransaccionP2P.class)
.match(match)
.sort(sort)
.group("orden._id", group)
.project(projections);
Iterator<TransaccionP2P> iterator = pipeline.aggregate(TransaccionP2P.class);
List<TransaccionP2P> transacciones = new ArrayList<>();
iterator.forEachRemaining(transacciones::add);
I have managed to construct this pipeline with the stackoverflow articles I found because I couldn't find a good example in Morphia documentation but I have these problems:
Morphia doesn't seem to have support for $replaceRoot so I have to create the projection field by field.
The transaction variable in the group aggregate is not recognized in the projection step.
Here is the projection exception:
com.mongodb.MongoCommandException: Command failed with error 17276: 'Use of undefined variable: transaction'.
I haven't managed to find the correct way of making the projection for 'transaction' to be recognized.
Can you help me?
Thanks
I am new to MongoDB and Spring mongotemplate. I would like to build a query using mongotemplate whose equivalent in Postgres would be
select * from feedback
where feedback.outletId in (
select outletId from feedback
where feedback.createdOn >= '2013-05-03'::date
)
Is this even possible in MongoDB?
Well there is no concept of inner queries in MongoDB so basically it can be achieved by 2 queries but probably you already know that and want a 'better' solution. Since you asked if it is possible, I think it can be achieved by aggregation however that can be tricky.
db.feedback.aggregate([
{$project : {
'outletId' : 1,
'feedback._id' : '$_id',
'feedback.createdOn' : '$createdOn',
'feedback.a' : '$a'
}},
{$group : {
_id : $outletId,
feedbacks : {$addToSet : '$feedback'}
}},
{$match : {
'feedbacks.createdOn' : {
$gte : ISODate('2013-05-03')}
}},
{$unwind : '$feedback'}]);
You can add one more $project stage in the end to turn child object into values as it was in the document. I know it doesn't look pretty, but I would explain it stage by stage,
first we project a document with putting all the needed field inside a child field called feedback,
in second stage we grouped it by outletId and put all the child feedback into an array named feedbacks,(so for each outletid we get all feedbacks).
in third stage we use $match to filter out where there is not even a single feedback in array which createdOn field is greater than set date,
after those outletIds are filtered out we call unwind to get each child in feedbacks array as a single document.
Now if we talk about mongoTemplate, yes it accepts all these parameter for aggregation including the nesting of child in feedback in first stage. just see some example of TypedAggregation
if you are saving the createdOn field as a string instead of timestamp or ISODate even normal mongo queries won't work on that when you need to find range as its working in your postgres example.
Hope it helps.