I have a user collection with documents having org array with multiples values. The problem here is that the Spring API returns 3 documents. But the Org data is null instead of individual array data. The native query is working perfectly returning org data with values.
I am using Spring Data version 1.10.1.RELEASE
Following is my native query
db.users.aggregate([
{ $unwind: "$Org" }
])
The above query returns 3 documents because org array contains 3 data.
Following is my equivalent SpringData API
Aggregation aggregation = newAggregation(
unwind("Org"),
);
AggregationResults<UserDTO> groupResults =
mongoTemplate.aggregate(aggregation, Users.class, Users.class);
return groupResults.getMappedResults();
Related
I'm using Mongo Aggregation with Spring Data to retrieve data from 4 collections.
My problem is that I want to add a filter based on the createdAt on 3 of these collections.
Here the examples:
Animal
{"_id":123,"name":"Lucky","age":3,createdAt:"2022-02-01 10:00:00.000Z"}
{"_id":456,"name":"Joe","age":5,createdAt:"2022-02-10 20:03:00.000Z"}
Cat
{"idAnimal":123,"toy":6,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"toy":2,createdAt:"2022-02-10 20:05:00.000Z"}
Pet
{"idAnimal":123,"meal":3,"medicine":false,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"meal":4,"medicine":true,createdAt:"2022-02-10 20:05:00.000Z"}
What I mean to do is to get all the Animals with a Pet collection created gte(2022-03-01). The expected result would be the first cat, Lucky.
In my code I tried this
final List<Criteria> criteria = new ArrayList<>();
criteria.add(new Criteria().orOperator(
Criteria.where("createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Cat.createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Pet.createdAt").gte("2022-03-01").lte("2022-03-02")
));
and my Aggregation set up is:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.lookup("cat", "_id", "idAnimal", "Cat"),
Aggregation.lookup("pet", "_id", "idAnimal", "Pet"),
Aggregation.unwind("$_id"),
Aggregation.match(new Criteria().andOperator(criteria.toArray(new Criteria[0]))
)
);
I tried to swap the criteria, querying the Pet first, but didn't go as expected. When I run this, I get no result.
Do you have any tips? Is it possibile to execute an aggregation on Dates over multiple collections?
Thank you in advance!
What does the a JSON blob with search filters, allowed operators: $eq, $ne, $in, $nin, $gt, $lt, $gte, $lte, $exists in the Swagger documentation that is shown in the DataStax Document API Swagger UI, it's not that documented so I want to ask if the query string is based on MongoDB?
The Document API exposed on top of Cassandra is provided by the open source project Stargate, indeed developed by Datastax and embedded in their Saas solution Astra.
The JSON query String than you created is parsed and converted in a proper CQL query under the hood.
Source code doesn't lie you can find the full code here and specially parsing of the where clause here
public List<FilterCondition> convertToFilterOps(
List<PathSegment> prependedPath,
JsonNode filterJson) {
List<FilterCondition> conditions = new ArrayList<>();
if (!filterJson.isObject()) {
throw new DocumentAPIRequestException("Search was expecting a JSON object as input.");
}
ObjectNode input = (ObjectNode) filterJson;
Iterator<String> fields = input.fieldNames();
while (fields.hasNext()) {
String fieldName = fields.next();
if (fieldName.isEmpty()) {
throw new DocumentAPIRequestException(
"The field(s) you are searching for can't be the empty string!");
}
...
The query string is pretty similar in spirit to what you'd find with Mongo.
Here are some sample where clauses to give an idea:
{"name": {"$eq": "Eric"}} - simple enough, matches documents that have a field name with value Eric
{"a.age": {"$gt": 0}} - You can also reference nested fields in a document
{"friends.[0].name": {"$in": ["Cassandra"]}} - Array elements are referenced using [], this would match if the document's first friend is named Cassandra.
{"friends.*.age": {"$gte": 24}} - Wildcard * can be used to match any element in an array, or any field at a particular level of nesting. This matches any friend whose age is >= 24.
My mongo version is 3.4
I'm running a complex aggregation and writing the result to a collection using $out stage.
What I wish to do is, if that aggregation is ran before and result exists on target collection, just return the saved result without rerunning the whole pipeline.
Problem 1 is, when $out is used, aggregation doesn't return any results, so I have to send a separate query to the target collection to retrieve them.
Problem 2 is, I need another query to check if aggregation is ran before.
Is it possible to combine all 3 queries to a single pipeline, so it would:
check outCollection, if result with given user_id exists, return it
if doesn't, run pipeline and save the result to outCollection
return aggregation result
my current pipeline looks like:
db.getCollection('sourceCollection').aggregate([
{
$match: {
user_id: "myUserId"
}
},...//perform various steps
{
$out: "outCollection"
}],{
allowDiskUse: true
})
Result is saved to outCollection in a document like:
{ user_id: "myUserId", aggResult: [...] }
how to perform below operation (it is the actual mongo query) using MatchOperation in Spring data mongodb ?
$match: { "docs": { $ne: [] } }
here docs is an array field and want to check that it is not empty.
I also had a similar problem, but I solved is as below.
MatchOperation mathOpertaion = match(Criteria.where("docs")
.elemMatch(new Criteria().exists(true)));
I started to use MongoDB database in my application and for data access I have chosen Spring Data for MongoDB.
I skimmed API reference and documentation and I can see that there is map-reduce integration but what about aggregation framework? I can see that it supports group by operation, which would indicate that it supports $group operator judging from this: http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/, but what about other operators, are that not supported for now?
I am asking this question because I wanted to know what kind of integration with MongoDB Sping Data provides so I know what to expect, so to speak.
Spring Data 1.3.0.RC1 is available and it does support the aggregation framework.
For example:
The shell aggregation comand:
db.eft_transactions.aggregate(
{$match:
{
service:"EFT",
source:"MARKUP",
}
},
{$group:
{
_id:"$card_acceptor_id",
tran_count:{$sum:1},
amount_sum:{$sum:"$amount"}
}
}
)
is run like this from java:
AggregationOperation match = Aggregation.match(Criteria.where("service").is("EFT").and("source").is("MARKUP"));
AggregationOperation group = Aggregation.group("card_acceptor").and("amount_sum").sum("amount").and("tran_count").count();
Aggregation aggregation = newAggregation(match, group);
AggregationResults<StoreSummary> result = this.mongoTemplate.aggregate(aggregation, "eft_transactions", StoreSummary.class);
The documentation is here
NOTE: We recently had to switch to using the BUILD-SNAPSHOT build of version 1.3.0. This change necessitated the change to 2 of the above lines which have changed to:
AggregationOperation group = Aggregation.group("card_acceptor").sum("amount").as("amount_sum").count().as("tran_count");
Aggregation aggregation = Aggregation.newAggregation(match, group);
The Spring Data MongoOperations.group() method is mapped to db.collection.group() MongoDB command and not the $group aggregation function. Currently there is no support in Spring Data MongoDB for aggregation framework. Map reduce, as you have mentioned, is supported though
Aggregation aggregation = newAggregation(
match(Criteria.where("salesyear").is(year)),
group("brand","salesyear").sum("numberOfCars").as("total"),
sort(Sort.Direction.ASC, previousOperation(), "brand")
);
Here is how to get the sum of a particular field.
private Map<String, Long> getTotalMap(){
/*
db.pDSSummaryModel.aggregate([{
$group: {
_id: null,
total: {
$sum: '$totalUniqueCustomerCount'
}
}
}])
*/
Aggregation aggregations = newAggregation(
group("null").sum("totalUniqueUserCount").as("userTotal")
.sum("totalUniqueCustomerCount").as("customerTotal"),
project("customerTotal", "userTotal")
);
AggregationResults<DBObject> results = mongoTemplate.aggregate(aggregations, "pDSSummaryModel", DBObject.class);
List<DBObject> fieldList = results.getMappedResults();
Map<String, Long> map = new HashMap<>();
if(fieldList != null && !fieldList.isEmpty()) {
for(DBObject db: fieldList){
map.put("userTotal", parseLong(db.get("userTotal").toString()));
map.put("customerTotal", parseLong(db.get("customerTotal").toString()));
}
}
return map;
}