Mongo aggregation based on Dates between on more Collections - mongodb

I'm using Mongo Aggregation with Spring Data to retrieve data from 4 collections.
My problem is that I want to add a filter based on the createdAt on 3 of these collections.
Here the examples:
Animal
{"_id":123,"name":"Lucky","age":3,createdAt:"2022-02-01 10:00:00.000Z"}
{"_id":456,"name":"Joe","age":5,createdAt:"2022-02-10 20:03:00.000Z"}
Cat
{"idAnimal":123,"toy":6,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"toy":2,createdAt:"2022-02-10 20:05:00.000Z"}
Pet
{"idAnimal":123,"meal":3,"medicine":false,createdAt:"2022-03-01 10:00:40.000Z"}
{"idAnimal":456,"meal":4,"medicine":true,createdAt:"2022-02-10 20:05:00.000Z"}
What I mean to do is to get all the Animals with a Pet collection created gte(2022-03-01). The expected result would be the first cat, Lucky.
In my code I tried this
final List<Criteria> criteria = new ArrayList<>();
criteria.add(new Criteria().orOperator(
Criteria.where("createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Cat.createdAt").gte("2022-03-01").lte("2022-03-02"),
Criteria.where("Pet.createdAt").gte("2022-03-01").lte("2022-03-02")
));
and my Aggregation set up is:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.lookup("cat", "_id", "idAnimal", "Cat"),
Aggregation.lookup("pet", "_id", "idAnimal", "Pet"),
Aggregation.unwind("$_id"),
Aggregation.match(new Criteria().andOperator(criteria.toArray(new Criteria[0]))
)
);
I tried to swap the criteria, querying the Pet first, but didn't go as expected. When I run this, I get no result.
Do you have any tips? Is it possibile to execute an aggregation on Dates over multiple collections?
Thank you in advance!

Related

How to trim a value by a particular length and then apply lookUp in mongoDB

The main issue i'm facing is doing multi document joins. For example, when I try and run a query on orders by customer I can't find an easy way to join orders to customers on a customer id because in the orders object the customer id has _user$ appended to the beginning of the user ID and I don't know how to truncate that in an aggregation pipeline.
eg; In the order object, the customer is defined as _p_customer:"_User$8qXjk40eOd"
But in the user table the _id is just 8qXjk40eOd and therfore the default lookup function in charts cannot match the two.
Note: I'm using parse server and it stores pointers in the above mentioned way.
This is how the data is being stored in the mongoDB order collection.
customer field in the above image is the pointer to the _User collection
This is how the data is stored in the mongoDB _User collection.
My requirement is to write mongo query that find all users and their related orders count. I'm not able to do that because I don't how to remove _User$ from the order row and join it the _User via _id.
before lookup in aggregate substr the field you want to use in $lookup
db.collection.aggregate([
{
$addFields :{
nUser : { $substr: [ "$_p_customer", 6, -1 ] } // _User$ has 6 characters
}
},
{
$lookup :{
from:"usersCollection",
localField:"nUser",
foreignField : "_id",
as : "UserObject"
}
}
])

How to filter on collection in deployment level changestream in mongodb java sdk

I am trying to setup a deployment level change stream with a pipeline filter on collection name using MongoDB java SDK. Here is the code snippet.
List<Bson> pipeline = Collections.singletonList(Aggregates.match(Filters.or(
Filters.eq("namespace", "db1.c1"),
Filters.eq("namespace", "db1.c2"))));
client.watch(pipeline)
.cursor()
.forEachRemaining(doc -> {
System.out.println(doc);
});
But this query does not match any document. Following variations of the pipeline document does not work either.
List<Bson> pipeline =
Collections.singletonList(Aggregates.match(Filters.or(
Document.parse("{'namespace': 'db1.c1'}"),
Document.parse("{'namespace': 'db1.c2'}"))));
Surprisingly pipeline works on other fields of the changestream document. For example, this works:
List<Bson> pipeline = Collections
.singletonList(Aggregates.match(Filters.and(
Document.parse("{'fullDocument.seq': 4}"),
Filters.in("operationType", Arrays.asList("insert")))));
I am not sure why this would be the case. I would appreciate any help in this regard.
in Order to get changestream events on DB or collection level you need to use filters on below attributes :
ns.db The name of the database.
ns.coll The name of the collection.
In your case if you are interested only for collection C1 & c2 then consider having filter something like below.
List<Bson> pipeline = Collections.singletonList(Aggregates.match(Filters.or(
Filters.eq("ns.coll", "c1"),
Filters.eq("ns.coll", "c2"))));
client.watch(pipeline)
.cursor()
.forEachRemaining(doc -> {
System.out.println(doc);
});
Please Note: above filter is on collections only you can add filter on DB for collection form specific DB like (ns.db = "db1" AND
ns.coll : $in :["c1", "c2"] )
MongoDB doc reference: https://docs.mongodb.com/manual/reference/change-events/#change-stream-output

Spring Data Mongo - Custom AggregtionOption not working

I tried creating a Custom AggregationOperation based on https://github.com/krishnaiitd/learningJava/tree/master/spring-boot-sample-data-mongodb
When I used a custom aggregation in my aggregation for a lookup, it threw an exception saying the "as" field is not found on the entity.
If anybody has tried using custom AggregationOperation please share your code or let me know where I am going wrong.
Below is my code,
String lookup = "{ $lookup : { from: 'ITEM', localField : 'item_id', foreignField : '_id', as : 'item' } }";
TypedAggregation<Order> aggregation = Aggregation.newAggregation(Order.class,
new CustomAggregationOperation(lookup),
unwind("item", false));
The exception:
org.springframework.data.mapping.PropertyReferenceException: No property item found for type Order!
A TypedAggregation is a special Aggregation that holds information of the input aggregation type.
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/aggregation/TypedAggregation.html
That means that Spring will verify after each stage that your documents have not changed the structure.
Since you are trying transform original document, you need to use a standard Aggregation.
Aggregation aggregation = Aggregation.newAggregation(
new CustomAggregationOperation(lookup),
unwind("item", false)
);
List<Order> result = mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Order.class), Order.class).getMappedResults();

Different output for native and SpringData Query

I have a user collection with documents having org array with multiples values. The problem here is that the Spring API returns 3 documents. But the Org data is null instead of individual array data. The native query is working perfectly returning org data with values.
I am using Spring Data version 1.10.1.RELEASE
Following is my native query
db.users.aggregate([
{ $unwind: "$Org" }
])
The above query returns 3 documents because org array contains 3 data.
Following is my equivalent SpringData API
Aggregation aggregation = newAggregation(
unwind("Org"),
);
AggregationResults<UserDTO> groupResults =
mongoTemplate.aggregate(aggregation, Users.class, Users.class);
return groupResults.getMappedResults();

Mongo db using result of query in another query using $in

I have the following model in mongo db:
User collection
{
_id:12345,
name:"Joe",
age:15,
}
Addresses collection
{
_id:7663,
userId:12345,
Street:"xyz",
number:"1235",
city:"New York",
state:"NY"
}
Now I want to get all the addresses of users above the age of 20. What I thought was to query all the ids of users above 20 and with the result of this query use the $in operator to find the addresses.
My question is, is there a way to turn this into one query? Is there a better way to query this?
(obs: this is just an example, with my problem I cannot embed addresses into users)
In Mongo shell you can use the result of one query in another. For example:
use database // the name of your database
db.coll1.find({_id:{$nin:db.coll2.distinct("coll1_id")}})
Here collection coll1 contains an _id field. Then you can check for any ids that are not in collection coll2's list of the coll1_id field. So this is a way to clean up two tables if you have records in coll1 which have no reference via the coll1_id field in coll2.
Another approach doing the same thing:
use database // the name of your database
temp = db.coll2.distinct("coll1_id");
db.coll1.find({_id:{$nin:temp}})
The first example does it in one command, the second does it in two, but the concept is the same. Using results from one query in another. Many different ways to do this. Also, the .toArray() method can be useful to create arrays if you're doing more than just using distinct().
Use the aggregation framework where the $lookup pipeline stage provides the functionality to join the two collections:
db.user.aggregate([
{ "$match": { "age": { "$gt": 20 } } },
{
"$lookup": {
"from": "addresses",
"localField": "_id",
"foreignField": "userId",
"as": "address"
}
}
])
The above will create a new array field called address (as specified in the $lookup as option) and this contains the matching documents from the from collection addresses. If the specified name already exists in the input document, the existing field is overwritten.