Spring data MongoDb Aggregation lookup from String to ObjectId - mongodb

I'm using Spring (boot) data 2.2.7 with mongodb 4.0.
I've set 3 collections that I'm trying to join via an aggregation lookup operation.
catalog
stock
operations
catalog
{
"_id" : ObjectId("5ec7856eb9eb171b72f721af"),
"model" : "HX711",
"type" : "DIGITAL",
....
}
mapped by
#Document(collection = "catalog")
public class Product implements Serializable {
#Id
private String _id;
#TextIndexed
private String model;
....
stock
{
"_id" : ObjectId("5ec78573b9eb171b72f721ba"),
"serialNumber" : "7af646bb-a5a8-4b86-b56b-07c12a625265",
"bareCode" : "72193.67751691974",
"productId" : "5ec7856eb9eb171b72f721af",
......
}
mapped by
#Document(collection = "stock")
public class Component implements Serializable {
#Id
private String _id;
private String productId;
....
the productId field refers to the _id one in the catalog collection
operations
{
"_id" : ObjectId("5ec78671b9eb171b72f721d3"),
"componentId" : ""5ec78573b9eb171b72f721ba",
.....
}
mapped by
public class Node implements Serializable {
#Id
private String _id;
private String componentId;
....
the componentId field refers to the _id one in the stock collection
I want to query operations or stock collection to retreive the corresponding Node or Component object list ordered by the Product.model field (in the catalog collection.)
While the goal is to code in Java I've tried to make the request first in the Mongo shell but I can't even get it working as I'm trying to join (lookup) a string with an ObjectId : Node.componentId -> Component._id
Component.productId -> Product._id
For the relationship Component(stock) -> Product(Catalog) I've tryed
LookupOperation lookupOperation = LookupOperation.newLookup()
.from("catalog")
.localField("productId")
.foreignField("_id")
.as("product");
TypedAggregation<Component> agg =
Aggregation.newAggregation(
Component.class,
lookupOperation
);
AggregationResults<Component> results = mongoTemplate.aggregate(agg, "stock", Component.class);
return results.getMappedResults();
but it returns the whole components records without product info.
[{"_id":"5ec78573b9eb171b72f721b0","uuId":"da8800d0-b0af-4886-80d1-c384596d2261","serialNumber":"706d93ef-abf5-4f08-9cbd-e7be0af1681c","bareCode":"90168.94737714577","productId":"5ec7856eb9eb171b72f721a9","created":"2020-05-22T07:55:31.66","updated":null}, .....]
thanks for your help.
Note:
In addition to #Valijon answer to be able to get the result as expected the returned object must include a "product" property either nothing is returned (using JSON REST service for example)
public class ComponentExpanded implements Serializable {
private String product;
....
with
AggregationResults<ComponentExpanded> results =
mongoTemplate.aggregate(agg,mongoTemplate.getCollectionName(Component.class), ComponentExpanded.class);

The problem resides in the mismatch of types between productId and_id as you have observed.
To join such data, we need to perform uncorrelated sub-queries and not every "new" feature makes it immediately into abstraction layers such as spring-mongo.
Try this:
Aggregation agg = Aggregation.newAggregation(l -> new Document("$lookup",
new Document("from", mongoTemplate.getCollectionName(Product.class))
.append("let", new Document("productId", new Document("$toObjectId", "$productId")))
.append("pipeline",
Arrays.asList(new Document("$match",
new Document("$expr",
new Document("$eq", Arrays.asList("$_id", "$$productId"))))))
.append("as", "product")),
Aggregation.unwind("product", Boolean.TRUE));
AggregationResults<Component> results = mongoTemplate.aggregate(agg,
mongoTemplate.getCollectionName(Component.class), Component.class);
return results.getMappedResults();
MongoPlayground Check here how shell query looks like.
Note: For Java v1.7, you need to implement AggregationOperation like below:
AggregationOperation l = new AggregationOperation() {
#Override
public Document toDocument(AggregationOperationContext context) {
return new Document(...); // put here $lookup stage
}
};

Related

Get likes count and whether user is liked record from MongoDB

I am trying to get count of likes and if user is liked this post in Mongo.
I managed to get this via native query with facets, but problems is how can i map this two fields on my custom java class (LikeStatus.class)?
thanks in advance!
please code below:
POJO:
public class LikeStatus {
String entityId;
LikedEntityType entityType;
long likesCount;
boolean isLikedByUser;
}
Document class:
public class Like {
#Id
private String id;
#EqualsAndHashCode.Include
private String entityId;
#EqualsAndHashCode.Include
private String profileId;
#EqualsAndHashCode.Include
private LikedEntityType entityType;
private LocalDateTime createdAt = LocalDateTime.now();
}
Query i used in Mongo:
> db.likes.aggregate({$facet:
{count:[
{$match:{entityId:"entityId"},
$match:{entityType:"OFFER"}}, {$count:"count"}],
isliked:[{$match:{profileId:"profileId4"}}, {$count:"isliked"}]}}).pretty();
and gives me result:
{
"count" : [
{
"count" : 3
}
],
"isliked" : [
{
"isliked" : 1
}
]
}
I managed to find solution which is suited my needs, hope it will be useful who faced with the same kind of queries in Mongodb and it will give some idea how it can be solved)
Java solution: i used Facet object to collect two aggregation request in one query like this:
In repository layer i created query:
default Aggregation getLikeStatus(String entityId, String entityType, String profileId){
FacetOperation facet = Aggregation.facet(match(where(ENTITY_ID_FIELD).is(entityId).and(ENTITY_TYPE_FIELD).is(entityType)),
Aggregation.count().as(LIKES_COUNT_FIELD)).as(LIKES_COUNT_FIELD)
.and(match(where(ENTITY_ID_FIELD).is(entityId)
.and(ENTITY_TYPE_FIELD).is(entityType)
.and(PROFILE_ID_FIELD).is(profileId)),
Aggregation.count().as(IS_LIKED_BY_USER_FIELD)).as(IS_LIKED_BY_USER_FIELD);
ProjectionOperation project = project()
.and(ConditionalOperators.ifNull(ArrayOperators.ArrayElemAt.arrayOf(LIKES_COUNT_FIELD).elementAt(0)).then(0)).as(LIKES_COUNT_FIELD)
.and(ConditionalOperators.ifNull(ArrayOperators.ArrayElemAt.arrayOf(IS_LIKED_BY_USER_FIELD).elementAt(0)).then(0)).as(IS_LIKED_BY_USER_FIELD)
.andExclude("_id");
return newAggregation(facet, project);
}
then in service layer it returns Document object which is mapped on my custom class LikeStatus fields:
Document status = template.aggregate(likeRepo.getLikeStatus(entityId, entityType, profileId), Like.class, Document.class).getUniqueMappedResult();
my custom POJO:
#Data
#NoArgsConstructor
#AllArgsConstructor
public class LikeStatus {
String entityId;
LikedEntityType entityType;
long likesCount;
boolean isLikedByUser;
}
Also i post native query solution in Mongo for reference:
db.likes.aggregate([
{$facet:
{"likesCountGroup":[
{$match:{entityId:"entityId", entityType:"TYPE"}},{$count:"likesCount"}
],
"isUserLikedGroup":[
{$match:{entityId:"entityId", entityType:"TYPE", profileId:"604cd12c-1633-4661-a773-792a6ec22187"}},
{$count:"isUserLiked"}
]}},
{$addFields:{}},
{$project:{"likes":{"$ifNull":[
{$arrayElemAt:["$likesCountGroup.likesCount", 0]},0]},
"isUser":{"$ifNull:[{$arrayElemAt["$isUserLikedGroup.isUserLiked",0]},0]}}}]);

Spring data mongo: Projection not working when path includes a key of a HashMap

Problem
When I try to project a value which is inside a java.util.Map then I get below exception. But when I run the generated shell query in roboMongo then It works. I would be very grateful if somebody could point out the problem.
org.springframework.data.mapping.context.InvalidPersistentPropertyPath: No property Germany found on com.nntn.corona.snapshot.repo.model.StatWithDelta!
Query Code In Java
spring boot parent: 2.0.5.RELEASE
Criteria matchCriteria = Criteria.where("timestamp").gte(startDate);
MatchOperation match = Aggregation.match(matchCriteria);
SortOperation sort = sort(new Sort(Sort.Direction.ASC, "timestamp"));
// #formatter:off
ProjectionOperation projection = project()
.andExpression("timestamp").as("timestamp")
.andExpression("countries.germany.total").as("total")
.andExpression("countries.germany.today").as("today");
// #formatter:on
Aggregation aggregation = newAggregation(match, sort,projection);
AggregationResults<Document> result = mongoTemplate.aggregate(aggregation, SnapshotEntry.class,
Document.class);
return result.getMappedResults();
Data Model
Java representation
#Document(collection = "snpashots")
public class SnapshotEntry {
#Id
private String id;
#Type(type = "org.jadira.usertype.dateandtime.joda.PersistentDateTime")
#Temporal(TemporalType.TIMESTAMP)
private DateTime timestamp;
private Map<String, StatWithDelta> countries;
private StatEntity total;
private StatEntity today;
private String source;
private String previousRecordId;
#Data
#NoArgsConstructor
#AllArgsConstructor
public class StatWithDelta {
private StatEntity total;
private StatEntity today;
}
}
Json representation
{
"_id" : "21-03-2020",
"timestamp" : ISODate("2020-03-21T09:26:00.965Z"),
"countries" : {
"germany" : {
"total" : {
"born" : NumberLong(81008),
"dead" : NumberLong(3255),
"sick" : NumberLong(30000)
},
"today" : {
"born" : NumberLong(50),
"dead" : NumberLong(10),
"sick" : NumberLong(12)
}
}
},
"_class" : "com.nntn.snapshot.repo.model.SnapshotEntry"
}
The problem is in TypedAggregation. This is a special aggregation where Spring holds information of the input aggregation type.
To avoid it, use raw aggregation (as if you run in MongoDB shell) this way:
AggregationResults<SnapshotEntry> result = mongoTemplate.aggregate(aggregation,
mongoTemplate.getCollectionName(SnapshotEntry.class),
SnapshotEntry.class);

MongoDB - FindById is not working and giving null

I am using Spring Boot(V2.2.2.RELEASE) + Spring Data Mongo Example. In this example, I've records like below
{
"_id" : ObjectId("5cb825e566135255e0bf38a4"),
"firstName" : "John",
"lastName": "Doe"
}
My Repository
#Repository
public interface EmployeeRepository extends CrudRepository<Employee, ObjectId>{
Employee findById(String id);
}
Code
Employee findById = employeeRepository.findById("5cb825e566135255e0bf38a4");
System.out.println(findById);
Even below code not working
Query query = new Query(Criteria.where("id").is(new ObjectId("5cb825e566135255e0bf38a4")));
List<Employee> find = mongoTemplate.find(query, Employee.class);
Seems there might be two issues
ObjectId should be used
employeeRepository.findById(new ObjectId("5cb825e566135255e0bf38a4"))
ID field goes with underscore
new Query(Criteria.where("_id").is(new ObjectId("5cb825e566135255e0bf38a4")))
I'm not a Java guy, so might miss smth, but at least give it a try :)
Using the input document with _id: ObjectId("5cb825e566135255e0bf38a4") you can use either of the approaches. Assuming there is the document in the employee collection you can query by the _id's string value.
public class Employee {
#Id
private String id;
private String name;
// constructors (with and with arguments)
// get methods
// override toString()
}
// Spring-data app using MongoTemplate
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "test");
Query query = new Query(where("id").is("5cb825e566135255e0bf38a4"));
Employee emp = mongoOps.findOne(query, Employee.class);
-or-
MongoRepository
interface extends CrudRepository and exposes the capabilities of the
underlying persistence technology in addition to the generic
persistence technology-agnostic interfaces such as CrudRepository.
#Repository
public interface EmployeeRepository extends MongoRepository<Employee, String> {
#Query("{ 'id' : ?0 }")
Optional<Employee> findById(String id);
}
// Spring-data app using the Repository
Optional<Employee> empOpt = employeeRepository.findById("5cb825e566135255e0bf38a4");
if (empOpt.isPresent()) {
System.out.println(empOpt.get());
}
else {
System.out.println("Employee not found!");
}
I too faced the issue. Interestingly things are working fine with version 2.1.7RELEASE
<documentRepository>.findById(<StringID>) treats as String Id in 2.2.2RELEASE
while 2.1.7RELEASE transforms it to ObjectId and hence the find query works.

Spring Boot: Get an object from MongoDB document?

I am using Spring Boot 2.1.3.RELEASE and MongoDB. And I am looking for a way to retrieve a part of a document.
Repository:
#Repository
public interface CompanyRepository extends MongoRepository<Company, String> {
}
Object:
#Data
#Document
public class Company {
public GeneralInfo info;
public Map<String, List<Employee>> officeIdEmployeeMap;
#Data
public class GeneralInfo {
#Id
public String companyId;
public String name;
}
#Data
public class Employee {
public String firstName;
public String lastName;
}
}
I need to get only GeneralInfo objects and then if some conditions are true get the List<Employee> from the officeIdEmployeeMap, not the whole map.
Can this be done by MongoRepository?
This is possible with MongoTemplate.
finalString officeName = "IT";
final Query query = new Query();
query.addCriteria(Criteria.where("officeIdEmployeeMap." + officeName).exists(true));
query.fields().exclude("officeIdEmployeeMap");
final Company company = mongoTemplate.findOne(query, Company.class, "collection");
This will return the Company object but only GeneralInfo will be set.
In CompanyRepository add the following method:
#Query(value="{}", fields = "{'info':1}")
public List<Company> getCompanies();
Call this when you need to get only General Info otherwise, call findAll() to get all details. You may write a new Query in the repository restricting the selected fields and optionally the criteria.
In case you want to achieve both the scenarios (get GeneralInfo only and get both) in the same query conditionally, instead of annotation create a query like below:
#Autowired
private MongoTemplate mongoTemplate;
Query query = new Query();
boolean someCondition = true; // Put your condition here
if(someCondition) {
query.fields().exclude("officeIdEmployeeMap");
}
return mongoTemplate.find(query, Company.class);
Assuming your company data in the database looks like this:
{ "_id" : ObjectId("5d3afb75b2ebf8a9ec2f906f"), "info" : { "companyId" : "1", "name" : "abc" }, "officeIdEmployeeMap" : { "g1" : [ { "firstName" : "A", "lastName" : "B" }, { "firstName" : "C", "lastName" : "D" } ] } }

Using Spring Data MongoDB to find and update document using compound _id field

I have a custom key class in my project as below.
I have implemented Serializable, implemented hashCode and equals too.
public class SalesKey implements Serializable {
Date timestamp;
String hostName;
public Date getTimestamp() {
return timestamp;
}
public void setTimestamp(Date timestamp) {
this.timestamp=timestamp;
}
public void setHostName(String hostName) {
this.hostName=hostName;
}
public String getHostName() {
return hostName;
}
#Override
public int hashCode() {
..
}
#Override
public boolean equals(Object obj) {
...
}
}
I use the custom key class in my MongoDB Document class as below.
#Document(collection = "Sales")
public class Sales {
#Id
SalesKey salesKey;
//Other fields
}
I am able to save the documents into mongodb using spring data mongodb.
Sales sales=new Sales();
SalesKey salesKey=getSalesKey();
sales.setSalesKey(salesKey)
//set other fields for sales object
//Save sales Document object to mongodb
The Save is successful.
I am also able to fetch by using the below query in my SalesRepository interface which extends MongoRepository.
After constructing, the salesKey as before.
#Query(value = "{ '_id' : ?0 }")
public Sales filterBySalesKey(SalesKey salesKey);
When I try to update using a criteria API as below, for updating specific fields in the document.
I first use the Criteria as below.
Criteria salesKeyCriteria = Criteria.where("_id").is(salesKey);
With the above criteria I then execute a update statment,
But, my update fails with the error,
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.demo.model.SalesKey
I am using Spring Data MongoDB framework and MongoDB 3.2.1.
Can anyone help me what is the codec and how can I add one to make my update successful.
Since SalesKey is your Mongo _id, you could do the following with MongoTemplate:
#Autowired
private MongoTemplate mongoTemplate;
...
Query query = new Query(Criteria.where("_id").is(salesKey));
mongoTemplate.updateFirst(query,Update.update("field", "new-value"),Sales.class);
Following up on your comment on mongo explain
If I look at the update explain in mongo shell:
$ db.sales.explain().update({"_id" : { "timestamp" : ISODate("2016-08-25T12:12:59.251Z"), "hostName" : "myhostname" }},{value:"new-value"})
It shows me the winning plan using IDHACK:
"winningPlan" : {
"stage" : "UPDATE",
"inputStage" : {
"stage" : "IDHACK"
}
},
Example of a Sales document:
{ "_id" : { "timestamp" : ISODate("2016-08-25T12:12:59.251Z"), "hostName" : "value" }, "value" : "new" }