How can I translate
$group : {
_id : 1 ,
likes : { $avg: { $size: ["$likes"] }},
count: { $sum: 1 },
}
into a spring-data-mongodb aggregation query?
You can use Following Code :
ArrayList<String> list = new ArrayList<>();
list.add("$likes");
BasicDBObject idObj = new BasicDBObject("_id", 1);
BasicDBObject likesObj = new BasicDBObject("$avg", new BasicDBObject("$size", list));
BasicDBObject countObj = new BasicDBObject("$sum", 1);
idObj.append("likes", likesObj).append("count", countObj);
BasicDBObject group = new BasicDBObject("$group", idObj);
List<DBObject> pipeline = new ArrayList<>();
pipeline.add(group);
MongoTemplate mongoTemplate = new MongoTemplate(new MongoClient(), "dbname");
AggregationOutput aggregationOutput = mongoTemplate.getCollection("collectionName")
.aggregate(pipeline);
List<DBObject> dbObjects = (List<DBObject>) aggregationOutput.results();
//convert dbObjects into appropriate output
You can use also following way for spring data mongodb
List<AggregationOperation> stages = new ArrayList<>();
GroupOperation groupOperation = group()
.count().as("count")
.avg(ArrayOperators.Size.lengthOfArray("likes")).as("likes");
stages.add(groupOperation);
AggregationResults<ResultDTO> result = mongoOperation.aggregate(newAggregation(stages),
"collectionName", ResultDTO.class);
public class ResultDTO {
private Integer count;
private Double likes;
//getter setter
}
Related
I have a problem with migrating an entity which has DBRef fields. If I change any field in this entity, my DBRefs are gone. What should I do?
Query query = new Query();
List<Criteria> criteriaList = new ArrayList<Criteria>();
criteriaList.add(Criteria.where("created").exists(false));
criteriaList.add(Criteria.where("updated").exists(false));
query.addCriteria(new Criteria().orOperator(criteriaList.toArray(new Criteria[criteriaList.size()])));
List<Asset> results = mongoTemplate.find(query, Asset.class);
results.forEach(asset -> {
asset.setCreated(asset.getCreatedBy().getCreated());
mongoTemplate.save(asset);
});
in summary, its problem is the MongoTemplate. mongoTemplate.save(...) creates a new entity and saves it to DB. As a consequence, if you want to change/update a field in your entity, you should use mongoTemplate.updateFirst(...). My solution is as the following.
public void createAssetCreateOrUpdateDateIfNot(MongockTemplate mongockTemplate) {
MongoTemplate mongoTemplate = mongockTemplate.getImpl();
Query queryGetAssetsCreatedOrUpdatedIsNot = new Query();
List<Criteria> criteriaList = new ArrayList<Criteria>();
criteriaList.add(Criteria.where("created").exists(false));
criteriaList.add(Criteria.where("updated").exists(false));
queryGetAssetsCreatedOrUpdatedIsNot
.addCriteria(new Criteria().orOperator(criteriaList.toArray(new Criteria[criteriaList.size()])));
List<Asset> results = mongoTemplate.find(queryGetAssetsCreatedOrUpdatedIsNot, Asset.class);
if (results.size() > 0) {
for (Asset asset : results) {
Date createdDate = (asset.getCreatedBy().getCreated() != null)
? asset.getCreatedBy().getCreated()
: new Date();
Date updateDate = (asset.getModifiedBy().getCreated() != null)
? asset.getModifiedBy().getCreated()
: new Date();
Query queryGetAssetViaId = new Query();
queryGetAssetViaId.addCriteria(Criteria.where("id").is(asset.getId()));
Update update = new Update()
.set("created", createdDate)
.set("updated", updateDate);
mongoTemplate.updateFirst(queryGetAssetViaId, update, Asset.class);
}
}
}
this is my method, when the server runs this mothod, it will take 18s.
but when i run the same query, use NoSQLBoost, it just takes 5s.
public Integer queryUserEduTotal(UserAdminV1StepListRequest body) throws ParseException {
Criteria criteria = Criteria.where(Column.SYSTEM_STATUS).is(true);
criteria.and(UserColumn.APP_ID).is(body.getAppId());
criteria.and(UserColumn.STATUS).is(2);
Criteria criteria1 = Criteria.where(UserColumn.USER_TYPE).exists(false);
Criteria criteria2 = Criteria.where(UserColumn.USER_TYPE).is(UserTypeEnum.USER_REGISTER.getKey());
Criteria criteria3 = new Criteria();
criteria3.orOperator(criteria1, criteria2);
criteria.andOperator(criteria3, Criteria.where(UserColumn.SYSTEM_CREATE_TIME).gte(sdf.parse(body.getStart())), Criteria.where(UserColumn.SYSTEM_CREATE_TIME).lte(sdf.parse(body.getEnd())));
LookupOperation lookup = lookup("user_edu_item_info", "_id", UserEduItemColumn.USER_ID, "edu_info");
Criteria criteria4 = Criteria.where(Column.SYSTEM_STATUS).is(true);
criteria4.and(UserColumn.APP_ID).is(body.getAppId());
criteria4.and("edu_info._id").exists(true);
Aggregation typedAggregation1 = Aggregation.newAggregation(
match(criteria),
lookup,
match(criteria4),
Aggregation.count().as(Constant.COUNT)
);
AggregationResults<BasicDBObject> aggregationResults = mongoTemplate.aggregate(typedAggregation1, User.class, BasicDBObject.class);
int count = 0;
if (aggregationResults.getMappedResults().size() == 0) {
count = 0;
} else {
String document = JSON.toJSONString(aggregationResults.getMappedResults().get(0));
JSONObject result = JSON.parseObject(document);
count = result.getInteger(Constant.COUNT);
}
return count;
}
and, i run the same code on the local by using the java program.it takes 5s too.
i dont know why the server takes so much time.
MongoDB configuration:
MongoClientOptions.Builder builder = new xxx()
builder.connectionsPerHost(100);
builder.minConnectionsPerHost(10);
builder.connectTimeout(30000);
builder.threadsAllowedToBlockForConnectionMultiplier(10);
builder.serverSelectionTimeout(30000);
builder.socketTimeout(0);
builder.maxWaitTime(120000);
builder.heartbeatConnectTimeout(30000);
builder.heartbeatSocketTimeout(30000);
builder.heartbeatFrequency(10000);
builder.minHeartbeatFrequency(500);
builder.localThreshold(15);
this is my local java code
public class Test {
public static void main(String[] args) {
// Link code omitted
MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory);
search(mongoTemplate);
try {
mongoDbFactory.destroy();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void search(MongoTemplate mongoTemplate){
Criteria criteria = Criteria.where("systemStatus").is(true);
//criteria code omitted, sane whit the server code
Aggregation typedAggregation1 = Aggregation.newAggregation(
match(criteria),
lookup,
match(criteria4),
Aggregation.count().as("count")
);
AggregationResults<BasicDBObject> aggregationResults = mongoTemplate.aggregate(typedAggregation1, User.class, BasicDBObject.class);
}
}
I'm using this method to send multiple list to client.
public ResponseEntity<?> getFiveLastRequestOfEachVehicleType() {
ResponseContent content = getResponseContent();
Map<String, List<Request>> map = new HashMap<>();
GroupBy groupBy = new GroupBy();
groupBy.initialDocument("vehicleTypeEnum");
query = new Query();
queryFieldsFilterLastFiveRequest(query, VehicleTypeEnum.NEISAN);
map.put("NEISAN", mongoOperations.find(query, Request.class));
query = new Query();
queryFieldsFilterLastFiveRequest(query, VehicleTypeEnum.BADSAN);
map.put("BADSAN", mongoOperations.find(query, Request.class));
query = new Query();
queryFieldsFilterLastFiveRequest(query, VehicleTypeEnum.BUJE);
map.put("BUJE", mongoOperations.find(query, Request.class));
content.setData(map);
return getReturn(content);
}
And this method apply criteria to each query
private void queryFieldsFilterLastFiveRequest(Query query, VehicleTypeEnum vehicleTypeEnum) {
query.addCriteria(Criteria.where("vehicleTypeEnum").is(vehicleTypeEnum));
query.addCriteria(Criteria.where("unlock").is(true));
query.fields()
.include("id")
.include("goodsTypeTitle")
.include("originCityTitle")
.include("price");
query.limit(5);
}
I wonder if there is any way to retrieve all list in one request to database using MongoOperations.
I would go to grouping manually later on, first query for all vehicleType :
private void queryFieldsFilterLastFiveRequest(Query query, VehicleTypeEnum... vehicleTypeEnum..) {
query.addCriteria(Criteria.where("vehicleTypeEnum").in(vehicleTypeEnum));
query.addCriteria(Criteria.where("unlock").is(true));
query.fields()
.include("id")
.include("goodsTypeTitle")
.include("originCityTitle")
.include("price");
query.limit(5);
}
Here the is has been changed by in, and accept a list of VehicleTypeEnum
Then to use it :
public ResponseEntity<?> getFiveLastRequestOfEachVehicleType() {
ResponseContent content = getResponseContent();
Map<String, List<Request>> map;
query = new Query();
queryFieldsFilterLastFiveRequest(query, VehicleTypeEnum.NEISAN, VehicleTypeEnum.BADSAN, VehicleTypeEnum.BUJE);
map = mongoOperations.find(query, Request.class).stream()
.collect(Collectors.toMap(r -> r.getVehicleTypeEnum().name(),r -> r));
content.setData(map);
return getReturn(content);
}
Here i use the stream api to group result by VehiculeTypeEnum name
How can i GroupBy data, based on date range in spring data Mongo db using
ArithmeticOperators.Subtract
I have wide range of timed data, and i should summarize data, based on average in interval time:
I partitioned data based on interval time and run this aggregation for each interval. this method is too time consuming operation when i use for loop and change start and until throw loop
MatchOperation match = match(Criteria.where("type").is(id).andOperator(Criteria.where("time").gt(start),
Criteria.where("time").lt(until)));
matchOperationL.add(match);
GroupOperation as = group("type").avg("value").as("mean");
ProjectionOperation previousOperation = project("mean").and("type").previousOperation();
Aggregation avg = Aggregation.newAggregation(match, as, previousOperation);
AggregationResults<Data2> aggregate = mongotemplate.aggregate(avg, Data1.class, Data2.class);
how can i use
ArithmeticOperators.Subtract
to solve my problem?
my EntityClass
public class Data1 {
#Id
private String id;
//value of data
private Number So;
//recorded time
private long time;
}
Can any one help me to convert this query to spring data form?
db.collection.aggregate([ {$group: { _id:{'$add': {'$trunc': { '$subtract' :[ {'$divide' : ['$time', 15000 ]}, { '$mod' : [{'$divide' :['$time',15000]},1] } ]},'$time'}}, avg : {$avg : "$So"} }}]
#Veeram second option is like this:
AggregationOperation matchTime = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
return new BasicDBObject("$match", BasicDBObject
.parse("{ 'time': {$gt: " + s + ", $lt: " + e + "} } }"));
}
};
AggregationOperation matchSId = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
return new BasicDBObject("$match", BasicDBObject
.parse("{'sd' : '" + sd + "'}"));
}
};
AggregationOperation groupOperation = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
return new BasicDBObject("$group",
BasicDBObject.parse("{ _id:{ '$trunc': { '$subtract' :[ {'$divide' : ['$time', " + dist
+ " ]},{ '$mod' : [{'$divide' :['$time'," + dist
+ " ]},1] } ]}},avg : {$avg : '$So'}}}"));
}
};
AggregationOperation sortOperation = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
return new BasicDBObject("$sort", BasicDBObject.parse("{'_id':1}"));
}
};
Aggregation agg = Aggregation.newAggregation(matchSd,matchTime, groupOperation, sortOperation);
AggregationResults<Result> aggregate = mongoOps.aggregate(agg, Data.class, Result.class);
I want to extract value of a embedded key(detokenObject.0.client.COUNTRY) from Mongodb document using java.
I have tried below piece of code.
DBCursor cursor1 = coll1.find();
while (cursor1.hasNext()) {
BasicDBObject dbObject = (BasicDBObject)cursor1.next();
BasicDBObject dbObject1= ((BasicDBObject)dbObject.get("detokenObject"));
BasicDBObject dbObject2= ((BasicDBObject)dbObject1.get("0"));
System.out.println("value is:::"+((BasicDBObject)dbObject2.get("client")).get("COUNTRY"));
}
But after running this, getting ClassCastException(com.mongodb.BasicDBList cannot be cast to com.mongodb.BasicDBObject)
any solution?
The error ClassCastException(com.mongodb.BasicDBList cannot be cast to com.mongodb.BasicDBObject) says it all, you are trying to cast a BasicDBList to BasicDBObject on detokenObject field, which is a list.
Use
BasicDBList dbList = (BasicDBList) dbObject.get("detokenObject");
instead of
BasicDBObject dbObject1 = ((BasicDBObject)dbObject.get("detokenObject"));
and then get its element client value by its index with dbList.getIndex(0);
i.e. your final code should look like:
DBCursor cursor1 = coll1.find();
while (cursor1.hasNext()) {
BasicDBObject dbObject = (BasicDBObject) cursor1.next();
BasicDBObject dbList = (BasicDBList) dbObject.get("detokenObject");
BasicDBObject dbObject1= (BasicDBObject) dbList.getIndex(0);
System.out.println("value is::: " + (BasicDBObject) dbObject1.get("client")
.get("COUNTRY"));
}
I have tried below solution to chained the keys and it worked for me.
public static Object getField(DBObject obj, String fieldName) {
String[] fieldSections =fieldName.split("\\.");
int i = 1;
Object value = obj.getfieldSections[0]);
while(i < fieldSections.length && value instanceof DBObject) {
value = ((DBObject)value).get(fieldSections[i]);
i++;
}
return value;
}