MongoDb: Aggregation using Groovy language - mongodb

Iam using groovy scripting under SpagoBI. I want to use aggregation.
I want for example to execute the following aggregation:
db.myCollection.aggregate(
[
{
$group : {
_id : { day: { $dayOfMonth: "$recvTime" } }
}
}
]
)
I tried:
DBObject projectFields = new BasicDBObject('$dayOfMonth',"recvTime");
DBObject project=new BasicDBObject('$project',projectFields)
DBObject groupFields = new BasicDBObject( "_id",project);
DBObject group = new BasicDBObject('$group', groupFields);
iterable = db.getCollection('myCollection').aggregate(group)
I got this error:
An unexpected error occured while executing dataset: { "serverUsed" : "192.168.1.160:27017" , "errmsg" : "exception: invalid operator '$project'" , "code" : 15999 , "ok" : 0.0}
Any ideas?
Updates:
the query executed in Mongo shell
db['cygnus_/kurapath_enocean_power_enocean'].aggregate(
... [
... {
... $group : {
... _id : { day: { $dayOfMonth: "$recvTime" } }
... }
... }
... ]
... );
{ "_id" : { "day" : 9 } }
{ "_id" : { "day" : 8 } }
{ "_id" : { "day" : 7 } }
{ "_id" : { "day" : 4 } }
{ "_id" : { "day" : 3 } }
the data stored in mongo db:
db['cygnus_/kurapath_enocean_power_enocean'].find()
{ "_id" : ObjectId("55e81e9631d7791085668331"), "recvTime" : ISODate("2015-09-03T10:19:02Z"), "attrName" : "power", "attrType" : "string", "attrValue" : "2085.0" }
{ "_id" : ObjectId("55e81e9631d7791085668332"), "recvTime" : ISODate("2015-09-03T10:19:02Z"), "attrName" : "power", "attrType" : "string", "attrValue" : "2085.0" }
{ "_id" : ObjectId("55e81e9831d7791085668333"), "recvTime" : ISODate("2015-09-03T10:19:04Z"), "attrName" : "power", "attrType" : "string", "attrValue" : "2077.0" }

From the error, the aggregation is not expecting the $project operator so you should change the projectFields and project variables to show the actual pipeline expressions i.e.
DBObject dateFields = new BasicDBObject("$dayOfMonth", "$recvTime");
DBObject dateObject = new BasicDBObject("day", dateFields);
DBObject groupFields = new BasicDBObject( "_id", dateObject);
DBObject group = new BasicDBObject('$group', groupFields);
iterable = db.getCollection('myCollection').aggregate(group);

In SpagoBI you have 2 ways to create a dataset of type MongoDB:
1) use a dataset of type Query and use JS language
2) use a dataset of type java class and write your code in java
For solution 1 you should create a dataset of type query, select a datasource MongoDB and write on the field QUERY the query in js following the specification written here (http://wiki.spagobi.org/xwiki/bin/view/spagobi_server/data_set#HQueryDataSet28Mongo29)
The button “Edit script” is used to modify the query string and you can find some tip in the wiki. So if the query is SQL you can use a js o groovy script to change the query string (for example apply some logic to create parameters or table names)
You can express your query in this way:
Var query = db.myCollection.aggregate(
[
{
$group : {
_id : { day: { $dayOfMonth: "$recvTime" } }
}
}
]
)

Related

Spring data MongoDb query based on last element of nested array field

I have the following data (Cars):
[
{
"make" : “Ferrari”,
"model" : “F40",
"services" : [
{
"type" : "FULL",
“date_time" : ISODate("2019-10-31T09:00:00.000Z"),
},
{
"type" : "FULL",
"scheduled_date_time" : ISODate("2019-11-04T09:00:00.000Z"),
}
],
},
{
"make" : "BMW",
"model" : “M3",
"services" : [
{
"type" : "FULL",
"scheduled_date_time" : ISODate("2019-10-31T09:00:00.000Z"),
},
{
"type" : "FULL",
“scheduled_date_time" : ISODate("2019-11-04T09:00:00.000Z"),
}
],
}
]
Using Spring data MongoDb I would like a query to retrieve all the Cars where the scheduled_date_time of the last item in the services array is in-between a certain date range.
A query which I used previously when using the first item in the services array is like:
mongoTemplate.find(Query.query(
where("services.0.scheduled_date_time").gte(fromDate)
.andOperator(
where("services.0.scheduled_date_time").lt(toDate))),
Car.class);
Note the 0 index since it's first one as opposed to the last one (for my current requirement).
I thought using an aggregate along with a projection and .arrayElementAt(-1) would do the trick but I haven't quite got it to work. My current effort is:
Aggregation agg = newAggregation(
project().and("services").arrayElementAt(-1).as("currentService"),
match(where("currentService.scheduled_date_time").gte(fromDate)
.andOperator(where("currentService.scheduled_date_time").lt(toDate)))
);
AggregationResults<Car> results = mongoTemplate.aggregate(agg, Car.class, Car.class);
return results.getMappedResults();
Any help suggestions appreciated.
Thanks,
This mongo aggregation retrieves all the Cars where the scheduled_date_time of the last item in the services array is in-between a specific date range.
[{
$addFields: {
last: {
$arrayElemAt: [
'$services',
-1
]
}
}
}, {
$match: {
'last.scheduled_date_time': {
$gte: ISODate('2019-10-26T04:06:27.307Z'),
$lt: ISODate('2019-12-15T04:06:27.319Z')
}
}
}]
I was trying to write it in spring-data-mongodb without luck.
They do not support $addFields yet, see here.
Since version 2.2.0 RELEASE spring-data-mongodb includes the Aggregation Repository Methods
The above query should be
interface CarRepository extends MongoRepository<Car, String> {
#Aggregation(pipeline = {
"{ $addFields : { last:{ $arrayElemAt: [$services,-1] }} }",
"{ $match: { 'last.scheduled_date_time' : { $gte : '$?0', $lt: '$?1' } } }"
})
List<Car> getCarsWithLastServiceDateBetween(LocalDateTime start, LocalDateTime end);
}
This method logs this query
[{ "$addFields" : { "last" : { "$arrayElemAt" : ["$services", -1]}}}, { "$match" : { "last.scheduled_date_time" : { "$gte" : "$2019-11-03T03:00:00Z", "$lt" : "$2019-11-05T03:00:00Z"}}}]
The date parameters are not parsing correctly. I didn't spend much time making it work.
If you want the Car Ids this could work.
public List<String> getCarsIdWithServicesDateBetween(LocalDateTime start, LocalDateTime end) {
return template.aggregate(newAggregation(
unwind("services"),
group("id").last("services.date").as("date"),
match(where("date").gte(start).lt(end))
), Car.class, Car.class)
.getMappedResults().stream()
.map(Car::getId)
.collect(Collectors.toList());
}
Query Log
[{ "$unwind" : "$services"}, { "$group" : { "_id" : "$_id", "date" : { "$last" : "$services.scheduled_date_time"}}}, { "$match" : { "date" : { "$gte" : { "$date" : 1572750000000}, "$lt" : { "$date" : 1572922800000}}}}]

Spring data mongodb - MappedResults returns blank list

Mongodb query is:
db.getCollection('country2').aggregate([{"$lookup":{"from":"state2","localFiel
d":"_id","foreignField":"countryId","as":"state2"}}]).pretty()
returns this result :
{
"_id" : ObjectId("5b2b520333a04107677b5ce1"),
"countryName" : "India",
"state2" : [
{
"_id" : ObjectId("5b2b515b33a04107677b5cdb"),
"stateName" : "UP",
"countryId" : ObjectId("5b2b520333a04107677b5ce1")
},
{
"_id" : ObjectId("5b2b516033a04107677b5cdc"),
"stateName" : "MP",
"countryId" : ObjectId("5b2b520333a04107677b5ce1")
}
]
}
Aggregation query returns blank MappedResults though raw results contain the expected results.
State2 is my java pojo.
public List<BasicDBObject> getAllData() {
LookupOperation lookUpOprn = LookupOperation.newLookup()
.from("state2")
.localField("_id")
.foreignField("countryId")
.as("state2");
org.springframework.data.mongodb.core.aggregation.AggregationOptions options = Aggregation.newAggregationOptions().cursor(new BasicDBObject()).build();
Aggregation aggregation = Aggregation.newAggregation(lookUpOprn).withOptions(options);
List<BasicDBObject> list = mongoOperations.aggregate(aggregation, "country2", BasicDBObject.class).getMappedResults();
return list;
}
Can you please help.

substract two date end return value

I need help to build a query to substract two dates in mongodb.
I have some documents like above :
{"_id" : "32472034809", "center": "102030", dateArq : 141010, inDate : "ISODate("2014-06-06T02:57:19.000-03:00)", biDate : ISODate("2014-06-07T02:57:19.000-03:00)"}
And Im trying to write a query
db.teste.aggregation([{$match : {dateArq : 141010}},{$project : {$subtract : ["$biDate" "$inDate"]}}])
In fact, I want to do : for each _id I want to result biDate - inDate , because I need to see if dateArq keep in a line constante.
In Oracle I did
select dateArq, (biDate - inDate) diff from teste where dateArq = 141010
Tks for help
The document and aggregation pipeline provided had syntax problems, and you needed to put a field name for the result of the $subtract, but otherwise your pipeline works for me:
> db.test.findOne()
{
"_id" : "32472034809",
"center" : "102030",
"dateArq" : 141010,
"inDate" : ISODate("2014-06-06T05:57:19Z"),
"biDate" : ISODate("2014-06-07T05:57:19Z")
}
> db.test.findOnedb.test.aggregate([
{ "$match" : { "dateArq" : 141010 } },
{ "$project" : { "dateDiff" : { "$subtract" : ["$biDate", "$inDate"] } } }
])
{ "_id" : "32472034809", "dateDiff" : NumberLong(86400000) }

parsing aggregation output mongodb using java

{
"serverUsed" : "localhost/127.0.0.1:27017",
"result" : [{
"_id" : {
"$oid" : "529f131430044109e30fc6f9"
},
"html" : {
"table" : {
"tbody" : {
"Barge" : {
"Name" : "ANTVERPIA 56",
"Bargeno" : 6003696,
"Harbour" : "HH",
"Reportedpresent" : " ",
"Starting" : "06-12-2013 spil 2"
}
}
}
}
}]
}
I have this as result, how can i get string value of Name. In this case ANTVERPIA 56.
I have tried with this following code but it does not working, please help.
for (DBObject result1: output.results()){
String name1 = (String)result1.get("html.table.tbody.Barge.Name");
System.out.println(name1);
}
You cannot access nested objects by using "." in Java driver. You have to get DBObject for each nested json object. Following code should solve problem.
for (DBObject result : output.results()) {
DBObject htmlObj = (DBObject) result.get("html");
DBObject tableObj = (DBObject) htmlObj.get("table");
DBObject tbodyObj = (DBObject) tableObj.get("tbody");
DBObject bargeObj = (DBObject) tbodyObj.get("Barge");
String name = (String) bargeObj.get("Name");
}

Mongodb query (hashmap objects)

I want to query all the UserGroup's ID where admins's ID="25160228446835585906563830293" or users's ID ="25160228446835585906563830293".
this is a hashmap key and value pair in java obj hashmap<String,Date>
"25160228446835585906563830293" : ISODate("2013-03-26T04:51:36.731Z")
{ "_id" : ObjectId("51512958849ca4748271c640"),
"_class" : "com.pcd.app.model.UserGroup",
"groupName" : "sdfsadfsad",
"privacyType" : "PRIVACY_OPEN",
"approvalType" : "MEMBER_APPROVAL",
"groupDescription" : "test",
"admins" : {
"25160228446835585906563830293" : ISODate("2013-03-26T04:51:36.731Z"),
"25160228446835585906563830294" : ISODate("2013-03-26T04:51:36.731Z"),
"25160228446835585906563830295" : ISODate("2013-03-26T04:51:36.731Z")
},
"users" : {
"25160228446835585906563830296" : ISODate("2013-03-26T04:51:36.731Z")
}
}
I'd suggest you restructure your document to make it indexable and more easily searched in MongoDB.
Instead of using the id of the admin as a field, add each admin as an object of an array:
"admins" : [
{ id: "25160228446835585906563830293",
date: ISODate("2013-03-26T04:51:36.731Z") }
],
This will make searches more natural:
db.so.find( { "admins.id" :
{ $in: ['25160228446835585906563830293',
'25160228446835585906563830296']}})
You can use the $in (docs) operator to look for admins with an id that matches the list as you wanted (admins.id).
So, given a Java QueryBuilder, it might look something like this:
BasicDBList adminIds = new BasicDBList();
adminIds.addAll(ids); // the ids could be a List<String>
DBObject inClause = new BasicDBObject("$in", adminIds);
DBObject query = new BasicDBObject("admins.id", inClause);
You may want to use ensureIndex to build an index (docs).
Based on your original example, here's the full document for reference:
{
"_id" : ObjectId("51512958849ca4748271c640"),
"_class" : "com.pcd.app.model.UserGroup",
"groupName" : "sdfsadfsad",
"privacyType" : "PRIVACY_OPEN",
"approvalType" : "MEMBER_APPROVAL",
"groupDescription" : "test",
"admins" : [
{ id: "25160228446835585906563830293" ,
date: ISODate("2013-03-26T04:51:36.731Z") },
{ id: "25160228446835585906563830294" ,
date: ISODate("2013-03-26T04:51:36.731Z") },
{ id: "25160228446835585906563830295" ,
date: ISODate("2013-03-26T04:51:36.731Z") }
],
"users" : [
{ id: "25160228446835585906563830296",
date : ISODate("2013-03-26T04:51:36.731Z") }
]
}
If you are using mongodb java driver you can do the following:
BasicDBObject queryForAdminsID = new BasicDBObject("admins." + adminsID, new BasicDBObject("$exists", true));
// BasicDBObject queryForUsersID = new BasicDBObject("users." + usersID, new BasicDBObject("$exists", true));
cursor = coll.find(query); // coll is a DBCollection
try {
while(cursor.hasNext()) {
System.out.println(cursor.next());
}
} finally {
cursor.close();
}
where usersID and adminsID are your ids