Im trying to learn how to make sql like queries in Mongo and found this aggregation framework, when i run the code the average always turns out "0" all though the 'columns' have numbers in them, if i understand it right mongoDB saves all values as a strings( the number where integers at first in my sql server database which i transferred to mongoDB) so i dont have to care about types really when working with MongoDB? Can someone see why my average function dont work? The ordered quantity has number between 1.000000 - 7.000000.
DBCollection order = db.getCollection("Orderstatus");
DBObject match = new BasicDBObject("$match", new BasicDBObject("Company", "100") );
DBObject fields = new BasicDBObject("Facility", 1);
fields.put("Ordered quantity", 1);
fields.put("_id", 0);
DBObject project = new BasicDBObject("$project", fields );
DBObject groupFields = new BasicDBObject( "_id", "$Facility");
groupFields.put("average", new BasicDBObject( "$avg", "$Ordered quantity"));
DBObject group = new BasicDBObject("$group", groupFields);
AggregationOutput output = order.aggregate( match, project,group );
System.out.println(output);
The thing is that i thought that the numbers would be as integers since i got them from my sql server database where they are stored as integers using the code below. I see now that i use getString() when getting the values, is that why the numbers are strings in mongodb? how can i get them as integers?? i really want to be able to manipulate them as numbers!
StringBuilder orderstatus = new StringBuilder();
orderstatus.append("SELECT * FROM dbo.fact_orderstatus");
PreparedStatement t = connect.prepareStatement(orderstatus.toString());
DBCollection orderstat = db.getCollection("Orderstatus");
ResultSet v = t.executeQuery();
ResultSetMetaData rsm = t.getMetaData();
int column = rsm.getColumnCount();
while (v.next()) {
BasicDBObject orderObj = new BasicDBObject();
for(int x=1; x<column +1; x++){
String namn= rsm.getColumnName(x);
String custNum = (v.getString(x));
if (custNum != null && !custNum.trim().isEmpty()
&& custNum.length() != 0)
orderObj.append(namn, custNum);
}
orderstat.insert(orderObj)
The reason this is happening is that MongoDB does care about types of values you save.
If you want to store numbers, make sure they are not quoted otherwise they become stored as strings and you lose ability to manipulate them as numbers.
Javascript also differentiates between numbers and string, but MongoDB supports more number types than simple JavaScript. You can read more here.
Related
I am trying to construct a MongoDB equivalent of an SQL WHERE IN clause by iterating over a Java Object List and using one field in that list to fill in the IN data. Using generic example, I would like to build the following command in MongoDB syntax:
SELECT title FROM albums WHERE recorded_year IN ('1967','1968','1969','1970');
The SQL command IN data is extracted from the Album object recordedYear value using the following loop:
if (albums.size() <= 1000) {
sb.append("SELECT title FROM albums WHERE recorded_year");
sb.append(" IN (");
for (int i = 0; i < albums.size(); i++) {
sb.append("'");
sb.append(albums.get(i).getRecordedYear());
sb.append("'");
if (i < albums.size() - 1) {
sb.append(",");
} else {
sb.append(")");
}
}
}
Most Mongo/Java sites I visited seem to deal with command structures using hard-coded values which, while helpful, is completely impractical in real world applications. Any help in pointing to a good tutorial, or if someone has the actual code itself would be greatly appreciated.
Thanks.
But the issue I am having is understanding how to pass a Java Object
list to the to the getAllDocuments...
Make an array of elements which you want to match with the field using the in operator. For example, If you have a someObject.year field, then the array will have the year values; int [] matchYears = { 1989, 2001, 2012 }. Instead of an array you can also use a List collection.
The query:
Bson queryFilter = in("recordedYear", matchYears);
List<Document> result = new ArrayList<>();
collection.find(queryFilter).into(result);
result.forEach(System.out::println);
The queryFilter is built using the com.mongodb.client.model.Filters factory class.
The MongoDB documentation on using the $in operator.
NOTE: The int [] matchYears = { 1989, 2001, 2012 } can be also be created as
int [] matchYears = { javaObject1.recorded_year, javaObject2.recorded_year, ... }.
I didn't quite implement it with BSON, but the logic in the method appears to be working with the code below. Thank you again for your help.
public void getAlbumYears(MongoCollection<Document> collection, List<Album> albums) {
BasicDBObject inQuery = new BasicDBObject();
List<String> year = new ArrayList<>();
for(Album album : albums) {
year.add(album.getYear());
}
inQuery.put("year", new BasicDBObject("$in", year));
for (Document document : collection.find(inQuery)) {
System.out.println(document.toJson());
}
}
With the following code, I would iterate over the MongoDB collection to compute its count.(with loop, limit and sort by _id ascendingly)
Ignore the fact there are other ways to compute the count since I am using this way to do other things(This code just illustrates the fact that I can't fetch out all the documents).
But it doesn't yield the correct the count.
The total documents of my collection should be 12637833,
but with the following code, the count printed is 12602135.
That is, there are about 30 thousand gap between them.
Can someone help what may leads to this problem? Thanks!
DBObject query = new BasicDBObject();
DBObject sorter = new BasicDBObject("_id", 1);
ObjectId largestObjectId = null;
int count = 0;
while (true) {
DBCursor cursor = c.find(query).sort(sorter).limit(200000);
if (!cursor.hasNext()) {
break;
}
while (cursor.hasNext()) {
count++;
BasicDBObject document = (BasicDBObject) cursor.next();
if (document == null) {
continue;
}
largestObjectId = (ObjectId) document.get("_id");
}
query = new BasicDBObject("_id", new BasicDBObject("$gt", largestObjectId));
cursor.close();
}
System.out.println("Total Count is: " + count)
The mismatch in count of the documents can happen only in an environment where one thread reads the data and other thread writes something into the database(i.e. In a multi threaded application in which one thread writes to mongodb and another performs certain operation).
Moreover when we have DBCursor for iterating collection with 12637833 records, not all the records will be loaded initially into the memory(JVM memory) instead it would be lazily fetched from the database.
Interestingly in a multithreaded application you can find difference between DBCursor length and iterating the cursor and obtaining the count, since when we use length or toArray on a DBCursor will irrevocably turn DBCursor into an array. So extra precaution should be taken before invoking toArray or length in DBCursor since it will suddenly increase the records in the memory. Say if we have ten million records, then there will be a ten-million element array in memory. So always use skip() and limit() to minimize the results before using length or toArray in a DBCursor.
I have a problem.
I have collection of documents like this:
{
id (not _id),
type,
number
}
And what I want to do is aggregate cells with specific type and minimum number for this type for every id and delete them from this collection. Basically, single id can have few different number for specific type and I want to delete Document with the lowest value.
I tried to aggregate it using Java Driver 3 and mongoshell but I stucked on constructing it.
You can take reference from something like this
List<DBObject> pipeline=new ArrayList<DBObject>();
DBObject match = new BasicDBObject("$match", new BasicDBObject("date", sdf.format(new Date())).append("country", country).append("operator", operator).append("server_ip", server_ip));
DBObject unwind = new BasicDBObject("$unwind", "$details");
DBObject match2 = new BasicDBObject("$match", new BasicDBObject("details.type", "application_health"));
DBObject sort = new BasicDBObject("$sort", new BasicDBObject("details.datetime", -1));
DBObject limit = new BasicDBObject("$limit", 1);
pipeline.add(match);
pipeline.add(unwind);
pipeline.add(match2);
pipeline.add(sort);
pipeline.add(limit);
AggregationOutput outputoutput = collection.aggregate(pipeline);
First, let me say that I am a novice so may not ask this question as clearly as it could have been asked. here goes
From java I am calling a function FindName(int,string) (snippet below), I am getting the JSONParser errors on the DBObject ref statement. The code works fine if I hardcode the value pairs, but the goal is for the calling routine to pass different values and get the correct result.
The values being passed are valid, I even println to verify the values.
The issue seems to be that even though the values are valid int and string that mongodb does not seem to allow you to specify the names of the values that are passed to this function in the key-value pairs.
Therefore I am guessing that you need to transform the date, name values that are passed in, to a format that is acceptable to the key-value pair. The question is how to do that.
static String FindName(int date, String name)
MongoClient mongoClient;
try {
mongoClient = new MongoClient("xxx.xx.xx.xxx",27017);
}
catch (UnknownHostException e)
{
e.printStackTrace();
return name;
}
DB db = mongoClient.getDB("tpfdf");
DBCollection PNR = db.getCollection("PNR");
DBObject ref = (DBObject) JSON.parse("{PnrByDateFlightCustom: {date: date), pname: name'}}");
at com.mongodb.util.JSONParser.parse(JSON.java:208)
at com.mongodb.util.JSONParser.parseObject(JSON.java:241)
at com.mongodb.util.JSONParser.parse(JSON.java:205)
at com.mongodb.util.JSONParser.parseObject(JSON.java:241)
at com.mongodb.util.JSONParser.parse(JSON.java:205)
at com.mongodb.util.JSONParser.parse(JSON.java:155)
at com.mongodb.util.JSON.parse(JSON.java:93)
at com.mongodb.util.JSON.parse(JSON.java:74)
at MongoDB5.locatePassengerName(MongoDB5.java:38)
at MongoDB5.main(MongoDB5.java:11)
Update. After many trial and errors, I found that if you dynamically build the string and then execute the ref with the dynamic string, it works.
This worked. I do not know if this is the correct way to code this, but this works so will go with it for now.
String mdbStr = "{PnrByDateCustom: {date:" + value + "}}";
System.out.println("mdbStr = " + mdbStr);
DB db = mongoClient.getDB("tpfdf");
DBCollection PNR = db.getCollection("PNR");
DBObject ref = (DBObject) JSON.parse("{}");
// ref = (DBObject) JSON.parse("{PnrByDateCustom: {date: value} }" );
ref = (DBObject) JSON.parse(mdbStr);
I have a problem with mongoTemplate in Spring when I am trying to query using NearQuery with a Sort. The Sort does not work:
Query query = new Query();
query.with(new Sort(Direction.DESC, "timeStamp"));
Criteria criteria = new Criteria();
criteria.and("type").is("MeasurementPoint");
query.addCriteria(criteria);
NearQuery queryN = NearQuery.near(p).maxDistance(new Distance(distance, Metrics.KILOMETERS)).num(range).query(query);
GeoResults<MeasurementPoint> geoPoints = mongoTemplate.geoNear(queryN, MeasurementPoint.class);
I do not know what I am doing wrong but the geoResult returns me the first match, not the last one (Sorted DESC). So, I assume that the Sort is not working properly.
Any idea? Is it a bug?
Thanks!
Unfortunately, isn't possible sort geoNear results since it doesn't returns a cursor and the default sort is the distance with point. What you could do is sort the results manually in Java code. Note that the code bellow ignores the distance and sorts only by "timeStamp".
List<GeoResult<Person>> results = geoPoints.getContent();
Collections.sort(results, new Comparator<GeoResult<Person>>() {
#Override
public int compare(GeoResult<Person> o1, GeoResult<Person> o2) {
return o1.getContent().getTimeStamp() == 2.getContent().getTimeStamp() ? 0 :
(o1.getContent().getTimeStamp() > o2.getContent().getTimeStamp() ? 1 : -1) ;
}
});
An alternative approach is use $geoWithin and $centerSphere. Since you're limiting results with some distance (distance variable), that could work.
Query query = Query.query(Criteria.where("coords").withinSphere(new Circle(p, new Distance(distance, Metrics.KILOMETERS).getNormalizedValue())));
query.with(new Sort(Direction.DESC, "timeStamp"));
Criteria criteria = new Criteria();
criteria.and("type").is("MeasurementPoint");
query.addCriteria(criteria);
List<Person> geoPoints = mongoTemplate.find(query, MeasurementPoint.class);
You could find more information about $geoWithin and $centerSphere here:
http://docs.mongodb.org/manual/reference/operator/geoWithin/
http://docs.mongodb.org/manual/reference/operator/centerSphere/