Looping Over a Java Object and Passing One Field to a Mongo Query Similar to SQL WHERE...IN Clause - mongodb

I am trying to construct a MongoDB equivalent of an SQL WHERE IN clause by iterating over a Java Object List and using one field in that list to fill in the IN data. Using generic example, I would like to build the following command in MongoDB syntax:
SELECT title FROM albums WHERE recorded_year IN ('1967','1968','1969','1970');
The SQL command IN data is extracted from the Album object recordedYear value using the following loop:
if (albums.size() <= 1000) {
sb.append("SELECT title FROM albums WHERE recorded_year");
sb.append(" IN (");
for (int i = 0; i < albums.size(); i++) {
sb.append("'");
sb.append(albums.get(i).getRecordedYear());
sb.append("'");
if (i < albums.size() - 1) {
sb.append(",");
} else {
sb.append(")");
}
}
}
Most Mongo/Java sites I visited seem to deal with command structures using hard-coded values which, while helpful, is completely impractical in real world applications. Any help in pointing to a good tutorial, or if someone has the actual code itself would be greatly appreciated.
Thanks.

But the issue I am having is understanding how to pass a Java Object
list to the to the getAllDocuments...
Make an array of elements which you want to match with the field using the in operator. For example, If you have a someObject.year field, then the array will have the year values; int [] matchYears = { 1989, 2001, 2012 }. Instead of an array you can also use a List collection.
The query:
Bson queryFilter = in("recordedYear", matchYears);
List<Document> result = new ArrayList<>();
collection.find(queryFilter).into(result);
result.forEach(System.out::println);
The queryFilter is built using the com.mongodb.client.model.Filters factory class.
The MongoDB documentation on using the $in operator.
NOTE: The int [] matchYears = { 1989, 2001, 2012 } can be also be created as
int [] matchYears = { javaObject1.recorded_year, javaObject2.recorded_year, ... }.

I didn't quite implement it with BSON, but the logic in the method appears to be working with the code below. Thank you again for your help.
public void getAlbumYears(MongoCollection<Document> collection, List<Album> albums) {
BasicDBObject inQuery = new BasicDBObject();
List<String> year = new ArrayList<>();
for(Album album : albums) {
year.add(album.getYear());
}
inQuery.put("year", new BasicDBObject("$in", year));
for (Document document : collection.find(inQuery)) {
System.out.println(document.toJson());
}
}

Related

How to sort this array of results?

Here is my query that attempts to sort an array of MongoDB documents based on a derived field called expiresAt. expiresAt is a Date object that represents the date string stored in doc.expirationDate.
It fails with an error TypeError: ... .sort({}) is not a function (shell):5
db.tokens.find().map(function(doc) {
var expiryDate = new Date(doc.credentialsMap.linkedin.expirationDate);
doc.expiresAt = expiryDate;
return doc;
}).sort({'expiresAt': -1});
What am I doing wrong? It's unclear to me exactly what return type map provides. If it's a cursor then why isn't sort supported? It's clearly available in the docs.
cursor.map() returns a Java script array.
The way you're calling sort() assumes that the return value is a MongoDB cursor, that's why it fails.
You'd have to use the regular Array.sort syntax.
For example, to have your map results sorted in descending order, use the following:
db.tokens.find().map(function(doc) {
var expiryDate = new Date(doc.credentialsMap.linkedin.expirationDate);
doc.expiresAt = expiryDate;
return doc;
}).sort(function(a,b) { return b.expiresAt - a.expiresAt});

mongodb cursor map function in java

i have a collection 'placements', each document has fields: placement_id, program_id, category, ... i need to find all placements what has program_id = 3 and only return a list of placement_id,
i can do it from mongo command line like this:
db.placements.find({program_id:{$in: [3]}}, {placement_id:1, _id:0}).map( function(doc){return doc.placement_id})
it return placement_ids in a array:
[196, 197, 198...]
but how can i implement the above query in Java, i checked the mongodb java api's DBCursor class, it doesn't have any function as 'map' or 'forEach'.
The DBCursor.next returns DBOBject which has a get method which you can use to push the values to an array.
I am thinking of something like this. (Not tested)
List<Integer> placementIdList = new ArrayList<Integer>();
while (cursor.hasNext()) {
DBObject obj = cursor.next();
int id = obj.get("placement_id");
placementIdList.add(id);
}
UPDATE :
The implementation on the database side for the map implementation is very similar to above.
mongos> db.test.find().map
function ( func ){
var a = [];
while ( this.hasNext() )
a.push( func( this.next() ) );
return a;
}
So either you do it in db or in the Java layer.
Advantage of doing it in db would be transmitting a slimmer packet across the wire (lesser serialization/deserialization costs), whereas doing it in Java you can take advantage of the processing power of a JVM.

CouchDB Map/Reduce view query from Ektorp

I'm trying to execute a query from java against a Map/Reduce view I have created on the CouchDB.
My map function looks like the following:
function(doc) {
if(doc.type == 'SPECIFIC_DOC_TYPE_NAME' && doc.userID){
for(var g in doc.groupList){
emit([doc.userID,doc.groupList[g].name],1);
}
}
}
and Reduce function:
function (key, values, rereduce) {
return sum(values);
}
The view seems to be working when executed from the Futon interface (without keys specified though).
What I'm trying to do is to count number of some doc types belonging to a single group. I want to query that view using 'userID' and name of the group as a keys.
I'm using Ektorp library for managing CouchDB data, if I execute this query without keys it returns the scalar value, otherwise it just prints an error saying that for reduce query group=true must be specified.
I have tried the following:
ViewQuery query = createQuery("some_doc_name");
List<String> keys = new ArrayList<String>();
keys.add(grupaName);
keys.add(uzytkownikID);
query.group(true);
query.groupLevel(2);
query.dbPath(db.path());
query.designDocId(stdDesignDocumentId);
query.keys(keys);
ViewResult r = db.queryView(query);
return r.getRows().get(0).getValueAsInt();
above example works without 'keys' specified.
I have other queries working with ComplexKey like eg:
ComplexKey key = ComplexKey.of(userID);
return queryView("list_by_userID",key);
but this returns only a list of type T (List) - using CouchDbRepositorySupport of course - and cannot be used with reduce type queries (from what I know).
Is there any way to execute the query with reduce function specified and a complex key with 2 or more values using Ektorp library? Any examples highly appreciated.
Ok, I've found the solution using trial and error approach:
public int getNumberOfDocsAssigned(String userID, String groupName) {
ViewQuery query = createQuery("list_by_userID")
.group(true)
.dbPath(db.path())
.designDocId(stdDesignDocumentId)
.key(new String[]{userID,groupName});
ViewResult r = db.queryView(query);
return r.getRows().get(0).getValueAsInt();
}
So, the point is to send the complex key (not keys) actually as a single (but complex) key containing the String array, for some reason method '.keys(...)' didn't work for me (it takes a Collection as an argument). (for explanation on difference between .key() and .keys() see Hendy's answer)
This method counts all documents assigned to the specific user (specified by 'userID') and specific group (specified by 'groupName').
Hope that helps anybody executing map/reduce queries for retrieving scalar values from CouchDB using Ektorp query.
Addition to Kris's answer:
Note that ViewQuery.keys() is used when you want to query for documents matching a set of keys, not for finding document(s) with a complex key.
Like Kris's answer, the following samples will get document(s) matching the specified key (not "keys")
viewQuery.key("hello"); // simple key
viewQuery.key(documentSlug); // simple key
viewQuery.key(new String[] { userID, groupName }); // complex key, using array
viewQuery.key(ComplexKey.of(userID, groupName)); // complex key, using ComplexKey
The following samples, on the other hand, will get document(s) matching the specified keys, where each key may be either a simple key or a complex key:
// simple key: in essence, same as using .key()
viewQuery.keys(ImmutableSet.of("hello"));
viewQuery.keys(ImmutableSet.of(documentSlug1));
// simple keys
viewQuery.keys(ImmutableSet.of("hello", "world"));
viewQuery.keys(ImmutableSet.of(documentSlug1, documentSlug2));
// complex key: in essence, same as using .key()
viewQuery.keys(ImmutableSet.of(
new String[] { "hello", "world" } ));
viewQuery.keys(ImmutableSet.of(
new String[] { userID1, groupName1 } ));
// complex keys
viewQuery.keys(ImmutableSet.of(
new String[] { "hello", "world" },
new String[] { "Mary", "Jane" } ));
viewQuery.keys(ImmutableSet.of(
new String[] { userID1, groupName1 },
new String[] { userID2, groupName2 } ));
// a simple key and a complex key. while technically possible,
// I don't think anybody actually does this
viewQuery.keys(ImmutableSet.of(
"hello",
new String[] { "Mary", "Jane" } ));
Note: ImmutableSet.of() is from guava library.
new Object[] { ... } seems to have same behavior as ComplexKey.of( ... )
Also, there are startKey() and endKey() for querying using partial key.
To send an empty object {}, use ComplexKey.emptyObject(). (only useful for partial key querying)

How do I get the date a MongoDB collection was created using MongoDB C# driver?

I need to iterate through all of the collections in my MongoDB database and get the time when each of the collections was created (I understand that I could get the timestamp of each object in the collection, but I would rather not go that route if a simpler/faster method exists).
This should give you an idea of what I'm trying to do:
MongoDatabase _database;
// code elided
var result = _database.GetAllCollectionNames().Select(collectionName =>
{
_database.GetCollection( collectionName ) //.{GetCreatedDate())
});
As far as I know, MongoDB doesn't keep track of collection creation dates. However, it's really easy to do this yourself. Add a simple method, something like this, and use it whenever you create a new collection:
public static void CreateCollectionWithMetadata(string collectionName)
{
var result = _db.CreateCollection(collectionName);
if (result.Ok)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
collectionMetadata.Insert(new { Id = collectionName, Created = DateTime.Now });
}
}
Then whenever you need the information just query the collectionMetadata collection. Or, if you want to use an extension method like in your example, do something like this:
public static DateTime GetCreatedDate(this MongoCollection collection)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
var metadata = collectionMetadata.FindOneById(collection.Name);
var created = metadata["Created"].AsDateTime;
return created;
}
The "creation date" is not part of the collection's metadata. A collection does not "know" when it was created. Some indexes have an ObjectId() which implies a timestamp, but this is not consistent and not reliable.
Therefore, I don't believe this can be done.
Like Mr. Gates VP say, there is no way using the metadata... but you can get the oldest document in the collection and get it from the _id.
Moreover, you can insert an "empty" document in the collection for that purpose without recurring to maintain another collection.
And it's very easy get the oldest document:
old = db.collection.find({}, {_id}).sort({_id: 1}).limit(1)
dat = old._id.getTimestamp()
By default, all collection has an index over _id field, making the find efficient.
(I using MongoDb 3.6)
Seems like it's some necroposting but anyway: I tried to find an answer and got it:
Checked it in Mongo shell, don't know how to use in C#:
// db.payload_metadata.find().limit(1)
ObjectId("60379be2bec7a3c17e6b662b").getTimestamp()
ISODate("2021-02-25T12:45:22Z")

MongoDB C# offic. List<BsonObject> query issue and always olds values?

I have not clearly issue during query using two criterials like Id and Other. I use a Repository storing some data like id,iso,value. I have created an index("_id","Iso") to performs queries but queries are only returning my cursor if i use only one criterial like _id, but is returning nothing if a use two (_id, Iso) (commented code).
Are the index affecting the response or the query method are failing?
use :v1.6.5 and C# official.
Sample.
//Getting Data
public List<BsonObject> Get_object(string ID, string Iso)
{
using (var helper = BsonHelper.Create())
{
//helper.Db.Repository.EnsureIndex("_Id","Iso");
var query = Query.EQ("_Id", ID);
//if (!String.IsNullOrEmpty(Iso))
// query = Query.And(query, Query.EQ("Iso", Iso));
var cursor = helper.Db.Repository.FindAs<BsonObject>(query);
return cursor.ToList();
}
}
Data:
{
"_id": "2345019",
"Iso": "UK",
"Data": "Some data"
}
After that I have Updated my data using Update.Set() methods. I can see the changed data using MongoView. The new data are correct but the query is always returning the sames olds values. To see these values i use a page that can eventually cached, but if add a timestamp at end are not changing anything, page is always returning the same olds data. Your comments are welcome, thanks.
I do not recall offhand how the C# driver creates indexes, but the shell command for creating an index is like this:
db.things.ensureIndex({j:1});
Notice the '1' which is like saying 'true'.
In your code, you have:
helper.Db.Repository.EnsureIndex("_Id","Iso");
Perhaps it should be:
helper.Db.Repository.EnsureIndex("_Id", 1);
helper.Db.Repository.EnsureIndex("Iso", 1);
It could also be related to the fact that you are creating indexes on "_Id" and the actual id field is called "_id" ... MongoDB is case sensitive.
Have a quick look through the index documentation: http://www.mongodb.org/display/DOCS/Indexes