how rebuild lucene indexes over hibernate search jpa with mongodb - mongodb

I have accidentally deleted my index directory, now i am trying to rebuild all indexes.
I am using the hibernate search with JPA, lucene and MONGODB.
the following method is returning no results
public void rebuildIndex()throws Exception{
org.hibernate.search.jpa.FullTextEntityManager fem = org.hibernate.search.jpa.Search.getFullTextEntityManager(entityManager);
org.hibernate.search.query.dsl.QueryBuilder queryBuilder = fem.getSearchFactory().buildQueryBuilder().forEntity(Person.class).get();
org.apache.lucene.search.Query query = queryBuilder.all().createQuery();
FullTextQuery fullTextQuery = fem.createFullTextQuery(query, Person.class);
//fullTextQuery.initializeObjectsWith(ObjectLookupMethod.SKIP, DatabaseRetrievalMethod.FIND_BY_ID);
System.out.println(fullTextQuery.toString());
List<Person> results = fullTextQuery.getResultList();
fem.clear();
System.out.println(results.size());
for(Person p : results){
fem.index( p );
fem.flushToIndexes();
fem.clear();
}
//fem.createIndexer().startAndWait();
}
the method is returning no result. how should I get all data from mongoDb to rebuild index?

as hibernate search didn't work with criteria neither with JPQL and has his own JP-QL parser.
I couldn't create a findAll method to retrieve all objetcs
the only way was use a native mongoDb query:
org.hibernate.search.jpa.FullTextEntityManager fem = org.hibernate.search.jpa.Search.getFullTextEntityManager(entityManager);
Mongo mongo = new Mongo("127.0.0.1", 27017);
DB db = mongo.getDB("mainBase");
DBCollection dbCollection = db.getCollection("Persons");
DBCursor cursor = dbCollection.find();
Collection<String> ids = new ArrayList<String>();
String id = "";
while (cursor.hasNext()) {
id = cursor.next().get("_id").toString();
System.out.println(id);
ids.add(id);
}
System.out.println(">"+ids.size());
Person pes;
for(String p : ids){
pes = new Person();
pes.setId(p);
pes = find(pes);
System.out.println("indexing: "+pes.getId());
fem.index( pes );//index each element
fem.flushToIndexes();//apply changes to indexes
fem.clear();//free memory since the queue is processed
}

Related

Mongo 3.6.3 java driver aggregate hint - returns undefined field 'hint'

I am trying to pass a hint on an aggregate in MongoDB Java Driver 3.6.3. The aggregate API allows for adding a hint as in:
MongoCollection<Document> coll = database.getCollection("myCollection")
ArrayList<BasicDBObject> docList = new ArrayList<BasicDBObject>();
BasicDBObject hint = new BasicDBObject("$hint","reportjob_customerId_1_siiDocumentAttributes.deleted_1");
MongoCursor<Document> cursor = coll.aggregate(docList).hint(hint).allowDiskUse(batchContext.isAllowDiskUse()).iterator();
I've tried with $hint and hint, but Mongo always returns Command failed with error -1: 'unrecognized field 'hint'
How do I properly send the hint on the aggregate call?
Looks like there is no support to pass the index name directly to hint. So you have pass the index creation document to the hint which you can get by name and use key to get the index document.
MongoCollection<Document> coll = database.getCollection("myCollection");
Bson index = coll.listIndexes().into(new ArrayList<>()).stream().filter(item -> item.getString("name").equals("reportjob_customerId_1_siiDocumentAttributes.deleted_1")).findFirst().get().getString("key");
List<BasicDBObject> docList = new ArrayList<BasicDBObject>();
MongoCursor<Document> cursor = coll.aggregate(docList).hint(index).allowDiskUse(batchContext.isAllowDiskUse()).iterator();
Legacy driver had support for both index name and document.
MongoClient client = new MongoClient();
DB database = client.getDB("myDatabase");
DBCollection coll = database.getCollection("myCollection");
List<BasicDBObject> docList = new ArrayList<BasicDBObject>();
AggregationOptions options = AggregationOptions.builder().allowDiskUse(batchContext.isAllowDiskUse()).build();
DBCursor dbCursor = ((DBCursor) coll.aggregate(docList, options)).hint("reportjob_customerId_1_siiDocumentAttributes.deleted_1");
You can create a jira to have the option to pass index name in the new driver here

In mongodb version 3 using java api how do we give a hint while querying

In mongodb version 3 using java api how do we give a hint while querying
The query result is FindIterable which has MongoCursor.
How should I give a hint to use a particular index.
With older versions there is DBCursor which have API's for hint.
modifiers should be used:
BasicDBObject index = new BasicDBObject("num", 1);
BasicDBObject hint = new BasicDBObject("$hint", index);
FindIterable<Document> iterable = collection.find().modifiers(hint);
You need to use FindIterable.modifiers method to specify $hint:
private static void testHint(MongoDatabase db) {
MongoCollection<Document> col = db.getCollection("Stores");
FindIterable<Document> iterable = col.find(new Document("store","storenumbervalue"));
iterable.modifiers(new Document("$hint","index_name"));
MongoCursor curs = iterable.iterator();
while(curs.hasNext()){
System.out.println(curs.next());
}
}
index_name - actual index name in mongo.

How to use DBObject Query in distinct function of mongo template?

i want to find distinct value of a field with some Query criteria. my code is..
public List searchservice(String th_type) {
Query query = new Query();
query.addCriteria(Criteria.where("th_type").regex(th_type));
List list = operations.getCollection("doclist").distinct("th_type", query);
return list;
}
in mongo template a distinct function is defined
mongoTemplate.getCollection(collection).distinct(key, query)
bu my code is giving error because i am using simple Query object instead of DBObject Query. how can i use DBObject Query here?
Use a BasicDBObject for this:
public List searchservice(String th_type) {
BasicDBObject dbObject = new BasicDBObject();
dbObject.append("th_type", th_type);
DBCollection dBCollection = operations.getCollection("doclist");
List list = dBCollection.distinct("th_type", dbObject);
return list;
}
UPDATE:
With regex:
BasicDBObject regexQuery = new BasicDBObject();
regexQuery.put("th_type", new BasicDBObject("$regex", th_type));
List list = operations.getCollection("doclist").distinct("th_type",regexQuery);

MongoDB Taking Too Long time in C#.net

I am retrieving data from mongoDB using C# driver, It is taking a lot of time when i do to list Please help me
My Mongoquery is
var documentReportIds = new BsonValue[] { LatestReportIds };
var documentChennelIds = new BsonValue[] { Cid };
var documentPropertyIds = new BsonValue[] { Pid };
IMongoQuery query = new QueryDocument();
query = Query.And(Query.GTE("CheckInDate", startdate.Date.AddMinutes(330)),
Query.LTE("CheckInDate", endDate.Date.AddMinutes(330)));
query = Query.And(query, Query.EQ("SubscriberPropertyId", reportFilter.SubscriberPropertyId));
query = Query.And(query, Query.EQ("LengthOfStay", reportFilter.LOS));
query = Query.And(query, Query.In("ReportId", documentReportIds));
query = Query.And(query, Query.In("ChannelId", documentChennelIds));
query = Query.And(query, Query.In("PropertyId", documentPropertyIds));
MongoDBEntities<ScheduleOptimizationReportDetails> _obj = new MongoDBEntities<ScheduleOptimizationReportDetails>();
var list= _obj.GetSchedularOptimizationJoin(query);
Class from where it perform data retrieving
public class MongoDBEntities<T>
{
MongoDatabase db = MongoDBInstance.GetMongoDatabase;
public List GetSchedularOptimizationJoin(IMongoQuery query)
{
MongoCollection MCollection = db.GetCollection(“Subscription_OptimisedReports”);
MongoCursor cursor = MCollection.FindAs(query).SetFields(Fields.Include(“ScheduleLogId”, “SubscriberPropertyId”, “CheckInDate”, “ReportId”, “CreatedDate”));
List entities = cursor.ToList();
return entities ;
}
}
what is another option to select data in C#, I have also applied indexing on column.
Please help me how to solve it.
You can use the MongoDB.Driver.Linq package to help you create your querys with Linq expressions.

Copying a mongo collection using Java Driver

I want to copy the contents from one collection to another.
in mongod this can be done:
db.tempMongoItem.find().forEach( function(x){db.mongoItem.insert(x)} )
Using Java Mongo Driver, I try:
DB db = mongoClient.getDB("mydb")
CommandResult result = db.command("db.tempMongoItem.find().forEach( function(x){db.mongoItem.insert(x)} )")
But I get:
result = [serverUsed:localhost:27017, ok:0.0, errmsg:no such cmd: db.tempMongoItem.find().forEach( function(x){db.mongoItem.insert(x)} ), code:59, bad cmd:[db.tempMongoItem.find().forEach( function(x){db.mongoItem.insert(x)} ):true]]
Any ideas?
You need to emulate the same thing JS is doing in Java, which means getting a cursor and iterating over it, inserting each document into new collection.
Something like this (coll is current, coll2 is new collection):
DBCursor cursor = coll.find();
try {
while(cursor.hasNext()) {
coll2.insert(cursor.next());
}
} finally {
cursor.close();
}
Both coll and coll2 are assumed to be DBCollection type.
Since it appears you are copying within the same DB, there is another way to do this if you are using 2.6 MongoDB using aggregation framework $out stage:
db.collection.aggregate({"$out":"newCollection"});
Note that this is limited to outputting into the same DB that original collection is in.
The following JAVA code will copy the collection from source to destination for a given database name (using mongodb-driver 3.0.4)
/** Clone a collection.
*
* #param fromCollectionName - The name of collection to be cloned
* #param toCollectionName - The name of the cloned collection
* #param dbName - The name of the database
*/
public void cloneCollection(String fromCollectionName, String toCollectionName, String dbName) throws MongoException {
MongoCollection toCol = this.getCollection(toCollectionName, dbName);
if (toCol != null) {
throw new MongoException("The destination collection already exists.");
}
List<Document> ops = new ArrayList<>();
ops.add(new Document("$out",toCollectionName));
MongoCollection sourceCollection = this.getCollection(fromCollectionName, dbName);
sourceCollection.aggregate(ops);
}
public MongoCollection getCollection(String collection, String dbName) {
MongoClient mongo = new MongoClient(new ServerAddress("localhost", Integer.parseInt(port)));
MongoDatabase database = mongo.getDatabase(dbName);
return curdb.getCollection(collection);
}
Please note that this will not copy over the indices that you have created in source collection. You will have to copy the indices seperately
Following up on Asya's response, you can use Java 8 Lambda functions to do:
collSource.find().forEach((Block<Document>) collTarget::insertOne);