I am getting below error, when I try to use greater than or less than in mongoDB where clause: Any idea how to avoid this?
Response message: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: gt for class: Script117
Below is my Full Groovy Code till now:
Everything works except the GREATER THAN (or lt) query filter:
import com.mongodb.DB;
import org.apache.jmeter.protocol.mongodb.config.MongoDBHolder;
import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
import com.mongodb.DBCursor;
import com.mongodb.DBCollection;
import com.mongodb.*;
//======================================================================
MongoClient mongoClient = new MongoClient(new ServerAddress("${serverIP}", ${serverPort}));
DB db = mongoClient.getDB("${mongodb}");
DBCollection coll = db.getCollection("${collectionName}");
StringBuilder rs = new StringBuilder();
rs.append("Collection: \n");
BasicDBObject allQuery = new BasicDBObject();
BasicDBObject fields = new BasicDBObject();
fields.put("name", 1); //projected fields
fields.put("age", 1); //projected fields
fields.put("eyeColor", 1); //projected fields
fields.put("balance", 1); //projected fields
BasicDBObject whereQuery = new BasicDBObject();
//whereQuery.put("gender", "female");
whereQuery.put("age", new BasicDBObject("$gt", 30));
DBCursor cursor = coll.find(whereQuery, fields);
while( cursor.hasNext() ) {
DBObject obj = cursor.next();
rs.append(obj.toString());
rs.append("\n");
}
rs.toString();
//String variable at the LAST LINE of this Groovy script will be displayed in Results tab, even if no Display methods are invoked !!!
The error message: groovy.lang.MissingPropertyException, is thrown when:
An exception occurred while a dynamic property dispatch fails with an
unknown property.
Note: the Missing*Exception classes are named for consistency and to avoid conflicts with JDK exceptions of the same name.
The attempt to insert the "age" as key and the new BasicDBObject("$gt",30) as a value, cause to the exception you get, because the JVM looks for a property calls gt which is not exists.
The put method means: Sets a name/value pair in this object.
In the code you have you intend to append the search criteria, and not to put it.
Related to the code you have, I would recommend replacing the using of put method with append.
This is your code:
BasicDBObject whereQuery = new BasicDBObject();
//whereQuery.put("gender", "female");
whereQuery.put("age", new BasicDBObject("$gt", 30));
If you replace the:
whereQuery.put("age", new BasicDBObject("$gt", 30));
With:
whereQuery.append("age", new BasicDBObject("$gt", 30));
Another suggestion you can use is the Criteria.where syntax,
which is much more readable.
def query = Query.query(Criteria.where('gender').is('female').and('age')
.gt(30))
collection.find(query, fields)
Another small issue, because your code is written in Groovy you should not finish each line with ;
Have you tried using single quotes so groovy doesn't think you're doing string templating?
whereQuery.put("age", new BasicDBObject('$gt', 30))
// ^ ^ here
You should be avoiding:
Referring JMeter Variables as ${var}
Using $var structures
as they have special meaning in Groovy, so:
Replace these ${serverIP} with vars.get("serverIP"), ${mongodb} with vars.get("mongodb"), etc.
Replace double quotes with single quotes, especially in the line where you use $gt function
(Optional) semicolons at the lines end are not necessary
References:
Groovy Differences with Java
How to Load Test MongoDB with JMeter
Related
I'm trying to get observable mongo data using Rx 3.1.0, im trying to implement "Observable.from_" an error appear that "from_" is not recongized as observable method
from pymongo import MongoClient
from rx import Observable
def collect_data(dbname, collection):
clt = MongoClient('localhost', 27017)
db = clt[dbname]
col = db[collection]
colData= col.find().limit(10)
return Observable.from_(colData)
collect_data('test', 'money').subscribe(print(data))
an error appear that "from_" is not recongized as observable method
I'm trying to use rx.of
from pymongo import MongoClient
import rx
def collect_data(dbname, collection):
clt = MongoClient('localhost', 27017)
db = clt[dbname]
col = db[collection]
colData= col.find().limit(10)
return rx.of(colData)
collect_data('test', 'money').subscribe(lambda data: print('{0}'.format(data)))
I get
<pymongo.cursor.Cursor object at 0x000001BF824FE8C8>
find() returns a cursor which is an iterable; so use it in conjection with a for loop or similar. e.g.
for item in colData
...
or
list(colData)
instead of
Observable.from_(colData)
i should replace with the new statement of RxPY 3.1.0
rx.from_(colData).pipe(
ops.filter(...),
ops.distinct(...),
ops.take(...),
ops.map(...),
)
I am trying to build a basic search for my API backend. Users pass arbitrary queries and the backend is supposed to return results (obviously). I would prefer a solution that works with a local index as well as Elasticsearch.
On my entity I defined an analyzer like this:
#AnalyzerDef(name = "ngram",
tokenizer = #TokenizerDef(factory = StandardTokenizerFactory.class ),
filters = {
#TokenFilterDef(factory = StandardFilterFactory.class),
#TokenFilterDef(factory = LowerCaseFilterFactory.class),
#TokenFilterDef(factory = StopFilterFactory.class),
#TokenFilterDef(factory = NGramFilterFactory.class,
params = {
#Parameter(name = "minGramSize", value = "2"),
#Parameter(name = "maxGramSize", value = "3") } )
}
)
For the query, I tried the following:
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(this.entityManager);
Analyzer analyzer = fullTextEntityManager.getSearchFactory().getAnalyzer("ngram");
QueryParser queryParser = new MultiFieldQueryParser(ALL_FIELDS, analyzer);
queryParser.setDefaultOperator(QueryParser.AND_OPERATOR);
org.apache.lucene.search.Query query = queryParser.parse(queryString);
javax.persistence.Query persistenceQuery =
fullTextEntityManager.createFullTextQuery(query, MyEntity.class);
List<MyEntity> result = persistenceQuery.getResultList();
As far as I understand, I need to provide an analyzer for the Query so that the search query are "ngram-tokenized" and a match can be found. Before, I used SimpleAnalyzer and as a result, the search only matched full words which - I think - backs my theory (Sorry, I am still learning this).
The above code gives me a NullPointerException:
java.lang.NullPointerException: null
at org.hibernate.search.engine.impl.ImmutableSearchFactory.getAnalyzer(ImmutableSearchFactory.java:370) ~[hibernate-search-engine-5.11.1.Final.jar:5.11.1.Final]
at org.hibernate.search.engine.impl.MutableSearchFactory.getAnalyzer(MutableSearchFactory.java:203) ~[hibernate-search-engine-5.11.1.Final.jar:5.11.1.Final]
at org.hibernate.search.impl.SearchFactoryImpl.getAnalyzer(SearchFactoryImpl.java:50) ~[hibernate-search-orm-5.11.1.Final.jar:5.11.1.Final]
in the line
Analyzer analyzer = fullTextEntityManager.getSearchFactory().getAnalyzer("ngram");
You cannot retrieve the analyzer from Hibernate Search when using the Elasticsearch integration, because in that case there is no analyzer locally: the analyzer only exists remotely, in the Elasticsearch cluster.
If you only need a subset of the query syntax, try out the "simple query string" query: it's a query that can be built using the DSL (so it will work the same with Lucene and Elasticsearch) and that provides the most common features (boolean queries, fuzziness, phrases, ...). For example:
Query luceneQuery = queryBuilder.simpleQueryString()
.onFields("name", "history", "description")
.matching("war + (peace | harmony)")
.createQuery();
The syntax is a bit different, but only because it's aiming at end users and tries to be simpler.
EDIT: If simple query strings are not an option, you can create an analyzer manually: this should work even when using the Elasticsearch integration.
org.apache.lucene.analysis.custom.CustomAnalyzer#builder() should be a good starting point. There are several examples in the javadoc of that class.
Make sure you only create the analyzer once and store it somewhere, e.g. in a static constant: creating an analyzer may be costly.
I am trying to learn about mongodb aggregation. I've been able to get the commands to work for a single output. I am now working on a pymongo script to parse through a dirty collection and output sterilised data into a clean collection. I am stuck on how to define variables properly so that I can use them in the aggregation command. Please forgive me if this turns out to be a trivial matter. But I've been searching through online documents for a while now, but I've not had any luck.
This is the script so far:
from pymongo import MongoClient
import os, glob, json
#
var_Ticker = "corn"
var_Instrument = "Instrument"
var_Date = "Date"
var_OpenPrice = "prices.openPrice.bid"
var_HighPrice = "prices.highPrice.bid"
var_LowPrice = "prices.lowPrice.bid"
var_ClosePrice = "prices.closePrice.bid"
var_Volume = "prices.lastTradedVolume"
var_Unwind = "$prices"
#
#
client = MongoClient()
db = client.cmdty
col_clean = var_Ticker + "_clean"
col_dirty = var_Ticker + "_dirty"
db[col_dirty].aggregate([{$project:{_id:0,var_Instrument:1,var_Date:1,var_OpenPrice:1,var_HighPrice:1,var_LowPrice:1,var_ClosePrice:1,var_Volume:1}},{$unwind:var_Unwind},{$out:col_clean}])
This is the error that I get:
>>> db[col_dirty].aggregate([{$project:{_id:0,var_Instrument:1,var_Date:1,var_OpenPrice:1,var_HighPrice:1,var_LowPrice:1,var_ClosePrice:1,var_Volume:1}},{$unwind:var_Unwind},{$out:col_clean}])
File "<stdin>", line 1
db[col_dirty].aggregate([{$project:{_id:0,var_Instrument:1,var_Date:1,var_OpenPrice:1,var_HighPrice:1,var_LowPrice:1,var_ClosePrice:1,var_Volume:1}},{$unwind:var_Unwind},{$out:col_clean}])
^
SyntaxError: invalid syntax
If I take out the variables and use the proper values, the command works fine.
Any assistance would be greatly appreciated.
In Python you must wrap a literal string like "$project" in quotes:
db[col_dirty].aggregate([{"$project":{"_id":0,var_Instrument:1 ...
The same goes for "_id", which is a literal string. This is different from how Javascript treats dictionary keys.
Note that you should not put quotes around var_Instrument, since that is not a string literal, it's a variable whose value is a string.
How to have inline results from a mapReducet with the mongodb java driver 3.2?
with driver version 2.x I was doing:
DBColleciont coll = client.getDB(dbName).getCollection(collName);
coll.mapReduce(map, reduce, null, OutputType.INLINE, query);
the new 3.x driver has two mapReduce() methods returning MapReduceIterable which misses a method to specify the INLINE output mode.
MongoCollection<Documetn> coll = client.getDatabase(dbName).getCollection(collName)
coll
.mapReduce(map, reduce).
.filter(query);
You can create the map-reduce command manually:
String mapFunction = ...
String reduceFunction = ...
BsonDocument command = new BsonDocument();
BsonJavaScript map = new BsonJavaScript(mapFunction);
BsonJavaScript red = new BsonJavaScript(reduceFunction);
BsonDocument query = new BsonDocument("someidentifier", new BsonString("somevalue"));
command.append("mapreduce", new BsonString("mySourceCollection"));
command.append("query", query);
command.append("map", map);
command.append("reduce", red);
command.append("out", new BsonDocument("inline", new BsonBoolean(true)));
Document result = mongoClient.getDatabase(database).runCommand(command);
I think this is extremely ugly, but it is the only working solution I found so far using 3.2. (... and would be very interested in a better variant, too... ;-))
I think I found it...
I had a deeper look into mongodb's Java driver source and it seems that the INLINE output feature is implicitly accessible:
The class MapReduceIterableImpl<TDocument, TResult>(MapReduceIterableImpl.java), which is the default implementation of the Interface return type of mapReduce(),
holds a private boolean inline with initial value true.
The only place where this can ever be switched to false is the method collectionName(final String collectionName) for which the description is as follows:
Sets the collectionName for the output of the MapReduce
The default action is replace the collection if it exists, to change this use action(com.mongodb.client.model.MapReduceAction).
If you never call this method on the object instance after mapReduce(), it will remain true as initialized...meaning: if there is no output collection, it must be inline.
Later on, when you access your result with iterator(), first(), forEach(...) etc internally the execute() method gets called which hast the magic if condition:
if (inline) {
MapReduceWithInlineResultsOperation<TResult> operation =
new MapReduceWithInlineResultsOperation<TResult>(namespace,
new BsonJavaScript(mapFunction),
new BsonJavaScript(reduceFunction),
codecRegistry.get(resultClass))
.filter(toBsonDocument(filter))
.limit(limit)
.maxTime(maxTimeMS, MILLISECONDS)
.jsMode(jsMode)
.scope(toBsonDocument(scope))
.sort(toBsonDocument(sort))
.verbose(verbose)
.readConcern(readConcern);
....
} else {
MapReduceToCollectionOperation operation =
new MapReduceToCollectionOperation(namespace, new BsonJavaScript(mapFunction), new BsonJavaScript(reduceFunction),
collectionName)
.filter(toBsonDocument(filter))
.limit(limit)
.maxTime(maxTimeMS, MILLISECONDS)
.jsMode(jsMode)
.scope(toBsonDocument(scope))
.sort(toBsonDocument(sort))
.verbose(verbose)
.action(action.getValue())
.nonAtomic(nonAtomic)
.sharded(sharded)
.databaseName(databaseName)
.bypassDocumentValidation(bypassDocumentValidation);
...so it is instanciating MapReduceWithInlineResultsOperation when collectionName() has not been called.
I had no chance to test it because my NetBeans hates me at the moment, but I think it is pretty clear.
What do you think, did I miss something?
Would be glad if I could help you shift the code to API 3.x, great project!
I have written a scala program for creating new database, and adding documents/views into it.
object CouchDBTest extends App {
val dbSession = new Session("localhost", 5984)
val db = dbSession.createDatabase("couchschooltest")
val newC1 = new Document
newC1.put("Type", "Class")
newC1.put("ClassId", "C1")
newC1.put("ClassName", "C-2A")
newC1.put("ClassTeacher", "T1")
newC1.accumulate("Students", "S1");
newC1.accumulate("Students", "S2");
newC1.accumulate("Students", "S3");
db.saveDocument(newC1)
val viewDocClass = new Document
viewDocClass.addView("Class", "function(doc) {if(doc.Type == 'Class') { emit([doc.ClassId, doc.ClassName, doc.ClassTeacher, doc.Students], doc);}}")
db.saveDocument(viewDocClass)
}
When I run this code, it creates the new database in CouchDB and adds the class document in this database. However, it doesn't add the views into the database. It gives the runtime error while adding the viewDocClass as
Error adding document - null null
For this I used couchdb4j API