I am working with the neo4j-rest-graphdb an just tried to use Cypher for fetching a simple Node result.
CypherParser parser = new CypherParser();
ExecutionEngine engine = new ExecutionEngine(graphDbService);
Query query = parser.parse( "START referenceNode = node (0) " +
"MATCH referenceNode-[PRODUCTS_REFERENCE]->products-[PRODUCT]->product " +
"RETURN product.productName " +
"ORDER BY product.productId " +
"SKIP 20"
"LIMIT 10");
ExecutionResult result = engine.execute( query );
Iterator<Map<String, Object>> iterator = result.javaIterator();
What is the best practise to iterate through the result? The last line causes my service to hang for ~6 sec. Without the iterator at the end the application is quiet fast. I also tried the webadmin cypher terminal, the results are fetched within 50ms. Am i doing something wrong?
In your case all the cypher operations (graph-matching, filtering etc. would go over the wire which is awfully chatty and slow) you don't want that !
The neo4j-rest-graphdb supports remote execution of cypher out of the box:
Just do, something like shown in this testcase:
RestCypherQueryEngine queryEngine = new RestCypherQueryEngine(restGraphDatabase.getRestAPI());
final String queryString = "start n=node({reference}) return n";
Map params = MapUtil.map("reference",0);
final Node result = (Node) queryEngine.query(queryString, params).to(Node.class).single();
assertEquals(restGraphDatabase.getReferenceNode(), result);
If I understood you correctly, graphDbService is a REST graph database, correct?
If you want to use Cypher on the Server, you should instead use the CypherPlugin. Look here: http://docs.neo4j.org/chunked/snapshot/cypher-plugin.html
I hope this helps,
Andrés
Related
I'm profiling my application locally (using the Dev server) to get more information about how GAE works. My tests are comparing the common full Entity query and the Projection Query. In my tests both queries do the same query, but the Projection is specified with 2 properties. The test kind has 100 properties, all with the same value for each Entity, with a total of 10 Entities. An image with the Datastore viewer and the Appstats generated data is shown bellow. In the Appstats image, Request 4 is a memcache flush, Request 3 is the test database creation (it was already created, so no costs here), Request 2 is the full Entity query and Request 1 is the projection query.
I'm surprised that both queries resulted in the same amount of reads. My guess is that small and read operations and being reported the same by Appstats. If this is the case, I want to separate them in the reports. That's the queries related functions:
// Full Entity Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
// Projection Query
public ReturnCodes doQuery() {
DatastoreService dataStore = DatastoreServiceFactory.getDatastoreService();
for(int i = 0; i < numIters; ++i) {
String projectionPropName = DBCreation.PROPERTY_NAME_PREFIX + i;
Filter filter = new FilterPredicate(DBCreation.PROPERTY_NAME_PREFIX + i,
FilterOperator.NOT_EQUAL, i);
Query query = new Query(DBCreation.ENTITY_NAME).setFilter(filter);
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 0, Integer.class));
query.addProjection(new PropertyProjection(DBCreation.PROPERTY_NAME_PREFIX + 1, Integer.class));
PreparedQuery prepQuery = dataStore.prepare(query);
Iterable<Entity> results = prepQuery.asIterable();
for(Entity result : results) {
log.info(result.toString());
}
}
return ReturnCodes.SUCCESS;
}
Any ideas?
EDIT: To get a better overview of the problem I have created another test, which do the same query but uses the keys only query instead. For this case, Appstats is correctly showing DATASTORE_SMALL operations in the report. I'm still pretty confused about the behavior of the projection query which should also be reporting DATASTORE_SMALL operations. Please help!
[I wrote the go port of appstats, so this is based on my experience and recollection.]
My guess is this is a bug in appstats, which is a relatively unmaintained program. Projection queries are new, so appstats may not be aware of them, and treats them as normal read queries.
For some background, calculating costs is difficult. For write ops, the cost are returned with the results, as they must be, since the app has no way of knowing what changed (which is where the write costs happen). For reads and small ops, however, there is a formula to calculate the cost. Each appstats implementation (python, java, go) must implement this calculation, including reflection or whatever is needed over the request object to determine what's going on. The APIs for doing this are not entirely obvious, and there's lots of little things, so it's easy to get it wrong, and annoying to get it right.
I am trying to select rows which are older than 7 days from current date. Database used is DB2 version 9.
Can you please tell me how exactly can I use the datetime in the query? The date table field is of type timestamp.
I am able to manually run the query without issues. However, when I am using in the prepared statement,
The app hangs when executing the query result = pselect.executeQuery(); as a result of which we need to restart db2 instance in order to clear it.
Can you please help what might be the issue? I do not see any exceptions at all. Other parts of the code works fine if I remove the select_query part.
try{
String select_query = "SELECT URL_ID ,URLVAL FROM URL_TAB WHERE " +
"UPDATED_DATE < TIMESTAMP(CURRENT_DATE - 7 DAYS, '00.00.00')";
System.out.println("select_query=" + select_query);
conn = JDBCDataObjectFactoryManager
.getConnection("JDBCConnectionFactory-SDE");
pselect = conn.prepareStatement(select_query);
System.out.println("pselect=" + pselect);
try{
System.out.println("inside try");
result = pselect.executeQuery();
System.out.println("result=" + result);
}catch(Exception e){
System.out.println("inside catch");
System.out.println("error message==============>"+e.getMessage());
}
if ((result != null) && (result.next())) {
System.out.println("3 >>>>>>>>>>>>>>>>>>>>>>>>>");
url_id = result.getInt(1);
url = result.getString(2);
}//end if
There are two possibilities: either the query is in a lock wait, or it runs for so long that it appears to be hung.
Check what is the value of LOCKWAIT database configuration parameter --by default it is -1, which means infinity, and you normally want to set it to a more reasonable value, typically 30 or 60 seconds. If it is the lock wait that causes your application to "hang", you would get an exception instead, which will help you to debug further.
If the issue is caused by the poor query performance, you'll need to work with your DBAs to figure out the root cause and resolve it.
i tried to make this as simple as possible. i`m new to spring batch, i have a small isuue with understanding how to relate spring items together especially when it comes to multi-steps jobs however this is my logic not code(simplified) and i dont know to impliment it in spring batch so i thought this might be the right structure
reader_money
reader_details
tasklet
reader_profit
tasklet_calculation
writer
however please correct me if i`m wrong and provide some code if possible.
thank you very much
LOGIC:
sql = "select * from MONEY where id= user input"; //the user will input the condition
while (records are available) {
int currency= resultset(currency column);
sql= "select * from DETAILS where D_currency = currency";
while (records are available) {
int amount= resultset(amount column);
string money_flag= resultset(money_type column);
sql= "select * from PROFIT where Mtypes = money_type";
while (records are available) {
int revenue= resultset(revenue);
if (money_type== 1) {
int net_profit= revenue * 3.75;
sql = "update PROFIT set Nprofit = net_profit";
}
else (money_type== 2) {
int net_profit = (revenue - 5 ) * 3.7 ;
sql = "update PROFIT set Nprofit = net_profit";
}
}
sql="update DETAILS set detail_falg = 001 ";
}
sql = "update MONEY set currency_flag = 009";
}
to fit this into a 'conventional' spring batch configuration, you would need to flatten the three loops into one if possible.
perhaps a sql statement that would return it in one loop similiar to;
select p.revenue, d.amount from PROFIT p, DETAILS d, MONEY m where p.MTypes = d.money_type and d.D_currency = m.currency and m.id = :?
once you've "flattened" it, you then fall into the more 'conventional' read/process/write of a chunk pattern where the reader retrieves a record from the resultset, the processor performs the money_type logic, and the writer then executes the 'update' statement.
Check for the use of ItemReaderAdapter where you could place all your SQL in some kind of DAO that could return a list of aggregated object containing all the info you need for your calculation.
Or
You could use the CompositeItemReader pattern. You basicaly define multiple ItemReader into one master ItemReader. The read() method will invoke all the inner ItemReader before going to the Processor /writer phase.
I could post you some example.. but i have to leave :-(..
Leave a comment if you need some example
I have a Map function in MongoDB which I'm later using Reduce on. I use a collection which has a bunch of users in it and users own some channels. However, there are users that do not have any channels and the Map/Reduce function raises an error in my script.
map = Code("function () {"
" if(!this.channels) continue;"
" this.channels.forEach(function(z) {"
" emit(z, 1);"
" });"
"}")
When I use return instead of continue to quit the function it works flawlessly except that I don't want to end the loop. Is there any smart way around this?
Thanks for your advice and better widsom.
If you return from map, it returns only from map for this document. Maps for other documents will be executed regardless of that.
I suggest rewriting your map to this form
function () {
if(this.channels) {
this.channels.forEach(function(z) {
emit(z, 1);
});
}
}
I think, this form is more clear. It will emit something for users that have channels, and skip those that don't have any.
I was writing a search for one of our sites: (SITE A)
BooleanQuery booleanQuery = new BooleanQuery();
foreach (var field in fields)
{
QueryParser qp = new QueryParser(field, new StandardAnalyzer());
Query query = qp.Parse(search.ToLower() + "*");
if (field.Contains("Title")) { query.SetBoost((float)1.8); }
booleanQuery.Add(query, BooleanClause.Occur.SHOULD);
}
// CODE DIFFERENCE IS HERE
Query query2 = new TermQuery(new Term("StateProperties.IsActive", "True"));
booleanQuery.Add(query2, BooleanClause.Occur.MUST);
// END CODE DIFFERENCE
Lucene.Net.Search.TopScoreDocCollector collector = Lucene.Net.Search.TopScoreDocCollector.create(21, true);
searcher.Search(booleanQuery, collector);
hits = collector.TopDocs().scoreDocs;
this was working as expected.
since we own a few sites, and they use the same skeleton,
i uploaded the search to another site ( SITE B )
but the search stopped returning results.
after playing a round a bit with the code, i managed to make it work like so: (showing only the rewriten lines of code)
QueryParser qp2 = new QueryParser("StateProperties.IsActive", new StandardAnalyzer());
Query query2 = qp2.Parse("True");
booleanQuery.Add(query2, BooleanClause.Occur.MUST);
anyone knows why this is happening ?
i have checked the Lucene dll version, and its the same version in both sites (2.9.2.2)
is the code i have written in SITE A wrong ? is SITE B code wrong ?
is this my fault at all ? can production server infulance something like this ?
Doesn't they have individual indexes on disk? If they have been indexed differently, they would also return different results. One thing that comes to mind is if there is some sort of case sensitivity that matters, becayse a TermQuery will look for an EXACT match, where as the parser will try to tokenize/filter the search term according to the analyzer (and probably search for "true" instead of "True".