Upon starting gremlin console, the g and graph objects aren't being automatically instantiated. From reading it seems that I don't have something configured correctly to allow for this during startup.
I've checked my gremlin-server.yaml file, which includes
graphs: {
graph: conf/gremlin-server/dynamodb.properties}
I'm able to manually create the objects in the console, so it does work
gremlin> graph = TitanFactory.open('conf/gremlin-server/dynamodb.properties');
==>standardtitangraph[com.amazon.titan.diskstorage.dynamodb.DynamoDBStoreManager:[127.0.0.1]]
gremlin> mgmt = graph.openManagement();
==>com.thinkaurelius.titan.graphdb.database.management.ManagementSystem#433b546f
gremlin> g = graph.traversal(standard());
==>graphtraversalsource[standardtitangraph[com.amazon.titan.diskstorage.dynamodb.DynamoDBStoreManager:[127.0.0.1]], standard]
Is there something else I'm missing? I found this issue logged on the gremlin-javascript project, but am still stuck. I think my problem is with my configuration on the server and not my implementation of gremlin-javascript.
Ultimately I'm trying to traverse using gremlin-javascript, but I'm forced to create 'graph' and 'g' each time (slow).
This is the expected behavior when starting the Gremlin console, unless you start it with an init script that does this for you:
// scripts/initMyGraph.groovy
graph = TitanFactory.open('conf/gremlin-server/dynamodb.properties')
g = graph.traversal(standard())
Then start the console with:
bin/gremlin.sh scripts/initMyGraph.groovy
Graph and Traversal objects will be accessible as graph and g variables in the console, respectively.
Related
I am trying to call the language detection method of the translate client api from pyspark for each row in a file.
I created a map method as the following but the job seems to just freeze with no error. If I remove the call to the translate API it executes fine. Is it possible to call Google client API methods within pySpark map ?
mapping method to do translation
def doTranslate(data):
translate_client = translate.Client()
# Get the message information
messageId = data[0]
messageContent = data[6]
detectedLang = translate_client.detect_language(messageContent)
r = []
r.append(detectedLang)
return r
Figured it out!! your question led me in the right direction. thanks!
Turns out I was getting an exception from the call because I was going past the default quota for sizes of messages. I added a try/except block and determined this was the problem. Then cutting the message size down (I am just testing so dont want to mess with the quota) fixed the issue.
Currently I'm using Eclipse with Nokia/Red plugin which allows me to write robot framework test suites. Support is Python 3.6 and Selenium for it.
My project is called "Automation" and Test suites are in .robot files.
Test suites have test cases which are called "Keywords".
Test Cases
Create New Vehicle
Create new vehicle with next ${registrationno} and ${description}
Navigate to data section
Those "Keywords" are imported from python library and look like:
#keyword("Create new vehicle with next ${registrationno} and ${description}")
def create_new_vehicle_Simple(self,registrationno, description):
headerPage = HeaderPage(TestCaseKeywords.driver)
sideBarPage = headerPage.selectDaten()
basicVehicleCreation = sideBarPage.createNewVehicle()
basicVehicleCreation.setKennzeichen(registrationno)
basicVehicleCreation.setBeschreibung(description)
TestCaseKeywords.carnumber = basicVehicleCreation.save()
The problem is that when I run test cases, in log I only get result of this whole python function, pass or failed. I can't see at which step it failed- is it at first or second step of this function.
Is there any plugin or other solution for this case to be able to see which exact python function pass or fail? (of course, workaround is to use in TC for every function a keyword but that is not what I prefer)
If you need to "step into" a python defined keyword you need to use python debugger together with RED.
This can be done with any python debugger,if you like to have everything in one application, PyDev can be used with RED.
Follow below help document, if you will face any problems leave a comment here.
RED Debug with PyDev
If you are wanting to know which statement in the python-based keyword failed, you simply need to have it throw an appropriate error. Robot won't do this for you, however. From a reporting standpoint, a python based keyword is a black box. You will have to explicitly add logging messages, and return useful errors.
For example, the call to sideBarPage.createNewVehicle() should throw an exception such as "unable to create new vehicle". Likewise, the call to basicVehicleCreation.setKennzeichen(registrationno) should raise an error like "failed to register the vehicle".
If you don't have control over those methods, you can do the error handling from within your keyword:
#keyword("Create new vehicle with next ${registrationno} and ${description}")
def create_new_vehicle_Simple(self,registrationno, description):
headerPage = HeaderPage(TestCaseKeywords.driver)
sideBarPage = headerPage.selectDaten()
try:
basicVehicleCreation = sideBarPage.createNewVehicle()
except:
raise Exception("unable to create new vehicle")
try:
basicVehicleCreation.setKennzeichen(registrationno)
except:
raise exception("unable to register new vehicle")
...
I am looking to open up multiple connections using a netty client bootstrap in order to parse messages coming from multiple sources. The messages all have the same format, however, due to the amount of data that needs to be processed, I must run each connection on separate threads (This is assuming netty creates a thread per client channel, which I couldn't find a reference for - if that's not the case, how would this be achieved?).
This is the code that I use to connect to the data server:
var b = new Bootstrap()
.group(group)
.channel(classOf[NioSocketChannel])
.handler(RawFeedChannelInitializer)
var ch1 = b.clone().connect(host, port).sync().channel();
var ch2 = b.clone().connect(host, port).sync().channel();
The initializer calls RawPacketDecoder, which extends ReplayingDecoder, and is defined here.
The code works well without #Sharable when opening a single connection, but for the purpose of my application I must connect to the same server multiple times.
This results in the runtime error #Sharable annotation is not allowed pointing to my RawPacketDecoder class.
I am not entirely sure on how to get past this issue, short of reimplementing in scala an instantiable class of ReplayingDecoder as my decoder based directly on ByteToMessageDecoder.
Any help would be greatly appreciated.
Note: I am using netty 4.0.32 Final
I found the solution in this StockExchange answer.
My issue was that I was using an object based ChannelInitializer (singleton), and ReplayingDecoder as well as ByteToMessageDecoder are not sharable.
My initializer was created as a scala object, and therefore a single instance allowed. Changing the initializer to a scala class and instantiating for each bootstrap clone solved the problem. I modified the bootstrap code above as follows:
var b = new Bootstrap()
.group(group)
.channel(classOf[NioSocketChannel])
//.handler(RawFeedChannelInitializer)
var ch1 = b.clone().handler(new RawFeedChannelInitializer()).connect(host, port).sync().channel();
var ch2 = b.clone().handler(new RawFeedChannelInitializer()).connect(host, port).sync().channel();
I am not sure whether this ensures multithreading as wanted but it does allow to split the data access into multiple connections to the feed server.
Edit Update: After performing additional research on the subject, I have determined that netty does in fact create a thread per channel; this was verified by printing to console after the creation of each channel:
println("No. of active threads: " + Thread.activeCount());
The output shows an incremental number as channels are created and associated with their respective threads.
By default NioEventLoopGroup uses 2*Num_CPU_cores threads as defined here:
DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt(
"io.netty.eventLoopThreads",
Runtime.getRuntime().availableProcessors() * 2));
This value can be overriden to something else by setting
val group = new NioEventLoopGroup(16)
and then using the group to create/setup the bootstrap.
Titan version: 0.9.0.M2
I'm trying to implement a custom serializer for using a HashMap as a property value. I have written my class according to the instructions provided in here. It implemets AttributeSerializer,has an equals method, and finally a no argument constructor.
I have set my configuration options as follows:
attributes.custom.attribute10.attribute-class = java.util.HashMap
attributes.custom.attribute10.serializer-class = com.graph.HashMapSerializer
I have packaged the serializer into a jar and placed in the lib folder of the Titan distribution. I have tried to start Titan but see the error 'Need to set configuration value: root.attributes.custom.serializer-class' and also 'Could not instantiate configured serializer class' in the 'gremlin-server' logs. I have tried a bunch of options including changing the attribute number and placing in the jar in couple of different places but with no success.
Please kindly comment on whether I am doing this the right way and also possible solutions please. I have read in a Titan related post that some configuration keys might change from version to version, can this be the reason somehow?
Also can anyone please comment on how to specify the full 'custom-class' name , 'custom-serializer' name and where exactly to place the jar?
Thanks for your time.
Placing the jar under $TITAN_HOME/lib is fine, but I think your properties should look like this:
attributes.custom.attribute1.attribute-class = java.util.HashMap
attributes.custom.attribute1.serializer-class = com.graph.HashMapSerializer
The example in the documentation mentions that they already had 9 custom attributes configured, so that's why it was using attribute10. You can verify which serializers are set in your graph using the TitanManagement interface.
$ ./bin/gremlin.sh
\,,,/
(o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.hadoop
plugin activated: tinkerpop.tinkergraph
plugin activated: aurelius.titan
gremlin> graph = TitanFactory.build().set('storage.backend','berkeleyje').set('storage.directory','/home/vagrant/titan-0.9.0-SNAPSHOT-hadoop1/db/berkeley').open()
==>standardtitangraph[berkeleyje:/home/vagrant/titan-0.9.0-SNAPSHOT-hadoop1/db/berkeley]
gremlin> mgmt = graph.openManagement()
==>com.thinkaurelius.titan.graphdb.database.management.ManagementSystem#78226c36
gremlin> mgmt.get('attributes.custom.attribute10.attribute-class')
==>null
gremlin> mgmt.get('attributes.custom.attribute1.attribute-class')
==>java.util.HashMap
I'm using the packaged app version of Postman to write tests against my Rest API. I'm trying to manage state between consecutive tests. To faciliate this, the Postman object exposed to the Javascript test runtime has methods for setting variables, but none for reading.
postman.setEnvironmentVariable("key", value );
Now, I can read this value in the next call via the {{key}} structure that sucks values in from the current environment. BUT, this doesn't work in the tests; it only works in the request building stuff.
So, is there away to read this stuff from the tests?
According to the docs here you can use
environment["foo"] OR environment.foo
globals["bar"] OR globals.bar
to access them.
ie;
postman.setEnvironmentVariable("foo", "bar");
tests["environment var foo = bar"] = environment.foo === "bar";
postman.setGlobalVariable("foobar", "1");
tests["global var foobar = true"] = globals.foobar == true;
postman.setGlobalVariable("bar", "0");
tests["global var bar = false"] = globals.bar == false;
Postman updated their sandbox and added a pm.* API. Although the older syntax for reading variables in the test scripts still works, according to the docs:
Once a variable has been set, use the pm.variables.get() method or,
alternatively, use the pm.environment.get() or pm.globals.get()
method depending on the appropriate scope to fetch the variable. The
method requires the variable name as a parameter to retrieve the
stored value in a script.