Error in connection to mongodb atlas from siddhi - mongodb

I am new to Siddhi, and I am trying to connect to MongoDB Atlas to make an insertion to the database of a collection, but when I configure the parameters and run the code in siddhi editor, it seems that there is no error in the console but it does not add the record to MongoDB.
Here is the code:
#App:name("ConectionMongoDBAtlas")
#App:description("Description of conection to MongoDB Atlas")
#sink(type='mongodb',
-- mongodb.uri='mongodb://username:password#ac-qe2xpea-shard-00-00.cs3wyqb.mongodb.net:27017,ac-qe2xpea-shard-00-01.cs3wyqb.mongodb.net:27017,ac-qe2xpea-shard-00-02.cs3wyqb.mongodb.net:27017/siddhi?ssl=true&replicaSet=atlas-4drk5v-shard-0&authSource=admin&retryWrites=true&w=majority',
uri='mongodb+srv://username:password#cluster0.cs3wyqb.mongodb.net/siddhi?retryWrites=true&w=majority',
collection.name = 'siddhiCollection',
database.name = 'siddhi'
-- secure.connection = 'true',
-- trust.store = 'C:/Users/luis.ortega/Downloads/siddhi-tooling-5.1.0/resources/security/client-truststore.jks',
-- key.store.password = 'mongodb',
-- sslEnabled = 'true',
-- trustStore = 'C:/Users/luis.ortega/Downloads/siddhi-tooling-5.1.0/resources/security/cloud.mongodb2',
-- keyStorePassword = 'mongodb',
-- #map(type='json')
-- #payload('{"name":"{{name}}", "age":{{age}}}')
)
#primaryKey("name")
#index('age')
define table siddhiCollection(name string, age int);
#sink(type = 'log')
define stream BarStream(message string);
#info(name= 'query1')
define stream InsertStream (name string, age int);
from InsertStream
insert into MongoCollection;
I tried to configure mongodb with the store annotation as it is in the documentation and also with the sink annotation.
We don't know if the database SSL certificate issue is a problem, I even added the certificate to the client-trutstore.jks.

I have tried the connection to the MongoDB (but not for the Atlas) with the following steps and it was successful.
Copy the mongo driver[1] to the /lib directory.
Create a database as 'test' and add the user admin to the database.
db.createUser({user: "admin", pwd: "admin", roles : [{role: "readWrite", db: "test"}]});
Then simulate the siddhi application using tooling UI after deploying the siddhi application[2].
[1] https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver/3.4.2
[2]
#App:name("store-mongodb")
#Store(type="mongodb",mongodb.uri='mongodb://admin:admin#localhost:27017/test') #PrimaryKey("name") #Index("amount:1", "{background:true}") define table SweetProductionTable (name string, amount double);
/* Inserting event into the mongo store */ #info(name='query1') from insertSweetProductionStream insert into SweetProductionTable;
If there are connection issues to the database, there should be error logs in the carbon log file of the tooling. You can check the log file, 'carbon.log' from the location /wso2/server/logs. This is needed to be checked only if you have observed the logs in the console of the browser.

Related

VOMongoRepository fails to connect to MongoDB replicaset with user credentials (Pharo/Voyage)

I am trying to save a root Object (MyDocument) into a mongoDB with authentication enabled and a ReplicaSet consisting of 3 Nodes (as inserted into mongoUrls)
With this call:
(VOMongoRepository
mongoUrls: {'127.0.0.1:27017' . '127.0.0.1:27018' . '127.0.0.1:27019'}
database: 'myDB'
username: 'myUser'
password: 'myPass') enableReplication
I receive a VOMongoConnectionError without any deeper information.
Trying the same with this:
VOMongoRepository
mongoUrls: {'myUser:myPass#127.0.0.1:27017/?replicaSet=myRepl' }
database: 'myDB'
I then receive a VOMongoError "not authorized for Query on myDB.MyDocument"
The credentials are double checked with mongo client and Compass, also the read/write permissions (actually the role is dbOwner).
Interestingly my testDocumentLifeCycle is able to create the object and to send a message to save, that returns without signaling an error, although it does not create the document in MongoDB. But the selectOne: is then returning the VOMongoError:
| doc |
MyDocument new
identity: 'me#there.com';
save.
user := MyDocument selectOne: [ :each | each identity = 'me#there.com'].
Just to mention: the above test for MyDocument class did work with a standalone mongod without authentication enabled. The only thing changed is the repository.
So what am I doing wrong?
Actually there is a bug in the replicaSet part of VoyageMongo. It is not using the credentials provided. It has been posted at https://github.com/pharo-nosql/voyage/issues/104

Define a database entry in DB2 db2dsdriver.cfg file that can access multiple databases

I currently use the CLI method of registering a node and then a database for that node (with CATALOG NODE / CATALOG DATABASE) to configure my DB2 CLI client to access our database server.
With a single registration of a database I can effectively register the default database but then when I connect in my application using SQLDriverConnect and use the "DATABASE=" option I can connect to other databases available on my server.
I would like to switch to the much easier to manage db2dsdriver.cfg configuration file, however I have been unable to configure it to allow a single to access multiple databases.
Some code to help clarify. My DB2 server instance has two databases defined like:
CREATE DATABASE DB_1 ON /opt/data/DB_1 USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM
CREATE DATABASE DB_2 ON /opt/data/DB_2 USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM
I register this server with my client CLI using these commands:
CATALOG TCPIP NODE DB_NODE remote example.server.com server 50000
CATALOG DB DB_1 as DB_1 at node DB_NODE
With that setup I can perform the following from my CLI application:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_1",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
or if I want to use the DB_2 database:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_2",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
Note I did not need to change the DSN, merely the "DATABASE" connection option.
Recently I found the db2dsdriver.cfg configuration file which I would rather use. To that end I created this and uncataloged my node and db from the cli:
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<configuration>
<dsncollection>
<dsn alias="DB_1" name="DB_1" host="server.example.com" port="50000"/>
</dsncollection>
<databases>
<database name="DB_1" host="server.example.com" port="50000"/>
<database name="DB_2" host="server.example.com" port="50000"/>
</databases>
</configuration>
I can connect with this:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_1",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
just fine, but now using this to connect to DB_2:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_2",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
results in this error:
SQL0843N The server name does not specify an existing connection. SQLSTATE=08003
Which I understand, but seems like a regression in functionality from the old node/db registration mechanism.
I'm attempting to determine if the functionality I was using is supported with the configuration file and I am doing something wrong or if it just doesn't work that way?
Thank you for your help.

Robot framework : Database library keywords not getting executed

I recently started working with Robot framework. So I had a requirement where I needed to connect with Postgres db.
So though I am able to connect with the db but then when I try to execute queries, the flow is getting stuck. Even the test is not failing. Following is what I did:
Connect To Database psycopg2 ${DBName} ${DBUser} ${DBPass} ${DBHost} ${DBPort}
${current_row_count} = Row Count Select * from xyz
The first statement is executing fine but then it gets stuck on second statement.
Can somebody help me out on this
To Execute Query and get data from result :
Connect To Database psycopg2 ${DBName} ${DBUser} ${DBPass} ${DBHost} ${DBPort}
${output} = Query SELECT * from xyz;
Log ${output}
${DataResults}= Get from list ${output} 0
${DataResults}= Convert to list ${DataResults}
${DataResults}= Get from list ${DataResults} 0
${DataResults} convert to string ${DataResults}
Disconnect From Database
You are not executing your query.... read below a bit documentation and an example ;)
In the example you can see example variable but introduce your data ;)
Name: Connect To Database Using Custom Params
Source: DatabaseLibrary
Arguments:
[ dbapiModuleName=None | db_connect_string= ]
Loads the DB API 2.0 module given dbapiModuleName then uses it to connect to the database using the map string db_custom_param_string.
Example usage Example usage: :
Connect To Database Using Custom Params pymssql database='${db_database}' , user='${db_user}', password='${db_password}', host='${db_host}'
${queryResults} Query ${query}
Disconnect From Database

MongoDB E11000 duplicate key error on mydb.testlookup.$name dup key:{:dummy123} in meanstack using angular-fullstack generator

Hi I am trying to create a simple project using angular-fullstack generator I have running my MongoDB and nodejs in windows, everything installed and running perfectly. I have created one schema as follows
'use strict';
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var TestlookupSchema = new Schema({
name: String,
ccode: String,
description: String,
info: String,
active: Boolean
});
module.exports = mongoose.model('Testlookup', TestlookupSchema);
I didn't touch any other default schemas that comes along with generator demo app.
whenever I am inserting any record into this collection I am getting below error
E11000 duplicate key error index: mydb.testlookup.$name dup key: {:dummy123}
I am using windows 7 as operating system
NodeJS 4.xx
Mongodb 3.x
What might be causing this error?
I got same problem but I resolved it by deleting index. Actually when generate angular-fullstack app it will create Thing schema there will be name field and when you create another schema which has same name field, so it will create Index. If you enter same data like in your case "dummy123" in name field for both schemas it will give duplicate key entry index error - E11000
Solution for this if you are in windows
Gotto Mongo Shell
command prompt - mongo.exe
use mydb
db.mydb.getIndexes()
You will find name as index just drop and recreate it
db.mydb.dropIndex( "name")
Now you restart your node app using grunt serve you wont get that problem again

Grails+Mongodb = do not persist object. Why?

I have first expirence in grails+mongodb. And I have a problem on saving object. First I connected local mongo db in such way to grails:
grails {
mongo {
host = "localhost"
//port = 27107
//username = "login"
//password="pwd"
databaseName = "db"
}
}
I don't know why, but if I specify port and login+password (I create such user with such password), grails give me an error, that cann't connect to mongodb. This is a log of mongodb on daemon start:
22:47:04 [initandlisten] MongoDB starting : pid=918 port=27017 dbpath=/var/lib/mongodb 64-bit host=ubnt-VirtualBox
Ok, with out port and login+pass it works. Next I create domain:
class Cover {
String name
String url
static constraints = {
name(blank: false)
url(blank: false)
}
static mapping = {
collection "cover"
database "covers"
}
}
I try to save it:
Cover cover = new Cover()
cover.name = title.text()
cover.url = url
println("Try to save object: ${cover.toString()}")
cover.save()
println("After save object: ${cover.toString()}")
What I have in output:
Try to save object: com.mydomain.Cover : (unsaved)
After save object: com.mydomain.Cover : 23
When I run shell with command 'mongo', then try to:
>use covers
>db.cover.find()
I see that there is nothing in collection. But when I look at db.cover.next_id.find() I see 23. So it seems like id increments. I can't figure out why object didn't save to mongodb. Why? Before I try to connect to mongodb and save via java driver and saving works (so mongodb seems to install correctly).
Also when I run shell I see such info >connecting to: test what means test? Where I could configure it before? May be shell connects to one db and grails to another?
For me it seems it's saving without problems unless you've some constraint errors...
Have you tried to print the errors after saving/validating?
println cover.errors
Also you could try to recover all the instances from the application without looking in mongo with
println Cover.list()
If with this last line you can see your saved instances, maybe grails is mixing the databases as you have in databaseName=db and you later say that Cover should be saved to covers database
I have found an error, it discovers for me that source in src/groovy have to access to grails sources in specific way, so to save cover I need to create service in grails, where I cna implemet save method, and they in place where I want to call save I should get service in such way
def ctx = ServletContextHolder.servletContext.getAttribute(GrailsApplicationAttributes.APPLICATION_CONTEXT)
CoverService coverService = ctx.coverService
And now it works.