How to read from a replicaset mongo by mongodb-erlang - mongodb

1. {ok,P}= mongoc:connect({rs, <<"dev_mongodb">>, [ "dev_mongodb001:27017", "dev_mongodb002:27017"]}, [{name, mongopool}, {register, mongotopology}, { rp_mode, primary},{ rp_tags, [{tag,1}]}], [{login, <<"root">>}, {password, <<"mongoadmin">>}, {database, <<"admin">>}]).
2. {ok, Pool} = mc_topology:get_pool(P, []).
3. mongoc:find(Pool, {<<"DoctorLBS">>, <<"mongoMessage">>}, #{<<"type">> => <<"5">>}).
I used latest version in github, and got an error at step 3.
It seems my selector is not valid, is there any example of how to use mongodb-erlang ?
My mongodb version is 3.2.6, auth type is SCRAM-SHA1.
mongoc:find(Pool, <<"mongoMessage">>, #{<<"type">> => <<"5">>}).
I tried this in rs and single mode, still got this error.
Is there any other simple way to connect and read?
I just need to read some data once from mongo when my erlang program start, no other actions.

Todays version of mongo does not support tuple colldb due to new query api introduced in mongo 2.6
You should connect to DoctorLBS database instead, and than use
mongoc:find(Pool, <<"mongoMessage">>, #{<<"type">> => <<"5">>}).

Related

How sync mongodb collection with elasticseatch index using logstash?

I'm tring to use logstash-7.6.2 to sync my mongo with elasticsearch. I'm using dbschema jdbc driver.
input {
jdbc{
jdbc_driver_class => "com.dbschema.MongoJdbcDriver"
jdbc_driver_library => "/home/user/mongojdbc2.3.jar,/home/user/mongo-java-driver-3.12.6.jar,/home/user/gson-2.8.6.jar"
jdbc_user => ""
jdbc_password => ""
jdbc_connection_string => "jdbc:mongodb://localhost:27027/test"
statement => "db.mycollection.find()"
}
}
output {
elasticsearch {
hosts => ["http://localhost:9220"]
manage_template => false
index => "testing"
}
stdout { codec => rubydebug }
}
But I catch the next error:
Error: Java::JavaSql::SQLException: No suitable driver found for
jdbc:mongodb://localhost:27027/test Exception:
Sequel::DatabaseConnectionError Stack:
java.sql.DriverManager.getConnection(java/sql/DriverManager.java:689)
java.sql.DriverManager.getConnection(java/sql/DriverManager.java:247)
I also tried to use the mongo native java driver and the unity jdbc driver. I also tried to use different version of mongo, tried from localhost and from a remote server. I tried to use different versions of logstash. Everything comes down to this error.
I've meet the same problems syncing MongoDB with Elastic and I want to share my experience with you. What I've tried:
Longstash input MongoDB plugin. The easist way to integrate, needs minimum configuration. But it doesn't support "arrays of objects", it simply igrores this datastructure. You may flatten them on filter stage using ruby script (converting arr: [{a: 1, b: 2}, {a: 3, b: 4}] -> arr_1_a = 1, arr_1_b = 2, ... and so on).
Logstash + jdcb mongo driver. The problem is that all mentions and instructions about this connection are a little bit outdated and all of them do not contain link to necessary Mongo Library. And libraries used in these instructions had been updated many times. Surely we have an option to search for older versions, but they do not support recent MongoDB versions. I've spent few days iterating through tens of different options (wisecoders dbschema, unity driver, many others). I've opened every library to investigate correct path to Java class, but none of them worked: every time I've meet different errors (and the most popular was No suitable driver found for jdbc:mongodb as yours).
Monstache. It was my last hope and it worked flawlessly. The only thing I had to do is to convert my Mongo instance to ReplicaSet (it can be single-node for dev puproses) and prepare simple .toml config for monstache. Also it has ready-to-go Docker image.
So I can personally recommend to use monstache for Mongo + Elastic syncing.
Hope this helps you.

pymongo - MongoClient retryWrites=false is not working

I'm currently working on a simple python CRUD script to check MongoDB out. Turns out I'm liking it a lot, but I have found myself being unable to work with MongoDB transactions. Everytime I try to start a transaction an exception is thrown saying:
This MongoDB deployment does not support retryable writes. Please add retryWrites=false to your connection string.
And, eventhough I've already added that option to my connection string:
self._client = MongoClient('mongodb://localhost/?retryWrites=false')
self._db = self._client.workouts
self._collection = self._db.workouts
That error is still popping up when running the following lines of code:
with self._client.start_session() as s:
with s.start_transaction():
self._collection.delete_one({'_id': id}, session=s)
next = self._collection.find_one({'_id': next_id}, session=s)
return next
What can I do?
I'm running python 3.7.3, pymongo 3.9.0 and MongoDB 4.0.12.

MongoDB "no SNI name sent" error from heroku java application

I followed this tutorial to deploy a sample application to Heroku. I just added the below method in MyResource class and returned the result from it instead of "Hello World" from getIt() method. I'm connecting to an atlas free tier cluster:
static String getMessage() {
MongoClient mongoClient = new MongoClient(new MongoClientURI("mongodb://<USER>:<PASSWORD>#cluster0-shard-00-00-2lbue.mongodb.net:27017,cluster0-shard-00-01-2lbue.mongodb.net:27017,cluster0-shard-00-02-2lbue.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin"));
DB database = mongoClient.getDB("mastery");
DBCollection collection = database.getCollection("summary");
DBObject query = new BasicDBObject("_id", new ObjectId("5c563fa2645d6b444c018dcb"));
DBCursor cursor = collection.find(query);
return (String)cursor.one().get("message");
}
This is the driver I'm using:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver</artifactId>
<version>3.9.1</version>
</dependency>
This is my import:
import com.mongodb.*;
The application works fine from my local system. But I face the below error when I deploy the application to Heroku and hit the service:
INFO: Exception in monitor thread while connecting to server cluster0-shard-00-01-2lbue.mongodb.net:27017
com.mongodb.MongoCommandException: Command failed with error 8000 (AtlasError): 'no SNI name sent, make sure using a MongoDB 3.4+ driver/shell.' on server cluster0-shard-00-01-2lbue.mongodb.net:27017. The full response is { "ok" : 0, "errmsg" : "no SNI name sent, make sure using a MongoDB 3.4+ driver/shell.", "code" : 8000, "codeName" : "AtlasError" }
at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:179)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:299)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.lang.Thread.run(Thread.java:748)
What is this SNI name? I can understand that the drivers are able to pick it from my machine, but not from Heroku machine. But I'm clueless on how to go about solving this! Is there a way to configure Heroku to reveal the SNI name when the driver asks for it? Can we get this value manually from somewhere in Heroku and directly feed it to the MongoDB drivers? Any help is appreciated.
EDIT:
It turned out that the client mentions the SNI name of the server it wishes to connect to as part of TLS security. And there seems to be a way to manually indicate the name in python driver. Is there any way to do this from java? Still puzzled why this is not an issue when running the app locally.
The code I was using to connect to the cluster turned out to be wrong. I followed the directions from the docs and it mentioned this:
To connect to an Atlas M0 (Free Tier) cluster, you must use Java
version 8 or greater and use a Java driver version that supports
MongoDB 3.4.
So I changed the java version to 1.8 in system.properties file:
java.runtime.version=1.8
It was earlier set to 1.7. I was also getting a deprecation warning on one of the methods I had used. So I again followed the docs to use the latest code and it worked like a charm.
The real takeaway here is to refer to the official docs everytime :)

Spring Data Embedded Mongo: 'unknown top level operator: $expr' on server

when I run any query containing $expr operation against Embedded Mongo I get the following error:
UncategorizedMongoDbException: Query failed with error code 2 and error message 'unknown top level operator: $expr' on server
The command runs fine against my local instance of mongo.
This is the version of embedded mongo I'm using: testCompile('de.flapdoodle.embed:de.flapdoodle.embed.mongo:2.1.1')
This is the query for reference:
Criteria.where("$expr").ne(Arrays.asList("$val.a", "$val.b"))
Found it.
flapdoodle was downloading a version of Mongodb that didn't have that feature by default.
You can override the default version by specifying the following in your
src/test/resources/application.properties
spring.mongodb.embedded.version=3.6.4
spring.mongodb.embedded.features=SYNC_DELAY,NO_HTTP_INTERFACE_ARG,ONLY_WITH_SSL

Type error on mongodb 3

When i try to insert data on mongodb 3 through command line it's showing following error
use video;
switched to db video
db.movies.insertOne({ "title": "Jaws", "year": 1975, "imdb": "tt0073195" });
2018-03-26T12:42:42.233+0530 E QUERY TypeError: Property 'insertOne' of object video.movies is not a function at (shell):1:11`
but video db also not created
Please help me to rectify this problem.
MongoDB supports db.collection.insertOne() from version 3.2, please check your mongodb version by using the mongo shell command
db.version()
References:
insertOne
version
Try with db.movies.insert instead of db.movies.insertOne and check If it's working fine. If it's working then your mongo version is less than 3.2. If not then share your mongoDb console Screenshot.