No updatedExisting from getLastError in MongoLab - mongodb

I am running updates against a database in MongoLab (Heroku) and cannot get information from getLastError.
As an example, below are statements to update a collection in a MongoDB database running locally in my machine (db version v2.0.3-rc1).
ariels-MacBook:mongodb ariel$ mongo
MongoDB shell version: 2.0.3-rc1
connecting to: test
> db.mycoll.insert({'key': '1','data': 'somevalue'});
> db.mycoll.find();
{ "_id" : ObjectId("505bcc5783cdc9e90ffcddd8"), "key" : "1", "data" : "somevalue" }
> db.mycoll.update({'key': '1'},{$set: {'data': 'anothervalue'}});
> db.runCommand('getlasterror');
{
"updatedExisting" : true,
"n" : 1,
"connectionId" : 4,
"err" : null,
"ok" : 1
}
>
All is well locally.
Now I switch to a database in MongoLab and run the same statements to update a document. getLastError is not returning an updatedExisting field. Hence, I am unable to test if my update was successful or otherwise.
ariels-MacBook:mongodb ariel$ mongo ds0000000.mongolab.com:00000/heroku_app00000 -u someuser -p somepassword
MongoDB shell version: 2.0.3-rc1
connecting to: ds000000.mongolab.com:00000/heroku_app00000
> db.mycoll.insert({'key': '1','data': 'somevalue'});
> db.mycoll.find();
{ "_id" : ObjectId("505bcf9b2421140a6b8490dd"), "key" : "1", "data" : "somevalue" }
> db.mycoll.update({'key': '1'},{$set: {'data': 'anothervalue'}});
> db.runCommand('getlasterror');
{
"n" : 0,
"lastOp" : NumberLong("5790450143685771265"),
"connectionId" : 1097505,
"err" : null,
"ok" : 1
}
> db.mycoll.find();
{ "_id" : ObjectId("505bcf9b2421140a6b8490dd"), "data" : "anothervalue", "key" : "1" }
>
Did anyone run into this?
If it matters, my resource at MongoLab is running mongod v2.0.7 (my shell is 2.0.3).
Not exactly sure what I am missing.
I am waiting to hear from their support (I will post here when I hear back) but wanted to check with you fine folks here as well just in case.
Thank you.

This looks to be a limitation of not having admin privileges to the mongod process. You might file a ticket with 10gen as it doesn't seem like a necessary limitation.
When I run Mongo in auth mode on my laptop I need to authenticate as a user in the admin database in order to see an "n" other than 0 or the "updatedExisting" field. When I authenticate as a user in any other database I get similar results to what you're seeing in MongoLab production.
(Full disclosure: I work for MongoLab. As a side note, I don't see the support ticket you mention in our system. We'd be happy to work with you directly if you'd like. You can reach us at support#mongolab.com or http://support.mongolab.com)

Related

elasticsearch 6 not allowing multiple types when trying to pipeline with mongo-connector

I am trying to push data from mongodb3.6 to elasticsearch6.1 using mongo-connector.
My records are:
db.administrators.find({}).pretty()
{
"_id" : ObjectId("5701d81893dc484c812b4fc1"),
"name" : "Test Naupada",
"username" : "adminn",
"ward" : "56a6129f44fc869f215fe3fe",
"password" : "nadmin"
}
rs0:PRIMARY> db.sub_ward_master.find({}).pretty()
{
"_id" : ObjectId("56a6129f44fc869f215fe3fe"),
"wardCode" : "3",
"wardName" : "Naupada",
"wardgeoCodes" : [],
"cityName" : "thane"
}
When I run mongo-connector I am getting following error:
OperationFailed: (u'1 document(s) failed to index.', [{u'index': {u'status': 400, u'_type': u'administrators', u'_index': u'smartjn', u'error': {u'reason': u'Rejecting mapping update to [smartjn] as the final mapping would have more than 1 type: [sub_ward_master, administrators]', u'type': u'illegal_argument_exception'}, u'_id': u'5701d81893dc484c812b4fc1', u'data': {u'username': u'adminn', u'ward': u'56a6129f44fc869f215fe3fe', u'password': u'nadmin', u'name': u'Test Naupada'}}}
Any help any one?
Thanks
ES 6 does not allow to create more than one type in any single index.
There's an open issue in the mongo-connector repo to support ES 6. Until that's solved, you should go with ES 5 instead.
You can do it in ES6 by creating a new index for different document type (ie different collection in mongoDB) and use -g flag to direct it to new index.
For example:
mongo-connector -m localhost:27017 -t localhost:9200 -d elastic2_doc_manager -n {db}.{collection_name} -g {new_index}.{document_type}.
Refer mongo-connector-wiki

Cannot access authenticated MongoDB collection from ReactiveMongo Play app

I have a MongoDB server where I have enabled authentication and created users with DB-specific permissions. The user for this app is defined as shown below i.e. geoAdmin has read, readWrite and dbOwner permissions for the relevant database:
MongoDB shell version: 3.0.0
connecting to: 192.168.2.89/test
> use geo_db
switched to db geo_db
> db.getUser("geoAdmin")
{
"_id" : "geo_db.geoAdmin",
"user" : "geoAdmin",
"db" : "geo_db",
"roles" : [
{
"role" : "read",
"db" : "geo_db"
},
{
"role" : "dbOwner",
"db" : "geo_db"
},
{
"role" : "readWrite",
"db" : "geo_db"
}
]
}
The following query works OK i.e. connecting to the remote server from my local mongo client:
mint:~ $ mongo 192.168.2.89:27017 -u geoAdmin -p secret --authenticationDatabase geo_db
MongoDB shell version: 3.0.0
connecting to: 192.168.2.89/test
> use geo_db
switched to db geo_db
> db.LAD_DEC_2013_GB_BFE.findOne({},{'properties.LAD13NM':1})
{
"_id" : ObjectId("54ffe2824f0787ec1293017f"),
"properties" : {
"LAD13NM" : "Hartlepool"
}
}
I then connect to the same remote host from a ReactiveMongo Play app on the same local client, with this URL in the app config file:
# ReactiveMongo
mongodb.uri = "mongodb://geoAdmin:secret#192.168.2.89:27017/geo_db"
But when my app tries to read from the same collection, I get a MongoDB "code = 13" error:
[DetailedDatabaseException: DatabaseException['not authorized for query on geo_db.LAD_DEC_2013_GB_BFE' (code = 13)]]
The app works fine if I connect to a local MongoDB which does not have authentication enabled.
Any ideas what might be going wrong here?
ReactiveMongo 0.11.7.play23 is supporting mongo 3.0 auth-protocols, but is still using the old as default.
With ReactiveMongo 0.11.7.play23 -plugin, you can make it authenticate with mongo 3.0, by adding "?authMode=scram-sha1" to the end of your mongodb.uri.
E.g.:
mongodb.uri = "mongodb://geoAdmin:secret#192.168.2.89:27017/geo_db?authMode=scram-sha1"
mongo 2.6 uses MONGODB-CR auth protocol and 3.0 uses MONGODB-SHA-1 by default
reactivemongo use MONGODB-CR auth protocol(not sure)
downgrade mongodb 3.0 auth mechanisms to MONGODB-CR
login mongo noauth
remove all user
update the version document for the authSchema.
ex.
db.getSiblingDB("admin").system.users.remove( {} )
db.getSiblingDB("admin").system.version.update(
{ _id: "authSchema" },
{ $set: { currentVersion: 3 } }
);
add authSource parameter to mongodb url
ex.
mongodb.uri = "mongodb://geoAdmin:secret#192.168.2.89:27017/geo_db?authSource=geo_db"

elasticsearch jdbc river polling--- load data from mysql repeatedly

When using https://github.com/jprante/elasticsearch-river-jdbc I notice that the following curl statement successfully indexes data the first time. However, the river fails to repeatedly poll the database for updates.
To restate, when I run the following, the river successfully connects to MySQL, runs the query successfully, indexes the results, but never runs the query again.
curl -XPUT '127.0.0.1:9200/_river/projects_river/_meta' -d '{
"type" : "jdbc",
"index" : {
"index" : "test_projects",
"type" : "project",
"bulk_size" : 100,
"max_bulk_requests" : 1,
"autocommit": true
},
"jdbc" : {
"driver" : "com.mysql.jdbc.Driver",
"poll" : "1m",
"strategy" : "simple",
"url" : "jdbc:mysql://localhost:3306/test",
"user" : "root",
"sql" : "SELECT name, updated_at from projects p where p.updated_at > date_sub(now(),interval 1 minute)"
}
}'
Tailing the log, I see:
[2013-09-27 16:32:24,482][INFO ][org.elasticsearch.river.jdbc.strategy.simple.SimpleRiverFlow] next run, waiting 1m
[2013-09-27 16:33:24,488][INFO ]> [org.elasticsearch.river.jdbc.strategy.simple.SimpleRiverFlow] next run, waiting 1m
[2013-09-27 16:34:24,494][INFO ]> [org.elasticsearch.river.jdbc.strategy.simple.SimpleRiverFlow] next run, waiting 1m
But the index stays empty. Running on a macbook pro with elasticsearch version stable 0.90.2, HEAD and mysql-connector-java-5.1.25-bin.jar in the river pligns directory.
I think if you switch your strategy value from "simple" to "poll" you may get what you are looking for - it has worked for me with jdbc on that version of elasticsearch against MS SQL.
Also - you will need to select a field as _id (select primarykey as _id) as this is used in the elasticsearch river for determining what records are added/deleted/updated.

MongoDB with Authentication

I am trying to get MongoDB running on my localhost (Windows) with authentication.
To do so, I first have to add a user, right?
I did so by starting the daemon using this command:
C:\[…]\mongod.exe -f C:\[…]\mongo.config
mongo.config contains the following:
# Basic database configuration
dbpath = C:\[…]\db\
bind_ip = 127.0.0.1
port = 20571
# Security
noauth = true
# Administration & Monitoring
nohttpinterface = true
After that I connected via this command:
C:\[…]\mongo.exe --port 20571 127.0.0.1
There I added a user:
> use admin
switched to db admin
> db.addUser('test', 'test')
{ "n" : 0, "connectionId" : 1, "err" : null, "ok" : 1 }
{
"user" : "test",
"readOnly" : false,
"pwd" : "a6de521abefc2fed4f5876855a3484f5",
"_id" : ObjectId("50db155e157524b3d2195278")
}
To check if everything worked I did the following:
> db.system.users.find()
{ "_id" : ObjectId("50db155e157524b3d2195278"), "user" : "test", "readOnly" : false, "pwd" : "a6de521abefc2fed4f5876855a3484f5" }
Which seemed OK to me.
After that I changed "noauth = true" to "auth = true" in the mongo.config file and restarted the daemon.
Now I expected to be able to connect with user and password:
C:\[…]\mongo.exe --port 20571 -u test -p test 127.0.0.1
Which denied access to me with this message:
MongoDB shell version: 2.0.4
connecting to: 127.0.0.1:20571/127.0.0.1
Wed Dec 26 16:24:36 uncaught exception: error { "$err" : "bad or malformed command request?", "code" : 13530 }
exception: login failed
So here's my question: Why does the login fail?
I can still connect without providing user and password, but can't access any data because "unauthorized db:admin lock type:-1 client:127.0.0.1". Which is actually what I expected.
As Andrei Sfat told me in the comments on the question I made 2 major errors.
First, I thought I could pass the IP to the Client as a simple argument. But you have to use --host for that.
Instead, the parameter I thought was the IP address actually should be the db name.
So the correct command to connect to a Server is as follows:
C:\[…]\mongo.exe --port 20571 -u test -p test --host 127.0.0.1 admin
Second, users are per database. As I only added the user "test" to the db "admin", it only works there.
Obviously the auth = true configuration wasn't load successfully. Did you forget the -f paramter when you restart the mongod.exe?
C:\[…]\mongod.exe -f C:\[…]\mongo.config

mongo dbname --eval 'db.collection.find()' does not work

Why does this work:
# mongo dbname
MongoDB shell version: 1.8.3
connecting to: nextmuni_staging
> db.collection.find()
{ "foo" : "bar" }
> bye
While this does not work:
# mongo localhost/dbname --eval 'db.collection.find()'
MongoDB shell version: 1.8.3
connecting to: localhost/dbname
DBQuery: dbname.collection -> undefined
It should be exactly the same, no?
Thanks!
The return val of db.collection.find() is a cursor type. Executing this command from within the shell will create a cursor and show you the first page of data. You can start going through the rest by repeating the 'it' command.
I think the scope of variables used during the execution of an eval'd script is only for the lifetime of the script (data can be persisted into collections of course) so once the script terminates those cursor variables no longer exist and so you would be able to send another eval script to page the data. So the behaviour you get during a shell session wouldn't really work from an eval script.
To get close to the behaviour you could run something like this:
mongo dbname --eval "db.collection.find().forEach(printjson)"
That shows you that the command does execute and produce a cursor which you can then iterate over sending the output to stdout.
Edit: I think the point I was trying to make was that the command you are issuing is working its just the output is not what you expect.
The printjson functions covers a lot of ground when scripting with mongo --eval '...'. Rather than chaining .forEach you can simply wrap your call.
$ mongo --eval 'db.stats_data.stats()' db_name
MongoDB shell version: 2.4.14
connecting to: db_name
[object Object]
$ mongo --eval 'db.stats_data.stats().forEach(printjson)' db_name
MongoDB shell version: 2.4.14
connecting to: db_name
Tue Jan 10 15:32:11.961 TypeError: Object [object Object] has no method 'forEach'
$ mongo --eval 'printjson(db.stats_data.stats())' db_name
MongoDB shell version: 2.4.14
connecting to: db_name
{
"ns" : "db_name.stats_data",
"count" : 5516290,
"size" : 789938800,
"avgObjSize" : 143.20110073980882,
"storageSize" : 1164914688,
"numExtents" : 18,
"nindexes" : 3,
"lastExtentSize" : 307515392,
"paddingFactor" : 1.0000000000000457,
"systemFlags" : 1,
"userFlags" : 0,
"totalIndexSize" : 1441559616,
"indexSizes" : {
"_id_" : 185292688,
"owner_id_key_idx" : 427678384,
"onwer_metric_key_idx" : 828588544
},
"ok" : 1
}