Can i use replica set name to connect via mongo-connector - mongodb

I would like to know, is there a way we can replicate from one mongo replica set to another via mongo-connector? As per mongo documentation we can connect two mongo instances via mongo-connector by using a command as in the example below, but I would like to pass replica set name or use a configuration file instead of passing server:port name in command line.
Mongo Connector can replicate from one MongoDB replica set or sharded cluster to another using the Mongo DocManager. The most basic usage is like the following:
mongo-connector -m localhost:27017 -t localhost:37017 -d mongo_doc_manager
I also tried config.json option by creating below config.json file but it has failed.
{
"__comment__": "Configuration options starting with '__' are disabled",
"__comment__": "To enable them, remove the preceding '__'",
"mainAddress": "localhost:27017",
"oplogFile": "C:\Dev\mongodb\mongo-connector\oplog.timestamp",
"verbosity": 2,
"continueOnError": false,
"logging": {
"type": "file",
"filename": "C:\Dev\mongodb\mongo-connector\mongo-connector.log",
"__rotationWhen": "D",
"__rotationInterval": 1,
"__rotationBackups": 10,
"__type": "syslog"
},
"docManagers": [
{
"docManager": "mongo_doc_manager",
"targetURL": "localhost:37010",
"__autoCommitInterval": null
}
]
}

yes its possible to connect to a replica set or a shard server using mongo connector.
{
mongo-connector -m <mongodb server hostname>:<replica set port> \
-t <replication endpoint URL, e.g. http://localhost:8983/solr> \
-d <name of doc manager, e.g., solr_doc_manager>
}
you can also also pass a connection string to the mongo-connector such as
{
mongo connector -m mongodb://db1.example.net,db2.example.net:2500/?replicaSet=test&connectTimeoutMS=300000
}
to specify specifc config files you can use
{ mongo-connector -c config.json }
where config.json is your config file.

I'm able to resolve my issue by entering backslash '\' for my windows directory path.Here is my updated config file for reference. Thanks to ShaneHarveyNot able to use Configuration file for connecting to mongo-connector
{
"__comment__": "Configuration options starting with '__' are disabled",
"__comment__": "To enable them, remove the preceding '__'",
"mainAddress": "localhost:27017",
"oplogFile": "C:\\Dev\\mongodb\\mongo-connector\\oplog.timestamp",
"noDump": false,
"batchSize": -1,
"verbosity": 2,
"continueOnError": false,
"logging": {
"type": "file",
"filename": "C:\\Dev\\mongodb\\mongo-connector\\mongo-connector.log",
"__format": "%(asctime)s [%(levelname)s] %(name)s:%(lineno)d - %(message)s",
"__rotationWhen": "D",
"__rotationInterval": 1,
"__rotationBackups": 10,
"__type": "syslog",
"__host": "localhost:27017"
},
"docManagers": [
{
"docManager": "mongo_doc_manager",
"targetURL": "localhost:37017",
"__autoCommitInterval": null
}
]
}

Related

OpsManager mongodb deployment issue adding PLAIN auth

I'm trying to enable PLAIN authentication security over a mongodb replica shard managed with OpsManager following their documentation https://docs.opsmanager.mongodb.com/v4.0/tutorial/enable-ldap-authentication-for-group/ .
The issue I'm facing is at the automation-agent trying to get mongoS status while restarting after enabling security. Please see the error output below:
<mongos_5> [09:18:19.711] Failed to compute states :
<mongos_5> [09:18:19.711] Error calling ComputeState : <mongos_5> [09:18:19.632] Error getting current config from running mongo using conn params = mongos01:27017 (local=false) :
<mongos_5> [09:18:19.632] Error getting pid for mongos01:27017 (local=false) :
<mongos_5> [09:18:19.632] Error running command for runCommandWithTimeout(dbName=admin, cmd=[{serverStatus 1} {locks false} {recordStats false}]) :
result={"$clusterTime":{"clusterTime":6808443558471663617,"signature": {"hash":"e44BxV30B7dTpampo4VZsVuio7E=","keyId":6808441655801151517}},"code":13,"codeName":"Unauthorized",
"errmsg":"command serverStatus requires authentication","ok":0,"operationTime":6808443558471663617} connection=&{mongos01:27017 (local=false) 2 true 0xc4207b21a0 2020-03-26 09:18:19.627337419 +0000 UTC 0xc4207bdef0 <nil> }
identityUsed= : command serverStatus requires authentication
I noticed that even if opsmanager is not able to get the status the security was enabled successfully and PLAIN authentication mechanism works but the status hangs at
Start the process ... Start MongoDB process
I tried this over the API following mongodb-labs repo https://github.com/mongodb-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/security_ldap_cluster.json but also manually following mongodb docs but everytime I'm facing the same error.
After all I enabled LDAP(PLAIN) only for mongo in mongoconfig file (see below the ops manager API snippet call example), and avoid enable in opsmanager for the agents also.
{
"args2_6": {
"net": {
"port": 28001
},
"replication": {
"replSetName": "rs0"
},
"storage": {
"dbPath": "/data/mongo"
},
"systemLog": {
"destination": "file",
"path": "/data/mongo/mongodb.log"
},
"security": {
"authorization": "enabled"
},
"setParameter": {
"saslauthdPath": "",
"authenticationMechanisms": "PLAIN,MONGO-CR,SCRAM-SHA-256",
}
}, ...

How to enable Javascript in Druid

I have been using Druid for the past week and wanted to enable javascript for some postAggregations.
I think I followed the outlined steps and updated the common.runtime.properties file in ../con f/druid/_common/ to include druid.javascript.enabled=true. I then stopped the current processes and re-ran the Quickstart procedures, but it still says that JavaScript is disabled:
{
"error" : "Unknown exception",
"errorMessage" : "Instantiation of [simple type, class io.druid.query.aggregation.post.JavaScriptPostAggregator] value failed: JavaScript is disabled. (through reference chain: java.util.ArrayList[0])",
"errorClass" : "com.fasterxml.jackson.databind.JsonMappingException",
"host" : null
}
I am currently running it in the 'Quickstart' configuration - single local machine. Any pointers? Thanks!
JavaScript Query For druid Aggregation. Save the file as .body and hit the curl request.
This is a sample query for Average value.
curl -X POST "http://localhost:8082/druid/v2/?pretty" \ -H
'content-type: application/json' -d #query.body
{
"queryType":"groupBy",
"dataSource":"whirldata",
"granularity":"all",
"dimensions":[],
"aggregations":[{"name":"rows","type":"count","fieldName":"rows"},
{"name":"TargetDOS","type":"doubleSum","fieldName":"Target DOS"}],"postAggregations":[
{
"type": "javascript",
"name": "Target DOS Average",
"fieldNames": ["TargetDOS", "rows"],
"function": "function(TargetDOS, rows) { return Math.abs(TargetDOS) / rows; }"
}], "intervals":[ "2006-01-01T00:00:00.000Z/2020-01-01T00:00:00.000Z" ]}
The part you are missing is likely that the quickstart reads configs from conf-quickstart rather than conf. So try editing conf-quickstart/druid/_common/common.runtime.properties.

Access Parse Dashboard without SSL

I recently created Parse Dashboard on my server.My server does't have SSL
,how can i connect without ssl
How to solve Parse Dashboard can only be remotely accessed via HTTPS using a Configuration file
First create a parse-dashboard-config.json and file should contain the following code
{
"apps": [
{
"serverURL": "http://example.com:1337/parse",
"appId": "yourappid",
"masterKey": "yourmasterkey",
"appName": "appname"
}
],
"users": [
{
"user":"user",
"pass":"pass"
}
]
}
On terminal
parse-dashboard --config parse-dashboard-config.json --allowInsecureHTTP

ServiceAddress is empty when using Consul Registrator

[
{
"Node": "consul-staging-a-1.org",
"Address": "10.0.11.221",
"ServiceID": "mesos-slave-staging-a-1.org:determined_bartik:5000",
"ServiceName": "service1",
"ServiceTags": null,
"ServiceAddress": "",
"ServicePort": 4003
},
{
"Node": "consul-staging-a-1.org",
"Address": "10.0.11.221",
"ServiceID": "mesos-slave-staging-a-1.org:angry_hypatia:5000",
"ServiceName": "service1",
"ServiceTags": null,
"ServiceAddress": "",
"ServicePort": 4007
}
]
This is what I get from querying the Consul service API (/v1/catalog/service/service1).
Commands that I used to start registrator and services:
docker run -d -v /var/run/docker.sock:/tmp/docker.sock -h $HOSTNAME gliderlabs/registrator consul://consul-staging-a-1.org:8500
docker run -d -p 4003:5000 -e "SERVICE_NAME=service1" docker-training/hello-world
docker run -d -p 4007:5000 -e "SERVICE_NAME=service1" docker-training/hello-world
Any steps I'm doing wrong? How do you assign the hostname to ServiceAddress field?
I had similar issue and found that before v6 Consul Registrator leaves ServiceAddress empty, but in v6 (at the moment the latest) it is "0.0.0.0". I've tried to use "-ip" option but it did not help, somehow it assigns internal IP address of the container. Found related issues at:
https://github.com/gliderlabs/registrator/issues/240
In my case I fixed it by binding container to IP address like:
docker run -d -p 10.0.0.3:4003:5000 -e "SERVICE_NAME=service1" docker-training/hello-world

Centrifuge not using MongoDB?

I just installed centrifuge (https://centrifuge.readthedocs.org/en/latest/) and created a configuration.json file and placed it in /var/www/ folder.
When I try to run centrifuge centrifuge config = /var/www/configuration.json, the server starts. However when I go to the default path http://localhost:8000 in the admin panel it keeps saying DataStructure used as SQLite.
Here's my configuration.json file
{
"password": "admin",
"cookie_secret": "secret",
"api_secret": "secret",
"structure": {
"storage": "centrifuge.structure.mongodb",
"settings": {
"host": "localhost",
"port": 27017,
"name": "centrifuge",
"pool_size": 10
}
},
state: null
}
I checked and the MongoDB server is running on port 27017.
It seems you are starting Centrifuge using incorrect command line arguments. Try copy and paste into your terminal:
centrifuge --config=/var/www/configuration.json