So, I am following a MongoDB tutorial on Pluralsight and I've been able to create a, b and c database on the same machine. After a successful creation of all three, I run mongo on port 30000 which is the port for my primary database.
>mongo --port 30000
It displayed connecting to the port and then I typed
db.getMongo()
It made a connection to the address
And I typed in a javascript object as done by the guy on Pluralsight which goes
>var democonfig={ _id: "demo", members: [{ _id: 0, host: 'localhost: 30000', priority: 10}, { _id: 1, host: 'localhost: 40000'}, { _id: 2, host: 'localhost: 50000', arbiterOnly: true}] };
After I pressed enter, I tried to run rs.initiate with the file democonfig
rs.initiate(democonfig)
This is the error I get:
{ "ok" : 0, "errmsg" : "Bad digit \" \" while parsing 30000", "code" : 93 }
This is how my replicaSet bat file looks like.
cd \Pluralsight\
md \Pluralsight\db1
md \Pluralsight\db2
md \Pluralsight\db3
#REM Primary
start "a" c:\MongoDB\bin\mongod.exe --dbpath ./db1 --port 30000 --replSet "demo"
#REM Secondary
start "b" c:\MongoDB\bin\mongod.exe --dbpath ./db2 --port 40000 --replSet "demo"
#REM Arbiter
start "c" c:\MongoDB\bin\mongod.exe --dbpath ./db3 --port 50000 --replSet "demo"
I ran into the same issue on Pluralsight's: "Introduction to MongoDb" tutorial. Below is what I used in the "configuring a replica set" section:
{
"_id": "demo",
"members": [
{
"_id": 0,
"host": "localhost:30000",
"priority": 10
},
{
"_id": 1,
"host": "localhost:40000"
},
{
"_id": 2,
"host": "localhost:50000",
"arbiterOnly": true
}
]
}
Solved it! Just removed all the spaces in between the javascript code and it ran fine.
I just removed space between localhost: and port number (localhost:30000) and the same thing for other 2 hosts. It worked fine.
Related
I have the following docker-compose file:
services:
pgdatabase:
image: postgres:13
environment:
- POSTGRES_USER=root
- POSTGRES_PASSWORD=root
- POSTGRES_DB=ny_taxi
volumes:
- "./data:/var/lib/postgresql/data:rw"
ports:
- "5432:5432"
pgadmin:
image: dpage/pgadmin4
environment:
- PGADMIN_DEFAULT_EMAIL=admin#admin.com
- PGADMIN_DEFAULT_PASSWORD=root
volumes:
- "./data_pgadmin:/var/lib/pgadmin"
ports:
- "8080:80"
I'm trying to connect to postgres using pgadmin but I'm getting the following error:
Unable to connect to server: could not translate host name "pgdatabase" to address: Name does not resolve
Running docker network ls I get:
NAME DRIVER SCOPE
bridge bridge local
docker-sql-pg_default bridge local
host host local
none null local
Then running docker network inspect docker-sql-pg_default I get
[
{
"Name": "docker-sql-pg_default",
"Id": "bfee2f08620b5ffc1f8e10d8bed65c4d03a98a470deb8b987c4e52a9de27c3db",
"Created": "2023-01-24T17:57:27.831702189Z",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.24.0.0/16",
"Gateway": "172.24.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"8f53be84a95c9c0591df6cc6edb72d4ca070243c3c067ab2fb14c2094b23bcee": {
"Name": "docker-sql-pg-pgdatabase-1",
"EndpointID": "7f3ddb29b000bc4cfda9c54a4f13e0aa30f1e3f8e5cc1a8ba91cee840c16cd60",
"MacAddress": "02:42:ac:18:00:02",
"IPv4Address": "172.24.0.2/16",
"IPv6Address": ""
},
"bf2eb29b73fe9e49f4bef668a1f70ac2c7e9196b13350f42c28337a47fcd71f4": {
"Name": "docker-sql-pg-pgadmin-1",
"EndpointID": "b3a9504d75e11aa0f08f6a2b5c9c2f660438e23f0d1dd5d7cf4023a5316961d2",
"MacAddress": "02:42:ac:18:00:03",
"IPv4Address": "172.24.0.3/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "default",
"com.docker.compose.project": "docker-sql-pg",
"com.docker.compose.version": "2.13.0"
}
}
]
I tried to connect to the gateway IP 172.24.0.1 and the IP of postgres base 172.24.0.2 but I got timeout error. Why my network isn't running?
Basically I solved my problem using the following steps:
First I accessed the pg-admin container and I used ping to verify if the pgadmin could reach the postgres.
Since pgadmin was reaching postgres I used sudo netstat -tulpn | grep LISTEN to verify in my host machine the ports that are in use. surprisingly I have two instances of pgadmin running on 8080 (one bugged).
I used docker-compose down to stop the servers and used docker system prune to delete all images/containers...
I verified the used ports again and one pgadmin still running on 8080.
I used pidof to check the PID of running (bugged) pgadmin.
Then I used kill -9 to kill the proccess.
Last, I used docker-compose up -d and I was able to communicate pgadmin with postgres via pgadmin interface.
I'm testing using docker compose for a replicated set on my local machine and am having trouble with getting Compass connected and functioning with the local set. Everything has deployed fine and I can see the exposed ports listening on the host machine. I just keep getting ECONNREFUSED.
Here is the Docker Compose file.
services:
mongo1:
hostname: mongo1
container_name: mongo1
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30001:27017
volumes:
- md1:/data/db
restart: always
command: mongod --replSet mongo-net
mongo2:
hostname: mongo2
container_name: mongo2
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30002:27017
volumes:
- md2:/data/db
restart: always
command: mongod --replSet mongo-net
mongo3:
hostname: mongo3
container_name: mongo3
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30003:27017
volumes:
- md3:/data/db
restart: always
command: mongod --replSet mongo-net
mongoinit:
image: mongo
restart: "no"
depends_on:
- mongo1
- mongo2
- mongo3
command: >
sh -c "mongosh --host mongo1:27017 --eval
'
db = (new Mongo("localhost:27017")).getDB("test");
config = {
"_id" : "mongo-net",
"members" : [
{
"_id" : 0,
"host" : "localhost:27017",
"priority": 1
},
{
"_id" : 1,
"host" : "localhost:27017",
"priority": 2
},
{
"_id" : 2,
"host" : "localhost:27017",
"priority": 3
}
]
};
rs.initiate(config);
'"
networks:
mongo-network:
volumes:
md1:
md2:
md3:
The containers and replication set deploy fine and are communicating. I can see the defined ports exposed and listening.
My issue is trying to use Compass to connect to the replicated set. I get ECONNREFUSED.
What's odd is I can actually see the client connecting on the primary logs, but get no other information about what the connection was refused/DCed.
{
"attr": {
"connectionCount": 7,
"connectionId": 129,
"remote": "192.168.32.1:61508",
"uuid": "f78c3803-fc7c-4655-9b77-94329bd41a7d"
},
"c": "NETWORK",
"ctx": "listener",
"id": 22943,
"msg": "Connection accepted",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.032+00:00"
}
}
{
"attr": {
"client": "conn129",
"doc": {
"application": {
"name": "MongoDB Compass"
},
"driver": {
"name": "nodejs",
"version": "4.8.1"
},
"os": {
"architecture": "arm64",
"name": "darwin",
"type": "Darwin",
"version": "22.1.0"
},
"platform": "Node.js v16.5.0, LE (unified)|Node.js v16.5.0, LE (unified)"
},
"remote": "192.168.32.1:61508"
},
"c": "NETWORK",
"ctx": "conn129",
"id": 51800,
"msg": "client metadata",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.034+00:00"
}
}
{
"attr": {
"connectionCount": 6,
"connectionId": 129,
"remote": "192.168.32.1:61508",
"uuid": "f78c3803-fc7c-4655-9b77-94329bd41a7d"
},
"c": "NETWORK",
"ctx": "conn129",
"id": 22944,
"msg": "Connection ended",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.044+00:00"
}
}
I have a mongo docker instance running on a remote server, what is the correct way to access the command line from my local machine?
If i login to the remote host, i can access this by:
$ docker exec -it mongo-dev mongo ccc-mongo
but i am unsure how to do this from my local machine.
I tried this:
$ ssh -L 4321:localhost:27017 khine#ccc1 -f -N
Are you sure you want to continue connecting (yes/no)? yes
khine#ccc1's password:
khine#dhegdheer:~/Sandboxes/$ mongo --port 4321
MongoDB shell version: 2.4.9
connecting to: 127.0.0.1:4321/test
channel 2: open failed: connect failed: Connection refused
Wed Sep 9 15:36:44.386 DBClientCursor::init call() failed
Wed Sep 9 15:36:44.388 Error: DBClientBase::findN: transport error: 127.0.0.1:4321 ns: admin.$cmd query: { whatsmyuri: 1 } at src/mongo/shell/mongo.js:147
exception: connect failed
on my remote machine i have 3 mongo instances running
khine#ccc1 /ccc $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
22a32b4f6a1d redis:2.8 "/entrypoint.sh redi 7 days ago Up 7 days 6379/tcp redis-web
167b022ab793 mongo:2.4 "/entrypoint.sh mong 7 days ago Up 7 days 27017/tcp mongo-web
ab84ea6cb44a redis:2.8 "/entrypoint.sh redi 2 weeks ago Up 2 weeks 6379/tcp redis-www
04dcc306af04 redis:2.8 "/entrypoint.sh redi 2 weeks ago Up 2 weeks 6379/tcp redis-dev
02c0c18307dc mongo:2.4 "/entrypoint.sh mong 2 weeks ago Up 2 weeks 27017/tcp mongo-www
61df69ec7edb mongo:2.4 "/entrypoint.sh mong 2 weeks ago Up 2 weeks 27017/tcp mongo-dev
running docker inspect, i get this:
khine#ccc1 /ccc $ docker inspect 61df69ec7edb
[{
"AppArmorProfile": "",
"Args": [
"mongod"
],
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"mongod"
],
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": [
"/entrypoint.sh"
],
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"MONGO_VERSION=2.4.14"
],
"ExposedPorts": {
"27017/tcp": {}
},
"Hostname": "61df69ec7edb",
"Image": "mongo:2.4",
"HostConfig": {
"Binds": [
"/ccc/mongo-data/dev:/data/db"
],
"CapAdd": null,
"CapDrop": null,
"CgroupParent": "",
"Name": "/mongo-dev",
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "172.17.0.34",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"LinkLocalIPv6Address": "fe80::42:acff:fe11:22",
"LinkLocalIPv6PrefixLen": 64,
"MacAddress": "02:42:ac:11:00:22",
"PortMapping": null,
"Ports": {
"27017/tcp": null
}
},
"Path": "/entrypoint.sh",
"ProcessLabel": "",
"ResolvConfPath": "/var/lib/docker/containers/61df69ec7edb6995f06d797f5b2eed420d0c4daa4cd089c3b9174900d72d0b13/resolv.conf",
"RestartCount": 0,
"State": {
"Dead": false,
"Error": "",
"ExitCode": 0,
"FinishedAt": "0001-01-01T00:00:00Z",
"OOMKilled": false,
"Paused": false,
"Pid": 15346,
"Restarting": false,
"Running": true,
"StartedAt": "2015-08-26T06:01:55.361817334Z"
},
"Volumes": {
"/data/db": "/ccc/mongo-data/dev"
},
"VolumesRW": {
"/data/db": true
}
}
]
if i add the IP address for the instance, i get this warning
$ Warning: remote port forwarding failed for listen port 4321
any advice much appreciated.
I'm using docker-compose to start 3 MongoDB servers which should be in a replica set.
I first start 3 MongoDB servers, then I configure the replica set.
This is how I would do the replica set config in a bash script:
mongo --host 127.0.0.1:27017 <<EOF
var cfg = {
"_id": "rs",
"version": 1,
"members": [
{
"_id": 0,
"host": "127.0.0.1:27017",
"priority": 1
},
// snip...
]
};
rs.initiate(cfg);
rs.reconfig(cfg)
EOF
Here I'm trying to replicate the configuring of the replica set using docker-compose.
# docker-compose.yml
mongosetup:
image: mongo:3.0
links:
- mongo1:mongo1
command: echo 'var cfg = { "_id": "rs", "version": 1, "members": [ { "_id": 0, "host": "127.0.0.1:27017", "priority": 1 }, { "_id": 1, "host": "mongo2:27017", "priority": 1 },{ "_id": 2, "host": "mongo2:27017", "priority": 1 } ] }; rs.initiate(cfg);' | mongo mongo1
Unfortunately that creates this error: yaml.scanner.ScannerError: mapping values are not allowed here.
What's the recommended approach?
Is it possible to store the cfg object in a separate file that docker-compose reads?
I fixed the problem by putting the config in setup.sh which I called from entrypoint.
mongosetup:
image: mongo:3.0
links:
- mongo1:mongo1
- mongo2:mongo2
- mongo3:mongo3
volumes:
- ./scripts:/scripts
entrypoint: [ "/scripts/setup.sh" ]
setup.sh
#!/bin/bash
MONGODB1=`ping -c 1 mongo1 | head -1 | cut -d "(" -f 2 | cut -d ")" -f 1`
mongo --host ${MONGODB1}:27017 <<EOF
var cfg = {
"_id": "rs",
"version": 1,
"members": [
{
"_id": 0,
"host": "${MONGODB1}:27017",
[cut..]
EOF
I have created a setup with docker-compose that starts 3 MongoDBs in a replica set and ElasticSearch with mongodb-river.
It's available at https://github.com/stabenfeldt/elastic-mongo
I already have MongoDB and installed Elasticsearch with Mongoriver. So I set up my river:
$ curl -X PUT localhost:9200/_river/database_test/_meta -d '{
"type": "mongodb",
"mongodb": {
"servers": [
{
"host": "127.0.0.1",
"port": 27017
}
],
"options": {
"secondary_read_preference": true
},
"db": "database_test",
"collection": "event"
},
"index": {
"name": "database_test",
"type": "event"
}
}'
I simply want to get events that have country:Canada so I try:
$ curl -XGET 'http://localhost:9200/database_test/_search?q=country:Canada'
And I get:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
I am searching the web and I read that I should first index my collection with Elasticsearch (lost the link). Should I index my Mongodb? What should I do to get results from an existing MongoDB collection?
The mongodb river relies on the operations log of MongoDB to index documents, so it is a requirement that you create your mongo database as a replica set. I assume that you're missing it, so when you create the river, the initial import sees nothing to index. I am also assuming that you're on Linux and you have a handle on the shell cli tools, so try this:
Follow these steps:
Make sure that the mapper-attachments Elasticsearch plugins is also installed
Make a backup of your database with mongodump
edit mongodb.conf (usually in /etc/mongodb.conf, but varies on how you installed it) and add the line:
replSet = rs0
"rs0" is the name of the replicaset, it can be whatever you like.
restart your mongo and then log in its console. Type:
rs.initiate()
rs.slaveOk()
The prompt will change to rs0:PRIMARY>
Now create your river just as you did in the question and restore your database with mongorestore. Elasticsearch should index your documents.
I recomend using this plugin: http://mobz.github.io/elasticsearch-head/ to navigate your indexes and rivers and make sure your data got indexed.
If that doesnt work, please post which versions you are using for the mongodb-river-plugin, elasticsearch and mongodb.