My mongo slave is dead because it stopped unexpectedly due to not enough space and It wont start due to
mongodb.service: Main process exited, code=exited, status=14/n/a
I tried to fix the error with following suggestions:
https://askubuntu.com/questions/823288/mongodb-loads-but-breaks-returning-status-14
but it lead to next error code:
mongodb.service: Main process exited, code=exited, status=100/n/a
which I tried to fix with following
https://dba.stackexchange.com/questions/220411/sudo-service-mongod-start-returns-error-100
this it it log output
2021-05-01T18:25:30.987+0000 I - [initandlisten] Fatal assertion 28579 UnsupportedFormat: Unable to find metadata for table:index-3-848131710157586571 Index: {name: _id_, ns: local.me} - version too new for this mongod. See http://dochub.mongodb.org/core/3.4-index-downgrade for detailed instructions on how to handle this error. at src/mongo/db/storage/wiredtiger/wiredtiger_index.cpp 241
The command sudo service mongodb start wont work because the status command shows that the service is dead.
I figured out that it would be easier to resync the data from scratch. I found the documentation
https://docs.mongodb.com/manual/tutorial/resync-replica-set-member/#resync-a-member-of-a-replica-set
but I am not fully aware what commands run to execute this operation.
My dbPath = "/mnt/mongo/mongodb", MongoDB shell version v3.4.14, and my database has about 2.5T. Could you give my some guidance how to execute initial sync mongo replica?
From my understanding i should
sudo rm -r /mnt/mongo/mongodb/*
sudo service mongodb start
After some time everything should get back to normal(?)
Correct me if I am wrong...
Related
I want to setup PostgreSQL 12 with PostGIS 3 on Ubuntu 20.04 for the purpose of creating an OSM Tile Server. I want to have 2 different clusters, one for a regular PSQL database and another for OSM data. I can't seem to get the one for the OSM data up and running:
When I run pg_lsclusters, I get the following:
Ver Cluster Port Status Owner Data directory Log file
12 main 5433 online postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log
12 osm_psql_db 5432 down postgres /var/lib/postgresql/12/2TB1/osm_psql_db /var/log/postgresql/postgresql-12-osm_psql_db.log
When I run journalctl -xe, I get the following:
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: Dependency failed for PostgreSQL Cluster 12-osm_psql_db.
-- Subject: A start job for unit postgresql#12-osm_psql_db.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit postgresql#12-osm_psql_db.service has finished with a failure.
--
-- The job identifier is 9566 and the job result is dependency.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: postgresql#12-osm_psql_db.service: Job postgresql#12-osm_psql_db.service/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: var-lib-postgresql-12-osm_psql_db.mount: Job var-lib-postgresql-12-osm_psql_db.mount/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: dev-disk-by\x2dlabel-osm_psql_db.device: Job dev-disk-by\x2dlabel-osm_psql_db.device/start failed with result 'timeout'.
Mar 13 11:47:43 cdil-MS-7B92 PackageKit[27900]: daemon quit
Mar 13 11:47:43 cdil-MS-7B92 systemd[1]: packagekit.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit packagekit.service has successfully entered the 'dead' state.
Any idea what could be holding me up?
*** EXTRA INFO JUST IN CASE ***
In terms of how I set up everything, I installed the following packages:
sudo apt install postgresql-12 postgresql-contrib postgis postgresql-12-postgis-3
Because the OSM data is quite large, I want to store that particular cluster on another hard disk. It's called "2TB1" and it's been mounted to /var/lib/postgresql/12/2TB1 because I realized that the postgres user needed access to the data_directory folder and all parent folders leading up to it.
To do so I modified the permissions of the new hard drive:
sudo chown -R postgres:postgres /var/lib/postgresql/12/2TB1
Next, I created the new db cluster instance:
sudo pg_createcluster 12 osm_psql_db -d /var/lib/postgresql/12/2TB1/osm_psql_db -p 5432
I start the new instance:
sudo pg_ctlcluster 12 osm_psql_db start
I get the following error:
A dependency job for postgresql#12-osm_psql_db.service failed. See 'journalctl -xe' for details.
For anyone that stumbles upon the same issue... I tracked the problem down to the *.service file referencing the wrong mount point for the database cluster location. Here's what I did:
Enable the new service (not sure if this is needed, but what the heck...)
sudo systemctl enable postgresql#12-osm_psql_db
Edit the postgresql#12-osm_psql_db.service
sudo systemctl edit --full postgresql#12-osm_psql_db.service
Change
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/%I
To
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/12/2TB1/osm_psql_db
As part of the service script, %I expands to VERSION/CLUSTER which in my case would have been 12/osm_psql_db. Since I was choosing to place the DB on another SSD and the database can't reside in the root directory of a disk, the mount location in the *.service file needed to be updated to 12/2TB1/osm_psql_db. This would not be necessary if you were storing all your databases on a single hard disk.
I have installed mongodb but when trying to start mongodb.service
but it shows the error
Failed to start mongodb.service: Unit mongodb.service not found.
what might be the reason how to resolve this.
Thanks!!!!!
I am trying to setup Postgres cluster with 3 machines to get high availability with automatic failover.
postgres-01 --> master
postgres-02 --> sync replica
postgres-03 --> async replica
When I tried to use synchronous_standby_names='postgres-02' in the postgresql.conf it fails to restart the postgres with the following error
LOG: invalid value for parameter "synchronous_standby_names": "postgres-02"
DETAIL: syntax error at or near "-"
FATAL: configuration file "/pgsql/postgresql.conf" contains errors
postgresql-10.service: main process exited, code=exited, status=1/FAILURE
Failed to start PostgreSQL 10 database server.
-- Subject: Unit postgresql-10.service has failed
-- Defined-By: systemd
Removing the '-' from the hostname fixes the problem, But is this really required.
You'll have to quote the name:
synchronous_standby_names = '"postgres-02"'
You should have at least two synchronous standby servers, else your system will stop functioning if the single synchronous standby server goes down.
Digitalocean disabled my droplet's internet access. After fixing the error (rollback to older backup) they restored the internet access. But afterwards I constantly get an error when deploying, I can't seem to get my Postgres database up and running.
I'm getting an error each time I try to deploy my application.
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
So I used SSH to login to my server and check if my Postgres was actually running with:
pg_lsclusters
Results into:
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Postgres server status
So my Postgres server seems to be down. I tried putting it 'up' again with:
pg_ctlcluster 9.5 main start After doing so I got the error: Insecure directory in $ENV{PATH} while running with -T switch at /usr/bin/pg_ctlcluster line 403.
And /usr/bin/pg_ctlcluster on line 403 says:
system 'systemctl', 'is-active', '-q', "postgresql\#$version-$cluster";
But I'm not to sure what the problem could be here and how I could fix this.
Update
I also tried updating the permissions on /bin to 755 as mentioned here. Sadly that did not fix my problem.
Update 2
I changed the /usr/bin to 755. Now when I try pg_ctlcluster 9.5 main start, I get this:
Job for postgresql#9.5-main.service failed because the control process exited with error code. See "systemctl status postgresql#9.5-main.service" and "journalctl -xe" for details.
And inside the systemctl status postgresql#9.5-main.service:
postgresql#9.5-main.service - PostgreSQL Cluster 9.5-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2018-01-28 17:32:38 EST; 45s ago
Process: 22473 ExecStart=postgresql#%i --skip-systemctl-redirect %i start (code=exited, status=1/FAILURE)
Jan 28 17:32:08 *url* systemd[1]: Starting PostgreSQL Cluster 9.5-main...
Jan 28 17:32:38 *url* postgresql#9.5-main[22473]: The PostgreSQL server failed to start.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Control process exited, code=exited status=1
Jan 28 17:32:38 *url* systemd[1]: Failed to start PostgreSQL Cluster 9.5-main.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Unit entered failed state.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Failed with result 'exit-code'.
Thanks!
You better not mix systemctl and pg_ctlcluster. Let systemctl makes the calls to pg_ctlcluster with the right user and permissions. You should start your postgresql instance with
sudo systemctl start postgresql#9.5-main.service
Also, check the errors in the startup log. You can post them too, to help you figure out what's going on.
Your systemctl status also outputs that the service is disable, so, when the server reboots, you will have to start the service manually. To enable it run:
sudo systemctl enable postgresql#9.5-main.service
I hope it helps
It is mainly because /etc/hosts file is somehow changed.I have removed extra space inside /etc/hosts file.Use cat /etc/hosts
Add these lines into the file
127.0.0.1 localhost
127.0.1.1 your-host-name
::1 ip6-localhost ip6-loopback
And I have given permission 644 to /etc/hosts file.It is working for me even after the reboot of the system.
I am getting this error instead of my shell prompt (running Ubuntu 16.04LTS).
I'm pretty new to MongoDB. I had to downgrade from version 3.4 to 3.2:
https://docs.mongodb.com/manual/release-notes/3.4-downgrade-standalone/
I followed those instructions (I thought) pretty faithfully, but now can't access the shell to confirm, etc.
scorpian555#scorpo:~$ mongo
MongoDB shell version: 3.2.15
connecting to: test
2017-07-13T14:41:31.434-0400 W NETWORK [thread1] Failed to connect to
127.0.0.1:27017, in(checking socket for error after poll), reason:
errno:111 Connection refused
2017-07-13T14:41:31.434-0400 E QUERY [thread1] Error: couldn't
connect to server 127.0.0.1:27017, connection attempt failed :
connect#src/mongo/shell/mongo.js:229:14
#(connect):1:6
exception: connect failed
There was a socket file that the error seems to be referencing at /tmp/mongodb-27017.sock
Per these, I ended up removing that socket file, then tried to reboot the mongod daemon:
'Failed to unlink socket file" error in MongoDB 3.0
MongoDB - Permission denied for socket: /tmp/mongodb-27017.sock
service mongod restart
service mongod status
scorpian555#scorpo:~$ service mongod status
● mongod.service - High-performance, schema-free document-oriented
database
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor
preset: enabled)
Active: active (running) since Thu 2017-07-13 14:58:08 EDT; 4s ago
Docs: https://docs.mongodb.org/manual
Main PID: 4763 (mongod)
CGroup: /system.slice/mongod.service
└─4763 /usr/bin/mongod --quiet --config /etc/mongod.conf
Jul 13 14:58:08 scorpo systemd[1]: Started High-performance, schema-
free document-oriented database.
Per those, I removed the socket file and ran:
scorpian555#scorpo:~$ ls -ld /tmp
drwxrwxrwt 17 root root 20480 Jul 13 14:58 /tmp
I think I'm a little lost here, I was under the impression that, upon restart of the daemon, it would recreate this temporary file, at which point I would check the permissions.
I can't find the link at the moment, but had found elsewhere instructions to make sure the owner of the mongod.sock file was mongod (not root) and I would have to chown it if it wasn't mongod.
I'm pretty new to MongoDB, like I said. I found this as another possible issue, but wanted to see if I should dive down this rabbit hole or whether I might be looking in the wrong place?
Connection refused to MongoDB errno 111
Thanks.
My issue was that, while I started the mongod service, I did not actually run the daemon with sudo as needed.