We previously had three Clickhouse nodes perfectly synced within Zookeeper until one of them was lost.
The Clickhouse node was rebuilt exactly as it was before (with Ansible) and the same create table command was run which resulted in the following error.
Command:
CREATE TABLE ontime_replica ( ... )
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/ontime_replica', '{replica}', FlightDate, (Year, FlightDate), 8192)
The error is:
Received exception from server:
Code: 253. DB::Exception: Received from localhost:9000, 127.0.0.1. DB::Exception: Replica /clickhouse/tables/01/ontime_replica/replicas/clickhouse1 already exists..
We're currently using Zookeeper version 3.4.10 and I would like to know if there's a way to remove the existing replica within Zookeeper, or simple let Zookeeper know that this is the new version of the existing replica.
Thank you in advance!
My approach to the solution was incorrect. Originally, I thought I needed to remove the replica within Zookeeper. Instead, the following commands within the Clickhouse server solve this problem.
Copy the SQL file from another, working node. The file is in /var/lib/clickhouse/metadata/default
chown clickhouse:clickhouse <database>.sql
chmod 0640 <database>.sql
sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data
service clickhouse-server start
Related
Recently because of an unknown issue our MongoDB hosted on a GCP compute VM has stopped, we were unable to restart it because it's throwing MongoDB.service is not found. so we reinstalled the MongoDB after taking a backup of all .wt files in the DB path. once we've reinstalled the MongoDB we copied the files back and we can't see the data in the DB. we tried the --repair flag but still no use.
is there a way we can get this working?
The other thing is we've taken the VM snapshot from the day before the crash. there we can see the data in MongoDB only if we run the method rs.slaveOk(). I think we can't use that DB as the primary db. is there a way that we can use this as a primary db.
I'm relatively new to the concept of replica set, Master/slave any suggestions and questions are welcome
Thanks
If you see the data from the snapshot with rs.slaveOk() it is easy to recover , you can reconfigure the member as standalone PRIMARY with this steps:
Get the current config:
cfg = rs.conf()
printjson(cfg)
Set in the temp variable cfg only the available member(in the example it is the first in the cfg with id:0 ):
cfg.members = [cfg.members[0]]
Reconfigure the replicaSet only with the available member:
rs.reconfig(cfg, {force : true})
(Don't forget to add the {force:true} option since it is the only way to reconfigure from SECONDARY)
If all is good with this member and it successfully elect as PRIMARY , you can add other new members with rs.add() ...
I have set up my MongoDB on local machine with replicaSet, and I have this error:
An error occurred while loading navigation: 'not master and
slaveOk=false': It is recommended to change your read preference in
the connection dialog to Primary Preferred or Secondary Preferred or
provide a replica set name for a full topology connection.
If I don't exec this command on secondary servers,
rs.slaveOk()
After using the command, the problem is solved, but only temporarily. After restarting the servers, the above error pops up again, I again have to stop it using the command.
How can I somehow define the slaveOk in a config file, so that I don't have to allow slaveOk each time I start the server?
Your connection string should look something like this:
mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=myRepl
If you are connecting from shell
mongo "mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=myRepl"
OR
mongo --host myRepl/mongodb0.example.com.local:27017,mongodb1.example.com.local:27017,mongodb2.example.com.local:27017
Where myRepl is the replica set name, mongodbX.example.com.local:27017 are your nodes.
We use Postgres as a docker container, and each time we restart the machine, the data and tables are lost.
Even data files under the base and global folders.
Even more, now we've faced some RAM problems, so I'm not able to connect to Postgres and VM instance itself in now way -- neither by ssh, psql client nor redash. The last one says the following:
Error running query: could not send SSL negotiation packet: Resource temporarily unavailable.
Is there any way to somehow solve the problem?
UPD. I've tried to create a disk snapshot, but it doesn't help.
UPD2. BTW, the pgdata folder is set to be a container volume and could be used outside of it.
I was attempting to migrate my standalone mongo server to a single node replica set using the following instructions:
https://docs.mongodb.com/manual/tutorial/convert-standalone-to-replica-set/
Upon restarting the docker container with the -replSet argument and initiating within the mongo shell, I noticed that the replica set was created, but virtually all of my database collections disappeared. The /data/db folder argument hasn't changed and I haven't assigned a different config file. Is there any hope in recovering my data?
I was using MongoDB version 2.6.6 on Google Compute Engine and used the click to deploy method.
rs0:SECONDARY> db.createUser({user:"admin", pwd:"secret_password", roles:[{role:"root", db:"admin"}]})
2015-07-13T15:02:28.434+0000 Error: couldn't add user: not master at src/mongo/shell/db.js:1004
rs0:SECONDARY> use admin
switched to db admin
rs0:SECONDARY> db.createUser({user:"admin", pwd:"secret_password", roles:["root"]})
2015-07-13T15:13:28.591+0000 Error: couldn't add user: not master at src/mongo/shell/db.js:1004
I had a similar problem with mongo 3.2:
Error: couldn't add user: not master :
When trying to create a new user, with root role.
I was using only a local copy of mongo.
In my mongod.conf file I had the following uncommented:
replication:
replSetName: <node name>
Commenting that out and restarting fixed the problem. I guess mongo thought it was part of a replication set, and was confused as to who the Master was.
Edit:
I've also found that if you ARE trying to setup a replication set, and you get the above error, then run:
rs.initiate()
This will start a replication set, and set the current node as PRIMARY.
Exit, and then log back in and you should see:
PRIMARY>
Now create users as needed.
I ran into this error when scripting replica set creation.
The solution was to add a delay between rs.initiate() and db.createUser().
Replica set creation is seemingly done in background and it takes time for the primary node to actually become primary. In interactive use this doesn't cause a problem because there is a delay while typing the next command, but when scripting the interactions the delay may need to be forced.
MongoDB will be deployed in a cluster of Compute Engine instances (also known as a MongoDB replica set). Each instance will use a boot disk and separate disk for database files.
Primary and master nodes are the nodes that can accept writes. MongoDB’s replication is “single-master:” only one node can accept write operations at a time.
Secondary and slave nodes are read-only nodes that replicate from the primary.
Your error message looks like you are trying to add the user on the secondary. Try adding the user in the primary.
I ran into this issue when I thought I was running mongo 3.4 but it was mongo 3.6. Uninstalling 3.6 and installing 3.4 fixed my issue.