I'm aware, that the kafka server can be shutdown using shell script kafka-server-stop.sh and zookeeper can be shutdown using zookeeper-server-stop.sh
But, how do we stop connect-distributed.sh gracefully, I didn't find any stop shell script for connect-distributed.
Unfortunately, there isn't a stop script.
Best options other than kill command would be to use systemctl to manage the service, or use pre-built Docker images to run the server that can be stopped.
Related
I have Kafka running with a Sql connector. Currently this is running on a Linux server and I am using a Putty connection to configure and run everything. I can start the worker process in standalone mode fine and everything works as expected. However, it feels like I should be able to leave my terminal session and keep the worker running like a service. Currently I end my terminal session and just reconnect with putty but again, this doesn't feel like the right approach. Does anyone know how to get the worker to run like a service?
./connect-standalone.sh '../config/worker.properties' '../config/connector.properties'
First I would like to tell you that, running Kafka-connect in a standalone mode is not a good choice, and you can also run Kafka-connect in distributed mode on a single machine.
If you don't want to create a service then you can use the screen utility.
Ex.
Create a Screen
screen -S kafka-connect
Run kafka-connect command
./connect-standalone.sh '../config/worker.properties' '../config/connector.properties'
Detech the screen using ctrl+A+D
List and Resume screen
screen -ls
screen -r kafka-connect
Type exit inside the screen to the terminal attached screen.
Service
Create a new file kafka-connect.service inside /etc/systemd/system directory.
kafka-connect.service
[Unit]
Description=Kakfka-connect
After=network.target
[Service]
User=ubuntu
Group=ubuntu
Environmet="KAFKA_HEAP_OPTS=-Xmx1G -Xms1G"
Environment="KAFKA_OPTS=-javaagent:/home/ubuntu/prometheus/jmx_prometheus_javaagent-0.15.0.jar=8080:/home/ubuntu/prometheus/kafka-connect.yml"
ExecStart=/home/ubuntu/kafka_2.13-2.7.0/bin/connect-distributed.sh /home/ubuntu/kafka_2.13-2.7.0/config/connect-distributed.properties
[Install]
WantedBy=multi-user.target
In ExecStart that is the command to start Kafka-connect service as distributed mode, you can change that also If you haven't install jmx_exporter then you can remove Environment="KAFKA_OPTS=-javaagent:/home/ubuntu/prometheus/jmx_prometheus_javaagent-0.15.0.jar=8080:/home/ubuntu/prometheus/kafka-connect.yml" this line from service.
Every time I stop the kafka server and start it again it doesn't start properly and I have to restart my whole machine and start the kafka server.
Does anybody know how I can restart kafka server without having to restart my machine?
Actually I would like to terminate the consumer from last session.
Thank you,
Zeinab
If your Kafka broker is running as a service (found under /lib/systemd/system/) from a recent Confluent Platform release, you can stop it using:
systemctl stop confluent-kafka.service
or if you'd like to restart the service,
systemctl restart confluent-kafka.service
Otherwise, you can stop your broker using
./bin/kafka-server-stop.sh
and re-start it:
./bin/kafka-server-start.sh config/server.properties
If you want to stop a specific consumer, simply find the corresponding process id:
ps -ef | grep consumer_name
and kill that process:
kill -9 process_id
Or simply:
sudo systemctl restart kafka
My mongodb run under Linux 6. I use the command db.shutdownServer() to close the database but the mongod process does not stop.
Stopping mongo directly with service mongod stop do a clean shutdown?
Thanks for your help
Proper ways to shutdown mongod is described in the documentation. They are:
Use shutdownServer()
From the mongo shell
use admin
db.shutdownServer()
Use --shutdown
From the Linux command line
mongod --shutdown
Use CTRL-C
When running the mongod instance in interactive mode, issue Control-C
Use kill
From the Linux command line
kill mongoProcessID
kill -2 mongodProcessID
So you need to figure out how /etc/init.d/mongodb stop actually stops the process on your Linux distribution. For example, on Debian it uses the wrapper which behaves similar to killall which is a proper method.
As per the documentation of mongodb, mongod --shutdown will work on linux system, but, on Mac, the --shutdown switch is not recognised. However, per the documentation, Ctrl+C will cleanly shutdown the db server. When you do a Ctrl+C on the same terminal where the db server is running, it initiates a dozen or so signalProcessingThread which indicates that the shutdown is proper and smooth. At the end, you can see that the process exits with code:0. Per the convention, Ctrl+C is awkward, but is clean, although not seemingly graceful.
I have a mongodb cluster up and running. I want to setup a client (mongos) to connect to the config servers from ubuntu. Most instructions just say to run this command:
mongos --configdb cfg0.example.net:27019,cfg1.example.net:27019,cfg2.example.net:27019
Is this command running as a service? Will the process still be running when I exit the shell? What happens if the process goes down? What is the proper way of running this client as a service?
You would use --fork or an init script to make this run as a service post terminal session shut down.
If the process goes down then your application cannot connect to the sharded set. It will be unable to connect at all to your DB. This is (not the only reason) why you should have good redundancy in mongos instances.
I tend to have one mongos per app server personally, however, it is all down to preference. Another option is to have a load balanced set of mongos instances.
I've completed this tutorial and successfully deployed a 3 node replica set. I can connect to it from other hosts and all is good. The question I have is that in the tutorial it states
Start MongoDB
Once the configuration files have been edited, start the database process manual:mongod on each instance by:
Log on onto the instance
Run the following command to start the process:
mongod --config /etc/mongod.conf
This should start the manual:mongod process
To me this seems as though the replica set is running as a user process and not as a system service as in the command
sudo service mongodb start
So what happens if one of the machines reboots? Is that process dead? How can I configure the whole replica set to run as a service?
On machine reboots, the mongod process will stop and you have to restart it.
In system scripts, I am not sure if on box restarts, mongod restart is automatically taken care of or not. But you can have service scripts for mongod process, which you get automatically, when you install using mongodb apt-get/yum packages.