Here my zkServer.cmd file :
#echo off
setlocal
call "%~dp0zkEnv.cmd"
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*
endlocal
The skServer.sh script will run the zkEnv.sh script which in-turn will look for a script '../conf/zookeeper-env.sh'
create a file on the conf folder called zookeeper-env.sh
Paste this into the file and restart Zookeeper:
JMXLOCALONLY=false
JMXDISABLE=false
JMXPORT=4048
JMXAUTH=false
JMXSSL=false
First obtain the hostname (or reachable IP eg. lan/public/NAT address):
hostname -i
# or find ip
ip a
next add following options to ZOOMAIN (assumed hostname my.remoteconsole.org and desired port 8989)
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=8989
-Djava.rmi.server.hostname=my.remoteconsole.org
More details about available options in java docs (http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html).
ADD org.apache.zookeeper.server.quorum.QuorumPeerMain in server-start.
The class org.apache.zookeeper.server.quorum.QuorumPeerMain will start a JMX manageable ZooKeeper server. This class registers the proper MBeans during initalization to support JMX monitoring and management of the instance.
In addition to above answer by Marcell du Plessis, if you are running zookeeper as a systemd service, then you can specify jmx port in the environment variable.
[Unit]
Description=Apache Kakfa Zookeeper
Requires=network.target
After=network.target
[Service]
Type=simple
User=user
Group=users
ExecStart=/your-zookeeper-install-path/bin/zkServer.sh start
ExecStop=/your-zookeeper-install-path/bin/zkServer.sh stop
TimeoutStopSec=180
Restart=on-failure
Environment="JMX_PORT=9999"
[Install]
WantedBy=multi-user.target
Alias=zookeeper.service
Related
Using reference to https://docs.microfocus.com/itom/MP_for_Apache_Kafka:1.10/Kafka/Kafka_JMX,
I created the jmx_local.config and modified the Kafka start up script)
The Kafka start up script picks the jmx_local.coonfig but the port is not getting exposed.
This is what I see on grepping the java process:
"/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin/java -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.config.file=/usr/local/etc/kafka/jmx_local.conf kafka.Kafka /usr/local/etc/kafka/server.properties"
cat /usr/local/etc/kafka/jmx_local.conf
Dcom.sun.management.jmxremote.port=9395
Dcom.sun.management.jmxremote.authenticate=false
Dcom.sun.management.jmxremote.ssl=false
Also tried with port 10167 but the port is not enabled. Also modified as 'com.sun.management.jmxremote.port=9395'
I could see the other jmx properties.
Any suggestion please
I did grep -rl "jmxremote" /usr/local/Cellar/kafka/2.6.0, and found the jxm config was considered from bin/kafka-run-class.sh. So added 'Dcom.sun.management.jmxremote.port=9395' in bin/kafka-run-class.sh and restarted the kafka service.
To find if the port is available:
netstat -an | grep 1099
With my team we are trying to move our micro-services to openj9, they are running on kubernetes. However, we encounter a problem on the configuration of JMX. (openjdk8-openj9)
We have a connection refused when we try a connection with jvisualvm (and a port-forwarding with Kubernetes).
We haven't changed our configuration, except for switching from Hotspot to OpenJ9.
The error :
E0312 17:09:46.286374 17160 portforward.go:400] an error occurred forwarding 1099 -> 1099: error forwarding port 1099 to pod XXXXXXX, uid : exit status 1: 2020/03/12 16:09:45 socat[31284] E connect(5, AF=2 127.0.0.1:1099, 16): Connection refused
The java options that we use :
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.port=1099
-Dcom.sun.management.jmxremote.rmi.port=1099
We are using the last adoptopenjdk/openjdk8-openj9 docker image.
Do you have any ideas?
Thank you !
Regards.
I managed to figure out why it wasn't working.
It turns out that to pass the JMX options to the service we were using the Kubernetes service descriptor in YAML. It looks like this:
- name: _JAVA_OPTIONS
value: -Dzipkinserver.listOfServers=http://zipkin:9411 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.rmi.port=1099
I realized that the JMX properties were not taken into account from _JAVA_OPTIONS when the application is not launch with ENTRYPOINT in the docker container.
So I pass the properties directly into the Dockerfile like this and it works.
CMD ["java", "-Dcom.sun.management.jmxremote", "-Dcom.sun.management.jmxremote.authenticate=false", "-Dcom.sun.management.jmxremote.ssl=false", "-Dcom.sun.management.jmxremote.local.only=false", "-Dcom.sun.management.jmxremote.port=1099", "-Dcom.sun.management.jmxremote.rmi.port=1099", "-Djava.rmi.server.hostname=127.0.0.1", "-cp","app:app/lib/*","OurMainClass"]
It's also possible to keep _JAVA_OPTIONS and setup an ENTRYPOINT in the dockerfile.
Thanks!
I have configured my superset in Virtual env want to run it as a service
I have tried using below config but its not working
[Unit]
Description=superset service
After=network.target
[Service]
Type=simple
User=superset
Group=superset
Environment=PATH=/home/ubuntu/code/superset:$PATH
Environment=PYTHONPATH=/var/superset/superset:$PYTHONPATH
ExecStart=/home/ubuntu/code/superset/superset runserver
[Install]
WantedBy=multi-user.target
Virtual Env folder is Superset
I get the below error
/etc/init.d/superset: 1: /etc/init.d/superset: [Unit]: not found
Usage: service < option > | --status-all | [ service_name [ command |
--full-restart ] ] /etc/init.d/superset: 5: /etc/init.d/superset: [Service]: not found
Actually the superset runserver is used for development mode and it is highly recommended other tools like gunicorn for production.
Anyway, the main problem is that superset path on the virutalenv is $VENV_PATH/bin/superset (in general the applications that treat like binary applications like superset or airflow, etc servers on this path: $VENV_PATH/bin and the easy way to find the path of any application on Linux systems is to use which command that in this case, you can use which superset to find the superset path ).
This is the superset service file that I use it on the production, hope to useful:
[Unit]
Description = Apache Superset Webserver Daemon
After = network.target
[Service]
PIDFile = /home/superset/superset-webserver.PIDFile
User = superset
Group = superset
Environment=SUPERSET_HOME=/home/superset
Environment=PYTHONPATH=/home/superset
WorkingDirectory = /home/superset
ExecStart =/home/superset/venv/bin/python3.7 /home/superset/venv/bin/gunicorn --workers 8 --worker-class gevent --bind 0.0.0.0:8888 --pid /home/superset/superset-webserver.PIDFile superset:app
ExecStop = /bin/kill -s TERM $MAINPID
[Install]
WantedBy=multi-user.target
I have following supervisord config(copied from this answer):
[program:myprogram]
process_name=MYPROGRAM%(process_num)s
directory=/var/www/apps/myapp
command=/var/www/apps/myapp/virtualenv/bin/python index.py --PORT=%(process_num)s
startsecs=2
user=youruser
stdout_logfile=/var/log/myapp/out-%(process_num)s.log
stderr_logfile=/var/log/myapp/err-%(process_num)s.log
numprocs=4
numprocs_start=14000
Can i do same thing with systemd?
A systemd unit can include specifiers which may be used to write a generic unit service that will be instantiated several times.
Example based on your supervisord config: /etc/systemd/system/mydaemon#.service:
[Unit]
Description=My awesome daemon on port %i
After=network.target
[Service]
User=youruser
WorkingDirectory=/var/www/apps/myapp
Type=simple
ExecStart=/var/www/apps/myapp/virtualenv/bin/python index.py --PORT=%i
[Install]
WantedBy=multi-user.target
You may then enable / start as many instances of that service using by example:
# systemctl start mydaemon#4444.service
Article with more examples on Fedora Magazine.org: systemd: Template unit files.
I have the following in my supervisord configuration:
[program:nodejs]
command=node server.js
autostart=true
autorestart=true
stderr_logfile=/home/user/logs/nodejs.err.log
stderr_outfile=/home/user/logs/nodejs.out.log
I don't see log files being created in /logs directory. I see that temporary log files are created in /tmp/nodejs_some_random_string.log. What am I am missing in order to have log files created properly as specified in the configuration?
supervisord has no configuration key with stderr_outfile - check out child-process-logs
use stdout_logfile or stderr_logfile
Try like this:
[program:nodejs]
command=node server.js
autostart=true
autorestart=true
stderr_logfile=/home/user/logs/nodejs.err.log
stdout_logfile=/home/user/logs/nodejs.out.log
see example configuration