Zookeeper / Exhibitor recurring JMX Error - apache-zookeeper

I don't know why this is occurring, but occasionally I will get this series of repeating errors and the zookeeper instances will go into a bad state.
Tue Feb 16 07:05:04 EST 2016 ERROR ZooKeeper Server: Using config: /opt/zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Tue Feb 16 07:05:04 EST 2016 ERROR ZooKeeper Server: JMX enabled by default
Tue Feb 16 07:05:04 EST 2016 INFO Process started via: /opt/zookeeper/zookeeper-3.4.6/bin/zkServer.sh
Tue Feb 16 07:05:03 EST 2016 INFO Kill attempted result: 0
Tue Feb 16 07:05:03 EST 2016 INFO Attempting to start/restart ZooKeeper
Tue Feb 16 07:05:03 EST 2016 INFO Attempting to stop instance
Tue Feb 16 07:05:03 EST 2016 INFO Restarting down/not-serving ZooKeeper after 60037 ms pause
Tue Feb 16 07:04:33 EST 2016 INFO ZooKeeper down/not-serving waiting 30026 of 40000 ms before restarting
Tue Feb 16 07:04:05 EST 2016 INFO ZooKeeper Server: Starting zookeeper ... STARTED
Tue Feb 16 07:04:04 EST 2016 ERROR ZooKeeper Server: Using config: /opt/zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Tue Feb 16 07:04:04 EST 2016 ERROR ZooKeeper Server: JMX enabled by default
The exhibitor stuff uses shared storage on a NAS. The servers are centOs 6.6. It is a three node ensemble, and the one noticible problem I have seen is that the "ensemble" connection string inside of Exhibitor GUI all of a sudden becomes different between the three nodes (one node may "forget" about some of the other nodes in the ensemble).
I don't even know where to look to dig into these causes. Any help or direction will be greatly appreciated. Its trully odd...
update versions
zk: 3.4.6
Exhibitor: 1.5.5

Related

Redis standalone stop working (timeouts) on master replica sync

We use a redis cache in out kubernetes cluster which stops working really randomly. It's a Standalone version based on this image: bitnami/redis:6.0.15
As custom parameters we use:
MASTER true
REDIS_AOF_ENABLED no
Every time when the redis stop working I see the following logs:
Jul 5 13:30:27 redis-0 redis 1:M 05 Jul 2022 11:30:27.060 * 10000 changes in 60 seconds. Saving...
Jul 5 13:30:27 redis-0 redis 1:M 05 Jul 2022 11:30:27.090 * Background saving started by pid 364
Jul 5 13:31:34 redis-0 redis 364:C 05 Jul 2022 11:31:34.307 * DB saved on disk
Jul 5 13:31:34 redis-0 redis 364:C 05 Jul 2022 11:31:34.341 * RDB: 431 MB of memory used by copy-on-write
Jul 5 13:31:34 redis-0 redis 1:M 05 Jul 2022 11:31:34.488 * Background saving terminated with success
Jul 5 13:32:35 redis-0 redis 1:M 05 Jul 2022 11:32:35.022 * 10000 changes in 60 seconds. Saving...
Jul 5 13:32:35 redis-0 redis 1:M 05 Jul 2022 11:32:35.052 * Background saving started by pid 365
-----
Jul 5 13:32:40 redis-0 redis 1:S 05 Jul 2022 11:32:40.436 * Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
Jul 5 13:32:40 redis-0 redis 1:S 05 Jul 2022 11:32:40.436 * REPLICAOF 178.20.40.200:8886 enabled (user request from 'id=71457 addr=10.0.16.46:14072 fd=12 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=47 qbuf-free=32721 argv-mem=24 obl=0 oll=0 omem=0 tot-mem=61488 events=r cmd=slaveof user=default')
Jul 5 13:32:41 redis-0 redis 1:S 05 Jul 2022 11:32:41.316 * Connecting to MASTER 178.20.40.200:8886
Jul 5 13:32:41 redis-0 redis 1:S 05 Jul 2022 11:32:41.316 * MASTER <-> REPLICA sync started
Jul 5 13:32:41 redis-0 redis 1:S 05 Jul 2022 11:32:41.362 * Non blocking connect for SYNC fired the event.
Jul 5 13:32:41 redis-0 redis Error 1:S 05 Jul 2022 11:32:41.409 # Error reply to PING from master: '-Reading from master: Connection reset by peer'
Jul 5 13:32:42 redis-0 redis 1:S 05 Jul 2022 11:32:42.316 * Connecting to MASTER 178.20.40.200:8886
Jul 5 13:32:42 redis-0 redis 1:S 05 Jul 2022 11:32:42.317 * MASTER <-> REPLICA sync started
Jul 5 13:32:42 redis-0 redis 1:S 05 Jul 2022 11:32:42.366 * Non blocking connect for SYNC fired the event.
Jul 5 13:32:42 redis-0 redis Error 1:S 05 Jul 2022 11:32:42.415 # Error reply to PING from master: '-Reading from master: Connection reset by peer'
Jul 5 13:32:43 redis-0 redis 1:S 05 Jul 2022 11:32:43.317 * Connecting to MASTER 178.20.40.200:8886
Jul 5 13:32:43 redis-0 redis 1:S 05 Jul 2022 11:32:43.317 * MASTER <-> REPLICA sync started
Jul 5 13:32:43 redis-0 redis 1:S 05 Jul 2022 11:32:43.366 * Non blocking connect for SYNC fired the event.
Jul 5 13:32:43 redis-0 redis Error 1:S 05 Jul 2022 11:32:43.416 # Error reply to PING from master: '-Reading from master: Connection reset by peer'
Jul 5 13:32:44 redis-0 redis 1:S 05 Jul 2022 11:32:44.320 * Connecting to MASTER 178.20.40.200:8886
Jul 5 13:32:44 redis-0 redis 1:S 05 Jul 2022 11:32:44.320 * MASTER <-> REPLICA sync started
Jul 5 13:32:44 redis-0 redis 1:S 05 Jul 2022 11:32:44.370 * Non blocking connect for SYNC fired the event.
Then I see that the queue increase, but I need to kill the pod to restart redis otherwise it will not work anymore.
next: GET 6126674261995698486,
inst: 1,
qu: 0, // queue => waiting operations
qs: 17,
aw: False,
rs: ReadAsync,
ws: Idle,
in: 0, // bytes waiting from input stream
in-pipe: 0,
out-pipe: 0,
serverEndpoint: redis.default.svc.cluster.local:6379,
mc: 1/1/0,
mgr: 10 of 10 available, // tread pool
clientName: production-9bbd94544-nlmv7,
IOCP: (Busy=0,Free=1000,Min=5,Max=1000), // no busy threads
WORKER: (Busy=14,Free=32753,Min=256,Max=32767),
v: 2.2.4.27433```
```Timeout performing GET (3000ms),
next: 2865582319381864083,
inst: 0,
qu: 0,
qs: 333,
aw: False,
rs: ReadAsync,
ws: Idle,
in: 0,
in-pipe: 0,
out-pipe: 0,
serverEndpoint: redis.default.svc.cluster.local:6379,
mc: 1/1/0,
mgr: 10 of 10 available,
clientName: production-58c7874fd8-tdcpz,
IOCP: (Busy=0,Free=1000,Min=1,Max=1000),
WORKER: (Busy=3,Free=32764,Min=256,Max=32767),
v: 2.2.4.27433
next: GET 6126674261995698486,
inst: 47,
qu: 0,
qs: 21368,
aw: False,
rs: ReadAsync,
ws: Idle,
in: 0,
in-pipe: 0,
out-pipe: 0,
serverEndpoint: redis.default.svc.cluster.local:6379,
mc: 1/1/0,
mgr: 10 of 10 available,
clientName: production-9bbd94544-nlmv7,
IOCP: (Busy=0,Free=1000,Min=5,Max=1000),
WORKER: (Busy=162,Free=32605,Min=256,Max=32767),
v: 2.2.4.27433```
Has anyone an idea?
Thank you.
Can you check if the redis service is running? kubectl get service/redis. It seems like the service is unable to receive traffic, which is possible if there are no pods to receive it.

freeradius startup error (code=exited, status=1/FAILURE)

I am new to freeradius. I have installed freeradius on centos-7 and planned to manage it using daloradius.
After installing freeradius it was working fine and started normally by "systemctl start radiusd".
but after installing and configuring MariaDB, freeradius is not starting while working with "radiusd -X" is working.
following is the error message:
[root#radiusx1 ~]# systemctl start radiusd
Warning: radiusd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Job for radiusd.service failed because the control process exited with error code. See "systemctl status radiusd.service" and "journalctl -xe" for details.
[root#radiusx1 ~]# systemctl status radiusd
● radiusd.service - FreeRADIUS high performance RADIUS server.
Loaded: loaded (/usr/lib/systemd/system/radiusd.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2017-03-02 17:21:00 PKT; 10s ago
Process: 26712 ExecStartPre=/usr/sbin/radiusd -C (code=exited, status=1/FAILURE)
Process: 26708 ExecStartPre=/bin/chown -R radiusd.radiusd /var/run/radiusd (code=exited, status=0/SUCCESS)
Mar 02 17:20:59 radiusx1.xoultech.com systemd[1]: Starting FreeRADIUS high performance RADIUS server....
Mar 02 17:21:00 radiusx1.xoultech.com systemd[1]: radiusd.service: control process exited, code=exited status=1
Mar 02 17:21:00 radiusx1.xoultech.com systemd[1]: Failed to start FreeRADIUS high performance RADIUS server..
Mar 02 17:21:00 radiusx1.xoultech.com systemd[1]: Unit radiusd.service entered failed state.
Mar 02 17:21:00 radiusx1.xoultech.com systemd[1]: radiusd.service failed.
Please help me to sort out this issue.
Any help will be appreciated.
That's likely a permissions issue. -X runs the process as the current user, whereas -C will change to the configured user/group.
You can use this invocation to perform a config check with debugging info written to stdout.
/usr/sbin/radiusd -C -lstdout -xxx
-C config check
-lstdout override configured logging destination and set to stdout.
-xxx set debugging to level 3. Should show any useful output.
Report back with the output if it doesn't make sense to you, and i'll interpret.
I match same issue, after check by Arran's command that will let you know reason, but suggest you to enable radius by "radius -X" under root environment, that will give you root cause, as follow my issue:
Under user environment:
[cisco#frank ~]$ /usr/sbin/radiusd -C -lstdout -xxx
Wed May 17 04:47:36 2017 : Info: radiusd: FreeRADIUS Version 3.0.1, for host x86_64-redhat-linux-gnu, built on Mar 5 2014 at 05:31:12
Wed May 17 04:47:36 2017 : Debug: Server was built with:
Wed May 17 04:47:36 2017 : Debug: accounting
Wed May 17 04:47:36 2017 : Debug: authentication
Wed May 17 04:47:36 2017 : Debug: ascend binary attributes
Wed May 17 04:47:36 2017 : Debug: coa
Wed May 17 04:47:36 2017 : Debug: control-socket
Wed May 17 04:47:36 2017 : Debug: detail
Wed May 17 04:47:36 2017 : Debug: dhcp
Wed May 17 04:47:36 2017 : Debug: dynamic clients
Wed May 17 04:47:36 2017 : Debug: proxy
Wed May 17 04:47:36 2017 : Debug: regex-pcre
Wed May 17 04:47:36 2017 : Debug: session-management
Wed May 17 04:47:36 2017 : Debug: stats
Wed May 17 04:47:36 2017 : Debug: tcp
Wed May 17 04:47:36 2017 : Debug: threads
Wed May 17 04:47:36 2017 : Debug: tls
Wed May 17 04:47:36 2017 : Debug: unlang
Wed May 17 04:47:36 2017 : Debug: vmps
Wed May 17 04:47:36 2017 : Debug: Server core libs:
Wed May 17 04:47:36 2017 : Debug: talloc : 2.0.*
Wed May 17 04:47:36 2017 : Debug: ssl : OpenSSL 1.0.1e-fips 11 Feb 2013
Wed May 17 04:47:36 2017 : Info: Copyright (C) 1999-2014 The FreeRADIUS server project and contributors
Wed May 17 04:47:36 2017 : Info: There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
Wed May 17 04:47:36 2017 : Info: PARTICULAR PURPOSE
Wed May 17 04:47:36 2017 : Info: You may redistribute copies of FreeRADIUS under the terms of the
Wed May 17 04:47:36 2017 : Info: GNU General Public License
Wed May 17 04:47:36 2017 : Info: For more information about these matters, see the file named COPYRIGHT
Wed May 17 04:47:36 2017 : Info: Starting - reading configuration files ...
Wed May 17 04:47:36 2017 : Debug: including dictionary file /etc/raddb/dictionary
Wed May 17 04:47:36 2017 : Debug: including configuration file /etc/raddb/radiusd.conf
Wed May 17 04:47:36 2017 : Error: Unable to open file "/etc/raddb/radiusd.conf": Permission denied
Wed May 17 04:47:36 2017 : Error: Errors reading or parsing /etc/raddb/radiusd.conf
[cisco#frank ~]$ radiusd -X
radiusd: FreeRADIUS Version 3.0.1, for host x86_64-redhat-linux-gnu, built on Mar 5 2014 at 05:31:12
Copyright (C) 1999-2014 The FreeRADIUS server project and contributors
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE
You may redistribute copies of FreeRADIUS under the terms of the
GNU General Public License
For more information about these matters, see the file named COPYRIGHT
Starting - reading configuration files ...
including dictionary file /etc/raddb/dictionary
including configuration file /etc/raddb/radiusd.conf
Unable to open file "/etc/raddb/radiusd.conf": Permission denied
Errors reading or parsing /etc/raddb/radiusd.conf
Under root environment:
[root#frank ~]# radiusd -X
radiusd: FreeRADIUS Version 3.0.1, for host x86_64-redhat-linux-gnu, built on Mar 5 2014 at 05:31:12
Copyright (C) 1999-2014 The FreeRADIUS server project and contributors
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE
You may redistribute copies of FreeRADIUS under the terms of the
GNU General Public License
For more information about these matters, see the file named COPYRIGHT
Starting - reading configuration files ...
including dictionary file /etc/raddb/dictionary
including configuration file /etc/raddb/radiusd.conf
including configuration file /etc/raddb/proxy.conf
including configuration file /etc/raddb/clients.conf
......
client 10.75.13.13 {
require_message_authenticator = no
secret = "cisco"
shortname = "asr9006-3"
limit {
max_connections = 16
lifetime = 0
idle_timeout = 30
}
}
Failed to add duplicate client asr9006-3
/etc/raddb/clients.conf[297]: Failed to add client 10.75.13.13
After correct config issue by root, then restart or reboot, then check again under user:
[cisco#frank ~]$ systemctl status radiusd
radiusd.service - FreeRADIUS high performance RADIUS server.
Loaded: loaded (/usr/lib/systemd/system/radiusd.service; enabled)
Active: active (running) since Wed 2017-05-17 05:10:15 CST; 10min ago
Process: 2438 ExecStart=/usr/sbin/radiusd -d /etc/raddb (code=exited, status=0/SUCCESS)
Process: 1956 ExecStartPre=/usr/sbin/radiusd -C (code=exited, status=0/SUCCESS)
Process: 1951 ExecStartPre=/bin/chown -R radiusd.radiusd /var/run/radiusd (code=exited, status=0/SUCCESS)
Main PID: 2439 (radiusd)
CGroup: /system.slice/radiusd.service
└─2439 /usr/sbin/radiusd -d /etc/raddb

RemoteException in connecting to Tomee (OpenEJB)

Working from the hello world sample, what can I do on tleilax to allow doge to connect to the locally running OpenEJB server? I've read through the documentation on how startup OpenEJB, but must be missing a step from the manual.
On tleilax, deploying Hello.jar:
thufir#tleilax:~$ openejb deploy Hello.jar
Mar 07, 2015 10:34:30 PM org.apache.openejb.client.EventLogger log
INFO: RemoteInitialContextCreated{providerUri=ejbd://localhost:4201}
Mar 07, 2015 10:34:31 PM org.apache.openejb.client.EventLogger log
INFO: ConnectionOpened{uri=ejbd://localhost:4201}
Mar 07, 2015 10:34:31 PM org.apache.openejb.client.EventLogger log
INFO: ServerAdded{server=ejbds://127.0.0.1:4203} ClusterMetaDataUpdated{provider=ejbd://localhost:4201, version=1425796337772, uris=2}
Mar 07, 2015 10:34:31 PM org.apache.openejb.client.EventLogger log
INFO: ServerAdded{server=ejbd://127.0.0.1:4201} ClusterMetaDataUpdated{provider=ejbd://localhost:4201, version=1425796337772, uris=2}
Mar 07, 2015 10:34:31 PM org.apache.openejb.client.EventLogger log
INFO: ServerRemoved{server=ejbd://localhost:4201} ClusterMetaDataUpdated{provider=ejbd://localhost:4201, version=1425796337772, uris=2}
Application deployed successfully at "Hello.jar"
App(id=/home/thufir/apache-openejb-4.7.1/apps/Hello.jar)
EjbJar(id=Hello, path=/home/thufir/apache-openejb-4.7.1/apps/Hello.jar)
Ejb(ejb-name=HelloBean, id=HelloBean)
Jndi(name=HelloBeanRemote)
Jndi(name=global/Hello/HelloBean!org.acme.Hello)
Jndi(name=global/Hello/HelloBean)
some output from tleilax openejb showing the deploy:
INFO: OpenWebBeans Container has started, it took 241 ms.
Mar 07, 2015 10:34:32 PM org.apache.openejb.assembler.classic.Assembler startEjbs
INFO: Created Ejb(deployment-id=HelloBean, ejb-name=HelloBean, container=My Stateless Container)
Mar 07, 2015 10:34:32 PM org.apache.openejb.assembler.classic.Assembler startEjbs
INFO: Started Ejb(deployment-id=HelloBean, ejb-name=HelloBean, container=My Stateless Container)
Mar 07, 2015 10:34:32 PM org.apache.openejb.assembler.classic.Assembler createApplication
INFO: Deployed Application(path=/home/thufir/apache-openejb-4.7.1/apps/Hello.jar)
scanning tleilax for open ports:
thufir#doge:~$
thufir#doge:~$ nmap 192.168.1.3
Starting Nmap 6.46 ( http://nmap.org ) at 2015-03-07 22:42 PST
Nmap scan report for 192.168.1.3
Host is up (0.00092s latency).
Not shown: 998 closed ports
PORT STATE SERVICE
22/tcp open ssh
119/tcp open nntp
Nmap done: 1 IP address (1 host up) scanned in 0.06 seconds
thufir#doge:~$
the bound services on tleilax:
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager start
INFO: ** Bound Services **
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: NAME IP PORT
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: httpejbd 127.0.0.1 4204
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: multipoint 127.0.0.1 4212
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: admin 127.0.0.1 4200
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: ejbd 127.0.0.1 4201
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: multicast 239.255.2.3 6142
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: ejbds 127.0.0.1 4203
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager printRow
INFO: multipulse 239.255.2.3 6142
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager start
INFO: -------
Mar 07, 2015 10:32:17 PM org.apache.openejb.server.SimpleServiceManager start
INFO: Ready!
So, the server is ready on tleilax. Why can't the client connect properly?
thufir#doge:~$
thufir#doge:~$ java -classpath .:NetBeansProjects/HelloClient/dist/HelloClient.jar:apache-openejb-4.7.1/lib/openejb-client-4.7.1.jar:netbeans-8.0.2/enterprise/modules/ext/javaee-api-7.0.jar org.acme.HelloClient
Mar 07, 2015 10:49:46 PM org.apache.openejb.client.EventLogger log
INFO: RemoteInitialContextCreated{providerUri=ejbd://192.168.1.3:4201}
Mar 07, 2015 10:49:46 PM org.apache.openejb.client.EventLogger log
WARNING: ConnectionFailed{uri=ejbd://192.168.1.3:4201 cause=java.io.IOException: Cannot connect to server 'ejbd://192.168.1.3:4201'. Check that the server is started and that the specified serverURL is correct.}
Mar 07, 2015 10:49:46 PM org.apache.openejb.client.EventLogger log
WARNING: BootstrappingConnection{provider=ejbd://192.168.1.3:4201}
Mar 07, 2015 10:49:46 PM org.apache.openejb.client.EventLogger log
WARNING: ConnectionFailed{uri=ejbd://192.168.1.3:4201 cause=java.io.IOException: Cannot connect to server 'ejbd://192.168.1.3:4201'. Check that the server is started and that the specified serverURL is correct.}
Mar 07, 2015 10:49:46 PM org.apache.openejb.client.EventLogger log
SEVERE: ConnectionStrategyFailed{strategy=StickyConnectionStrategy, cluster=org.apache.openejb.client.ClusterMetaData#7d261f7f, server=ejbd://192.168.1.3:4201}
Exception in thread "main" javax.naming.NamingException: Cannot lookup '/HelloBeanRemote'. [Root exception is java.rmi.RemoteException: Unable to connect; nested exception is:
java.io.IOException: Cannot connect to server 'ejbd://192.168.1.3:4201'. Check that the server is started and that the specified serverURL is correct.]
at org.apache.openejb.client.JNDIContext.lookup(JNDIContext.java:405)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at org.acme.HelloClient.main(HelloClient.java:16)
Caused by: java.rmi.RemoteException: Unable to connect; nested exception is:
java.io.IOException: Cannot connect to server 'ejbd://192.168.1.3:4201'. Check that the server is started and that the specified serverURL is correct.
at org.apache.openejb.client.Client.processRequest(Client.java:167)
at org.apache.openejb.client.Client.request(Client.java:141)
at org.apache.openejb.client.JNDIContext.request(JNDIContext.java:211)
at org.apache.openejb.client.JNDIContext.lookup(JNDIContext.java:399)
... 2 more
Caused by: java.io.IOException: Cannot connect to server 'ejbd://192.168.1.3:4201'. Check that the server is started and that the specified serverURL is correct.
at org.apache.openejb.client.SocketConnectionFactory$SocketConnection.failure(SocketConnectionFactory.java:334)
at org.apache.openejb.client.SocketConnectionFactory$SocketConnection.open(SocketConnectionFactory.java:308)
at org.apache.openejb.client.SocketConnectionFactory.getConnection(SocketConnectionFactory.java:162)
at org.apache.openejb.client.ConnectionManager.getConnection(ConnectionManager.java:112)
at org.apache.openejb.client.AbstractConnectionStrategy.connect(AbstractConnectionStrategy.java:105)
at org.apache.openejb.client.AbstractConnectionStrategy.connect(AbstractConnectionStrategy.java:81)
at org.apache.openejb.client.ConnectionManager.getConnection(ConnectionManager.java:88)
at org.apache.openejb.client.Client.processRequest(Client.java:165)
... 5 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.openejb.client.SocketConnectionFactory$SocketConnection.open(SocketConnectionFactory.java:300)
... 11 more
thufir#doge:~$
thufir#doge:~$
For what it's worth, I checked that the HelloClient runs locally from tleilax fine:
thufir#tleilax:~$
thufir#tleilax:~$ java -classpath .:HelloClient.jar:apache-openejb-4.7.1/lib/openejb-client-4.7.1.jar:netbeans-8.0.2/enterprise/modules/ext/javaee-api-7.0.jar org.acme.HelloClient
Mar 07, 2015 10:56:46 PM org.apache.openejb.client.EventLogger log
INFO: RemoteInitialContextCreated{providerUri=ejbd://127.0.0.1:4201}
Mar 07, 2015 10:56:46 PM org.apache.openejb.client.EventLogger log
INFO: ConnectionOpened{uri=ejbd://127.0.0.1:4201}
Mar 07, 2015 10:56:46 PM org.apache.openejb.client.EventLogger log
INFO: ServerAdded{server=ejbds://127.0.0.1:4203} ClusterMetaDataUpdated{provider=ejbd://127.0.0.1:4201, version=1425796337772, uris=2}
Hello World!!!!
thufir#tleilax:~$
Of course, the client running locally on tleilax connects to 127.0.0.1, while the client on doge connects, or tries to connect, to the ip adress for tleilax: 192.168.1.3; the ip address is hardcoded in the sample.

MeteorJS Mongo connection timed out

I have a MeteorJS app that is not connecting to Mongo. It's been deployed using meteor deploy but for some reason it times out when trying to connect to the database :S I can still access the mongo database using meteor mongo and all of the data is present!
I am using Meteor 0.9.3.1 and here are my packages:
standard-app-packages
insecure
underscore
arunoda:npm#0.2.6
mrt:bootstrap-3
http
iron:router
mrt:q#0.9.7
mrt:moment#2.8.1
mrt:twilio-meteor
email
Here are the logs from meteor logs :
[Thu Oct 16 2014 17:39:25 GMT+0000 (UTC)] INFO Exception in setInterval callback: MongoError: connection to [production-db-a3.meteor.io:27017] timed out
at Object.Future.wait (/meteor/dev_bundles/0.3.55/lib/node_modules/fibers/future.js:326:15)
at _.extend._nextObject (packages/mongo/mongo_driver.js:858)
at _.extend.forEach (packages/mongo/mongo_driver.js:892)
at _.extend.map (packages/mongo/mongo_driver.js:902)
at _.extend.fetch (packages/mongo/mongo_driver.js:926)
at Cursor.(anonymous function) [as fetch] (packages/mongo/mongo_driver.js:741)
at Object.residentModel.getActiveResidents (app/shared/collections/residents.js:43:40)
at Object._.extend._run (app/server/services/deviceService/sensorChangeWatchdog.js:6:39)
at app/lib/watchdogFactory.js:7:12
at _.extend.withValue (packages/meteor/dynamics_nodejs.js:56)
- - - - -
at Object.toError (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/utils.js:110:11)
at commandHandler (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/cursor.js:679:63)
at null.<anonymous> (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/db.js:1843:9)
at g (events.js:180:16)
at emit (events.js:98:17)
at Base.__executeAllServerSpecificErrorCallbacks (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/connection/base.js:365:29)
at Server.<anonymous> (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/connection/repl_set/ha.js:445:10)
at Server.g (events.js:180:16)
at Server.emit (events.js:98:17)
at null.<anonymous> (/meteor/containers/450809a5-5a17-5e6e-8233-85e6ad895366/bundle/programs/server/npm/mongo/node_modules/mongodb/lib/mongodb/connection/server.js:501:88)
[Thu Oct 16 2014 17:41:05 GMT+0000 (UTC)] INFO STATUS running -> killed
[Thu Oct 16 2014 17:41:05 GMT+0000 (UTC)] NOTICE Starting application on port 19159
[Thu Oct 16 2014 17:41:05 GMT+0000 (UTC)] INFO STATUS killed -> starting
[Thu Oct 16 2014 17:41:06 GMT+0000 (UTC)] INFO STATUS starting -> running
[Thu Oct 16 2014 17:41:07 GMT+0000 (UTC)] INFO init

Apache tomcat server startup increase over 30000 ms in eclipse Kepler

when i run my JSP page on apache tomcat 7 in my eclipse, server output shows up in 30000ms or more some times it never starts up, at first output got in lessthan 700ms,
Any help or suggession on this problem...
sample output...
NFO: Starting Servlet Engine: Apache Tomcat/7.0.12
Mar 18, 2014 10:42:39 PM org.apache.catalina.util.SessionIdGenerator createSecureRandom
INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [156] milliseconds.
Mar 18, 2014 10:42:58 PM org.apache.coyote.AbstractProtocolHandler start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Mar 18, 2014 10:42:58 PM org.apache.coyote.AbstractProtocolHandler start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
Mar 18, 2014 10:42:58 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 30553 ms
Consider the following 2 suggestions:
1)As I commented above check if more then one application is running on the same server.This could be the primary reason.May be 3 or 4 applications might be running once you start your tomcat service.
2)May be some library that you are using for your application is creating some problem,for example the library might be initializing some file storage or may be something else.For this you will have to debug and check your stacktraces.