I have a free OpenShift account with the default 3 gears. On this I have installed the WildFly 8.1 image using the OpenShift web console. I set the minimal and maximal scaling to 3.
What happens now is that OpenShift will create 3 JBoss WildFly instances:
One on the entry node (which is also running HAProxy)
One on an auxiliary node
One on another auxiliary node
The weird thing is that the JBoss WildFly instance on the entry node is by default disabled in the load balancer config (haproxy.conf). BUT, OpenShift is still deploying the war archive to it whenever I commit in the associated git repo.
What's extra problematic here is that because of the incredibly low number of max user processes (250 via ulimit -u), this JBoss WildFly instance on the entry node cannot even startup. During startup JBoss WildFly will throw random 'java.lang.OutOfMemoryError: unable to create new native thread' (and no, memory is fine, it's the OS process limit).
As a result, the deployment process will hang.
So to summarize:
A JBoss WildFly instance is created on the entry node, but disabled in the load balancer
JBoss WildFly in its default configuration cannot startup on the entry node, not even with a trivial war.
The deployer process attempts to deploy to JBoss WildFly on the entry node, despite it being disabled in the load balancer
Now my question:
How can I modify the deployer process (including the gear start command) to not attempt to deploy to the JBoss WildFly instance on the entry node?
When an app scales from 2 gears to 3, HAproxy stops routing traffic to your application on the headgear and routes it to the two other gears. This assures that HAproxy is getting the most CPU as possible as the application on your headgear (where HAproxy is running) is no longer serving requests.
The out of memory message you're seeing might not be an actual out of memory issue but a bug relating to ulimit https://bugzilla.redhat.com/show_bug.cgi?id=1090092.
Related
in my Jboss web console, the topology view in Domain tab is empty, I don`t know why. Everything is up and running, I just can't see the domain topology in Jboss console. "Unable to load topology"
I just got the "Unable to load topology" error today.
I have separate multinode jboss domain configurations in my env.
one is 6 nodes and one is 3 nodes. all running 6.4.22.GA
The error came up for me when we were switching ldap user authentication hosts and attempting to do that while leaving the servers up/running as much as possible.
When the domain node was changed to the new ldap server and brought back up we got the topology error.
fix was to bounce jbossas-domain on the other nodes and point to the new ldap server. After we did that the jboss console was able to display the topology again.
In short my solution was to make sure all the nodes in the jboss domain had the same configuration and then bounce them.
I want to start two or more jboss eap 6.4 in the domain, but when I started the second domain I got this warning:
[Server:server-one] 15:34:35,606 WARN [org.hornetq.core.client]
(hornetq-discovery-group-thread-dg-group1) HQ212034: There are more
than one servers on the network broadcasting the same node id. You
will see this message exactly once (per node) if a node is restarted,
in which case it can be safely ignored. But if it is logged
continuously it means you really do have more than one node on the
same network active concurrently with the same node id. This could
occur if you have a backup node active at the same time as its live
node. nodeID=14bdbf74-f56c-11e4-a65f-738aa3641190
I cannot get this to work.
You must have copied the node from other node. Delete messagingjournal from data directory from all the nodes and restart all the nodes again.
I have configured a Soa Cluster with one admin node and two managed nodes and all server nodes configured in three different machines. once I deploy a Bpell to one managed node it automatically deploys in the other managed nodes as well(default behavior). once you go to soa enterprise manager those deployed Bpels can be viewed under [Soa -> managed node -> Defult ->..]. It is the same place where we deploy new Bpels. I accidentally undeploy all bpels (you can do it by right clicking a managed node and choosing un-deploy option).
Now I'm having a hard time to get back to previous state, how to deploy all those projects again to a specific managed node. I tried to restart the node hoping it would sync again, yet the managed server went to "admin" state (not the ok state).
is there anything needs to be done !!
Thanks, Hemal
You'll need to start the server from command line, it will work.
For managing 'managed servers' from EM or WLS console, there's one additional step that's needed during instalation process.
Please modify the nodemanager.properties of WLS and set the property startscriptenabled=true.
http://download.oracle.com/docs/cd/E12839_01/core.1111/e10105/start.htm#CIHBACFI
Got a general question about load balancing setup in JBoss (7.1.1.Final). I'm trying to setup a clustered JBoss instance with a master and slave node and I'm using the demo app here (https://docs.jboss.org/author/display/AS72/AS7+Cluster+Howto) to prove the load balancing/session replication. I've basically followed through to just before the 'cluster configuration' section.
I've got the app deployed to the master and slave nodes and if I hit their individual IPs directly I can access the application fine. According to the JBoss logs and admin console the slave has successfully connected to the master. However, if I put something in the session on the slave, take the slave offline, the master cannot read the item that the slave put in the session.
This is where I need some help with the general setup. Do I have to have a separate apache httpd instance sat in front of JBoss to do the load balancing? I thought there was a load balancing capability built into JBoss that wouldn't need the separate server, or am I just completely wrong? If I don't need apache, please could you point me in the direction of instructions to setup the JBoss load balancing?
Thanks.
Yes, you need a Apache or any other software or hardware that allows you to perform load balancing of the HTTP request JBoss Application Server does not provide this functionality.
For proper operation of the session replication you should check that the server configuration and the application configuration is well defined.
On the server must have the cache enabled for session replication (you can use standalone-ha.xml or standalone-full-ha.xml file for initial config).
To configuring the application to replicate the HTTP session is done by adding the <distributable/> element to the web.xml.
You can see a full example in http://blog.akquinet.de/2012/06/21/clustering-in-jboss-as7eap-6/
I have a Tapestry application (WAR, no EJB) that ...
... I want to deploy on 2 EC2 small instances (for failover).
... uses Spring Security
... is stateful (very small session state)
... should be deployed on Glassfish 3.1 (seems to have best cluster support?)
... and has an elastic load balancer with sticky session in front of it
How can I configure a cluster to achieve minimal ('no') interruptions for the user experience in case A) a node fails and B) I deploy a new version?
Everything is explained here: http://download.oracle.com/docs/cd/E18930_01/html/821-2426/docinfo.html#scrolltoc
Basically, you setup a DAS (=master), which control nodes with instances on it. You could do all of this on the same machine (1 DAS, 1 node with multiple instances), although it would be a good idea to have at least 2.
You should then have at least one load balancer (apache, physical load balancer, whatever).
A) if a node fails, the load balancer can redirect all traffic to the other node
B)
deploy application, disabled, with new version (see "application versioning")
mark server A as unavailable
enable the new version on server A
mark server A as available and server B as unavailable
enable the new version on server B
mark server B as avalilable