We are attempting to deploy new PHP code via Capistrano while running Opcache.
Capistrano creates a new deploy directory each time you deploy, then adjusts a symlink so that the webserver points to the new directory. Because Opcache caches by the real path of the file, that means that the newly deployed version of a site is cached completely separately from the old.
The problem we are running into is that Opcache runs out memory because each new deploy causes the full code base to be cached, and old code is never evicted. We could call opcache_reset(), but when the cache is reset, we briefly get 500 errors when the caches stampede. (We would also have the same errors if we tried to launch a new deploy without warming up the cache.)
Is there a better way to handle this? Some way to launch the new code while not filling up opcache until it runs out of memory (or empties itself because it has too many files) that allows us to avoid calling opcache_reset() on the live site? We are using (or trying to transition to, anyway) Nginx as our web server with PHP-FPM handling the PHP requests.
An option would be to call opcache_invalidate for each of the files in the old version of the site at the end of the deployment. You could prevent cache stampede by including the file following the invalidation.
A second option would be to setup fpm to have multiple pools, and to restart them one by one (they'll start with a clean opcache). You'll somewhat prevent the cache stampede only one server will have a clean cache at any given time, and the application will stay up because nginx will be able to balance the load on the various pool.
Another option is to delete the old versions of the script, so that opcache clears them from the cache once the revalidate_freq has passed, forcing it to load the new files from the filesystem.
Related
I am using following command to deploy code to my AEM instance "mvn clean install -Daem.host=localhost -Daem.port=1202 -Dmaven.test.skip=true
"
After deployment pages are taking too long to load at least 7 mins.
I found No errors/Exceptions in error log.
There could be couple of factors causing this slowness -
Amount of memory allocated to AEM instance, default setting is - CQ_JVM_OPTS='-server -Xmx1024m -XX:MaxPermSize=256M -Djava.awt.headless=true' which is actually not sufficient for optimal performance. I have been using double of this configurations and sometimes even more.
When you deploy your package with code, the bundles are processed and services are registered. Depending on number of services/components being registered the time can go up. Sometimes there are hooks within code that cause few system level bundles to cycle as well, if that happens it would actually cause all the other bundles dependent on system bundle to cycle and registering the services again.
your code deployment could be triggering some workflow that either consumes lot of resources or is causing delayed activation on your bundle. The first scenario could happen if your deployment has something like images which when deployed causes OOTB image workflow to trigger (there could be other based on your code). Second scenario could be that you have bundle activator either waiting for another bundle which gets deployed later (and/or stays installed and not active) or you are building some sort of caching that waits for pages to be deployed and processed. There are countless such scenarios that can cause this issue.
What you could do is check the status of the bundles in /system/console/bundles pre and post deployment you can identify bundle related issues there. Another thing you could try is to do selective deployment of the code to figure out what module is causing issue that then dive deeper in to that module.
Also look at recent request logs to identify the flow of page load to see if there are services, filters etc in picture that are causing delays.
Let me know if any of this approach helps you identify the root cause and in case you need further help, will be here to assist.
I am trying to use eclipse kepler for Java EE 7.I already installed JBoss Tools and added JBoss Wildfly successfully as a server. However my changes are not automatically deployed. Is there anyway the app can be deployed automatically just as when using glassfish?
Using Eclipse, click twice on your WildFly Server to edit the following properties:
Publishing: choose "Automatically publish after a build event". I like to change the publishing interval to 1 second too.
Application Reload Behavior: check the "Customize application reload ..." checkbox and edit the regex pattern to \.jar$|\.class$
That's it. Good luck!
Both #varantes and #Sean are essentially correct, but these answers are not full.
Unfortunately the only way in a Java server environment to have full, zero-downtime hot deployment is to use paid JRebel or free spring-loaded tool.
But for small project there are some ways to speed up work by partial hot-deployment. Essentially:
When enabled option Automatically publish when resource change
then changes inside *.html, *.xhtml files are immediately
reflected as soon as you refresh the browser.
To make hot deployment work for *.jsp files too, then you should
inside ${wildfly-home}/standalone/configuration/standalone.xml
make following change:
<jsp-config/>
replace with:
<jsp-config development="true"/>
restart the server and enjoy hot deployment of web files.
But when modifying *.java source files, then only partial hot deployment is possible. As #varantes stated in his answer, enabling Application Reload Behavior with regex pattern set to \.jar$|\.class$ is an option, but has serious downside: whole module is restarted, thus:
It takes some time (depending on how big is a module).
Whole application state is lost.
So personally, I discourage this solution. JVM supports (in debug mode) code-swapping for methods' bodies. So as long as you are modifying only bodies of existing methods, you are at home (zero downtime, changes are reflected immediately). But you have to disable automatic publishing inside server settings otherwise the application's state will still be destroyed by that republish.
But if you are heavily crafting Java code (adding classes, annotations, constructors) then unfortunately I can only recommend set publishing into Never publish automatically (or shutdown server) and when you finish your work in Java files, then restart by hand your module (or turn-on server). Up to you.
It works for small Java projects, but for bigger ones, JRebel is invaluable (or just spring-loaded), because all approaches described above are not sufficient. Also because of such problems, solutions like Rails/ Django /Play! Framework gained so huge popularity.
I am assuming you are using the latest version of Wildfly (8.0 Beta 1 as of writing).
In the standalone.xml config file, look for <jsp-config/>. Add the attribute development="true" and it should hot-deploy. The resulting config will look like this:
<jsp-config development="true"/>
Add attributes (development, check-interval, modification-test-interval, recompile-on-fail) in configuration file in xPath = //servlet-container/jsp-config/
<servlet-container name="default" default-buffer-cache="default" stack-trace-on-error="local-only">
<jsp-config development="true" check-interval="1" modification-test-interval="1" recompile-on-fail="true"/>
</servlet-container>
(It works in WildFly-8.0.0.Final)
Start server in debug mode and It will track chances inside methods. Other changes It will ask to restart the server.
Documentation says if you have a context file here:
$CATALINA_HOME/conf/Catalina/localhost/myapp.xml
it will NOT be replaced by a context file here:
mywebapp.war/META-INF/context.xml
It is written here: http://tomcat.apache.org/tomcat-6.0-doc/config/context.html
Only if a context file does not exist for the application in the $CATALINA_BASE/conf/[enginename]/[hostname]/, in an individual file at /META-INF/context.xml inside the application files.
But everytime I re-deploy the war it replaces this myapp.xml with the /META-INF/context.xml!
Why does it do it and how can I avoid it?
Thanx
Undeploy part of redeploy deletes app and the associated context.xml.
If you use maven tomcat plugin you can avoid deleting context.xml if you deploy your app with command like this:
mvn tomcat:deploy-only -Dmaven.tomcat.update=true
More info here: https://tomcat.apache.org/maven-plugin-2.0-beta-1/tomcat7-maven-plugin/deploy-only-mojo.html
You can use deploy-only with parameter mode to deploy the context.xml too.
The short answer:
Just make the TOMCATHOME/conf/Catalina/localhost dir read-only, and keep reading for more details:
For quick deployment mode (Eclipse dynamic web project, direct Tomcat
connection, etc.) on a local/non-shared Tomcat server you can just define your JDBC datasource (or any
other 'web resource') using the META-INF/context.xml file inside the
WAR file. Easy and fast in your local environment, but not suitable for staging, QA, or
production.
For build deployment mode (usually for staging, QA, or prod), JDBC
datasources and other 'web resources' details are defined by the
QA/production team, not the development team anymore. Therefore, they
must be specified in the Tomcat server, not inside the WAR file
anymore. In this case, specify them in the file
TOMCATHOME/conf/Catalina/localhost/CONTEXT.xml (change Catalina
by the engine, and localhost by the host, and CONTEXT by your context accordingly). However,
Tomcat will delete this file on each deployment. To prevent this
deletion, just make this dir read-only; in Linux you can type:
chmod a-w TOMCATHOME/conf/Catalina/localhost
Voila! Your welcome.
The long answer
For historical reasons Tomcat allows you to define web resources (JDBC datasources, and others) in four
different places (read four different files) in a very specific order of precedence, if you happen to define the same resource multiple times. The ones named in the
short answer above are the more suitable nowadays for each purpose, though you could still
use the others (nah... you probably don't want to). I'm not going to
discuss the other ones here unless someone asks for it.
On tomcat7, also woth autoDeploy=false the file will be deleted on undeploy. This is documented and not a bug (althought it avoids good automated deployments with server-side fixed configuration).
I found a workaround which solved the problem for me:
create a META-INF/context.xml file in your webapp that contains
on the Server create a second context "/config-context" in server.xml and put all your server-side configuration parameters there
on the application use context.getContext("/config-context").getInitParameter(...) to access the configuration there.
This allows a per-host configuration that is independent of the deployed war.
It should also be possible to add per-context configurations by adding contexts like "/config-context-MYPATH". In your app you can use the context path oth the app to calculate the context path of the config app.
According to the documentation (http://tomcat.apache.org/tomcat-8.0-doc/config/automatic-deployment.html#Deleted_files) upon redeploy tomcat detects the deletion (undeploy) of your application. So it will start a cleanup process deleting the directory and xml also. This is independent of auto deployment - so it will happen upon redeployment through manager and modification of war also. There are 3 exceptions:
global resources are never deleted
external resources are never deleted
if the WAR or DIR has been modified then the XML file is only deleted
if copyXML is true and deployXML is true
I don't know why, but copyXML="false" deployXML="false" won't help.
Secondly: Making the directory read only just makes tomcat throwing an exception and won't start.
You can try merging your $CATALINA_BASE/conf/Catalina/localhost/myapp-1.xml, $CATALINA_BASE/conf/Catalina/localhost/myapp-2.xml, etc files into $CATALINA_BASE/conf/context.xml (that works only if you make sure your application won't deploy its own context configuration, like myapp-1.xml)
If someone could tell what is that "external resources" that would generally solve the problem.
The general issue as described by the title is covered by Re-deploy from war without deleting context which is still an open issue at this time.
There is an acknowledged distinction between re-deploy which does not delete the context, and deploy after un-deploy where the un-deploy deletes the context. The documentation was out of date, and the manager GUI still does not support re-deploy.
Redeployment means two parts: undeployment and deployment.
Undeployment removes the conf/Catalina/yourhost/yourapp.xml because the
<Host name="localhost" appBase="webapps" unpackWARs="true"
autoDeploy="true"> <!-- means autoUndeploy too!!! -->
</Host>
Change the autoDeploy="false" and Tomcat has no order anymore to remove the conf/Catalina/yourhost/yourapp.xml.
There is an feature that allowes us to make those steps (undeploy/deploy) as one single step (redeploy) that do not remove the context.xml. This feature is available via the manager-text-interface, but the option is not available using the manager-html-interface. You might have to wait until the bug in tomcat is fixed. You can use the method described in this answer as an workaround.
I am using eclipse 3.5 with google app engine + spring framework to develop application. My problem is when I change the code and build the project, the new code doesn't come in to effect. I even deleted the old file but at runtime, the old version gets display in the browser. Why?
Your description is not explicit, so following are my assumptions:
You are changing jsp/js or view related files
You have Google app engine plugin for eclipse to deploy the code
Following might be one of the reasons:
Your view files are cached in the browser, so try deleting the browser cache
Google app engine might have cached your files, so try deleting the temp folder
Eclipse IDE wouldn't have deployed your changed code, so check the timestamp, if it still shows the old timestamp, then find out how to configure eclipse to detect your changes.
I found 3 workarounds for similar issue (changes to a servlet (.java file) were not taking effect).
Before editing the .java file, stop the application. It seems that if you edit it while running, the issue occurs, even if you edit it, stop then re-start.
Delete items in Temp folder e.g, (C:\Users\username\AppData\Local\Temp), then stop and re-start the app.
I think the culprit folder in the Temp folder is this:Jetty_127_0_0_1_8888_war__.g0qk00
Right click in in the console area and select 'Remove All Terminated'. Sometimes that reveals there are other instances running, which need to be stopped by clicking the terminate button.
I got the similar issue and the problem was due to not stopping the running server.
What i was doing was running the server, editing the java file, saving and again running the server. This created two instances of the server running and when checking into the browsers the old code was executed from the first instances.
So, the solution is terminate and relaunch the server, and your new changes will be in effect.
When deploying my project to SpringSource dm Server, every once in a while a JAR fails to deploy with the following message:
/mnt/myproject/springsource/work/com.springsource.server.deployer/packed/my.project.0.1.10.M.jar' cannot be unpacked.
java.util.zip.ZipException: error in opening zip file
There are 5 .war files in the project. If one of them fails, it's always the same one (which is also the last one to be copied into the pickup directory). However, usually all 5 will deploy without issues. It is the exact same set of files in all instances, taken from a maven repository, just deployed to new server instances.
The file that fails can be opened just fine by 7-Zip. If I stop Spring, clear the pickup directory, start Spring and copy the .war files to pickup again, it will usually work.
The usual deployment process is:
Start Spring
Wait until it reports Open for business with profile 'web'
Copy all 5 projects with a 2 second delay between each copy (scripted).
Similar issues java-util-zip-zipexception-error-in-opening-zip-file and jboss5-cannot-deploy-due-to-java-util-zip-zipexception-error-in-opening-zip-fil do not seem to apply.
You don't say which version of dm Server you are running, so I would recommend upgrading to 2.0.x to pick up fixes if you haven't already. You may also like to upgrade to Eclipse Virgo which is the continuation of the dm Server project.
My guess is that the heuristic in dm Server for determining when a file copy into pickup has terminated is playing up, possibly due to a slow or irratic copy operation. Is there anything unusual about your disk, such as encryption or remote mount, which may interfere with the copy operation?
One way to rule out the heuristic would be to place the files in the pickup directory when dm Server is not running and then start dm Server when the copy operation has definitely completed. If the problem reproduces, then there may be a problem in the JRE you are using.