I've got a bit of a problem with deployments on my project and after hours of searching the web I can't find an answer to this.
Situation:
I am working on a Web application that lives of uploads and other files that get generated during use.
To keep things simple I store these into: .../mywebapp/web/some subfolders/*
So far, so good.
My Problem:
Every time I redeploy my project on the actual server (after updating classes/jsp's)
Glassfish deletes the entire content of .../mywebapp/ during redeployment.
My Procedure so far:
Export the latest version of my webapp as .war.
Add the changed files into the .war file on the server (rename to .zip, then back to .war)
Redeploy the .war on my server using the admin console (locahost:4848)
My question is
This current procedure is very prone to dataloss (I could lose the files!)
Is there a straight forward way where I can upload changes to my server without the risk of losing all the files that have been added during runtime?
I see two choices:
move the data 'out of harm's way' (find some place for it that isn't
in the deployment directory; like a database)
Switch to directory deployment instead of archive deployment.
The better of these two choices is the first one... It is more portable than the other; every server out there supports deploying archives. A lot of servers support directory based deployment... but they all do it a bit differently... so a directory structure that deploys on A may not deploy on B.
I had this same issue, solved using XCOPY and Event Scheduler.
Effectively, you are continuously sync two folders
Run a scheduled task for the following batch file every X minutes
sync.bat:
xcopy "domain1\applications\%YOUR_APP_NAME%l\path\to\folder" "D:\folder\to\sync" /D /I /Y
xcopy "D:\folder\to\sync" "domain1\applications\%YOUR_APP_NAME%l\path\to\folder" /D /I /Y
Switches:
/D - Only copy newer files if the destination file exists
/I - If the destination does not exist, and you are copying more than one file, this switch assumes that the destination is a folder.
/Y - Overwrite without prompting
Related
I am trying to copy files via a .bat file on a scheduled basis. I know how to handle all of it except 1 piece - the issue is the source folder name is different each time. the first time it's C:\folder\new. Second time it will be C:\folder\new-0001, then C:\folder\new-0002. How do I account for the source folder changing?
This is Cerbot SSL renewal as standalone. Tomcat on Windows.
I plan to stop the webservice, renew the cert (which creates the new folder of new files), copy the files to Tomcat, then start the webservice.
I am running automated tests of our application on different versions of an OS build (Windows 7, Windows 10, etc...). My testing suite requires that I copy files to the Slave computers when there are changes in the tests (external to the build application). The test files are not in the Jenkins work space as they do not change frequently and therefore do not need to be copied to the Slave with each execution.
I am looking to be able to update the files on the Slaves, but not under the work space directory, so the Copy-To-Slave plugin will not work from my understanding.
I am looking to have batch files, testing resource files, DB generation scripts and others copied to the Slave computer by a Jenkins job. This job may monitor GIT, but not everything being copied is from GIT.
In essence, execute the following but to the Slave computer
xcopy C:\Testing*.* C:\Resources\Testing /s/v/e
The reason for this is our testing scripts look for certain files to execute (DB scripts for building the database for the current platform/DB Engine) and as these do not change too frequently, we only need to copy the files when they are changed, and leave the files in place for subsequent test runs. There is a large amount of files and GBs of data that does not need to be copied with each test run. There are also multiple executions of the application with the same testing files where the application has different configurations, but should produce the same results, so the test files do not need to be copied with each of these executions.
I found a configuration on the Copy-To-Slave plugin to add additional directories as destinations, that are relative to the file system root directory (C:\ in my case) which will solve my problem.
I'm trying to think of a good solution for automating the deployment of my .NET website to the live server via FTP.
The problem with using a simple FTP deployment tool is that FTPing the files takes some time. If I FTP directly into the website application's folder, the website has to be taken down whilst I wait for the files to all be transferred. What I do instead is manually FTP to a seperate folder, then once the transfer is completed, manually copy and paste the files into the real website folder.
To automate this process I am faced with a number of challenges:
I don't want to FTP all the files - I only want to FTP those files that have been modified since the last deployment. So I need a program that can manage this.
The files should be FTPed to a seperate directory, then copy+pasted into the correct destination onces complete.
Correct security permissions need to be retained on the directories. If a directory is copied over, I need to be sure that the permissions will be retained (this could probably be solved by rerunning a script that applies the correct permissions).
So basically I think that the tool that I'm looking for would do a FTP sync via a temporary directory.
Are there any tools that can manage these requirements in a reliable way?
I would prefer to use rsync for this purpose. But seems you are using windows OS here, some more effort is needed, cygwin stuff or something alike.
Sometimes we have huge amount of JAR files in jboss/server/web/tmp/vfs-nested.tmp directory.
For example today this directory contained over 350k jar files.
But on other hosts there are only 2 jar files in this directory.
What can be the root cause of this problem?
We use JBoss 5.1
UPDATE:
I found following information in release notes for JBoss 5.1.0.GA:
JBoss VFS provides a set of different
switches to control it's internal
behavior. JBoss AS sets
jboss.vfs.forceCopy=true by default.
To see all the provided VFS flags
check out the code of the
VFSUtils.java class.
So I do not understand what should I set?
Should I set -Djboss.vfs.forceNoCopy=true or -Djboss.vfs.forceCopy=false?
Or should I set both of them?
UPDATE 1:
I have read entire thread http://community.jboss.org/thread/2148?start=0&tstart=0
and now I am not shure that I should change either jboss.vfs.forceCopy or jboss.vfs.forceNoCopy.
According to this thread I will have OutOfMemory error instead of huge amount of files in tmp dir.
From here: http://sourceforge.net/project/shownotes.php?release_id=575410
"Excessive nestedjarNNN.tmp files in the tmp directory. The VFS unwraps nested jars by extracting the nested jar into a tmp file in the java tmp directory. This can result in a large number of files that fill up the tmp directory. You can disable this behavior by setting -Djboss.vfs.forceNoCopy=true on command line used to start jboss. This will be enabled by default in a future release, JBAS-4389."
jskaggz has a good answer. In addition, I have this in the beginning of my run.bat file:
rmdir /s /q c:\apps\jboss-5.1.0.ga\server\default\tmp
rmdir /s /q c:\apps\jboss-5.1.0.ga\server\default\work
rmdir /s /q c:\apps\jboss-5.1.0.ga\server\default\log
mkdir c:\apps\jboss-5.1.0.ga\server\default\tmp
mkdir c:\apps\jboss-5.1.0.ga\server\default\work
mkdir c:\apps\jboss-5.1.0.ga\server\default\log
echo --- Cleared temp folders ---
I've had problems with old copies of classes hanging around, so this seems to help.
We have solved this problem by exploded deployment ( works for war and ear) as described in jboss documentation http://docs.jboss.org/jbossas/docs/Administration_And_Configuration_Guide/5/html/ch03s01.html
That's way vfs is not used.
I had the same issue described above in production and resolved it with the following solution.
Added java options
-Djboss.vfs.cache=org.jboss.virtual.plugins.cache.IterableTimedVFSCache
-Djboss.vfs.cache.TimedPolicyCaching.lifetime=1440
My setup also defines additional deployment directories so I needed to add these additional directories to vfs.xml file located in $JBOSS_SERVER_HOME/conf/bootstrap/ in order to see the benefit.
The lifetime setting I think is in minutes so I set it to a day as I have a scheduled restart of the server overnight.
Prior to finding this solution I had also tried using -Djboss.vfs.forceNoCopy=true and -Djboss.vfs.forceCopy=false
This appeared to work but I noticed the application ran a lot slower - presumably because these settings turn vfs caching off.
My Jboss version is jboss-5.1.0.GA
and my application runs in a cluster on production.
Found a lot others having the same problem running in cluster (or farm) environments.
https://issues.jboss.org/browse/JBAS-7126 desribes to solve the problem having a farm directory as deployment directory.
I had the same problem using a 2nd deploy directory.
The jar files out of my applications coming from this 2nd deploy directory got copied until the disk was full.
Tried adding the 2nd deploy directory the same way as at https://issues.jboss.org/browse/JBAS-7126 described for the farm directory.
It works well!
We were facing the same issue and were able to circumvent the issue by using a farm directory as deployment directory.
After putting that process in place we were facing one more issue due to the nature of our DEV environment ( We have clustered environment and we have many developers deploying on the shared DEV environment ) of not getting a consistent results while we were deploying the EARs and WARs that way .We circumvented the issue by making sure that the EARs and JARs that are being deployed are TOUCHED (http://en.wikipedia.org/wiki/Touch_(Unix) ) on the servers to make sure that inconsistencies are avoided .
I've been desperately looking for the answer to this and I feel I'm missing something obvious.
I need to copy a folder full of data files into the TARGETDIR of my deployment project at compile time. I can see how I would add individual files (ie. right click in File System and go to Add->File) but I have a folder full of data files which constantly get added to. I'd prefer not to have to add the new files each time I compile.
I have tried using a PreBuildEvent to copy the files:
copy $(ProjectDir)..\Data*.* $(TargetDir)Data\
which fails with error code 1 when I build. I can't help but feel I'm missing the point here though. Any suggestions?
Thanks in advance.
Graeme
Went to this route.
Created a new project (deleted the default source file Class1)
Added the files/folders necessary to the project.
Added the project as project output in the installer, choosing the option content files.
This removes the complexity of having to zip/unzip the files as suggested earlier.
Try
xcopy $(ProjectDir)..\Data\*.* $(TargetDir)Data /e /c /i [/f] [/r] /y
/e to ensure tree structure fulfilment (use /s if you want to bypass empty folders)
/c to continue on error (let the build process finish)
/i necessary to create the destination folder if none exists
/y assume "yes" for overwrite in case of previously existing files
[optionnal]
/f if you wanna see the whole pathes resulting from the copy
/r if you want to overwrite even previously copied read-only files
The method is simpler on the project than on files, yes. Beside, on files, it copies only the modified/missing files on each build but forces you to maintain the project on each data pack modification. Depends on the whole data size and the variability of your data pack.
Also beware the remaining files if you remove some from your data pack and rebuild without emptying your target folder.
Good luck.
I solved the problem by a workaround:
Add a build action of packaging entire directory (could be filtered) to a ZIP file.
Add a reference to an empty ZIP file to deployment project.
Add a custom action to deployment project to extract the ZIP to destination folder.
It's simple and stable.
Your error is probably because your path has spaces in it and you don't have the paths in quotes.
ex copy "$(ProjectDir)..\Data*.*" "$(TargetDir)Data\"
I need to do a similar thing. Thinking a custom action...
I found a different workaround for this. I added a web project to my solution that points at the data directory I want included in the deployment project. The web project automatically picks up any new files in the data directory and you can refer to the project content in the deployment project.