ANT - speed improvements to existing script - deployment

I've built a set of generic deployment scripts which work great for the majority of our stuff. We've just however introduced our largest project to the setup and we're now finding times are far too varied and long for our liking.
The project size as it stands is 33,226files at a size of 400Mb plus. Times are currently taking between 13mins & 55mins (the last deployments time) depending on certain decisions made by ANT ( more below ).
In terms of the steps we currently do the following on x2 servers:-
1) ANT exports the project from SVN to both servers (made up of 3 parts).
2) It begins to shutdown the Web Services on Server #1.
This was the workaround we put in to stop Windows (2003) file locking failing the deployment.
3) ANT runs a "move" task on the current version (all parts) into temporary folders & moves the exported new version into its place.
4) Customised deployment code is run - one part being to move permanant features from the temporary folder into the new (i.e system files / Web Server Admin tools).
5) Delete the temporary folder.
6) Bring the Web Services back online
... rinse and repeat for the 2nd server steps 2 -> 6.
7) Save the ANT logs.
The main issue I'm having is that the ANT move task seems to make one of two decisions. It either:
a) Very simply swaps the versions over and moves on - taking a minute or two to handling it or
b) Goes through some kind of integrity check that it moves every file and folder from one place to the other. This floods the logs and takes a fair length of time to complete. Hence the 40+ minutes extra added on.
I can't find anything online that explains what causes ANT/OS to make that decision. Option A would be the ideal full-time situation.
I've tried copy, delete separately. I've tried the sync task. All seem to have this slow performance.
So really I'm asking what others with more experienced than me do with deployments of this scale. Do you have any hints / tips on how I could improve / speed this process up? Any ideas what the move is doing and if there is maybe a better way of doing all this?
Thanks a lot,
James

Thanks for the input all.
Just to add an answer to this one I've made the following changes which seem to have knocked a few minutes of it.
The first one was I've changed how the swapping happens off the back of the comment I mentioned before. It seems that ANT will try and do the following :-
"If the target directory does not already exist, Ant will do a rename of the directory. But if the target directory exists, it instead does a copy into the directory and delete from the source directory instead."
I think what's happened is ANT is trying to put the new version in before the old version has been completely removed. So instead of trying to rename the old I've now moved it into a temporary folder and deleted this at the end of the build. That seems to have stabled things on that front.
A few other things I've added to make ANT a bit smart :-)
1) I've set it up so ANT will not deploy any part of the build that is the same as what currently exists. So if part 1 is selected and that's on the Test environment already then it's removed from the build and SVN exports.
2) With the service shutdown / startup I've got ANT reading the response that comes back. If a service tells it that it's already started when calls, as sometimes happens if a service relies on others Windows automatically boots them up, then I've told ANT to hang around and move onto the next one.
Little steps like that seem to have improved things by a fair bit. I'd like to still try and get more out of them but these certaintly have given them a big step.
Thanks again,
James

even we faced this problem of auto deployment script taking longer time in our organization. So initially we had our script running in sequential like cleaning, stopping tomcats, updating, starting tomcats and making sure that all webapps are properly deployed.
So we have done following things like:
1. parallely cleaning and stopping all tomcats
2. do svn swith
3. parallely start all tomcats
4. make sure all webapps are deployed properly using jmx
here is the piece of code :
<target name="all_clean_parallel">
<parallel>
<antcall target="x1_clean"/>
<antcall target="x2_clean"/>
<antcall target="x3_clean"/>
</parallel>
</target>
<target name="all_start_parallel">
<parallel>
<antcall target="x1_start"/>
<antcall target="x2_start"/>
<antcall target="x3_start"/>
</parallel>
</target>
And the piece of code to check whether webapp is deployed propely or not with jmx help :
<macrodef name="mStatus">
<attribute name="aModule" />
<attribute name="aHost" default="localhost"/>
<attribute name="aPort" default="9012"/>
<attribute name="aMaxWait" default="240"/>
<attribute name="aTomcat" default=""/>
<attribute name="aState" default="1"/>
<sequential>
<waitfor maxwait="#{aMaxWait}" maxwaitunit="second" timeoutproperty="#{aHost}.#{aTomcat}.#{aModule}.#{aPort}.server.timeout" >
<and>
<jmx:equals
host="#{aHost}"
port="#{aPort}"
ref="#{aHost}.#{aTomcat}.#{aModule}.#{aPort}"
name="Catalina:j2eeType=WebModule,name=//localhost/#{aModule},J2EEApplication=none,J2EEServer=none"
attribute="state"
value="#{aState}"
/>
</and>
</waitfor>
<if>
<equals arg1="${#{aHost}.#{aTomcat}.#{aModule}.#{aPort}.server.timeout}" arg2="true" />
<then>
<var name="failBuild" value="true"/>
<echo message="*************************Host.Tomcat.Module = #{aHost}.#{aTomcat}.#{aModule} is not deployed into the tomcat" />
</then>
<else>
<echo message="#{aHost}.#{aModule} is deployed into the tomcat" />
</else>
</if>
</sequential>

Related

Continue the Wix setup after having a service that could not start

We have a setup in which we have a service that we try to install and run.
For some reason, the service cannot start(due to a port already in use). This isn't critical for us and should not stop the setup.
The service is declared like this:
<DirectoryRef Id="BIN">
<Component Id="MyService" Guid="*" SharedDllRefCount="yes">
<File Id="MyService.exe" Name="MyService.exe" KeyPath="yes" Vital="no" Compressed="default" DiskId="1" Source="$(var.DirDotfuscated)\MyService.exe" />
<ServiceControl Id="Install" Name="MyService" Start="install" Stop="install" />
<ServiceControl Id="Uninstall" Name="MyService" Stop="uninstall" Remove="uninstall" />
<ServiceInstall Id="NewServiceInstall2" Name="MyService" DisplayName="My Service" Type="ownProcess" Interactive="no" Start="auto" ErrorControl="normal" Description="My service" Vital="no" />
</Component>
<Component Id="Xms_HostService_Files" Guid="*" SharedDllRefCount="yes">
<File Id="MyService.exe.config" Name="MyService.exe.config" Vital="no" Compressed="default" DiskId="1" Source="$(var.DirDotfuscated)\MyService.exe.config" />
<File Id="MyServiceCommon.dll" Name="MyServiceCommon.dll" Vital="no" Compressed="default" DiskId="1" Source="$(var.DirDotfuscated)\MyServiceCommon.dll" />
<File KeyPath="yes" Id="MyServiceCore.dll" Name="MyServiceCore.dll" Vital="no" Compressed="default" DiskId="1" Source="$(var.DirDotfuscated)\MyServiceCore.dll" />
</Component>
</DirectoryRef>
When we execute the setup, we get this error:
And then, we only have the option to Retry(which will also fail) or cancel(that stops the setup).
We tried so many things(only put serviceInstall, not serviceControl, ...) but at some point we always have an error.
How should we manage this?
Attempted Answer (without ability to test):
What happens if you set the ServiceControl element's Wait attribute to "no"? I don't have a service exe to test with at the moment, but I believe that could work as you intend it.
Custom actions should generally be avoided for reliability reasons, but on the other hand - if you do need something special - that's what they are there for. Be prepared for most deployment problems to originate from your custom actions though: Why is it a good idea to limit the use of custom actions in my WiX / MSI setups?
Some further advice (which was not asked for :-) ):
You should not install multiple binaries with one component. You should use one file per component for many reasons. Windows Installer best practice specifically dictates to have only one binary per component, but in my opinion you should use one component per file in general to make minor upgrades and patching possible, and self-repair more reliable.
To better understand component reference counting: Change my component GUID in wix?
By eliminating hard-coded GUIDs you can take advantage of WiX's advanced auto-GUID creation concept. This will change the component GUID if the absolute installation path changes. This is correct behavior for component reference counting. Auto-magic. You either set Guid="*" or just leave out the Guid attribute entirely. A few installation locations need a hard coded GUID - the WiX compiler will warn you and explain why.
If you do change the component structure (to use one file per component) you should change the installation path to "break the link to past sins" with regards to component reference counting. This is a very complex topic to explain, but changing the installation path will sort all problems for you - if you also enable the auto component GUIDs I mentioned in the previous point. Keep the path stable from then on (until you have a major new version).
You can do it as simple as adding a sub folder with the the application's major (and minor?) version to the main installation folder hierarchy: "Program Files\MyCompany\MySoftware\5" instead of "Program Files\MyCompany\MySoftware".
I would only add the major version to the path and keep the installation path stable throughout your application's lifetime and then increment when you want to break the link to previous installers for a major new software version (for example if you want to install two versions side-by-side - your application must be built to handle this properly, i.e not overwriting shared settings in the registry from both versions etc...).
You might want to consider simplifying your WiX source file by only specifying values for attributes that are non-standard (otherwise rely on defaults). This can substantially simplify your WiX source files. Here is an example: Syntax for guids in WIX?
Just a quick sample inline (same as in link above - check it out), this is all that is required to install a normal file with default attributes / parameters - all other attributes default well - unless you want to override something:
<Component>
<File Source="..\File.dll" />
</Component>
Some links:
Windows Installer Best Practices (full list).
Windows Installer Best Practices - Organizing Applications into Components (specifically for component creation).
When component reference counting has gone haywire (missing files after upgrades, unexpected removal of shared files on uninstall, etc...): WiX 3.8: Two MSI using the same registry values. How to delete registry values only if both MSI are uninstalled?
Drop the ServiceControl element in lieu of a CustomAction element with #DllEntry="WixQuietExec" set, then use a standard means of starting the service like net start foo and ignore the result. See Quiet Execution Custom Action for details.

why is my eclipse RCP product locked and can't be updated?

I am building an Eclipse RCP-based product, and I am running into an issue where when I try to use the built-in p2 UI to install updates to the product, I get an error message in the dialog that "Insufficient access privileges to apply this update."
I have debugged into Eclipse and found that the 'root cause' is that there is a p2 .profile file that has xml that includes this snippet:
<iuProperties id='com.datical.db.ui.product' version='1.33.0.201412032223'>
<properties size='4'>
<property name='org.eclipse.equinox.p2.internal.inclusion.rules' value='STRICT'/>
<property name='org.eclipse.equinox.p2.type.root' value='true'/>
<property name='org.eclipse.equinox.p2.type.lock' value='3'/>
<property name='org.eclipse.equinox.p2.base' value='true'/>
</properties>
</iuProperties>
The relevant line is the one that says <property name='org.eclipse.equinox.p2.type.lock' value='3'/>
I'm not sure what I am doing wrong - I think I must have something awry in my product definition or my feature definition or in my install process that is causing this line to be there.
When I step through the Eclipse code (our target environment is 3.7/Indigo) I see that the profile is being written inside org.eclipse.equinox.internal.p2.engine:SurrogateProfileHandler:addSharedProfileBaseIUs (which is private static.) That is called from SurrogateProfileHandler:createProfile
The product's p2 repository is being built using the tycho plugins, version 0.15.
We finally discovered another piece of information that may be relevant. We were using a custom OSGI directory name. When we removed that, everything started working as expected.
It seems that you have a shared install where the product you would like to update is (potentially) used by many installations as a base - and therefore cannot be updated.
This kind of problem goes beyond what can be answered well on stackoverflow because you'd really need to provide an example project and exact steps to reproduce the problem.
The most plausible cause for the symptoms you are seeing is some kind of file system permission problem. This documentation mentions that you need write permission to the installation directory for running Eclipse with -initialize. Maybe you are lacking some permissions making some of the -initialize procedure fail and leave the installation in an inconsistent state.

Deploy website to drop folder on TFS build server

I am using TFS 2012 on a build server to do continuous integration, and also builds for other environments. I am deploying a .net 4.0 webforms solution containing two websites and a console app to a build server and in the drop folder I get a _PublishedWebsites folder containing the deployment package for the custom configuration I have specified (ie not Debug or Release). I get the correct .Web_Package for each website.
I am using the MSBuildArgumments setting in the build process as follows:
/p:DeployOnBuild=true /p:PublishProfile=Development/p:VisualStudioVersion=10.0
Everything works well, but the team lead wants the actual deployment in the drop-folder and not the deployment package.
I have looked at the MS Documentation but it does not appear to help in my case.
I have tried creating a script, but I dont want to go into a huge series of powershell scripts, and I struggle with powershell anyway. I just cant get it to do what I want.
My question is: can this be done, and what is the best way of doing it? If it does mean creating a script then so be it ( perhaps one line for each object) - I am really having a hard time working out which direction I should be going in.
I thought something a script containing one line for each deployment -3 lines like this:
_PublishedWebsites/<project name>.Web_Package/<project name>.deploy.cmd /T: /M:<site> /U:<user> /P:<password>
But where to put the script and how to call?
I ended up using the solution as described in this blog post:
http://blog.degree.no/2012/03/automatic-config-transformations/
To get what I needed, I needed to change the project files of the projects to be deployed - adding this extra node at the top of each.csproj file
<Target Name="TransformConfigFiles" AfterTargets="AfterBuild" Condition="'$(TransformConfigFiles)'=='true'">
<ItemGroup>
<DeleteAfterBuild Include="$(WebProjectOutputDir)\Web.*.config" />
</ItemGroup>
<TransformXml Source="Web.config" Transform="$(ProjectConfigTransformFileName)" Destination="$(WebProjectOutputDir)\Web.config" />
<Delete Files="#(DeleteAfterBuild)" />
</Target>
and adding this switch to the MSBuildArguments in the build definition:
/p:TransformConfigFiles=true
When I run my builds using the DeployOnBuild switch, my builds output two folders, one with the Package, and one with just all the website files (both under PublishedWebsites). Is that not what you want?
I have:
BuildDrop_PublishedWebsites\Website
BuildDrop_PublishedWebsites\Website_Package

Ant Deploy Windows build manifests SVN

I everyone, I am using Eclipse, Subclipse, and ANT. I would like to generate a build manifest with the files that have changed, added, updated, deleted, from the repo (with the individual version numbers on my current system).
<propertyfile file="${dist.dir}\deploymentManifest.txt"
comment="This file is automatically generated - DO NOT EDIT">
<entry key="buildtime" value="${builtat}"/>
<entry key="build" value="${svnversion}"/>
<entry key="version" value="${version}"/>
<entry key="systemLocation" value="${directory/filename.ext}"/>
</propertyfile>
How do I peel that information from the files in Eclipse? or how do I use ANY to retrieve this info?
Thanks,
Frank
Well, ${buildtat} could be taken from the <tstamp> task in Ant. The others could be parsed by doing a svn log --xml and then using the resulting XML from a <xmlproperties> task. Right off the top of my head (i.e. no error checking):
<!-- Gets the Time Stamp -->
<tstamp>
<format property="buildtat" pattern="MM/dd/yyyy HH:MM"/>
</tstamp>
<!-- Generates the revision information you need-->
<exec
executable="svn"
output="${svn.log.file}">
<arg line="log --xml -rHEAD/>
</exec>
<!-- Reads that information into a Property -->
<xmlproperty file="${svn.log.file}"/>
<echo message="Subversion Rev: ${log.logentry{revision}}"/>
However, I'd recommend you look at a continuous build system like Jenkins. Whenever you make a change in your Subversion repository, Jenkins picks up the change and automatically does a new build. Not only does this allow you to verify that your changes don't break your build, but Jenkins can do other things too like run JUnit tests. Jenkins then stores your build and the results of your tests and the whole build log in an easy to get to HTML page.
Where Jenkins will work for you is that Jenkins automatically exposes such things as the Subversion Revision as part of the build process. You can fetch the Subversion Revision, the Jenkins build number, the name of the Jenkins project and many other things as environment variables. Then, you could do this:
<property env="env."/>
<propertyfile file="${dist.dir}\deploymentManifest.txt"
comment="This file is automatically generated - DO NOT EDIT">
<entry key="buildtime" value="${env.BUILD_ID}"/>
<entry key="build" value="${env.SVN_REVISION}"/>
<entry key="version" value="${BUILD_NUMBER}"/>
<entry key="systemLocation" value="${directory/filename.ext}"/>
</propertyfile>
Take a look at Jenkins. It's fairly easy to understand and use.
It should take you about 5 minutes to download and maybe 10 minutes on a Linux system to get up and running. Windows is more complex and might take as long as 15 to 20 minutes to get up and running. You can run it on your desktop system for now, and play around with it.
It should take you maybe another half hour to figure out how to setup a project that can automatically do builds whenever someone does a commit.
Jenkins is web based, but comes with its own light weight web based application engine. All you need is Java 1.6 to run it. (And, if you're using Eclipse, you should already have that).

TFS2010 Build Definition to Deploy to multiple servers?

I've been looking into TFS2010 new build and deployment features with MSDeploy. So far everything is going well (although its been hard to find information about specific scenarios).
Can I modify my Build Definition to specify 2 or more servers to deploy to? What I need to do is deploy to multiple servers (as I have two in my testing environment which uses a NLB).
What I have now is a Build definition which Builds, runs my tests, and then Deploys to ONE of my testing servers (which has the MsDeployAgentService running on it). It works fine, and each web project is deployed as configured in its project file. The MSBuild Arguments I use are:
* /p:DeployOnBuild=True
* /p:DeployTarget=MsDeployPublish
* /p:MSDeployServiceURL=http://oawww.testserver1.com.au/MsDeployAgentService
* /p:CreatePackageOnPublish=True
* /p:MsDeployPublishMethod=RemoteAgent
* /p:AllowUntrustedCertificated=True
* /p:UserName=myusername
* /p:Password=mypassword
NB: I dont use /p:DeployIISAppPath="xyz" as it doesnt deploy all my projects and overrides my project config.
Can I add another build argument to get it to call more than one MSDeployServiceURL? Like something like a second /p:MSDeployServiceURL argument that specifies another server?
Or do I have to look for another solution, such as editing the WF?
I saw an almost exact same question here posted 2 months ago: TFS 2010 - Deploy to Multiple Servers After Build , so it doesn't look like I'm the only one trying to solve this.
I also posted on the IIS.NET forums where MSDeploy is discussed: http://forums.iis.net/t/1170741.aspx . It's had quite a lot of views, but again, no answers.
You don't have to build the project twice to deploy to two servers. The build process will build a set of deployment files. You can then use the InvokeProcess to deploy to multiple servers.
First create a variable named ProjectName. Then add an Assign activity to the "Compile the Project" sequence. This is located in the "Try to Compile the Project" sequence. Here are the properties of the Assign:
To: ProjectName
Value: System.IO.Path.GetFileNameWithoutExtension(localProject)
Here are the properties of our InvokeProcess activity that deploys to the test server:
Arguments: "/y /M:<server> /u:<domain>\<user> /p:<password>"
FileName: String.Format("{0}\{1}.deploy.cmd", BuildDetail.DropLocation, ProjectName)
You will need to change <server>, <domain>, <user>, and <password> to the values that reflect your environment.
If you need to manually deploy to a server you can run the command below from your build folder:
deploy.cmd /y /M:<server> /u:<domain>\<user> /p:<password>
I couldn't find the solution I was looking for, but here's what I came up with in the end.
I wanted to keep the solution simple and configurable within the TFS arguments while at the same time staying in line with the already provided MSBuildArguments method which has been promoted a lot. So I created a new Build Template, and added a new TFS WorkFlow Argument called MSBuildArguments2 in the Arguments tab of the WorkFlow.
I searched through the BuildTemplate WorkFlow for all occurances of the MSBuildArguments (there were two occurances).
The two tasks that use MSBuildArguments are called Run MSBuild for Project. Directly below this task, I added a new "If" block with the condition:
Not String.IsNullOrEmpty(MSBuildArguments2)
I then copied the "Run MSBuild for Project" task and pasted it into the new If's "Then" block, updating its title accordingly. You'll also need to update the new Task's ConmmandLineArguments property to use your new Argument.
CommandLineArguments = String.Format("/p:SkipInvalidConfigurations=true {0}", MSBuildArguments2)
After these modifications, the WorkFlow looks like this:
Save and Check In the new WorkFlow. Update your Build Definition to use this new WorkFlow, then in the build definition's Process tab you will find a new section called Misc with the new argument ready to be used. Because I'm simply using this new argument for deployment, I copied the exact same arguments I used for MSBuild Arguments and updated the MSDeployServiceURL to my second deployment server.
And that's that. I suppose a more elegant method would be to convert MSBuildArguments into an array of strings and then loop through them during the WorkFlow process. But this suits our 2 server requirements.
Hope this helps!
My solution to this is a new Target that runs after Package. Each project that needs to produce a package includes this targets file, and I chose to make the Include conditional on an externally-set "DoDeployment" property. Additionally each project defines the DeploymentServerGroup property so that the destination server(s) are properly filtered depending on what kind of project it is.
As you can see towards the bottom I'm simply executing the command file with the server list, pretty simple.
<!--
This targets file allows a project to deploy its package
As it is used by all project typesconditionally included from the project file
-->
<UsingTask TaskName="Microsoft.TeamFoundation.Build.Tasks.BuildStep" AssemblyFile="$(TeamBuildRefPath)\Microsoft.TeamFoundation.Build.ProcessComponents.dll" />
<!-- Each Server needs the Group metadatum, either Webservers, Appservers, or Batch. -->
<Choose>
<When Condition="'$(Configuration)' == 'DEV'">
<ItemGroup>
<Servers Include="DevWebServer">
<Group>Webservers</Group>
</Servers>
<Servers Include="DevAppServer">
<Group>Appservers</Group>
</Servers>
</ItemGroup>
</When>
<When Condition="'$(Configuration)' == 'QA'">
<ItemGroup>
<Servers Include="QAWebServer1">
<Group>Webservers</Group>
</Servers>
<Servers Include="QAWebServer2">
<Group>Webservers</Group>
</Servers>
<Servers Include="QAAppServer1">
<Group>Appservers</Group>
</Servers>
<Servers Include="QAAppServer2">
<Group>Appservers</Group>
</Servers>
</ItemGroup>
</When>
</Choose>
<!-- DoDeploy can be set in the build defintion -->
<Target Name="StartDeployment" AfterTargets="Package">
<PropertyGroup>
<!-- The _PublishedWebsites area -->
<PackageLocation>$(WebProjectOutputDir)_Package</PackageLocation>
<!-- Override for local testing -->
<PackageLocation Condition="$(WebProjectOutputDirInsideProject)">$(IntermediateOutputPath)Package\</PackageLocation>
</PropertyGroup>
<Message Text="Tier servers are #(Servers)" />
<!-- A filtered list of the servers. DeploymentServerGroup is defined in each project that does deployment -->
<ItemGroup>
<DestinationServers Include="#(Servers)" Condition="'%(Servers.Group)' == '$(DeploymentServerGroup)'" />
</ItemGroup>
<Message Text="Dest servers are #(DestinationServers)" />
</Target>
<!-- Only perform the deployment if any servers fit the filters -->
<Target Name="PerformDeployment" AfterTargets="StartDeployment" Condition="'#(DestinationServers)' != ''">
<Message Text="Deploying $(AssemblyName) to #(DestinationServers)" />
<!-- Fancy build steps so that they better appear in the build explorer -->
<BuildStep
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Message="Deploying $(AssemblyName) to #(DestinationServers)...">
<Output TaskParameter="Id" PropertyName="StepId" />
</BuildStep>
<!-- The deployment command will be run for each item in the DestinationServers collection. -->
<Exec Command="$(AssemblyName).deploy.cmd /Y /M:%(DestinationServers.Identity)" WorkingDirectory="$(PackageLocation)" />
<BuildStep
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Id="$(StepId)"
Status="Succeeded"
Message="Deployed $(AssemblyName) to #(DestinationServers)"/>
<OnError ExecuteTargets="MarkDeployStepAsFailed" />
</Target>
<Target Name="MarkDeployStepAsFailed">
<BuildStep
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Id="$(StepId)"
Status="Failed" />
</Target>
I am the author of the other similar post. I have yet to find a solution. I believe it is going to be modifying the workflow to add a postprocessing MSBUILD -sync task. That seems to be the most elegant, but was still hoping to find something a bit less intrusive.
I'm not sure if that could help you with TFS 2010, but I have a blog post for TFS 2012: Multiple web projects deployment from TFS 2012 to NLB enabled environment.