For a project I am working on, we have CI setup using Jenkins.
We now want to setup Continuous Delivery (CD) using Puppet.
Here is our dev environment specs
Windows 2008 Server
Jboss-SOA-P (jboss AS 5.1 app server) - 2 instances
Jenkins for CI
Installed Puppet Learning VM (as we are evaluating, so we don't have a license to install Puppet Enterprise).
My question is: How can I automate deployment of my application(s) on already installed Jboss servers (on Windows m/c) using Puppet?
For my organization, I am using the following tools to achieve Continuous Delivery and Continuous Integration
Foreman: A provisioning and ECN ( External Node Classifier )
Puppet Master :
This will be running in our main server
Puppet Agents:
On rest of the servers
Jenkins:
On main server
Nexus repository for maintaining staging and release repositories on another server
A nexus repository module installed in puppet master. It has the logic to connect to nexus repository, fetch the latest release from "release" repository.
Flow:
I have a jenkins job defined whose purpose is making sure the build doesn't break and at the time of release, I perform maven release
that in turn upload the latest version to nexus release repository.
Load nexus repository module in foreman and map "Nexus" class to all my servers. This is the crucial part that allows me to perform
cloud deployment with a single button click in foreman. To do this, you need to have your own puppet files that perform dbmigration, undeploy and deployment. I always write a deploy maven module for all my projects which will have only these puppet files that allows me to perform deployment in all servers in one shot.
Since you are already familiar with CI and CD, I hope my statements are self explanatory
Without knowing how you want your application deployed, it's hard to answer exactly how to do it. But I'll see if I can point you in the right direction:
There are existing modules on the forge to deploy JBoss with Puppet. I'd recommend looking through and seeing if you can find one that suits your requirements.
You could then integrate your source control to automatically deploy changes to your JBoss instance as they get checked in.
There's an example of using Puppet with Tomcat and Maven for continuous delivery here. It's a few years old, but the concepts still apply: http://www.slideshare.net/carlossg/2013-02-continuous-delivery-apache-con
Here's also an example from CloudBees for contionous delivery with Puppet and Jenkins (which has a lot of the engineers behind Jenkins) https://www.cloudbees.com/event/continuous-delivery-jenkins-and-puppet-debug-bad-bits-production
Plus a general PuppetLabs one here: https://puppetlabs.com/blog/whats-continuous-delivery-get-speed-these-great-puppetconf-decks
Also: I don't want to sound too salesly, and full disclosure, I work at PuppetLabs. But you can use Puppet Enterprise with up to 10 nodes for free, so I'd recommend that over using the Learning VM, which isn't really designed for hosting applications.
Related
I'm trying to use TFS 2015.1 on premise to build a CI pipeline for our dev & uat. I've created a vNext CI build, which builds fine. But when I want to add a deploy step for on prem IIS server, I only then see Azure Web Deployment options.
Ideally I wanted to add a step which uses the existing deploy (MS Deploy) profiles, which I'm able to use from VS2015 directly, using 'Publish'. However I see no option to do so.
How can I deploy the latest build to internal dev servers (not Azure)? I would like to use the MS Deploy option, unless there's a better way of doing it?
The fact that their is no option to starts to make me think there's probably a different way to accomplish it!
Thanks.
If you're able to upgrade to TFS 2015.2, web-based Release Management came out with it that works similarly to Build vNext with flexible and open-source tasks. You can also customize tasks.
Here's a link for IIS Web App Deployment from the vso-agent-task's GitHub repo where Microsoft stores updated versions of their tasks that you can download for web-based Build and Release Management.
I'll be publishing a blog about web-based RM with TFS 2015 Update 2 or VSTS on my website in the next few weeks. To give you an idea though, the starting point (for a web application) is a folder in your web project called WebDeploy (no significance - any name will do) that contains a PowerShell DSC script that configures the server, deploys the web files and then replaces any tokenised configs. To give you an idea see this post about how to use DSC to configure servers. (Only covers part of the final script though!) The next steps are:
In the build hub create a Website artifact - containing your web files and DSC script.
In the release hub for an environment use a Windows Machine File Copy task to deploy the artifact to a temp folder on the target node.
Then use a PowerShell on Target Machines task to execute the DSC script. After configuring the server the script copies the web files to their proper location, sorts out config using xReleaseManagement and cleans up the WebDeploy folder.
See this article for general details of the route I'm taking, but watch out as it has some errors eg the firewall instructions are incomplete (file and print sharing through the firewall needs to be enabled).
I can thoroughly recommend the PowerShell DSC route - I've had a few glitches but on the whole it feels very productive and the right way to be going.
I am doing PowerShell DSC POC. I configured the Pull server and one client machine. It is working fine and I am very much happy about PowerShell DSC feature.
Now I want to integrate this feature with our continuous Integration process. We are using Nolio for MSI deployment and do the other configurations. As of now I want to use DSC only for configurations and Nolio will continue for deployment process (due to reduce the Migration complexity). Later we planned to replace the Nolio with DSC including deployment. Here is my questions.
1) We have monthly releases. As per my understanding I need to install the MSI(will deploy websites) in all machines including Pull Server and Nodes. Then I will do the configuration settings using Pull Server configuration. Once I configured the Pull server how could I do the second deployment. Will Pull server node create any problem at the time of deployment like reverting the installed files as per old configurations? Is there any way to stop the Pull server settings at the time of deployment?
2) If I want to install MSI also from DSC, I am planning to do that like below.
Change the Pull server configuration to install the MSI from other configuration settings.
Install the MSI in Pull server and all Node machines.
Do all other configurations in Pull server.
Change the configuration to apply pull server configurations to
Node from Install MSI.
Is this good process?
Could you please anyone help me to achieve this? Please share if you have any other best practices.
Thanks in advance.
Is there a way in Bamboo to deploy artifacts from artifactory rather than only local published artifacts? I've found the Artifactory Plugin but as far as I could see, it only allows for deploying stuff into artifactory.
I'm using Bamboo 5.4.2
You can use your build server to deploy from Artifactory to your application server, that's a very detoured way to go. You already uploaded all the binaries to Artifactory why would you want to download them to the build server again?
You have number of ways to get the needed files to your application server right from Artifactory, without involving the CI server, and the selection depends on how complicated your requirements are. If all you need is to get the latest version of some artifact from Artifactory to app server, tools like LiveRebel are a great match. If you need to do more, e.g. deploy on sophisticated topology of clustered environment with sharded data schema upgrade without downtime, you might need something more free-style like Puppet, Chef, Ansible, or Salt.
In any way, Artifactory Properties and the REST API to work with them are your best friend. Using properties in your REST queries for artifacts allows expressing queries like "Give me all the artifacts that were produced by certain Bamboo build, but only those, which were staged, have the QA level of 'production' and matching the target deployment target".
As many apps do, we have a number of config and properties files for our Java applications. We have gone with the approach of keeping these files separate from our codebase (i.e. they are not included in the war files for deployment) but in a separate directory. However, I would still like to track changes to these files in a source control and deploy them using our CI.
I'm looking for strategies on how others have done this. Did you write a script to push the files to the app server(s). Does the script live on the CI server?
Our SCM is Mercurial which we have set up on its own server to use as a central repo. Our CI is Hudson (not Jenkins) set up on its own server and of course our app servers are separate from these as well. All servers are *nix OS.
Consider using configuration management tools like puppet or chef for managing all your application configuration files. Both tools use "manifests" or "receipes" which can placed under revision control and matched to each server deploying the application.
Another option is to consider is to develop an install package for your OS, see the following articles for more details:
http://www.sonatype.com/people/2011/11/bringing-java-and-linux-together-on-the-way-to-continuous-live-deployment/
The advantage of doing it this way is that the install can be configured to generate the correct configuration tailored for the environment it is deployed onto. A more important benefit is that it's simpler to manage and install.
We decided to use AMAZON AWS cloud services to host our main application and other tools.
Basically, we have a architecture like that
TESTSERVER: The EC2 instance which our main application is
deployed to. Testers have access to
the application.
SVNSERVER: The EC2 instance hosting our Subversion and
repository.
CISERVER: The EC2 instance that JetBrains TeamCity is installed and
configured.
Right now, I need CISERVER to checkout codes from SVNSERVER, build, if build is successful, unit test it, and after all tests pass, the artifacts of successful build should be deployed to TESTSERVER.
I have completed configuring CISERVER to pull the code, build, test and produce artifacts. But I couldn't manage how to deploy artifacts to TESTSERVER.
Do you have any suggestion or procedure to accomplish this?
Thanks for help.
P.S: I have read this Question and am not satisfied.
Update: There is a deployer plugin for TeamCity which allows to publish artifacts in a number of ways.
Old answer:
Here is a workaround for the issue that TeamCity doesn't have built-in artifacts publishing via FTP:
http://youtrack.jetbrains.net/issue/TW-1558#comment=27-1967
You can
create a configuration which produces build artifacts
create a configuration, which publishes artifacts via FTP
set an artifact dependency in TeamCity from configuration 2 to configuration 1
Use either manual or automatic triggering to run configuration 2 with artifacts produced by configuration 1. This way, your artifacts will be downloaded from build 1 to configuration 2 and published to you FTP host.
Another way is to create an additional build step in TeamCity for configuration 1, which publishes your files via FTP.
Hope this helps,
KIR
What we do for deployment is that the QA people log on to the system and run a script that deploys by pulling from the team city repository whenever they want. They can see in team city (and get an e-mail) if a new build happened, but regardless they just deploy when they want. In terms of how to construct such a script, the team city component involves retrieving the artifact. That is why my answer references getting the artifacts by URL - that is something any reasonable script can do using wget (which has a Windows port as well) or similar tools.
If you want an automated deployment, you can schedule a cron job (or Windows scheduler) to run the script at regular intervals. If nothing changed, it doesn't matter much. I question the wisdom of this given that it may mess up someone testing by restarting the system involved.
The solution of having team city push the changes as they happen is not something that team city does out of the box (as far as I know), but you could roll your own, for example by having something triggered via one of team city's notification methods, such as e-mail. I just question the utility of that. Do you want your system changing at random intervals just because someone happened to check something in? I would think it preferable to actually request the new version.