I have a .net app which has an MVC3 front end and 2 windows services.
It all depends on 2 RavenDB installations which can either be ran as windows services or IIS - I'm not bothered here.
The services are built using TopShelf and testing is done with straight NUnint. I use Github as my repo.
Ideally on each successful 'Release' build I'd like a build, test, wipe directories and RavenDb data dirs and then deploy (web and services) and then start the processes to run finishing off on a console app i'm building which can run in a default set of data.
How would you manage the deployment here? I have no CI server yet. I have a completely fresh server that I can do as I please with. I haven't done CI/CD for a long time and I suspect the weapons have changed.
Should I be looking at MSBuild/NAnt? PSake, Rake? Team City?
How would you manage the post build processes?
I've been using Jenkins with psake and it's working very well. To be honest psake does most of the work, with Jenkins simply pulling down the source and then invoking my psake script, but as Nick Nieslanik says you can easily have Jenkins call MSBuild/NUnit/etc directly if you like.
Jenkins
From a peruse of Jenkins vs. CruiseControl(.NET) threads on StackOverflow, the general consensus appeared to be to go with Jenkins. Having not really tried CruiseControl I can't vouch for that, but I will say that Jenkins is very good. I found Jenkins easy to set up. I had a quick peek at CruiseControl.NET and found Jenkins easier to get started. I haven't looked at TeamCity at all so can't speak to that.
CI: Hudson with .Net vs CruiseControl.Net
What is the difference between Hudson and CruiseControl for Java projects?
How and why do I set up a C# build machine?
Jenkins has a nice plugin system, and a lot of plugins, including one for Powershell which makes it easy to call through to a psake script.
Psake
So far I think psake is awesome. It's based on rake syntax, but is a wee bit more native to Windows than rake. As it's sitting on top of powershell, you can leverage a lot of the handy Windows admin functionality that comes with that. For example, see this post
for a great example of setting up and tearing down IIS app pools and sites direct from your psake tasks. I think that's great, and I'm not sure how you'd go about doing that in MSBuild, Nant, or rake. Basic file system operations are also it's bread and butter -- seems better than having a bunch of angle brackets just to copy some files somewhere.
As for MSBuild and Nant, I think they're both pretty powerful, but editing XML files for this kind of thing just feels painful. Powershell is a proper scripting language with deep Windows integration. psake is a DSL for building and other tasks on top of that. It's a good combo.
That said, for actually building, I just farm that out to msbuild from within psake and call it on the solution/project files I want to build. psake has a built in command for calling msbuild and specifying which version to use, etc. (Truth be told the most pain so far is from msbuild borking up on solution files that build just fine in Visual Studio.)
As you're working with RavenDB, you might be interested to know that they're using psake for building RavenDB (and Rhino-ESB).
For some good psake tips in general, see this post.
Long story short, personally I'd recommend Jenkins and psake. That combo will integrate nicely with git, msbuild, NUnit, IIS, and probably even windows services.
I'd use Jenkins as the job execution engine. Then I could create a set of MSBuild scripts to perform the core-build and use the Jenkins plug in model to add pre & post build tasks as necessary (i.e. NUnit execution and result parsing, Powershell script execution for some deployment) etc. Jenkins has great integration with Github using Post/Pre-commit hooks that allow you to set up CI builds quite simply.
Related
I'm porting some Jenkins builds into a highly restricted ADO Pipeline environment. When making some CAT files the MakeCAT utility was being used and when verifying INF files the InfVerif tool was being used. In our company's highly restricted ADO environment I can't seem to access tools directly anywhere outside the build directories for the job and was told there wouldn't be a compromise on that.
The best I could figure was directly downloading the files and their dependencies as Secure Files and kluding together tool directories for each required tool. That is a dirty hack and skates around legal grey-areas with tool licensing, so I'm not a fan of that approach. But that said I used DUMPBIN /IMPORTS to see which each respective tool required:
InfVerif.exe:
msvcrt.dll
ntdll.dll
api-ms-win-core-libraryloader-l1-1-0.dll
KERNEL32.dll
VERSION.dll
ADVAPI32.dll
MakeCAT.exe:
msvcrt.dll
KERNEL32.dll
WINTRUST.dll
USER32.dll
When it comes to creating drivers and driver-related files what are we expected to utilize in ADO Pipelines with these kinds of restrictions? I don't mind using alternative tooling so long as it accomplishes the same exact goals.
ps: I went ahead and copied all the DLLs in each tool's directory and played "delete DLLs until this tool starts to break," to narrow down what was actually needed to be packaged on a local system. InfVerif needed no additional DLLs and MakeCAT only needed wintrust.dll added. Mind you this was constrained to our own usage of each tool and your usage may differ from ours and require additional dependencies to package.
Generally the point of build agents is that running jobs utilise the capabilities of the agent to create the build.
It would be quite obscure to ask developers to self-package every single dependency into the pipeline and kludge together executables and dll's for common SDK tools.
Presumably these build agents have stuff like .NET SDK's installed for the builds to utilise. This is no different. You should ask the team managing the build agents to install the relevant SDKs and ensure the paths are configured and available for the build agent.
(e.g echo PATH inside build agent should include directories where those SDK's are installed)
Asking developers to check in packaged exe's and dlls is more of a security risk. Who knows where they come from and if they are safe?
Is there an equivalent to Powershell make? The equivalent of make/rake/cake/py-make, even Gulp...etc? I primarily desire a task runner for build automation. I want to be able to select and compose tasks like I can in Gulp.
Ideally, the solution would be native to Powershell. There are very tight security restrictions on software installations for the project. The power of Google has failed me.
nothing built-in, but there are different solutions like psake or invoke-build. maybe something else.
You can also use VScode for that, you can define tasks (basically run scripts or something) and run them on save or on check out, etc
I am working on a project where we need to repeat certain steps with powershell to deploy stuff. i would like to create a process/install guidance (steps supported with UI) with WIX but after the msi has finished i have an entry in programs and features. I just need it to execute the powershell and the end without registering in windows. i might be using the wrong tooling or whatever, any suggestions are welcome.
Definitely not recommended unless you want to track the deployment of these scripts on different systems by checking the entries in ARP (Add/Remove Programs), and even then it clogs up the Add/Remove view of your computers. Most system administrators hate this approach, it is better to just write to your own registry key and read it back from every machine.
What are the scripts doing? Are you actually installing files.
I have been tasked with looking into our deployments, and seeing where they can be streamlined. Right now we have 4 different configurations (Debug/Dev, Test, Staging, Release) and 4 *.config files. We have a task that will overwrite app/web.config with the appropriate *.config pre-build time based on the active configuration. An MSI is created, and we do a full deployment of the component on release night.
This is not entirely ideal because if we change something in a config file, or fix the spelling in a specific view we have to re-deploy the entire thing. Not to metion that the MSI will occasionally require a reboot. One other option that has been brought up is instead of creating MSIs we could create custom deployment/rollback scripts and have the ability to do incremental release.
Has anyone here tried deployments both ways? What are some of the pros/cons you have found? Is there a third way we haven't thought of?
edit: Just to clarify a few things...We don't deploy to customers. All software is deployed to our servers. (a few sites, and a lot of windows services). We never change things in production. We actually use the built in system within VS to create the MSI, so that part isn't the terrible part. To me it just doesn't make sense to redeploy an entire website if you had to change 1 view. We also have to deploy to multiple servers. Right now that is done by running the MSI on each one.
MSI pros:
Application/service/site gets installed and registered like most other Windows apps, and shows up in Add/Remove programs
Some built-in support for re-installing, upgrading
Has some built-in support for installing Windows services/IIS sites/lower-level Windows features
MSI cons:
Seems really cryptic once you get "under the hood"
Seems more difficult to customize than using a custom script
Script pros:
Easier to customize, although certain steps might require lots of/cryptic scripting (working with IIS, lower-level computer administration)
Don't have to deal with low-level weirdness of MSI
Script cons:
.bat scripting is not the most readable or writable language. (Powershell is better, but then you have to worry about whether Powershell is installed on the target machine).
Low-level operations require a lot of administrative scripting for commit/rollback behavior
No built in support for installing or rolling-back (MSI has some support built-in)
One thing I've come across that helps with MSIs is WiX (http://wix.sourceforge.net/), but even WiX seems pretty cryptic in a lot of ways. We use a combination of MSBuild and WiX to do automated builds and deployment/installs, and it works okay for us.
Overall, I'd probably lean more towards doing MSI/WiX (or other installer toolkit) deployments over scripts. MSIs are the standard way of doing installs on Windows, and once you get it working, you usually don't have to change too much. MSBuild or some other build framework (NAnt, etc.), can be useful for setting up the deployment (renaming files, doing string replacements, etc.), before putting together the final MSI package.
Running a dev company that build web apps for five years we struggled with this and tried a bunch of solutions. Here are a couple tips:
Always replace the entire web directory with your code (except if you have content generated by the web site, like a CMS). It's pretty fast to do this and incremental deployments can introduce phantom bugs if files are left around.
Have your build process (Nant, MSBuild, whatever) mod the .config files for each environment and build for what you push for. Alternately you can use registry settings so that the .config files are the same but that means a dedicated machine for each environment. May or may not be an issue.
Don't make changes in production. If you need to make changes (spelling errors on site) make those top priority to get changed in dev so that you don't overwrite them with the next push.
If you aren't using MSI's then make sure you have a rollback process. Keeping a copy of the site just before you changed it really helps when something unexplained goes sideways during a roll-out.
I don't know that these tips point to MSI or script. I think it's a matter of which you are most comfortable with. MSI's can be hard to customize, but easy to run and manage. Microsoft has lots of tools for managing roll-outs of MSI's across an organization or farm. Scripts may require custom tools and custom tooling or lots of manual work on the production end.
We ran scripts with Nant and a custom deployment harness. These days (VS2008) building deployment packages is much easier.
Your best option is to get a decent MSI builder to do the job with - i'm talking about InstallShield etc (there are a couple, so do look around). While these invariably cost, they can save you a huge amount of time/money/pain further down the track. Having said that, the pain is not totally eliminated, just reduced :)
Anything tricky you need to do can be done as a custom task within the msi - and you can even do this with the setup builder that comes with Visual Studio (if you are using VS).
I have a suggestion for your config files - include all four in the msi, and then have a public property which can be set from the command line. You can then modify that public property to install the appropriate config file (and have the default value of that property set so that the release config gets installed). That way, your customers just use the msi and get the correct config file, but your test team can get their config file by changing the value of the public property; the command line they would use to do the install is this:
msiexec /i "MyInstaller.msi" CONFIG=test
You can do install scripts quite easily, but as already mentioned you also need to script the uninstall. Using install scripts precludes you from getting Windows certification for your product should you look at getting that done. But that doesn't mean you shouldn't use install scripts, they may be the perfect fit for your needs. Alternatively you may look at using a combined script/msi approach by having your scripts run as custom actions from within the msi.
I am currently writing a deployment script in MSBUILD, and after downloading several extensions, I have found myself looking at the build file and thinking:
What was the point in doing this in MSBUILD?
This deployment script is completely procedural: stop website, delete folder, copy files, change permissions, start website, etc. There is no fancy dependency stuff which I assume is the natural domain of tools like MSBUILD, NANT and MAKE.
The only reason I can see to use MSBUILD is that it comes as standard, and its easy to put the extensions into your SVN so builds 'just work'.
The problem with it is I have to spend all this time working out how to do 'basic stuff' in MSBUILD (locating extensions, working out syntax) which would be trivial (although more verbose) in Powershell or even command line.
So to sum up:
Are procedural tasks suited to MSBUILD or are you better of using something like Powershell?
Check out PSAKE and see what you think.
http://www.jameskovacs.com/blog/IntroducingPsake.aspx
http://powerscripting.wordpress.com/2009/01/25/episode-56-james-kovacs-talks-about-psake/
http://code.google.com/p/psake/
Experiment! Enjoy! Engage!
Jeffrey Snover [MSFT] Windows Management Partner Architect
MSBuild is not a scripting language and shouldn't be used as such. It's almost unfortunate that MSBuild has such a rich extensibility and is flexible enough to be used for just about anything. Use the tools that are most appropriate to the task, if you find yourself spending too much time creating functionality that is too limited and too low quality compared to what you'd be able to create with other technology, you should switch.
It really depends on your situation. If it were up to me, though, and you were using Visual Studio - I would say yes, stay with MSBuild for the sake of integration.
On the other hand, I would choose MSBUILD, as while the tasks are very procedural, it gives you the flexibility to extend this build process later on to handle more complex tasks.
msbuild comes with .NET. You have to add powershell to servers / users must add it - at least through Windows XP, server 2003. That may or may not be a problem in your environment.
I don't think procedural tasks are suited for writing in MSBUILD, simply because the shorter the msbuild, the better as far as I am concerned. I might use msbuild to call them, but would probably write an extension library to implement them.
I think it depends on how your release and deployment process flows, as to wither it makes sense to use an MSBuild extension or execute power-shell. MSBuild allows the flexiablity to handle all your process steps in one self contained execution flow.
If you need it to occur all at one time then MSBuild gives you control over the 'Events' or targets that can be overridden to meet your requirement.
If the requirement is deploy your artifacts after compiling your code then MSBuild is well suited to do this since you can use the 'AfterBuild' target that gets triggered during an MSBuild standard execution. It can make your process self contained.
Powershell cannot build your code. It would have to call MSBuild from within your script. To me it is a matter of having your build and deployment self-contained and therefore would be organized better.
MSBuild is the core Microsoft build platform and engine.