I am looking at getting a continuous integration/continuous deployment environment set up for my windows azure project and I was wondering if anyone had managed to (can point me in the right direction to) build and deploy a windows azure cloud service using powershell and Hudson and perhaps has sample scripts.
I can get the project to build using MSBuild64 (I'm running x64 Windows 2008 R2 Standard).
I know 32 bit development works, but assume 64 bit development is better as i understand it as problems will be ironed out on my local box as opposed to once deployed in the Azure environment which i believe is 64 bit. (Please feel free to correct my thinking here)
I assume i have to get the cspack.exe file to package the deployment first as in a manual deployment via the development portal.
Ideally i would like to deploy it locally (with the development simulation) run unit tests against it (perhaps against cloud storage for integration tests), deploy it to staging (run the acceptance/bdd) tests and then switch from staging to production.
Any help with anyones experience in this which will speed this research up for me would be appreciated
Many Thanks
Mark
This may help to get you started: http://blog.smarx.com/posts/building-running-and-packaging-windows-azure-applications-from-the-command-line
Also see http://scottdensmore.typepad.com/blog/2010/04/windows-azure-deployment-for-your-build-server-part-2-deploy-certs.html.
Mark, I am looking at something similar. The best solution at this point appears to be a custom solution that uses the REST APIs.
Erick
Related
Newbie to automated azure deployment here! I have the happy task of automating our deployment to the cloud. I have also done some reading and discovered that the 2 main tools are MSbuild and Powershell. Please could anyone tell me why i would use one over the other or indeed if there are any better ways to automate the deployment. Keeping in mind that my main concern is performance and i need this deplymrnt to be as fast as possible.
Any insight would be most welcome.
I'm a fan of using PowerShell for deployments. It's pretty quick to set up and the script can be pretty straight forward.
MSBuild can be great too. I use MSBuild from TFS Team Build to kick off a PowerShell script to do the deployment. Works like a champ.
A good starting point would be http://blogs.msdn.com/b/tomholl/archive/2011/12/06/automated-build-and-deployment-with-windows-azure-sdk-1-6.aspx. This blog does a great job of showing you how to build and deploy with Team Build.
If you don't want/need the Team Build and MSBuild part, then just look at his PowerShell script. That covers the basics of getting a deployment from your dev environment to Windows Azure.
You should use Web Deploy, it only takes about a minute to deploy a fix. See these links
http://blogs.msdn.com/b/cloud/archive/2011/04/19/enabling-web-deploy-for-windows-azure-web-roles-with-visual-studio.aspx
http://channel9.msdn.com/Blogs/funkyonex/Speed-Up-Azure-Deployments-with-the-New-Web-Deployment-Feature
At SplendidCRM, we had a similar need to automate deployments to Azure, but as our need was to service our live customers, we had to develop using C#. We have been watching Azure for many years, but it was not until they provided a DNS service did it make sense to make the move. Using the Azure Resource Manager (ARM) libraries, we were able to automate VM creation, SQL database creation and DNS name creation. In addition to the Microsoft documentation for ARM, we found it particularly useful to be able to get the Microsoft source code for the PowerShell scripts that wrap ARM. This is because the documentation does not always provide a complete set of settings.
In the end, we decided to release the Azure deployment code as part of a new Ultimate edition that combines order and customer management with software deployment.
Nutshell :
I'd like some help / info / resources r.e. setting up a Team Build, MsBuild, TFS 2010 automated deployment for my Web App to Azure (inc. all the DB bits).
Ideally I'd like to have a process that I can fire off from the VS2010 Team Explorer UI "Queue New Build", then just keep an eye on its progress, releasing me to work on something else. Options to delve into logging for any issues, and know that my process is robust, complete and totally non-manual, i.e. :
Backs up all my Live data (SQL Azure
and Azure Blobs)
Deploys any DB schema changes (contained in my DB project)
Deploys any data changes to my core data (e.g. config data etc - which I have in my Post-Deployment scripts)
Does things sensibly (e.g. using compression for deployment packages to save time & bandwidth)
Cover my silly backside (e.g. seamlessly rolling back failed changes)
Keeps app 100% running during deployments (failed or successful) e.g. sessions left intact, minimal chances of data loss
Keep detailed logs of processes progress at each stage for fixing any issues
Keep everything that should be Source Controlled well... source controlled
Background / Dream / Goal :
At my last FT job we had a pretty sweet automated deployment setup for our hosted Web Apps, using CC.Net (to manage the process), CCTray & the CC.NET web UI to monitor and control, Code Generation (CodeSmith + NetTiers templates for data access & entities), MSBuild, VS Databse Projects, a few .bat scripts, and some handy utilities like PsExec etc to help out with little bits and pieces.
I didn't set it up, but have some experience managing it, dealing with issues etc.
It was (98% of the time) a lovely experience to deploy. You'd make sure TFS was up to date, double click CCtray, right click on a project and then click "Force Build", sit back and watch Green => Yellow => Green.
Great !!
Current Situation :
I run my own Micro ISV, and my main project is an App on Azure (in Beta).
I'd very much like to replicate the kind of deployment experience I had before - I'm even considering moving out of Azure to dedicated servers - just because I know I can setup an automated deployment system there.
My main stopping point is the DB bits, seems like a nightmare. Maybe I'm missing some great free tool or library which would do the job, I really hope so, but I also could really do with someone experienced with this to point me in the right direction for a "Best Practice" solution to wrap up all the little bits neatly.
I have scoured Google, read and read, burnt hours and hours, but what I seem to find atm is half solutions, not quite right for my project and needs, based on expensive tools I can't afford (near $0 budget), or is plain over my head and a bit incomprehensible & scary.
Now I'm NO Sys Admin, but with enough time I can generally work out what I need to do for these sort of things.
However, I don't have ANY time right now, and the success of my whole project really depends on me being able to cut out the horrendous 40min+ manual deployment wastage I currently endure.
I want to be able to get some user feedback, find a bug, or code an improvement and confidently just fire off a deploy and crack on with something else.
The extra issues with development for Azure in its current state (as opposed to dedicated servers), and the currently fairly poor tooling support from MS (I know there's lots of improvements coming, but I need something right now) - has left me swimming in a sea of "I don't know"'s & "I'm not sure"'s and tends to end in one big :
"I give up + a manual deploy for almost an hour + a little sobbing inside as my dream of deployment heaven dies once again" :(
But I know people out there more proficient, knowledgeable and experienced than myself have cracked this one for themselves. I just can’t seem to find the info I am sure is out there.
So if anyone has some good resources, tips, links, comments, or opinions on this, I'd love to hear.
Details Of My Setup :
App up and running in Azure (which is in Beta - partly due to not having the auto deploy setup), running in a Production slot, I haven't bothered with a staging slot, as some issues with subdomains / DNS / the auto generated Url has made that look painful / not feasible.
Azure / App :
App is
1 Web Role
ASP.NET 4
MVC 2
EF 4
SQL Azure
Azure Blob Storage
1 Worker Role - this runs some scheduled tasks, and works with same DB and Blob Storage
SQL Azure
Azure Blob Storage
The 2 Roles communicate with the Azure queue system (or will do shortly)
Locally :
Datacenter 08 (DC) + Hyper V
- VM for TFS 2010
- VM for a Linux firewall
Dev Box 1 (Win 7)
- VS 2010 / VS 08
- SQL 08 R2 / 05
Dev Laptop 2
- as above.
I tend to run these together all the time (so I never need to stop to wait for anything) with the FANTASTIC free tool Synergy to bind Keyboard, Mouse, Clipboard together.
Some Of The Stuff I've Read :
I have read what I have found and some of its great stuff, so I am also posting these links here to help other's struggling with this stuff, but none of it quite seems to do the trick, or maybe I don't get the trick maybe I'm missing something ?
http://deploytoazure.codeplex.com/
How do I manage and publish a database with my MVC2 application on Azure?
How can I automate the "generate scripts" task in SQL Server Management Studio 2008?
http://www.koltovich.com/blog/DeployingAzureProjectFromTFS2010BuildServer.aspx
http://msdn.microsoft.com/en-us/library/ff803365.aspx
http://msdn.microsoft.com/en-us/library/gg432988.aspx
http://www.jimzimmerman.com/blog/2010/03/16/Deploying+An+Azure+Project+Using+TFS+2010.aspx
http://archive.msdn.microsoft.com/azurecmdlets
http://selfpacedazure.web.officelive.com/Documents/Windows%20Azure%20Platform%20Articles%20from%20the%20Trenches.pdf
http://msdn.microsoft.com/en-us/library/gg651132.aspx
http://social.technet.microsoft.com/wiki/contents/articles/overview-of-tools-to-use-with-sql-azure.aspx
http://msdn.microsoft.com/en-us/library/ms178078.aspx
http://blog.syntaxc4.net/post/2011/05/13/Continuous-Integration-in-the-Cloud.aspx
http://blog.syntaxc4.net/post/2009/12/31/Synchronizing-a-Local-Database-with-the-Cloud-using-SQL-Azure-Sync-Framework.aspx
http://social.technet.microsoft.com/wiki/contents/articles/overview-of-tools-to-use-with-sql-azure.aspx
http://social.technet.microsoft.com/wiki/contents/articles/developing-and-deploying-with-sql-azure.aspx
http://blogs.msdn.com/b/tomholl/archive/2011/02/23/using-msbuild-to-deploy-to-multiple-windows-azure-environments.aspx
http://www.scarydba.com/2011/04/25/sql-azure-deployments/
Disclaimer / Forum Abuse Minimisation Blurb :
Like I say I am NO Sys Admin, I am NO script magician, and NO CI guru, I am a simple minded web dev, so pls pls be nice if its mindlessly easy to you, or if I stoopidly am missing the point, I don't mean to be all "Does You Haz the codes?" But I’ve basically spent 6 months dreaming that one day soon someone will post a nice clear simple blog entry with an "Idiots guide" that solves all my woes, and an hour later I am in deployment heaven again, but I am still waiting (or Googling badly), and its breaking my little Developer's heart :(
P.S. I promise that If I get a good answer here I'll do my bit for the fantastic SO community and spend at least 8 hours scouring for questions I might be able to help with and contributing back.
Great.
It seems the new SQL Server (Code Named Denali), along with the new SQL Server Developer Tools (Code Named Juneau), and specifically the 2.0 release of DAC projects may well have filled the gap between development and deployment to SQL Azure.
The new v2.0 of the DAC framework expands the set of supported objects
to full support of SQL Azure schema objects and data types across all
DAC services: extract, deploy, and upgrade
From SQL Azure Import/Export
Also see :
Bob Beauchemins Blog Post suggesting schema upgrades on SQL Azure are now
supported with the new DACImportExportCli.exe utility.
Other suggestions the new DACs 2.0 solve the major issues with upgrade deployments to SQL Azure
And looks like it all should run side by side with current setup. Will check it out and update here on progress. Brilliant.
For the database deployments I use RedGate compare which works well with Azure. There is a command line edition which can be used as part of an automated build process. Regarding keeping the site always running, you should deploy to staging and then the production site is never down. Once deployed you can switch the staging over to prod.
I have found an example of running Tomcat in Windows Azure, but JBoss seems to be a bit trickier. Has anyone succeeded in running JBoss in Windows Azure, and if so, what should I do to get it running?
I would like to know if JBoss can be run in a worker role, not the VM role which is currently in beta.
I'd recommend trying it and asking questions as you get roadblocked. Most "does this run in Azure" questions are answered with "yes", but full-fledged examples and guides are going to be sparse as the platform is very new. So be the guy who writes it :)
You have two options with something like this:
1.) Run a worker role that invokes and runs similar to a windows service. (most tomcat on azure guides approach this way)
2.) Use a Azure Startup task to install, configure, start jBoss as a windows service and/or on top of tomcat
With the addition of full-trust, startup tasks, RDP, and eventually even VMrole .. anything that can run on a Windows 2008 server can run on Azure. So try installing on a Windows box, document those steps, then 1 by 1, figure out the "azure way" to accomplish those tasks. Approached this way, the initial setup will likely be pretty simple. From there, you'll learn a lot about Azure's offerings and can probably even make some changes to better leverage the platform for JBoss needs.
Once you get up there and have issues, ask away and you're more likely to find direct answers by the audience of MVPs, Azure team members, and cloud geeks here vs just a generic "have you done this" answer.
We've run into an issue with the New-Deployment Azure Powershell commandlet timing out; we've put in a bug report with MS. While they gave us an explanation for it (the path and timeout threshold used to upload through commandlets is different then what's used by the web portal); they don't have a fix for us.
We need to get this running so we can automate our build deployments, so we're looking into developing a custom commandlet to replace New-Deployment using the Azure SDK; hoping this path will not have the timeout issues the commandlet did. But before we go down that route; are there any other scriptable tools I can use to replace the New-Deployment functionality? I looked at Cloudberry for Windows Azure; but that doesn't have a scriptable interface yet.
Any constructive input is greatly appreciated.
If you are developing worker roles, then you can reuse the dynamic assembly loading system that comes with the execution framework of Lokad.Cloud.
Basically, you just have to upload a ZIP archive containing all your DLLs toward the Blob Storage and the framework takes care of the rest. Extra Bonus: worker restart takes about 10s (vs +10min for a classical deployment)
A similar behavior could be obtained with web roles too, but this part hasn't been implemented in Lokad.Cloud yet.
FYI, we solved this issue by writing a custom tool to upload the package with a higher timeout tolerance; as well as using CSManage (http://code.msdn.microsoft.com/Release/ProjectReleases.aspx?ProjectName=windowsazuresamples&ReleaseId=3233) to create the deployment after uploading.
I am currently working on a project in Microsoft Robotics Development Studio, and I am having a little trouble deploying the solution for general use.
My problem is this. I am making a simulation, so I need to deploy the simulation engine along with my project. This is dependent on the Ageia PhysX engine, DirectX, and the Microsoft XNA framework. I understand this, and I understand that the deployment tool for MRDS (dssdeploy) isn't set up to deploy simulations by default. So, if anyone has worked with MRDS before, and has deployed a simulation, I could use some help with this.
Unfortunately, dssdeploy does not work with simulation projects. It won't package up all the required simulation libraries. I believe the reason for this is due to license and/or install issues for PhysX, DirectX, and XNA. I think you can still zip up your own service with dssdeploy, but you will have to have your target user install RDS first, then un-deploy your service.
FYI, because of this and many other headaches using dssdeploy, I have stopped using it to distribute my code. I simply zip up my service, then tell the end user to run dssprojectmigration on the code after they un-zip it.