Here is the story, I am using RedGate SqlCompare to generate update scripts for my Dev env, each package contains only changes from current Dev version to Latest in source control.
Here is an example:
I create a table (package-0.1) -> Deploy to DevDB
I add Columns (package-0.2) -> Deploy to DevDB
I renamed some Column (package-0.3) -> Deploy to DevDB
But once I want to promote it to QA it causes me problem because it promotes only latest package-0.3 that contains only part of the changes (renaming of the column)
So I am looking for a way to deploy all the packages prior to current on Promotion if it is possible.
By now I solved that by creating custom package that contains all the change scripts, but is it possible to solve that with Octopus?
Thanks
Ihor
each package contains only changes from current Dev version to Latest
The way you do it is going to be painful for you as SQL Compare takes a state based approach. What you want to apply is the migrations based approach. You can see Alex's post on the difference between two approaches.
SQL Source Control 5 will come with a better migrations approach which will work with SQL Compare command line tool and DLM Automation tools. However, beta is closed right now unfortunately but I suggest you to contact the team through the e-mail address provided there.
The other option you have is ReadyRoll which has the pure migrations based approach. You can see this post on its octopus deploy integration.
Related
We are using Visual Studio Team Services.
We have a Prod-Branch which is builded by our Prod-Build-Definition and deployed by our Prod-Release-Definition to our Test / Integration and Production Environments.
With each Prod-Release deployed to the customer, we create a Prod-Rel-Version-x.x.x Branch from the Prod-Branch (in Case we need that for a Hotfix).
During the Sprint we are developping on a Dev-Branch which is builded by our Dev-Build-Definition and deployed by our Dev-Release-Definition to our DEV Environment for Developer Tests.
After the Sprint (or from time to time) the Dev-Branch is merged to the Main-Branch and then to the Prod-Branch. From there it is deployed to the different Stages for Testing by the customer.
When there is a Hotfix-Case, we fix the bug on the Prod-Rel-Version-x.x.x Branch and would like to reuse our existing Prod-Build-Definition to build this Hotfix-Version and deploy to the different stages by the existing Prod-Release-Definition for testing and going live with this version.
How can we reuse our Prod-Build-Definition with this different Branch (Prod-Rel-Version-x.x.x Branch instead of the Prod-Branch)?
When I look at the build definition, I think i would be possible, just be editing the Server Path (Repository > Mappings) from $/NameOfOurApp/Prod to $/NameOfOurApp/Prod-Rel-Version-x.x.x)...that should do the trick or not? But from what I read, it's not possible to use Build-Variables in Server Mappings, so I cannot change this variable for example in the Queue new Build Dialog...
What's the best way to accomplish my scenario?
The only way to do this is to create a single build definition which downloads all the branches. Then use variables in the tasks to select the version to build. This will become very messy (and slow) very fast.
Instead it's much easier to clone the build definition. Alternatively you can create a Build Definition Template from an existing build definition and use that to create a new Build Definition.
A much, much better solution however, is not to rely on so many branches.You only need the branch when you really need to make a hotfix, and you only need the stages branches when you have a lot of findings in higher stages. By improving teh way you work, you'll be able to get rid of the branches, simplifying the work for all.
Update
VSTS and TFS 2018 now support the use of variables in the workspace definition.
We are using RedGate combined with SQL Test (tSQLt). In order to unit test, we install the framework on each database.
Is there a way to use the tSQLt framework in such a way where your unit tests and framework objects can reside in one central location which can then be used by multiple databases?
We are also using RedGate's SQL Source Control with TFS as our repository to track schema changes. These changes get promoted in the following environment order: Development --> Test --> Production.
Needless to say, the addition of the framework combined with the tests themselves represent large amount of new SQL objects (tables, stored procedures, etc) now in our databases. Ideally we would like these objects to reside only in Development and Test and avoid cluttering our production database. We could skip merging the tSQLt changes to Production, but then we would have unmerged changes sitting around in the Test environment's source control until the end of time.
Any thoughts on getting around this problem?
As you're using SQL Source Control to manage your database changes, checking in your tSQLt tests is the right thing to do. If you want to ensure that these don't get pushed to staging or production, you need to ensure that the tools you use to push the changes exclude the tSQLt tests. If you are using Redgate SQL Compare for this, use the option "Ignore tSQLt framework and tests". See the product documentation for a detailed explanation. If you are using a different tool or process, post a comment and I'll amend this answer.
There is currently no way to install tSQLt in a separate database. I have started the process of making tSQLt database agnostic, but that is basically a complete rewrite, so it will take a while.
In the meantime, you can exclude tSQLt from SQL Source Control: https://redgate.uservoice.com/forums/39019-sql-source-control/suggestions/4901910-faster-way-to-exclude-all-tsqlt-content
If you still want your tests in source control but don't want to promote them to the higher environments, that is the default behaviour in Redgate's DLM Automation Suite. You can either use one of the build server plugins (like TeamCity or TFS for build/test then Octopus Deploy for release) or do it all in PowerShell using SQL Release. https://documentation.red-gate.com/display/SR1/SQL+Release+documentation
If you have a license for Redgate's SQL Toolbelt, you might already be licensed for the Automation tools (this is a change to previous licensing); http://www.red-gate.com/products/sql-development/sql-toolbelt/#automation
I am using Sitecore 6.6.0, we have multiple environments
Local
DEV
QA
PROD
I have to deploy few changes directly from Local to Prod (Don't ask me why directly to PROD, even if it is for QA, my question remains same), what I am doing is create a package on my local with all items and separately create folder structure for all files related to the fix an deploy that to PROD.
There is always a chance of human error, since I will have to remember all associated items and files for a fix, so is there a better automated way, which will not skip any changed Items or Files?
On the other note I am using Bit-bucket for source controlling sitecore code what about sitecore DBs? most of the sitecore developments stays in DBs. What is the best approach to source control sitecore DBs?
Update
Installed packages from nuget
After installing Unicorn from nuget and unicorn.default.config, I get the following error
Attempt by method 'Unicorn.Data.DataProvider.UnicornDataProvider..ctor(Unicorn.Data.ITargetDataStore, Unicorn.Data.ISourceDataStore, Unicorn.Predicates.IPredicate, Rainbow.Filtering.IFieldFilter, Unicorn.Data.DataProvider.IUnicornDataProviderLogger, Unicorn.Data.DataProvider.IUnicornDataProviderConfiguration, Unicorn.Predicates.PredicateRootPathResolver)' to access method 'System.Action`1<System.__Canon>..ctor(System.Object, IntPtr)' failed.
Further after following the ReadMe on Github
When I do a sync on site/unicorn.aspx.
[P] Auto-publishing of synced items is beginning.
ERROR: Method not found: 'Sitecore.Publishing.Pipelines.Publish.PublishResult Sitecore.Publishing.Publisher.PublishWithResult()'. (System.MissingMethodException)
at Unicorn.Publishing.ManualPublishQueueHandler.PublishQueuedItems(Item triggerItem, Database[] targets, IProgressStatus progress)
at Unicorn.Pipelines.UnicornSyncEnd.TriggerAutoPublishSyncedItems.Process(UnicornSyncEndPipelineArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Unicorn.ControlPanel.SyncConsole.Process(IProgressStatus progress)
Solution:
For older sitecore versions (pre 7.2 iirc) you need to disable the auto
publish config file as it relies on a method added later by sitecore.
https://github.com/kamsar/Unicorn/issues/103
In order to track the database changes you are making, you will first need to install software that will be able to help you serialize your changes and store in source control. Team Development for Sitecore (TDS) and Unicorn are the two most popular options.
You will also want to make sure you have your own local database where you are making your changes so you can isolate those changes from your QA, PROD, etc. allowing you to maintain the same level of isolation you do for developing code.
Automation of this process helps reduce the human error you mention for the deployment by introducing a repeatable and known process. Here are a few blogs that can help you get started:
Jason Bert - Continuous Deployment (Git/TDS/TeamCity)
Jason St-Cyr - Automating with TeamCity and TFS (TFS/TDS/Team Build)
Andrew Lansdowne - Auto deploy Sitecore Items using Unicorn and TeamCity (Unicorn/TeamCity)
Brian Beckham - TDS and Build Configurations
You may also want to look into configuration transforms to support different values in your Sitecore Include patch files. SlowCheetah plugin will let create the transforms in Visual Studio (it might be in Visual Studio 2015 now...). TDS can pick up those transforms automatically and execute them on the build server for you, or you can do it with Visual Studio itself to create published packages.
For Sitecore versioning and deployment Unicorn is also a good option.
https://github.com/kamsar/Unicorn
Cheers,
Bo
Trying to make my life easier, Currently we have 4 developers working in Visual Studio 2012 and we are using TFS 2012 for source control. The project we work on is a multi-tenant web application (single source directory with multiple dbs) that is a mixture of legacy, asp and vb6 com components, coupled with new C# code. We use TFS for source control and for managing User Stories and Bugs. Because of the way our site works it can not be ran or debugged locally only on the server.
Source Control is currently setup with a separate branch for each developer that's working directory is mapped to a shared network path on the dev server that has a web site pointed to it in IIS. Dev01-Dev05 etc. The developers work on projects in their branch test it using their dev website, then check in changes to their own branch and merge those into the trunk. The trunk's work space is mapped to the main dev website so that the developers can test their changes against the other customer's dev domains to test against customizations and variances in functionality based on the specific dbs the are connected to.
Very long explanation but basically each dev has a branch and a site, that are then merged into the trunk with its own site.
In order to deploy our staging server:
I compile the trunk's website via a bat file on the server
Run a windows app I built to query TFS for changesets associated with
specific WorkItems in a certain status, and copy all the files for
those changesets from the publish folder to a deployment folder.
Run another bat file on the server to use RedGate's Deployment Manager
to create a package from those new files
Go to the DM site on our network to create and deploy that release (haven't been able to get the command line tools to work for this, so I have to do it manually)
Run any SQL scripts that have been saved off in Folders that match ticket numbers on each database (10 or so customer dbs) to support the release
I have tried using TFS automated build stuff and never really got it to build the website correctly. Played around with Cruise Control also with little success. Using a mishmash of skunk works projects to do this is very time consuming and unreliable at best.
My perfect scenario would be:
Gated Checkin, Attempt build/publish every time a developer merges into the trunk, rejects and notifies developer if the build fails.
End of the day collect the TFS Items of a certain status and deploys files associated with them to the staging site
Deploy SQL scripts for those TFS items across all the customer dbs in staging
Eventually* run automated regression UI tests, create new WorkItems or emails to devs if failed
Update TFS WorkItems to new state so QA/Customers know their items are ready to test in our staging environment
Send report of what items were deployed successfully
How can I get here so that I am not spending hours preparing and deploying releases to staging and eventually production? Pretty open to potential solutions, things that would be hard to change would be the source control we are using, can't really switch to subversion or something else so we are pretty stuck with TFS.
Thanks
Went back in and started trying to get TFS to build/publish my web solution. I was able to get a build to complete successfully. adding msbuild argument /p:DeployOnBuild=True and setting the msbuild platform to x86 seemed to do the trick on that.
Then I found https://github.com/red-gate/deployment-manager-tfs which gives you a build process template to do the package and deployment using the redgate tools. After playing with that for a bit I finally got it to create, package and deploy my build to our staging environment.
Next up will be to modify the template to run some custom scripts to collect only the correct items to deploy, deploy all the sql files and then to set the workitems to the appropriate statuses after completion.
Really detailed description of your process. Thanks for sharing!
I believe you can set up TFS to have gated check-in on a single branch, which if you can setup on trunk would make sure that the merges built successfully. That could trigger msbuild, if you can get that working or a custom build job.
If you can get that working then you'd be able to use that trunk code as the artifact to send to Deployment Manager. That avoids having to assemble the files for deployment through the TFS change sets, as you'd be confident that the trunk could always build.
Are you using Deployment Manager to deploy the database from source control as well as the application?
That could be a way to further automate the process. SQL Source Control and SQL CI allow you to source control the structure of a database, keep a database up to date on each check-in, and run database unit tests. They also produce database packages for Deployment Manager, so you can deploy a release that contains both the application and the database.
If you want to send me the command you're using in step 4 to deploy the release using Deployment Manager I can help out with that. The commands I use are:
DeploymentManager.exe --create-release --server=http://localhost:81 --project="Project Name" --apiKey=XXXXXXXXXXX--version=1.1
DeploymentManager.exe --deploy-release --server=http://localhost:81 --project="Project Name" --apiKey=XXXXXXXXXXX--version=1.1 --deployto=CI-Environment-Name
That will create a release version 1.1 using the latest available packages for that project. You can optionally specify the package to be used when creating the release with
--packageversion=<package name>=<version>
--packageversion="application=1.5
I've spent the past few months developing a webApi solution that I'm ready to push up to Azure and hook into an Azure SQL Database. It was built with EF Code First.
I'm wondering what standard approaches there are to making changes to the database while in production. I've been using database initializers up to this point but they all blow away data and re-seed.
I have a feeling this question is too broad for a concise answer, so I'd like to ask: what terminology / processes / resources should a developer look into when designing a continuous integration workflow for a solution built with EF Code First and ASP.NET WebAPI, hosted as an Azure Service and hooked up to Azure SQL?
On the subject of database migration, there was an interesting article on ASP.NET about this subject: Strategies for Database Development and Deployment.
Also since you are using EF Code First you will be able to use Code First Migrations here for database changes. This will allow you to better manage the changes you make to the database.
I'm not sure how far you want to go with continuous integration but since you are using Azure it might be worth it to have a look at Continuous delivery to Windows Azure by using Team Foundation Service. Although it relies on TFS in the cloud it's of course also possible to configure it with for example Jenkins. However this does require a bit more work.
I use this technic:
1- Create a clone database for your development environment if it doesn't exist.
2- Make the necessary changes in your dev environment and dev
database.
3- Deploy to your staging environment.
4- If you added some static datas
that should also exist in your prod database, use a tool like
SQLDataExaminer to find the data differences and execute the
insert, update, deletes for according rows. Use Schema Compare in VS2012 to find differences between your dev
and prod environment by selecting source as dev and target as prod.
And execute the script in your prod.
5- Swap the environments