For example when a developer makes changes in any of the database elements in a business critical database it should force them to commit the code before applying the changes to database itself. I came across Redgate sql source control which matches my expectation somehow. Still do we have any more tools or effective database practices that I am missing here?
If you use SQL Source Control or a tool like it (eg, ReadyRoll or VS Database Projects) I'd recommend also using DLM Dashboard.
The reason for this is that no tool can enforce changes to go through a process if people are given (too many) rights and are able to apply changes to production. It's then up to these people to correctly follow the process.
Although DLM Dashboard doesn't enforce changes to your database, it will alert you on changes made to production, warning you when out-of-process changes (aka "drift").
DLM Dashboard is free, which is another reason to use it!
Related
I'm leading a small software development team (4 people), and have just broken ground on a source-controlled SQL Server 2008 database project, with isolated development databases for each developer. I'm still implementing this one step at a time, but I'm envisioning each developer having their own database, with a naming scheme something like <ProjectName>_DEVELOPMENT_<TFSUserName>. This was all recommended per the MSDN articles I've been reading, but someone let me know if that sounds way off.
Anyway, we have a shared application solution that we've been developing for some time. In the past, we had no database version control, and just modified our database directly from SQL Server Management Studio when new reference data needed to be populated, or when we were testing functionality -- one change immediately affected everyone else. So with this new change, I'm wondering what the best way would be to have each person connect to their isolated development databases from the application solution. Prior to isolated databases, our connection to the database was specified in our application's web.config as a connection string. If we're each going to have our own database, the only way I can see it working is for each developer to set their connection string in their local solution to point to their personal database. But changing the web.config will check out that file in the solution, so developers will always have to specifically uncheck that file when checking in application changes to the baseline. Is there a less clunky way for each developer to use their isolated database when doing application testing?
I recommend that you not make the database names username-specific. Instead make the database the same name for each developer and always reference it via localhost (localhost\<ProjectName>_DEVELOPMENT). Then the same connection string will work for every developer.
MSDN's suggestion to use username-specific databases is better for a shared development environment. It's definitely not ideal for a localized environment.
Hi i have configured the basics of cruise control to make releases, and automated nunit test using just MSBuild. Now i'm wondering if is possible to deploy/versioning databases with this?
I'm a beginner at CCNet .So if is possible some suggestions or tutorials (if there are) . Also if someone knows a free tool for database deployment/versioning let me know.. i will be grateful.
Thanks in advance
Hugh
It isn't free but SQL Source Control from RedGate can do what you're looking for, assuming it's a SQL Server database. It has a commandline interface that you can use in CCNet tasks. The easy approach of just migrating up is... easy, the changes are applied to your database schema / data. There was an issue with v2x of the tool that they've overcome with 3, which is that if you were to rename a table column then it would delete the column and create a new one with the right name. Obviously that's quite a big problem if you've got data you want to keep, so with v3 there's the concept of migrations and this allows you to specify alter scripts so instead of dropping the column you could script the change non-destructively.
As far as I know, at this time, they don't have anything that allows you to roll back your version.
Otherwise you could take a look at database migration tools, there seemed to be some promise for these in .Net at least. There is also this post that has some other tools (again for .net) and then there's this https://stackoverflow.com/search?q=database+migration+tool which is not restricted to any language but is general database migrations
If you're still looking for ways to version and migrate databases, one such tool is dbdeploy.net . I've hosted it on github after forking it and doing some work. Latest version is fully up to date and has some interesting features (done by someone who also uses it and sent a pull request).
This is a question for those of you developing on a team of devs where all of you have separate databases. You're versioning your database using source control and other tools which will automatically bring dev databases up to date to the latest version of the database (schema, data, SP's, functions, etc.).
OK Great! But wait! What if you are developing on version 4.0 of your software, but now you need to switch branches to the 3.2 branch to fix a bug? The schema could be (almost assuredly is) very different by now...
I suppose if you went through the extra effort to write rollback scripts along with your change scripts, this could work. But that seems like a lot of work - is it really worth it?
Much easier would be to create a new 3.2-branch database and work with that while working on the 3.2-branch code. It doesn't seem reasonable to me to require that each developer has exactly one database to work with.
I'm going on a limb and assume that you are versioning the database as a binary? If all your database assets were in the form of constructive code (eg SQL scripts and/or text data dumps), the solution would be simple, as suggested by Mark: store these assets as part of the development branch. To work on version 3.2, switch the branch, re-run the create scripts and presto, 3.2 database. Merging would be just as easy as with regular code (or just as painful, depending on your version control system of choice).
Here are some suggestions to work in this mode:
If creating the database instances from text is too slow, make a cache on a shared disk volume, keyed by the contents of all the schema / data files (or the MD5 sum thereof).
Write a pre-commit hook to ensure that the schema and data dumps in the developer's instance are the same as the ones under version control. This prevents people from making changes to their dev database with an interactive tool, and then forgetting to commit them.
You mention change scripts; treat them as a liability. While they may be required by your deployment scenario (eg for customers who want to upgrade in-place), they duplicate information from the version history of the database, and per Murphy's law duplication means desynchronization sooner or later. Try to auto-generate the change scripts from the versioned database assets using "diff"; or if this cannot be achieved, dedicate some serious unit tests to database upgrades.
I am curious if there are any solutions out there, preferably free, that can have a central database to publish data to in a versioned manner.
For example,
Client 1 decides to edit a persons profile so it gets a local copy on its machine to make changes to. When they are happy with there edit they publish the results to the central database. Just like how you would do a submit in perforce.
Client 2 tries to edit the same local copy but when they go to submit they have to resolve conflicts.
The central database must store compressed differences between versions of the data.
At any point someone can look at all versions of the data submitted.
Check out OffScale DataGrove.
This product tracks changes to the entire DB - schema and data. You can tag versions in any point in time, and return to older states of the DB with a simple command. It also allows you to create virtual, separate, copies of the same database so each team member can have his own separate DB. All the virtual copies are tracked into the same repository so it's super-easy to revert your DB to someone else's version (you simply check-out their version, just like you do with your source control). This means all your DBs can always be synchronized.
Disclaimer - I work at OffScale :-)
"Version control of databases" is a bit ambiguous for a title, because you are actually asking for a VCS using a database as repository "data store".
Subversion has such a model (either Berkeley DB or filesystem-based).
It also has a Copy-Modify-Merge model which is similar to the kind of locking mechanism you are describing.
(source: red-bean.com)
(source: red-bean.com)
The sql tools from redgate sort of offer some of this functionality, but not implemented in a way you describe. For example, sql data compare can compare the differences between data in 2 databases, and sql source control can be used as well.
However, getting a copy of the database on a local machine, making changes and resubmitting would be more of a manual process.
What database server are you using? If you are using MySQL and PHP, Doctrine has 'Versionable' behavior which can be applied to a model.
The documentation on this behavior is here:
http://www.doctrine-project.org/projects/orm/1.2/docs/manual/behaviors/en#core-behaviors:versionable
This is exactly what my product (yes I'm biased :)) DBmaestro Teamwork does.
It enforces and keep track on the changes of structure and content
It prevents two parallel changes on an object structure or content by two (as long they work on the same object - meaning, same database, same schema, ...)
It uses a baseline aware analysis which understand the nature of the change and knows if the change should be promoted or should be ignored (as it was made from another environment) or if there is a conflict
And much more…
I would encourage you to read a comprehensive, unbiased review on Database Enforced Management Solution by veteran Database expert Ben Taylor which he posted on LinkedIn https://www.linkedin.com/pulse/article/20140907002729-287832-solve-database-change-mangement-with-dbmaestro
Around here we have been working with a bunch of Visual Source Safe repositories for about 10 years or so.
Now I want to get rid of sourcesafe and move on to Team Foundation Server.
Do you have any tips or tricks for me before I embark on this migration? What are the things I have to be careful about?
I am sure this migration will mean that our working habits have to be modified in some way. Do you think that these changes could be a problem for the organization? Think about a group of about 20 .NET developers in a single site.
There are a few different ways you can migrate. The tool will pull your history, etc. over, but the more pragmatic and simple way is to lock VSS as a history archive and start fresh:
Have everyone check in all changes into VSS, make sure everything builds, etc.
Set all VSS databases to "locked" (read-only rights for all users)
Get Latest on the entire VSS database into a "clean" set of folders on a workstation
Check all of the files into TFS from the workstation
For any history prior to the conversion, folks need to go to VSS, but after a week or two it's realistically unlikely to happen all that often. And you know that the history in VSS is accurate and not corrupted by the conversion process.
Be aware that TFS does not support sharing files between different projects, as VSS does. If you have any such shared files the link between them will be broken during the migration, resulting in initially identical, but now distinct files in each project. Updates to one of these files in TFS will no longer propagate to the copies in the other projects.
If you do choose to use the VSSConverter.exe tool that ships with Visual Studio Team Foundation Server, then you should install TFS 2008 SP1 first as it includes a number of improvements as detailed on this blog by the migration tools team.
Some of the key features of the
release include:
Elimination of namespace conflicts. I
previously blogged about this as "the
rename problem" and we have fixed the
converter to correctly migrate files
with overlapping namespaces. This was
the biggest pain point for most users
trying to use previous versions of the
tool.
Automatic solution rebinding.
In this latest version, VS solution
files will be automatically upgraded
to the 9.0 version and checked back in
to version control. Previously users
were required to do this manually.
Correcting of timestamp
inconsistencies. The use of client
timestamps by VSS can lead to
revisions being recorded in the
opposite order that they actually
occurred in. The tool now recognizes
this issue and continues migrating
changes where it would previously
fail.
Improved logging. Although
we've fixed a lot of issues, providing
better, more detailed logging will
help users that do run into issues
diagnose the problems.
I just googled, but this walkthrough seems like a good reference, and it mentions the tool VSSConverter which should help you make the migration as painless as possible.
I would like to recommend one thing though: Backup. Backup everything before you do this. Should anything go wrong it's better to be safe than sorry.
My links aren't showing up. This is the address: http://msdn.microsoft.com/en-us/library/ms181247(VS.80).aspx
We are currently in the process of doing this at my day job. We are actually making the switch over in about a month. I am a main part of the migration and a big part of why we are getting off of SourceSafe. To help in the migration, I used the Visual Studio® Team System 2008 Team Foundation Server and Team Suite VPC Image. It was very useful. Right off the bat, the image contains a full working TFS installation for you to play and demo with. It also includes Hands on Labs and one of the labs is running the VSS -> TFS migration tool. If you have an MSDN subscription, once you have played with the image, the next step would be to install the TFS Small Team edition that comes with your subscription.
One thing to note is to make sure you get the latest Service Packs for Visual Studio 2008 and the .NET Framework installed on the image. The service packs fixed some annoying bugs and it definately increased the usability of the system. We have a farely large SourceSafe database with about 90+ projects and the migration tool took about 32 hours to complete. First I made a backup of our sourcesafe database for testing. Then I made the migration on the test sourcesafe database. Afterwards, I checked the source tree in TFS and everything transferred fine. We kept all the history for our source files from VSS which was great. No need to keep that stinking VSS database around after we go live.
We are taking the migration in steps. First the source control and letting our developers get use to using it. Then after that we will be migrating the QA and Business Analysts over to use the Work Item tracking features.
My advice is to take the migration in steps. Don't do too much at one time. Give time for people who will be using the system to train up.
VSS Converter is a far from perfect solution. And there are significant differences between the 2005 and the 2008SP1 version of the converter.
For example, in a VSS DB that's been in use for a long time, there will have been a large number of users contributing to VSS. Many of these users will have left the organisation a long time ago and therefore will no longer have domain accounts. TFS requires mapping VSS users to domain accounts, so you will have to decide whether you map old users to a single 'dummy' domain account or to a current team member.
In addition, VSS Converter 2008 requires these domain accounts to be valid TFS accounts. Whereas the 2005 converter does not enforce this.
If your VSS history contains significant folder Moves, then it's likely you will loose all history before this Move. For example, if you Move a folder to a new location, then Delete the previous parent, you will loose all history. See this article for more explanation:
http://msdn.microsoft.com/en-us/library/ms253166.aspx
In one migration I was involved with, we had a 10 year old VSS database that lost all history prior to 6 months ago. This was due to a significant tidy up that took place 6 months ago.
TFS conversion tool <-- Use this
I've used this tool for some times already, the results are pretty satisfatory as it comes with the history of changesets from SourceSafe if you desire too.
Anyway, using this tool you should always pay attention to errors and warnings in the log, and check if everything built okay / passed okay.
It's recomended to also run an Analysis on SS before running this.
Hope it helps
Good guidance there from my former colleage Guy Starbuck. Another thing to add with that approach - you may have decided over time that you want to refactor the way your application is organized (folders etc) and this will give you an oppurtunity to do so.
I've been in situations where we organized a solution haphazardly without thought (let alone major changes in the application) which led to a desire to organize things differently - and the move from VSS to TFS is a great oppurtunity to do so.
As far as the original question:
And: this migration will for sure mean that our working habits have to be modified in some way. Do you think that this changes could be a problem for the organization? Think to a group of about 20 .net developers, in a single site
I would say - yes your working habits will change but much more for the better.
You no should use "Check-out" Locks and "Get-Latest on Check-out".
You can now effectively Branch and Merge
You will now have "Changesets" all files checked-in at the same time will be grouped together. This makes historical change tracking much easier - but more importantly - rollbacks are much easier (ie find all files checked in at the same time and roll them back)
Associating Check-ins to Work Items. Don't overlook Work Items! The biggest mistake you can make is to only use TFS as a VSS replacement. The Build and Project Management features are excellent - you paid for them - USE THEM!
As far as details on how your experience will change, another former colleague of mine (and Team System MVP) Steve St. Jean wrote a detailed article on the differences: From VSS to TFS