I look after the merging, deployment and release of Products & Services in the Company I work for. I've slowly moved to this position from development so a lot of this is new to me (I guess!)
We have a deployment process, but no real Release procedure other than telling stakeholders and members of staff about these new services/features/bug fixes shortly before release.
I've heard things about ITIL Release Management, CMDB, versioning and other mumbo jumbo, but are they actually needed or am I going to end up being weighted down in by a load of crap.
The question I guess I'm trying to ask is: what's my first step? Am I making a mountain out of a molehill? How do I shape this department?
Steve
In my opinion following ITIL practices or implementing a CMDB are not necessary but best practices. The most important thing, and first step, in your case, is developing or documented sound processes about what you do. For documentation you can use "programmer friendly" tools like a wiki (MediaWiki, TikiWiki), but if you do not document your practices it is very difficult to implement continual improvement.
If you have already implemented and documented your processes you can study standards and best practices related to your department. ITIL and ISO 20000-1 are standards focused on the quality of services you offer. Services, not ongoing operations. ITIL can provide you with some useful (but not necessary) good practices like implementing a CDMB. If you implement a CMDB correctly your department will have in a database the configuration of the assets you use to provide your services. You will be able to store the configuration of the systems on your clients or what you want. The CMDB can associate its elements with incidents or known errors so the support department could provide the best service to your clients.
CMMI or CRUM are other standards/frameworks that will probably interest you.
About versioning, I think some sort of versioning is a must. GIT or Subversion are good options.
Other tools very interesting in my opinion is some kind of continuous integration, like Jenkins and some ticketing system like Trac or Mantis.
Related
Normally all our upgrades, feature or bug-fix, are incremental. This seems obvious because even to develop a bug-fix we take the latest code-base of our product. However, this leads to a requirement that our customers have to update the software to the latest version even to get a bug-fix, they get the features as well as bug-fixes.
Recently, customers are telling us that they want the ability to get bug-fix patches without having to use new feature enhancements. Basically they want the bug to be fixed but they don't want new features. This is a dilemma for us. This is possible if the bug-fix changes a different set of binaries than the features. However, many times the features and bug-fixes are in the same binaries.
Is it even possible to create an upgrade that can be applied on any software version independently? Do others do this? Are Microsoft or Office KB patches ever non-incremental? Any guidance or links to relevant articles are highly appreciated. Thank you.
What the customer wants is normal but that does not mean that this is an easy problem.
What you describe is at the heart of configuration management. SCM is not just using a tool like git, it is using such a tool to solve the needs present. There is no silver bullet, to meet configuration requests take a lot of work and preparation of your code. I see that you have not tagged this as configuration management; this is exactly what you need to research if you have not.
Back to your specific request: The customer wants to be able to release bug fixes without the features you are continually working on. For the purposes of this answer I will assume your business model is that you offer a product which is a living thing and that several customers 'subscribe' to.
Branching strategy based on customer
Customer X, Y, and Z all gets different branches. Along with a developing branch from which you can pull features/bug fixes to the customers that want them.
This solution I would say is often very challenging (SCM generally is), you have to deal with diverging codebases and the developers have to be very aware of your work-flow.
Release branches
Having a branch for every released version which you can patch with bug fixes while not bringing in the new features is what I suggest. You're question is a bit unclear what you mean by incremental, if this somehow refer to the distribution of the patches itself then that is another matter and this answer will be less useful.
Feature-flags
You deliver new features to the customer which do not want them but they are hidden behind logic in your application. This can be hard to combine with what you describe as 'complete revamp'. This can also have other advantages as it makes it easy to perform A-B testing.
It places requirements on your architecture and also discipline when it comes to what to allow, too many feature flags for every little thing and it can get un-maintainable.
Have a strategy
You need to plan for this or you can not satisfy your customers needs. And it may be very well that if you have customers who wants the wrong things you have to negotiate. But providing bug-fixes without new features I think is the minimum
The books I read on this is old, but here is one i remember anyway: Wayne A Babich: Software Configuration Management - Coordination for Team Productivity
Release Strategies on the Rene Moser blog may be helpful.
The LTS (Long Time Supported Versions) section seems applicable to your concerns.
Regarding your questions:
Is it even possible to create an upgrade that can be applied on any
software version independently? - Sure, but it takes extra time, effort and money (usually).
Do others do this? - Yes.
Are Microsoft or Office KB patches ever non-incremental? - Yes, for major releases; but not KB patches.
A follow up question: is it worthwhile to do the extra work to facilitate non-incremental software releases for your software users?
The answer to this questions depends on your customer base, which may require a long meeting with the software developers and those in Marketing. If the customers wanting non-incremental releases are significant in terms of their numbers and revenue, then the extra work is probably necessary to keep those customers.
We are a small Microsoft shop with 4 developers. We like the idea to have everything integrated and under the same SQL database but not to deal with too many features or a complexity we don't need.
The other option would be to use two distinct third party tools. We also wonder if VS2010 version control and bug/issue tracking capabilities are comparable with the best third party tools on the market.
With TFS 2010, there is a TFS Basic version/configuration options. You can use SQL Server Express as the database, the install is simple and quick and you can install it on a client OS. It includes version system, work items and build system (no sharepoint, reporting...). Price should not be high, the target users for TFS Basic are those used to use SourceSafe. Also if you have MSDN, there is a change that it goes with it, so you would not have to pay extra.
Some thoughts on TFVC in general -- note that I've never used TFVC specifically, but I've been in a similar situation a couple times. My main concern is that it's too little.
TFVC (Team Foundation Version Control) appears to be a client-server version control system. I don't know anybody who hasn't upgraded to DVCS yet. I've never used TFVC but I can't imagine what benefits it'd offer to outweigh the architectural disadvantage. (And before you ask: I only use it from my workstation in the office, where the network has never gone down, but I still use its distributed features every day.)
I also work at a small Microsoft shop with 4 developers and we have never once regretted using Mercurial. It's one of the few decisions we made that everybody seems to have loved. It's one of those moves, like switching to a language with GC, that you never want to even think about reversing.
In terms of support, I hope you found some way to get great support from Microsoft. Things will come up with any VCS and it looks like community support is a couple orders of magnitude worse than Hg or Git.
I can't say much about bug trackers -- I think they're all pretty much the same these days. I've installed a couple open-source ones in an afternoon, even with no experience. The major difference seems to be, if you go with a big-name one, you'll be able to find lots of tools and extensions that work with it. For example, there are a million and one extensions for reporting/testing/etc. for Bugzilla. TFS probably has similar things, for enough money.
Two other things I'd keep in mind:
First, even if you only want these 2 features today, you will want other features in the future, and it will be (sooner or later) something Microsoft doesn't offer. So it's best to make peace with 3rd-party tools ASAP.
Second, unless you miraculously happen to pick the perfect set of tools for your company's future growth for all time, you will at some point want to migrate away from whatever solution you pick today. So make sure it either provides a way to do a full export, or is popular enough that other projects are falling over themselves to write importers for it.
I guess this all sounds kind of negative for TFS. I didn't really mean it that way -- I'm sure it does some things really well. But unless you're rolling in dough already, save your money.
Good luck with whatever you choose!
TFS work ittm tracking is awesome. Whatever your workflow is and the information you want to gather, you can do it easily with TFS.
Most expect TFS to do what they do out of the box as if it were omniscient. Unfortunately, you do things slightly differently than me and I from everyone else. The key is making it easy for the many different roles in your organization to get and input information into whatever your work tracking system is.
There is a plateau you will reach with custom, cobbled together tools that you won't get from cross referencing information along the product development process. TFS removes that barrier. The second barrier is knowing how to consume the information and that takes training and experience in effectively managing projects.
TFS Version control is certainly the low point when compared to the industry right now. DVCS popularity exploded about the time TFS came out. That said, Brian Harry has stated that TFS "fully expect that we will be adding distributed version control to TFS." http://blogs.msdn.com/bharry/archive/2010/01/27/codeplex-now-supports-mercurial.aspx
Take that as you will.
TFS is a great system but after having deployed and used it for some time I realize that the TCO is just too high to justify for a small team.
I will start with a little background for perspective to the question, what legitimate questions can I pose in order to identify the correct source control tool for the real needs of my development teams?
The developers have used ClearCase since an IBM consulting lead project team installed it for use during a large in-house project. During this project the Rational Suite was used to perform tasks, testing, etc. Once this project was completed, only ClearCase was used as a method of storing source code. Unfortunately, at this time ClearCase is used by the developers as a mandatory tool. Until a global project was started at the beginning of 2009, other colleagues abroad used primarily SVN. The global project now requires these developers to use ClearCase Remote Client (CCRC).
I have observed the tool, though very powerful, appears too much for their development needs. The UCM process is implemented on-site, but there is much expressed frustration with the process steps and particularly when using CCRC. The other issue is other development tools require files to be writeable in order to complete development tasks, causing developers to hijack in order to work.
As many companies are now outsourcing and reevaluating all their strategies in order to remain competitive and financially viable, I would like to verify we have the right tool for the right reasons and one that will support not only development activities and needs today, but also for the future. I look forward to gaining a better understanding of this issue.
what legitimate questions can I pose in order to identify the correct source control tool for the real needs of my development teams?
IMO, some of these questions include:
What is the status quo?
Why is the status quo? [Perhaps political: an IBM consulting lead project]
Why might we want to change (what are the incentives)?
What are the alternatives that we could change to?
What would we gain, and what would be lose, by changing to an alternative?
What would be the cost (or effort) associated with changing it?
You can have a better understanding of the potential advantages for the current situation (ClearCase) by reading "What are the basic ClearCase concepts every developer should know ?"
Since CCRC is used to access ClearCase through a RCP interface and a remote Web-based view server, that looks like a cheap alternative to DVCS (Distributed VCS).
That means a possible alternative is to use a free DVCS (Git, Mercurial, ...)
I use UCM for the last 6 years, and find it very useful for large project with multiple inter-dependent "components" (i.e. "coherent set of files") with multiple parallel development.
However, I would not recommend such a heavy methodology (UCM) for small independent projects, with simple linear development life-cycle.
So, to complete the set of questions proposed by ChrisW
What is the size of typical project managed under VCS ?
What is their typology (inter-dependent on one another or rather autonomous ?)
What is the workflow of merge followed by those projects (a simple one might indicate Subversion as a possible candidate, a more complicated one would require more serious tools for heavy merging)
What is the network typology (is a central repository always the best configuration, or a local repo with distribution/replication features would not be more adapted ?)
Cheryl, without attempting to solve your solution - CCRC is obviously an option for your remote developers, but there also exists ClearCase Multisite. If there is a process issue, perhaps providing local rather than remote access to the infrastructure would be a short-term solution, rather than moving SCM.
My organization has begun slowly repurposing itself to a less product-oriented business model and more contract-oriented business model over the last year or two. During the past year, I was shifted into the new contracting business to help put out fires and fill orders. While the year as a whole was profitable (and therefore, by at least one measure, successful, we had a couple projects that really dinged our numbers for the year back around June.
I was talking with my manager before the Christmas holiday, and he mentioned that, while he doesn't like the term "post-mortem" (I have no idea what's wrong with the term, any business folks or managers out there know?), he did want to hold a meeting sometime mid-January where the entire contract group would review the year and try to figure out what went right, what went wrong, and what initiatives we can perform to try to improve profitability.
For various reasons (I'll go into more detail if it's requested), I believe that one thing our team, and indeed the organization as a whole, would benefit from is some form of organized code-sharing. The same things get done again and again by different people and they end up getting done (and broken) in different ways. I'd like to at least establish a repository where people can grab code that performs a certain task and include (or, realistically, copy/paste) that code in their own projects.
What should I propose as a workable common source repository for a team of at least 10-12 full-time devs, plus anywhere from 5-50 (very) part time developers who are temporarily loaned to the contract group for specialized work?
The answer required some cultural information for any chance at a reasonable answer, so I'll provide it here, along with some of my thoughts on the topic:
Developers will not be forced to use this repository. The barrier to
entry must be as low as possible to
encourage participation, or it will
be ignored. Sadly, this means
that anything which requires an
additional software client to be
installed and run will likely fail.
ClickOnce deployment's about as
close as we can get, and that's awfully iffy.
We are a risk-averse, Microsoft shop. I may be able to sell open-source solutions, but they'll be looked upon with suspicion. All devs have VSS, the corporate director has declared that VSTS is not viable going forward. If it isn't too difficult a setup and the license is liberal, I could still try to ninja a VSTS server into the lab.
Some of my fellow devs care about writing quality, reliable software, some don't. I'd like to protect any shared code written by those who care from those who don't. Common configuration management practices (like checking out code while it's being worked on) are completely ignored by at least a fifth of my colleagues on the contract team.
We're better at writing processes than following them. I will pretty much have to have some form of written process to be able to sell this to my manager. I believe it will have to be lightweight, flexible, and enforced by the tools to be remotely relevant because my manager is the only person who will ever read it.
Don't assume best practices. I would very much like to include things like mandatory code reviews to enforce use of static analysis tools (FxCop, StyleCop) on common code. This raises the bar, however, because no such practices are currently performed in a consistent manner.
I will be happy to provide any additional requested information. :)
EDIT: (Responsing to questions)
Perhaps contracting isn't the correct term. We absolutely own our own code assets. A significant part of the business model on paper (though not, yet, in practice) is that we own the code/projects we write and we can re-sell them to other customers. Our projects typically take the form of adding some special functionality to one of the company's many existing software products.
From the sounds of it you have a opportunity during the "post-mortem"to present some solutions. I would create a presentation outlining your ideas and present them at this meeting. Before that I would recommend that you set up some solutions and demonstrate it during your presentation. Some things to do -
Evangelize component based programming (A good read is Programming .NET Components - Jubal Lowy). Advocate the DRY (Don't Repeat Yourself) principle of coding.
Set up a central common location in you repository for all your re-usable code libraries. This should have the reference implementation of your re-usable code library.
Make it easy for people to use your code libraries by providing project templates for common scenarios with the code libraries already baked in. This way your colleagues will have a consistent template to work from. You can leverage the VS.NET project template capabilities to this - check out the following links VSX Project System (VS.Net 2008), Code Project article on creating Project Templates
Use a build automation tool like MSBuild (which is bundled in VS2005 and up) to copy over just the components needed for a particular project. Make this part of your build setup in the IDE (VS.NET 2005 and up have nifty ways to set up pre-compile and post-compile tasks using MSBuild)
I know there is resistance for open source solutions but I would still recommend setting up and using a continuous automation system like CruiseControl.NET so that you can leverage it to compile and test your projects on a regular basis from a central repository where the re-usable code library is maintained. This way any changes to the code library can be quickly checked to make sure it does not break anything, It also helps bring out version issues with the various projects.
If you can set this up on a machine and show it during your post-mortem as part of the steps that can be taken to improve, you should get better buy since you are showing something already working that can be scaled up easily.
Hope this helps and best of luck with your evangelism :-)
I came across this set of frameworks recently called the Chuck Norris Frameworks - They are available on NuGet at http://nuget.org/packages/chucknorris . You should definitely check them out, as they have some nice templates for your ASP.NET projects. Also definitely checkout Nuget.
organize by topic, require unit tests (feature-level) for check-in/acceptance into library; add a wiki to explain what/why and for searching
One question: You say this is a consulting group. What code assets do you have? I would think most of your teams' coding efforts would be owned by your clients as part of your work-for-hire contract. If you are going to do this you need to make absolutely certain that your contracts grant you rights to your employees' work.
Maven has solved code reuse in the Java community - you should go check it out.
I have a .NET developer that's devised something similar for our internal use for .NET assemblies. Because there's no comparable .NET Internet community, this tool will just access an internal repository in our corporate network. Otherwise will work rather much the way Maven does.
Maven could really be used to manage .NET assemblies directly (we use it with our Flex .swf and .swc code modules) is just .NET folk would have to get over using a Java tool and would probably have to write a Maven plugin to drive msbuild.
First of all for code organization check out Microsoft Framework Design Guidelines at http://msdn.microsoft.com/en-us/library/ms229042.aspx and then create a central Location source control for the new framework that your going to create. Set up some default namespaces, assemblies for cleaner seperation and make sure everyone gets a daily build.
Just an additional point, since we have "shared code" in my shop as well.
We found out this is very much a packaging issue:
Whatever code your are producing or tool you are using, what you should have is a common build tool able to package your sources into a "delivery component", with everything used to actually execute the code, but also the documentation (compressed), and the source (compressed).
The main interest into having a such a "delivery package unit" is to have as less files to deploy as possible, in order to ease the download of those units.
The build process can very well be managed by Maven or any other (ant/nant) tool you want.
When some audit team want to examine all our projects, we just deploy on their post the same packages we deploy on a production machine, except they will un-compressed the source files and do their work.
Since our source files also includes whatever files are needed to compile them (like for instance eclipse files), they even can re-compile those projects in their development environment).
That way:
Developers will not be forced to use this repository. The barrier to entry must be as low as possible to encourage participation, or it will be ignored: it is just a script to execute to get the "delivery module" with everything in it they need (a maven repository can be used for that too)
We are a risk-averse, Microsoft shop: you can use any repository you want
Some of my fellow devs care about writing quality, reliable software, some don't: this has nothing to do with the quality of code written in these packages modules
We're better at writing processes than following them: the only process involved in this is the packaging process, and it can be fairly automated
Don't assume best practices: you are not forced to apply any kind of static code analysis before packaging executable and source files.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Does your work environment use Harvest SCM? I've used this now at two different locations and find it appalling. In one situation I wrote a conversion script so I could use CVS locally and then daily import changes to the Harvest system while I was sleeping. The corp was fanatic about using Harvest, despite 80% of the programmers crying for something different. It was needlessly complicated, slow and heavy. It is now a job requirement for me that Harvest is not in use where I work.
Has anyone else used Harvest before? What's your experience? As bad as mine? Did you employ other, different workarounds? Why is this product still purchased today?
I had the benefit of using Harvest at a bank and you'll never find a more wretched hive of scum and villainy, backwards triple-forking undocumented check-in gauntlets that require 15 steps to make one simple change. Nevermind that they weren't even using branching. This is an evil tool don't let it get you in its clutches.
Chances are, your company has some sort of contract with CA - are you using a lot of other CA software in-house?
Edit: Guess so!
OK I'm going to answer this in a couple of episodes because its late here and Harvest is a big topic.
Firstly CA Harvest (which is what version 7 of the product is called, version 5 is CCC which I cant recall the expansion, version 12 is called CA SCM) is a lot more than just a SCM tool - in the same way ClearCase is a lot more than an SCM tool. SVN, CVS, git, hg are all base-standard SCM and little more.
What you get with Harvest is SCM + Policy. It gives you a place to store and version your code and wrap it all in a policy of how that code matures though your organization from dev to prod. Do you have a policy in your organization that a Lead Developer needs to sign off on the code before its released to QA ? Harvest allows you define the signoff as a policy, and enforces it - you cant migrate the code from the "Dev" state to the "QA" state until one of the people in the project designated as a Lead Dev does exactly that. Do you have a policy that any SQL code needs signoff by a DBA before it progresses ? Harvest allows you to define that policy, and enforces it - so you might need both Lead Dev and DBA signoff before code migrates.
Harvest is by no means a tool for most software organizations - it is typically used in the finance industry, or in business' where a very strong regulatory framework governs what they can do. Banks need to comply with Sarbannes-Oxley, which has very strong auditing requirements. Harvest provides the ability to define all kinds of controls and process around how changes to the Banks assets move through their lifecycle. I know large public transport organizations that are responsible for the safety and punctuality of millions of people every day, that need the tightly defined control mechanisms that a tool like Harvest provides. I also have seen Harvest used in environments where 1000's of developers use it everyday - yes, I'm not exaggerating, literally 1000's of devs in one organization, writing code for a worldwide retailer, pushing IT solutions out their door everyday to the stores around the world.
Harvest is not perfect, thought version 12 is much better. It has too many "that's just stupid"-moments, it does per-file versioning ala CVS, and CVS-like branching and directory versioning (or lack thereof), with all the fun we've come to know and fear. Once you know it and accept it though, its isn't inherently slower than any other SCM I've used. It just has a bigger job to do than just version your code.
Another big win, and its even bigger with version 12, is its integration with other CA tool (and ability to integrate with non-CA tools, but not many at the moment) - defect tracking with Quality Centre, trouble ticketing with Unicentre Service Desk, software deployment to the desktop with SDM. You can define bridges between these apps that result in a lot tighter integration of these concerns, with the usually positive effects on accuracy and timeliness.
If your dealing with getting software out to a worldwide enterprise, with thousands of desktops and servers, mainfame/midrange/middleware systems, iron-clad change control processes, complexity, regulations, contracts, auditors, just a whole bunch of complexity, Harvest is just one tool in a whole suite of tools your going to need. If you just want a simple SCM for a team of 10 devs supporting a few hundred customers, its not a great way to go.
I'll try to add something about how Harvest actually works next time - repositories, projects, views, packages, forms, processes etc. That might help explain why some organizations use it, and why its not for everyone.
I used Harvest during a short gig in the banking industry a few years ago. I agree that it was practically unusable, but the people in charge of QA seemed to love it.
I worked for a company that had two choices; ClearCase or Harvest. Subversion hadn't ever been considered, and the reason was that ClearCase (IBM) and Harvest (CA) both had longstanding mainframe contracts already.
We've used Harvest for about ten years (2000-2010) and even though we are now looking at replacing it I believe it has served us very well.
Harvest (let's stick with that name even though it's no longer it's official name), was the first major tool we implemented to support us in R&D and at the time none uf us knew much about the many aspects of application lifecycle (versioning of code, branching, automated testing, regression testing, quality assurance, deployment to numerous runtime environments and production, rollback, ememrgency fixes, maintenance updates etc.); today we know a lot more and our development processes serve us very well (not that there is not room for many improvements).
We do not have a very hierarchical organisation (we don't have a lot of inspectors that need to approve changes) but it's very helpful to have support for "checkpoints" - points in the development process where something need to happen (e.g. functional testing or integration testing).
The drawback (for us) with Harvest in regards to usability has been "what a programmer need to do to change x lines of code". Today (out there) there are a lot of easier and more efficient ways than Harvest to get write access to source code files, make your updates and then return the files again / move them to another aspect of the development process (testing,deployment etc.). Another drawback is the price tag; it's expensive.
Gain we've had with Harvest:
It support workflow and therefore we've been able to have a single system to manage code versioning, workflow and process automation. If possible it's easier to maintain and improve a single system than many.
In addition to providing cmd line access to internal processes (making it possible to script special solutions when so required by your processes) Harvest also is easily configured by graphic interface.
It has the concept of "Package" which makes it easy to attach plenty of meta data to code changes and to handle the changes independently of other changes (versioning on file level rather than change sets containing the complete code mass). This is helpful to handle indpendent emergency and maintenance changes.
If a developer is only a programmer and only think on the coding aspect of software development then I imagen he/she would might get very frustrated with Harvest.
If a developer is a developer and understand that software development is a lot more than coding and that the coding is only the very begining of a the lifecycle of software then I belive he would see a lot of benefits with Harvest.
I have been using HARVEST for the last 4 years and i love it. The kind of support it gives you to control the code movement is really fantastic. We use HARVEST to deploy applications on to Websphere. It also do an amazing work in deploying the plugins into the web server along with the application. When you want to have a process in place for moving the code in a big enterprise environment, i don't think any other tool can even come closer to HARVEST.