PLC Version Control - version-control

I need to come up with a CM process for PLC code.
Currently, the system is developed using RSLogix 5000. The build product is a monolithic file that can be loaded onto a PLC for execution and edited directly in the development environment. With multiple developers, this has become a problem. They're stepping on each others changes.
As an analogy, it's as if, when doing Java development, the only wway to edit and save the source would be to load up a *.jar file into your IDE, make the change, and then save it back to the jar file. This is less than ideal.
How can I coordinate changes between multiple developers working with PLC's?

If we are talking about one big binary files, then a VCS (centralized or decentralized) is not the best tool for the job.
An external referencial (a shared disk for instance) where a batch will copy and label the current PCL state is better.
See "Tracking Software History"
To avert discontinuities in the historical record of revisions, old versions of programs must be stored.
“We take it a step further, though. Using our MDT AutoSave, we actually go out and interrogate the equipment. Overnight or at whatever frequency is specified, the software reads the programs in the PLCs and then compares that information to the last known program. The version-control software will copy the new program and store it and [then] compare it to the last one.
Launching version control is fairly simple. Required is software installation and then hardware configuration. “You would need a server and a couple of weeks of engineering and you’re good to go,” Perysyn says. However, his company uses a “shrink-wrap approach” that involves installing the software and then customization by users filling in the blanks.
That being said, when you have multiple changes from multiple developers, you need an integration environment where a first delivery can be done and validated, before pushing it to the actual server.
See also this post.

I use Unity Pro, so this may not apply for other brands.
Unity can export an "archive" file which is XML which describes the PLC program and IO setup in its entirety. After commissioning changes, I create an export and check it in to my local Git repo. This gets me an annotated history of changes, but no visual comparison. I can always use UnityDiff for comparison.
Check out http://www.mdtsoft.com/ also

You need specialized versioning system for PLCs like VersionDog.
From the manufacturer:
"Special support with Smart Compares for SIMATIC S5, SIMATIC S7,
SIMATIC PCS 7, WinCC, WinCC flexible, InTouch, CoDeSys, TwinCAT,
Phoenix PC WORX, RSLogix, Schneider Modsoft, Schneider Concept,
Schneider Unity, SINUMERIK 840D, Bosch IndraWorks and more. Also robot
programs from ABB and Kuka and office related data formats like
Microsoft Word, Microsoft Excel and Adobe PDF are perfectly supported
by versiondog.
Update: Here is a screenshot showing ladder version compare. I guess that's what most PLC folks are interested in. We also use it to schedule e-mail report if PLC offline and online application versions are a match, as an alarm that something has been changed in PLC but not put into version control server.

About RSLogix5000 specifically, I have seen developers use an emulated PLC and make their changes online. The final product once developed is then put together with all the comments (as they are not contained in the PLC) and then commissioned. There are issues with changes that cannot be done online, such as AOIs. There are tools in place to stop two people editing the same logic online at once and to take ownership of sections. Backups can be done in the form of uploads, but there isn't any way to track changes.
It is a messy problem, messier still for when you are maintaining a system as you want an .ACD that you can go online with, as unless you are somehow doing a diff with the RSLogix compare tool you just see unreadable machine code like "+|Éû³´¬ÙÆW×晵‚>Ù,"
The most common revision control I have seen (sadly) is just saving the the latest file, then taking a copy and adding the current date to the file name, like the recommended control.com post described.

RSLogix5000 has always prohibited multiple users from opening and editing on the same .ACD simultaneously. However, if multiple users have identical .ACD files, open them, and all make connections to the same target controller, they each can edit on the controller simultaneously, but only if they are working on different routines. Other's edits appear automatically, if they were to look at another programmers routine.
Note that working online like this is usually done with the PLC running, even sometimes with the target system (some kind of machine) operating. This kind of arrangement for the purpose of completing work faster, or in some cases because the system is huge. No one develops like this, as it is really a debug tool and impractical for significant changes.
If one programmer finishes, and another is not done, the unfinished work of the other will be saved to the first programmer's .ACD when they save. Whoever saves last will have everyone's work.
Like others have mentioned in this thread, using file date is fairly reasonable. Some companies use a version control variable that is usually displayed on a connected HMI. Other companies use a separate document that documents who and what changes. Sometimes version notes are placed in a lengthy rung comment in the main routine.
My company uses a separate change log, and dated archive copies are maintained. Multiple programmers are only used in the most extreme cases. Someone is always designated to maintain the offline file integrity, usually the person who will be working the longest, or the project manager.
It is important to note that rung comments are not carried from one user to another before RSLogix5000 v21 because previous versions didn't store comments on the controller.
All this said, you might be trying to manage offline development. I haven't seen any sophisticated methods for this. Usually programmers write the needed routines separately, and a project manager will assemble them into a single project. The cleanest approach I've seen is where a project manager will create an architecture with global functionality, and assign routine work to others, giving them a copy of the .ACD to work with. They return the .ACD with changes, and the project manager copies and pastes their routines into the "master" project.

This is a very good question and it really depends on what you want it to do.
If you are only using Rockwell equipment it might be helpfull to look at their solution, I think it's called FactoryTalk AssetCentre.
Currently I am looking into using Bazaar from Canonical.
One thing that VonC pointed out is that a piece of software that can interogate the PLC is a deffinate plus, not a must in my oppinion but it sure as hell helps.
Am I reading your question properly and you have multiple developers working on the same PLC code at the same time? It's a scary thought but I know it sometimes needs to happen, Siemens PLC's are a bit easier to program with multiple developers but I would assign one person to consolidate and test all the changes before committing to the PLC. Any CVS system will let you create branches for every developer but how you would get them to consolidate their changes is the million dolar question.
Bart.

A simple thing to do would be to do a text diff on the .l5k files so you can easily see whether a developer has been messing with part of the file that is outside of their scope.

I saw this question just now from a link at stack exchange: Are There Realistic/Useful Solutions for Source Control for Ladder Logic Programs. Rather than have a link only answer, I'll dupe my answer here:
There is actually a canned solution - from GE-IP of all places. Check out Proficy Change Management. This product does version control from a PLC control systems point of view, rather than a pure version control of files point of view - it works as a layer sitting on top of a VCS (the scary part is that originally this VCS was Visual SourceSafe) and handles rights management, reporting and checkout/checkin.
While the product is from GE-IP, it is designed to support a variety of PLC and HMI systems out of the box.
Full disclosure, I used for work for a company selling and installing PCM (but that was 7 years ago). So if you ask me what it was like back then I'm likely to tell you where it all went wrong!

In my company we just started a trial with Copia.io
Check it out. Our first tests look very promising!
It brings, branching, merging, ladder diff etc... for multiple PLC platforms (Rockwell, Siemens, Codesys)..
PS. I work for a company that builds machines, we were looking for version-dog alike solutions with a bit more power in collaboration and diffing capabilities. I used tools like Mercurial, Git, Tortoise in past companies (not for PLC though).

Related

Compress Workspace for archiving App versions

Now(6:13pm Jun 1, 2012): I resign myself to learning git and github so that I can do version control. I won't need to mail copies of the (compressed) code to myself, but I still don't understand the mechanism after a day of looking at this stuff.
I get the SHA1 concept for uniquely identifying a file, and using the first 2 characters fo the hash as a directory name. But I'm still confused on the updates, pointers, merge business.
Previously: I have multiple versions of programs, so I can regress to an earlier one to solve a problem.
I used to like to compress the one I was using, and send it to myself via email, but today when I did that the compressed version was too small (49 kb instead of 6 mb). So I guess I am referencing the "workspace" (the extension on the app is ".xcworkspace").
I probably shouldn't waste too much time on this problem, since it is merely a backup, but on the other hand, having the full size is an indication that the whole app is self contained, instead of pointers elsewhere that may be inadvertently changed or destroyed.
Is there any way to "undo" my current version to have all the correct data, or is it really tough?
From personal experience I agree with other commentators that Git is the way to go, or even Mercurial. The learning curve bends down after a while especially if the needs are modest.
As to the need for a "Poor Man's Version Control", sometimes you do need one. For example, you work at a employer that does not allow downloading and use of non-corporate software and the centralized VCS is not allowed to be used for Ad Hoc, Experimental, or skunk work.
Related post: poor mans source control zip project files on build
I'm not sure how to get back any changes without knowing more about your set up, but I can recommend that you look into a slightly new setup: your email-an-archive-to-yourself system sounds like a poor man's Revision Control System, except worse than poor, because there are plenty of great RCS tools available for free.
I recommend you spend an hour or so and read about git. If you learn a few commands you can have a complete change history of your project, and jump back to any point in time you like. (And then change history, creating alternate timelines, become your own grandparent, and cause all sorts of problems/adventures.) Most of the time version control is used in the context of a development team, but it provides a lot of benefit even for a lone wolf.

ClearCase advantages/disadvantages [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Because I'm currently struggling to learn IBM Rational ClearCase, I'd like to hear your professional opinion.
I'm particularly interested in advantages/disadvantages compared to other version-control-systems like Subversion or Git.
You can find a good comparison between ClearCase and Git in my SO answer:
"What are the basic ClearCase concepts every developer should know?", illustrating some major differences (and some shortcomings of ClearCase)
File-centric operations
The most single important shortcoming of ClearCase is its old "file-centric" approach (as opposed to "repository-centric" like in SVN or Git or Perforce...)
That means each checkout or check-in is done file per file. The atomicity of operation is at file levels.
Combine that with a very verbose protocol and a network with potentially several nodes between the developer workstation and the VOB server, and you can end up with a fairly slow and inefficient file server (which ClearCase is at its core).
File-per-file operations means: slow recursive operations (like recursive checkout or recursive "add to source control", even by clearfsimport).
A fast LAN is mandatory to mitigate the side-effects of that chatty protocol.
Centralized VCS
The other aspect to take into account is its centralized aspect (even though it can be "distributed" with its multi-site replicated VOB feature)
If the network does not allow access to the VOBs, the developers can:
still work within snapshot views (but with hijacked files only)
wait for the restoration of the network if they are using dynamic views
Expensive Distributed VCS option
You can have some distributed VCS feature by replicating a Vob.
But:
you need a special kind of license to access it.
that license is expensive and add to the cost of the regular license
any vob that uses the replicated vob (admin vob, admin pvob, ...) must be replicated as well (meaning some projects not directly concerned with a distributed development will still have to pay multi-site license...)
Old and not user-friendly GUI
the GUI is very old school and impractical (mid-90's MFC look, completely synchronous GUI, meaning you have to wait for a refresh before clicking elsewhere): when browsing baselines, you cannot quickly look for one in particular.
the GUI on Unix is not exactly the same than on Windows (the latest 7.1 version is better but not there yet)
the installation process is quite complicated (although the latest Installer Manager introduced by CC7.1 is now a coherent GUI on Windows or Unix and does simplify the procedure)
the only real rich application has only been developed for CCRC (the Remote Client)
UCM inconsistencies and in coherencies
As mentioned in "How to Leverage ClearCase’s features", dynamic views are great (a way to see data through the network without having to copy them to the disk), but the main feature remain UCM: it can be a real asset if you have big project with complex workflow.
Some shortcomings on that front:
the dependencies between components is not well managed for a depth superior to one (because of the bug of "parasite baseline")
UCM still has some in coherencies and inconsistencies as documented in CM Crossroads
Limited policies with Base ClearCase
Using ClearCase without using UCM means having to define a policy to:
create branch (otherwise anyone can create any branch, and you end up with a gazillon of them, with merge workflow nightmare)
put labels (otherwise you forget to label some files, or you put a label where you were not supposed to, or you "move" (gasp) a label from one version to another: at least UCM baselines cannot be moved)
define changeset. ChangeSets only exist with UCM activities. With Base ClearCase, you are reduced to clever "cleartool find" requests...
No application rights
ClearCase right management is entirely built on system rights.
That means you need to register your user to the correct system group, which is not always easy to do when you have to enter a ticket to your IT service in order for them to make the proper registration.
Add to that an heterogeneous environment (users on Windows, and server on Unix), and you need to register your user on Unix as well as Windows! (with the same login/group name). Unless you put some sort of LDAP correspondence between the two world (like Centrify)
No advanced API
only CLI is complete ("cleartool" is the ClearCase Command Line Interface), meaning that any script (in Perl or other language) consists in parsing the output of those cleartool commands)
ClearCase Automation Library (CAL) exists, but is quite limited
Java API exists, but only for web views for the CCRC client.
View Storages not easily centralized/backed up
The View storages are the equivalent of the ".svn" of SubVersion, exept there is only one "view storage" per view instead of many .svn in all the directories of a SubVersion workspace. That is good.
What is bad is that each operations within a view (a simple "ls", checkout, checking, ...) will trigger a network request to the view_server process that manages your view server.
2 options:
declare your view storage on your workstation: great for scalability, you can have as many view as you want without taxing the LAN: all communications are directly done on your workstation. BUT if that machine dies on you, you loose your views.
declare your view storage on a centralized server: that means all view_server process will be created there and that all operations on a view by any user will have to communicate with that server. It can be done if the infrastructure is "right" (special high-speed LAN, dedicated server, constant monitoring), but in practice, your LAN will not support this mode.
The first mode means: you have to backup yourself your work in progress (private files or checked-out files)
The second mode means: your workstation can be unavailable, you can just log on another a get back your views (execpt for the private files of a snapshot view)
Side discussion about dynamic views:
To add to the "dynamic view" aspect, it has one advantage (it's dynamic) and one shortcoming (it's dynamic).
Dynamic views are great for setting a simple environment to quickly share a small development between a small team: for a small development effort, a dynamic view can help 2 or 3 developers to constantly stay in touch one with another, seeing instantly when one's commit breaks something in the other views.
For more complex development effort, the artificial "isolation" provided by snapshot view is preferable (you see changes only when you refresh - or "update" - your snapshot view)
For real divergent development effort or course, a branch is still required to achieve true code isolation (merges will be required at some point, which ClearCase handles very well, albeit slowly, file-by-file)
The point is, you can use both, for the right reasons.
Note: by small team I do not mean "small project". ClearCase is best used for large project, but if you want to use dynamic views, you need to setup up "task branches in order to isolate a small development effort per branch: that way a "small team" (a subset of your large team) can work efficiently, sharing quickly its work between its members.
If you use dynamic views on a "main" branch where everyone is doing anything, then any check-in would "kill you" as it could introduced some "build breaks" unrelated with your current development effort.
That would then be a poor usage of dynamic views, and that would forget its other usages:
additional way of accessing data, in addition of snapshot views, meaning it is a great tool to just "see" the files (you can for example use a dynamic view to tweak its config spec until you see what you want and then copy those select rules into your usual snapshot view)
a side view to make merges: you work with your snapshot view, but for merges you can use your dynamic "sister-view" ("sister" as in "same config spec"), in order to avoid having a failed merge because of checked-out files (on which you would be currently working on your snapshot view), or because of a snapshot view not completely up-to-date. Once the merge is complete, you update your regular snapshot view and resume your work.
Developing directly in a dynamic view is not always the best option since all (non-checked-out) files are read over the network.
That means the dll or jar or exe needed by your IDE would be accessed over the network, which can slow down considerably the compilation process.
Possible solutions:
one snapshot view with all in it
a snapshot view with dll or jar or exe in it (files which do not changes every five minutes: one update per day), and dynamic view with only the sources visible.
The cost is a fairly obvious disadvantage. Not just the license cost, but also the cost of a ClearCase guru's salary. Almost every company I'm aware of that uses ClearCase seems to have at least one person whose only purpose is to tame the unruly beast.
The very fact that it's complicated enough to require a full-time nanny is also worrying.
An absolute nightmare of a system. It made me wish we could go back to VSS! (Never mind any modern source-control system like Subversion or Git!)
It's slooooow.
If you use dynamic views and the network goes down you cannot access your working copy of the source. You can do nothing but sit and wait for it to be fixed.
If you use snapshot views you seem to run into conflicts and "hijacked" files all the time, so the files in your working copy are never quite the same as in the source repository.
Whenever you try a large update or deliver operation it invariably FAILS for one reason or another, requiring your ClearCase guru to spend a few hour/days figuring it out. Oh yes, you must have a dedicated, full-time ClearCase guru!
When it fails you often cannot roll back the operation, either, so you're stuck with an operation in progress and the developers are blocked.
When you look past the pretty(?) icons, the GUI is very poor - right down to things like being unable to resize windows to see full file paths!
Their support staff are quite reluctant to fix anything. Their first response is always "this is by design" and "can you work around it?" If they do ultimately provide a fix (after much arguing) it will be the most basic possible fix to the most immediate problem.
Basically, it's slow, complicated and unreliable as hell. Oh, and did I mention it's ridiculously expensive? The only way they can possibly sell it is by talking to decision-makers who have never used the product and never will! I'm quite sure that no developer in the world would ever buy it.
Atomic commits and changesets are my biggest gripes against ClearCase. Let's say you check in five files as part of a bug fix or refactoring. Then it is discovered that something got messed up and you need to revert. Good luck finding which five files they are and what version each one needs to be on. But let's take a step back. You have just finished editing those five files, and it's time to commit. The first four go through just fine. That last one requires a massive merge. The other four files are already checked in. They don't wait for you to finish your necessary changes in the last file. I sure hope that no one updated or is using a dynamic view. A continuous integration build server is going to fail too.
Sometimes we make a whole new directory full of files that need to be checked in, but we don't want to check them in until they are done. It's early and everything is still volatile, so why check things in that you might delete very soon? OK, fine so far. Now it's time to check in. You add the newly created folder to source control. Well, ClearCase isn't recursive, so only that single folder is checked in. With SVN, that folder and everything below it is added, as you choose. The developer needs to remember to add everything, otherwise, a lot of files are going to be missing.
ClearCase owns the files and folders so you cannot modify anything unless you have checked it out first. The eclipse plugin takes away a lot of the nuisance here. I can't tell you how many times I opened a file in vi to make a quick change, only to find that I had forgotten to check it out first. Checkout isn't recursive either.
Updates can be painfully slow without changesets. When you update with a snapshot view, every file updates, not just the modified files. I worked on a project with 20,000+ files. I would remote in to my work machine, start the update, then drive to work; get coffee; go to my desk while it was finishing up. That might sound like an exaggeration, but it sadly isn't.
Dynamic views are terrible unless you are in a very small team. And if that's the case, why do you even have ClearCase? I have seen countless people's views getting hosed because someone checked in files that broke the views of everyone else. You should always update and merge any conflicts on your own view. That way, the changes only affect you. With a dynamic view, you cannot merge down before pushing back up; you just commit and hope.
I know cost probably isn't a big concern, but the developers who make the money for the company would enjoy spending the $50k-$100k (depending on ClearQuest license, which is a common addition) on either fun events or new equipment (chairs, monitors, etc.). IBM recommends having staff to keep ClearCase going. Why not re-purpose those people to generate revenue for the company, instead of making sure things don't crash and burn?
Some of the reasons that I have heard for not switching:
Learning will take time and money
Learning SVN or Mercurial should take no more than a day. Only ClearCase suggests having a certain ratio of admins to developers.
Migration will be painful
This is why tools exist: cc2svn
It's not as easy with Mercurial
Security
There are no known gaping holes in SVN AFAIK, and the development team is dedicated to fixing anything that is found quickly. The Department of Defense seems OK with SVN.
No real productivity gain afterwards
It takes forever trying to track down bugs without changesets. I love being able to roll back until I can't see the bug. You can't do that in ClearCase.
Multisite
WANdisco solves that problem. It's not free though.
The only thing that ClearCase does better than the rest is branching individual files, while keeping the others on the same track as another branch.
Everything I have done in Clearcase always seems hard. Whereas, I've never had that impression with other systems(except maybe CVS on occasion).
I've used SVN, CVS, Clearcase, and Mercurial.
My experience with ClearCase was a disaster, and I will second Don's statement that it requires a resident expert-- unfortunately we had more than one. I had experience with CVS and other version control systems, I was familiar with the concepts, but I found the ClearCase documentation incomprehensible and had to ask for help several times; different experts gave me conflicting advice to the point where we actually broke cd. That is, after I issued a ClearCase command in a UNIX shell, the "cd" command failed with an error message.
The basic task of a version control system is really pretty simple. Honestly, I think that half a dozen commands should suffice, using a file scheme that plays well with others. To me ClearCase looks like the result of a marketing exec deliberately complicating the hell out of things to make the product look sophisticated and powerful. I've heard that it can be configured to behave in a simple, safe, reliable way, but again that requires the services of an expert-- out of the box it's like a motorized swiss army knife.
Everything I've experienced related in any capacity to ClearCase is inefficient, ugly, overly complex, slow, confusing, expensive and inconvenient.
It seems to attract managers and engineers that JUST HAVE GOT IT ALL WRONG.
Damn, IBM and Rational must have amazing sales guys to sell such a crappy product.
We are just migrating off CC onto Git for many of the reasons given here. I would like to add one reason to stay away from CC or any other commercial source control system.
Your vital business data is hostage to ClearCase. You can't get it out.
Your vital business data is the code, its version history and all metadata such as commit comments, who checked in and when.
All software will have a limited useful life. You should always ask yourself when you introduce a new system that swallows important business data, whether it is code, bugs, customer data or what not: How do I get my data out again? If you can't answer that question, you should not introduce that system.
When we migrated out we lost most of our history and all of our metadata. Essentially we only have history corresponding to released versions, but information about what changes were done in response to what customer requests is lost (we have that data in the customer support and bug ticket system, so it is not completely lost, but the coupling to the source code is gone).
This will be somewhere between a nuisance and a problem for us on short to medium term. In a couple of years time, it is not important anymore, but perhaps for 1-3 years it will matter.
(There are commercial tools to migrate CC to other SCM, but they were not deemed adequate to our needs, and I doubt it would have been feasible. The minimal export we did took long enough.)
The lesson learnt is: Never entrust vital business data to proprietary systems.
No atomic commits
Once you checked in files it is very hard to revert to a certain state, because atomic commits aren’t supported. When checking in multiple files, each file gets a new revision (similar to CVS) and not the check-in itself. I think this is a crucial feature, because you hardly want revert single files but complete commit actions (which should map tasks). With ClearCase you can only revert to certain states by using Labels. In practice using ClearCase Labels for each check-in is overkill and thus not done.
Crappy user interface
The GUI of ClearCase Explorer is just a big joke. Horrible in usability and ugly looking. Different and often necessary functions aren’t provided (e.g. recursively checking in worked on artifacts). Command line tool cleartool used with cygwin is much better, but still some things aren’t available like recursively adding new files/folders to source control. I have to laugh my head off if I read a 50 lines of code long script to workaround this.
High administration efforts
Administrating ClearCase beast is far from obvious or lightweight (in difference to other scm-systems like CVS, subversion or Git). Expect to put quite a few dedicated ClearCase experts to just keep it running.
Horrible performance
Nothing is worse as making your developers wait while interfacing with SCM-tool, it is like driving with hand brakes enabled. It slows down your brain and also your work. Getting fresh new files to your snapshot view takes around 30 minutes for 10K artifacts. An update (no artifacts were changed) for the same amount takes roughly 5 minutes. When experimenting a lot and jumping between different up-to-date views means a lot of waiting. It gets even worse, when you’re working on files and you want to check-in or update them. Check-out, check-in and adding to source control cycles take around 10-15 seconds which is obviously a nightmare. It gets very annoying when you’re refactoring renaming/moving types or methods (many files can be affected).
Lack of support of distributed development
Today software development is often a distributed thing (developers are spread around the world working on the same product/project). ClearCase definetely isn’t suitable for this, because it is badly suited for offline work. Doing a check-out (action before you can edit a file/folder) requires that you are network connected. Here you could use the hijack option but this is rather a workaround as a feature (you basically just unlock the file on the filesystem). If your development sites are far away from your ClearCase server the check-in/check-out latency can even increase so dramatically that it is not usable at all. There are workarounds for that like using ClearCase Multisite (scm DB replica technology), but you have to pay extra for it and is not trivial to adminstrate.
Git as alternative
Though being a big fan+supporter of Open Source I am still willing to pay money for good software. But looking at IBM-monster ClearCase I wouldn’t invest my money here, it has all these discussed shortcomings, and further more IBM doesn’t seem to invest money to improve their product significantly. Recently I had a look a Git scm which looks very good, especially for its branching+merging features, where ClearCase has its major strengths.
This information taken from http://www.aldana-online.de/2009/03/19/reasons-why-you-should-stay-away-from-clearcase/
Possibly the worst software ever made. I will not work for any firm that uses rational anything. Aside from CC completely crashing and restarting my workstation frequently on dynamic builds. What happens when you are pushing something to source control and CC does what it does best, crash? Is your code then put in lost+found, backed up somewhere maybe? No, it is gone forever. So if you are ever in the god-awful situation of using this giant piece of expensive software, keep duplicates of everything. Good job Rational / IBM. Way to capture the most important part of source control, reliability. Die slow.
Downsides of ClearCase - an addition to the most in-depth post here.
The merge tool is not worthwhile. It barely helps you, remembers no decisions you made, its just a glorified diff.
The merge tool has to check out directories to even CHECK if they need a merge. Its a bit insane.
I use BitKeeper at work (let's assume Git), and merging two repositories even if there are conflicts is so trivial and user friendly even with command line, while ClearCase having tons of GUI tools is a long and laborious process which is also extremely error prone.
All GUI tools require a ton of latency. Even seeing what can be done on a file requires a high speed connection. So right-clicking in the ClearCase tool on a file working from home could take a minute or two having high speed internet because of the extreme amount of networking requirements.
Someone can completely mess up the repository or check-ins if they make their view spec different than the team. Which is quite insane that nobody can just check out some branch; they need the appropriate view spec which will incidentally give them the right stuff. The whole concept can be nice and flexible but 99% of the time it just causes lots of pain. Did I mention you can't email your spec via Microsoft Outlook since CC tools don't accept UTF-8 so you can't copy-paste it?
I have absolutely nothing nice to say about CC. I used it for 2 years at 2 companies and dropped it in a heartbeat feeling happy the entire time. It is also impossible to just experiment with at home with your own projects, so you will still learn SVN or Git at home, and be forced to go through ClearCase pains at work. Nobody I know has ever used CC voluntarily. They only use it because some manager at work decided CC is the path to salvation and forced everyone to migrate to it. In fact my last company migrated from CVS to ClearCase, and after one year from ClearCase to SVN. It was that hated.
ClearCase is not just one thing that makes you say no. It's like living in a house infested with ants. Each ant is just a minor inconvenience at best, but the infestation will drive you mad.
I'm trying to consolidate a few comments into an actual post here. I'm not really here to persuade you that one is better than the other, except by way of making a few points:
If you're comparing git and ClearCase, I respectfully submit that you need to better define your needs - if you are considering ClearCase for a "good" reason, the git probably isn't even in the equation - it's far too new to trust for enterprise-level source control, imo.
ClearCase introduces a lot of concepts into the version control space that other systems don't have, so it can be pretty daunting and confusing. Especially if the only experience you have is reading the documentation, as appears to be the case for a few people here.
ClearCase is definitely not well suited to huge code bases supported by developers who are not on a LAN with a VOB server. You can have many replicated (multi-site) VOB servers to get them close to remote developers, but this isn't necessarily practical if those remote sites are just a single developer.
Do you want file versioning or repository versioning? This is a pretty important question, and one that will necessarily filter out an entire set of tools, making your job easier. Repository versioning has a lot of advantages (and it's not "new", like some posters claimed - commercial tools like Perforce have been around for more than a dozen years, and there may have been tools that did repository versioning even before Perforce), but it isn't a panacea.
With a sufficiently large installation of any source control system, you're going to need help. When considering tools, you need to consider how easy it will be to find people to help you (either job applicants who have experience, or consultants who will be there at a moments' notice to address any issues). There's no such thing as a maintenance-free SCM system, and assuming you have one will get you into more trouble than picking one that requires "too much" administration.
Don't pay too much attention to people who talk about how bad "dynamic views" are - bad SCM policies are bad, regardless of the tool you're using. Your configuration management policies and practices have to be separate from your choice of tool - no tool will stop people from smashing all over your codebase if you don't define sensible branching and merging policies. If someone suggests that having developers directly commit onto /main is ever a sensible idea, you might want to walk away from that conversation.
ClearCase is a fine tool, but it is a complicated tool. There is no getting around this - it does not have an "easy install" mode. :-) From a technical standpoint, there's nothing that git or SVN can do that ClearCase cannot (although often the terminology is different, since Open Source projects tend to just invent new taxonomy where there already existed one), but some things are definitely easier/harder for a given system, depending on their design. ClearCase "snapshot" views are basically the same thing you would have if you checked out a repository from SVN or CVS - it's a local copy of the source code on your machine, with pointers back into the central server for tools to query version history, etc. You can work with these views without any network connection to the ClearCase server at all once they have been created, and you can "recycle" them to avoid downloading your entire repository again when you want to move to work on another branch, for example. "Dynamic Views" are basically a ClearCase invention, and the standard operating mode for a LAN. They appear the same as checking out an SVN repository, but they don't actually copy any files until you make changes. In this way the view is available immediately, but it obviously cannot be worked with if the main clearcase server is unavailable, and is unpleasant to work with over a high-latency connection. They also have the convenience of being able to be mounted as a network drive on any machine with access to the server on which they were created, so if your windows workstation dies, you can just log onto another one, mount your view, and get back to work, since all the files are stored either in the VOB server (for files you haven't modified on this branch), or the view_server (for files you have created or modified just in this view).
Also, and this deserves its' own paragraph....clearmerge is nearly worth the price of admission alone. It's hands down the best merge tool that I've ever used in my life. I firmly believe a lot of bad practice in SCM has developed because of a lack of high-quality merge tools, so CVS users never learned to use branches properly and this fear of branching has propagated to the current day for no particularly good reason.
Ok, all that being said, if you're looking for reasons not to use ClearCase, they're not hard to find, although I think that's the wrong way to go about it. Really you should need to come up with good reasons TO use ClearCase, not reasons for NOT using ClearCase. You should come into any SCM situation assuming that ClearCase is too much tool or too complicated a tool for the job, and then see if you have some situation that encourages you to use it anyhow. Having IBM or Rational logos is not a good reason.. :-)
I would not even consider ClearCase unless you could say yes to all the following statements:
You do now, or will eventually have, more than 50 developers working on the same codebase.
Most of those developers are centrally located, or have high-throughput low-latency connections to a central location.
You have a set of SCM policies and can identify how to use ClearCase to enforce those policies (really you should consider this for any tool)
Money really is no object
My experience is mostly limited by CC, CVS and SVN. In principle, CC is technologically capable, enterprise ready and comparable by features with any modern VCS. But it has several flaws that make it unusable in any people-oriented environment. For process oriented environments it is probably more appropriate, though I doubt that such environments are appropriate by themselves. Maybe, in military, cosmic or medical software, I don't know. Anyway, I believe that even for these domains there are appropriate and still more friendly tools.
Beside being technically capable VCS, CC has several distinctive advantages:
Dynamic views
Nice version tree
Triggers
Good merge versioning, including renames
In my opinion, their use is limited excepting last one; and they don't compensate flaws. Dynamic view nice in theory, but not always available in practice. Version tree has much less use in other VCS, while necessary in CC because of proliferation of branches (see 6). Triggers, as I know, very detailed and capable, but I think that for most practical tasks SVN hooks are good enough. And now about disadvantages that mostly concerns usability:
CC totally fails in sense of usability for main user group: for developers. And that is the main reason why I think that it should never be used in any environment, be it enterprise or not. Even if it were free, it would nevertheless suck your company's money by wasting time of your developers and frustrating them. This point is composed from:
"Check out-Check In" with strict locking approach - it is counter-productive, refactoring unfriendly, and dangerous in repository organizations with single development branch for multiple developers. Meanwhile, the advantages of strict locking are negligible.
Poor performance and high load
It effectively cannot be used remotely without multi-site (due to 2). Multisite is expensive too. ClearCase Remote client is very limited. It don't even have cleartool (before version 7.1), leaving alone dynamic views.
It can hardly be used offline. Dynamic views are just not work. Snapshot views are effectively read only, because you cannot check out without access to repository (see 1). Hijack is poor option which in fact means that CC gives up any responsibility for hijacked file. And CC cannot show you difference with previous revision when offline. SVN is able to show difference with previous revision even being offline.
Overly complicated model, especially with UCM: VOBs, PVOBs, Projects, streams, branches, views, deliver, update, load, restore, rebase, merge, baseline, check in, check out. I think that half of this concepts are just superfluous and doesn't add value, while increasing both technical and conceptual complexity. Few developers understand even basic stuff about CC.
Proliferation of branches. For example, repository often organized with stream per developer (due to 1). It just has no sense in SVN or most other VCSs.
No repository wide revisions. Well, there are such revisions as understand, they called baselines. But when I see some file revision and want to get repository snapshot at the moment of that file revision, I will get some problems. I will need to do black magic with config spec to create a snapshot view, or find somehow through dynamic view if it is available.
Crappy user GUI. Version tree, even being nice, has mediocre usability. Merge tool is just a pity. Other "features": not resizeable windows, absence of incremental search in some places, mouse-centric interface, look and feel in 1995 style, strange work flow distributed between Client and Project Explorer etc.
CC provokes rare and vast check ins. You all know, that check ins must be small and frequent. But developers usually refrains from additional interactions with CC, hijack files and work in local VCS or even without VCS at all (which is more often, unfortunately). And then, after two weeks of development they begin commit comlex feature that adds 20 files and affects another 20 files. It lasts for a day or two, because they hijacked files and now need to perform manual merge with all new changes from repo and resolve all conflicts and discrepancies. During that process, code lies not compilable, because several files successfully got checked in and others do not. And after that it still lies not compilable because they forgot to add another 2 files to CC.
It is very expensive
It is very complex in terms of infrastructure and requires dedicated administrators
ClearCase seems extremely powerful, from the outside. But really, it's just that the number of commands and options you need to use for basic workflow is so high that these get hidden behind a few aliases or scripts, and you end up with something less powerful than CVS, with the usability of Visual Source Safe. And any time you want to do something a little more complicated than your scripts allow, you get a sick feeling in your stomach.
Compare this with Git, which seems complicated from the outside, but after a week working with it you feel completely in control. The repository model is simple to understand, and incredibly powerful. Because it's easy to get at the nuts and bolts, it's actually enjoyable to dig below the surface of your daily workflow.
For example, figuring out a trivial task such as how to just view a non-HEAD version of a file in a snapshot view took me a couple of hours and what I ended up with was a complete hack. Not the enjoyable sort of hack either.
But in Git, figuring out a seemingly complicated task such as how to interactively commit only some changes, (and leave the rest for later) was great fun, and all the time I have the feeling that the VCS is allowing me to organise code and history in a way that suits me, rather than history being an accident of how we used the VCS. "Git means never having to say 'you should have'".
At my work, I use Git for all sorts of lightweight tasks, even within ClearCase. For instance, I do TDD, and I commit to Git whenever a bunch of tests pass and I'm about to refactor. When the task's eventually done, I check in to ClearCase, and Git helps me review exactly what I'm changing. Just try to get ClearCase to produce a diff across a couple of files - it can't! Use Google to find out the various hacks people have tried to work around this. This is something version control should do out of the box, and it should be easy! CVS has had this for decades!
Nightmare to administer in secure environments
Outdated technology
Non-intuitive GUI
Expensive
Resource monster
Sellout to Microsoft
In my opinion? Only reason to have it? If you are religiously following RUP.
The support is terrible. We've had tickets open for years. Our eclipse guru actually fixed a bug in their eclipse plugin locally in about 30 minutes by disassembling the java file. But the ticket still hasn't got past level one support. Every so often they either try to sneakily close it or ping it back to us 'to try on the latest version' (even though we sent them a reproduction recipe which they could try for themselves.).
Do not touch with a barge pole.
Performance.
ClearCase is powerful, stable (IF properly maintained and supervised) but it's slow.
Geological sometimes.
Dynamic views views lead to horrible build times, snapshot views can take ages to update (lunch break for large projects) or checkout (go home for the day).
Clearcase is so annoying it actually drives people to write poetry about it:
http://digital-compulsion.blogspot.com/2007/01/poetic-pathetic-version-control.html
http://grahamis.com/blog/2007/01/24/if-it-was-free-no-one-would-download-it/
The developers will spend 1/2 their time figuring out clearcase before doing any work and once they've figured it out they'll install git locally and only push to the clearcase repo as needed.
You'll have to employ a dedicated Clearcase admin.
I would suggest SVN for toolset and Git for scaling/workflow. I'd also suggest avoiding CC where possible. (Not counting money, the fact it is such a pain to use that is requires a full time admin is a total joke)
I recently had to wrangle with a similar situation. Maybe you can learn from my story.
The team I was newly assigned to was using a heavyweight tool in an convoluted, error-prone manner. I first attempted to sell them on my tools and processes of choice. This attempt failed miserably. I was flabbergasted that they would pick such a burdensome environment over one that was both easier and more effective. Turns out that they wanted to be disciplined, and using a painful process felt disciplined to them. It sounds wierd, but it's true. They had a lot of other misconceptions too. After I figured out what they were after, we actually stuck with the same tool suite (Serena), but massively changed how it was configured.
My advice to you is to figure out what matters to your team. Ripping on ClearCase won't get you anywhere unless you speak to their interests. Also, find out why they don't want to use alternatives. Basically do a little requirements gathering and fit your tool choices to your needs. Depending on your options, who knows, Clear Case may end up being the best option after all.
I'm not totally against ClearCase ( it does have it's advantages ), but to list out the disadvantages:
License Limitations - I can't easily work from home, because I don't have access to the license server. Even with a snapshot view on my laptop I have to play tricks because I can't get a license. There is a special remote client, but adds tons of its own limitations to the mix
License Limitations again - Only so many seats to go around, and then no one can use it.
Unix tools out of date - ClearCase seems to run best on Unix systems, but the GUI tools suck there. Windows/Unix integration of ClearCase introduces all sorts of its own pains.
The biggest downfall for me is both the performance (especially if your VOB is multisite or offsite), and potentially lengthy downtimes.
If you're like me and work in a relatively small office as part of a large company (with no onsite IT), Clearcase servers going down can cost you the better part of a workday in non-productivity as well as getting the right people to get it fixed.
Bottom line, use it only if you really need it for what you are doing and make sure you have a beefy IT budget to maintain it.
ClearCase is perfectly usable if your willing to also use another version control system on top of it! personally I find using mercurial ontop of CC to work quite well.
no atomic checkins
As of the new version of version 7.1 CC provides atomic checkin as functionality IF you like that. Personally I would really not want it but apparently some people see that as "an essential feature". I NEVER would want one big bulk in one go as a sort of massive version. Then again... if you want it just turn it on.
so... no longer an argument.
We used UCM ClearCase integrated with ClearQuest (DR Tracking/change request system) for the last 4 years with more than 50 developers. We have over 50 UCM projects over thousand of streams that handled over 35K DRs and change requests. During this period we have officially made over 600 integration deliveries and while having up to 6 concurrent development and release efforts.
I am the main CM/ClearCase guy with a backup who is able to perform the regular delivery/merge and integration builds. The network and servers are supported by the IT team. All I can say is we have had virtually no problems coming from the CM side of this huge development effort and were never a show stopper. Our developers where trained with just the basic stuff and a simple steps were given to them whenever a new project (branch) was created at the request from the project management.
Too many developer complained about ClearCase because they lack the proper CM/IT/ClearCase/Process/Management support. Developers should focus on development not SCM or be a tool specialist. For a large software development, at least 5-7% of the budget should be spent on CM and tool support.
Running a JDK from a VOB in Linux.
Try it, you need to play with the LD_PRELOAD variable (I know!)
the point of "it needs a dedicated person" and "it is complicated" etc....
The core issue here with finding this a problem is that you have to define if you want to have configuration management performed in your organization (which is NOT version management). Configuration Management is like Project Management: even without a tool you still can do project managment and without a tool you can do Configuration Management. Lots of people have a hard time understanding this and lots of people think Configuration Management is equal to a tool which versions sources of software or something...... (therefore comparisons with subversions or other VERSION management systems)
ClearCase is a solution that is build for usage in a Configuration Management environment ERGO: there is a configuration manager (just like "there is a project manager").
So... if in your perception that dedicated person is there to manage a tool I think there is something very wrong. In my perception there is a dedicated person who does configuration management who from an end-user perpective only shows up when there is a problem with the tool but regards this as only 1% of his job.
So what you need to do (like in any other software project) go back to your requirements and put a list of requirements together on what your organisation wants with configuration management. AND YES like in any other software project you will have users (like e.g. developers) who totally not agree with other users (like e.g. management) on certain requirements. There lies the key imho on some reactions I read here.
And IMHO if you have the organization list of requirements AND a configuration manager in the mix.... the choice is pretty clear (see also the forum on www.cmcrossroads.com)
ClearCase is not a tool only for end-users entering their sources under version control like subversion or git. That is only 1% of why a configuration manager really wants a mature configuration management tool.
And... I think the choice of a CM system should never lay with developers equal to choosing the right project management tool or the right CRM system. Developers are end-users of a certain part of the functionality of the tool.
I will be maybe alone here, but ClearCase is not that bad as everyone says. It can handle huge repositeories. Dynamic view are pretty cool and powerful feature too. It is reliable, can be customized by adding triggers and constraints on a pef file basis, permissions, etc.
Unfortunatelly, it comes with a price, big price. It is costly, and to operate properly needs to be properly configured and maintained by dedicated IT team. It makes it really good for BigCo, but not so wise choice for SmallFirm.
I'm a big fan of DVCS and git, but can understand why would BigCo choose ClearCase over SVN and Git. What I can't understand why would anyone choose SVN over Git ;>
Dynamic Views. Must admire a fully functional translucent file system.
One big benefit is that the Intellectual Property is always in the corporate network. A laptop can be lost/stolen and no source code in jeopardy.
Another is the instant access to source code and changed files, no time is ever spent downloading anything.
It serves well for the purpose it has.

Arguments against zip files as source control [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
What arguments can be used against using zip files of source code as a form of version control?
In general each developer is working on their own program and has a responsibility for it. But there are times of course when other developers are involved in work on that program.
Each developer has their own naming convention for zip files ranging from appending the date, a number after the program name or even appending _old / _oldold _newversion etc… When there is collaboration on development of some code. It has to be checked who has the ‘latest’ version of the code – and where it resides, usually the correct version is identified.
There is no easy existing method to diff source trees and during development unwanted changes occasionally slip into code.
The zip file corresponding to software releases that have release to manufacturing are archived. This at least adds some traceability.
Also before RTM there the code is peer reviewed against the previously released version so quality assurance does exist.
Are there any formal white papers explaining the advantages of source control, making clear that the above isn’t a fully valid form of source control? Arguments exist here that since the end product (manufacturing releases) are under control and these are reviewed that there is no problem with the process. Developers do not have too much of a problem working with zip files in this way, but may not be aware of the advantages.
Creating and managing zip files is error-prone.
Real source control gives you tools to understand your code:
History browsing
Diffs between revisions
Annotation of source files to track the origin of a change
Real source control isn't difficult, there's lots of help out there.
The best argument is surely that using a version control system like Subversion or Mercurial is much, much easier and more secure than messing about with zip files. I doubt there has been much paper writing on the subject, as the use of
zip files for this purpose is fairly obviously wrong.
There are a number of SO questions on the general advantages of version control. For example How can I convince my department to implement a version control system? and https://stackoverflow.com/questions/250984/do-i-really-need-version-control
I assume you currently work at a company that practices this method of zip control, and you're looking for ammunition to help you change this practice. There are a lot of questions on StackOverflow about source control, and the community here are in near-total consensus on the benefits of proper source control and the horrors of working without it (for very good reason).
I'll add something here to benefit your battle: YOUR COMPANY IS #$#%&$## CRAZY!!! ZIP FILES??? ARE YOU ##$##% KIDDING ME???
I am assuming that this question was asked because the original poster is working in an office where the standard practice is to share zip files.
Zip files are obviously bad, for the reasons given by Ned Batchelder. The biggest reason I would suggest is that it's clunky, and difficult to merge changes, or get diffs between past revisions easily.
I would recommend you read A Visual Guide to Version Control for some good arguments about why version control systems are very useful, and a superior way of managing code.
I suspect there'll be as many white papers comparing zip files to proper source control as there'll be white papers comparing cutting one's genitals off with a rusty butter knife with buying a puppy.
Zip files work as a very basic form of version control. It's a way to separate "states" of the source. However, it's not a good form of version control because you have to do a lot of work to perform basic source control management tasks. For example:
Bob's team is working on a major feature that requires changing dozens of files. He works in his own private zip-controlled area for a while. He's created 30 new files, added features to 12 existing files, and made changes to existing behavior in 3 existing files over 4 months. How do you merge Bob's work with the main trunk that has also evolved over the last 4 months? Do you hand-diff thousands of lines of code and decide how to merge them? How do you ensure that anything that uses the 15 existing files isn't broken? How do you ensure that Bob's features or main trunk features aren't accidentally omitted?
Alice is investigating a bug in her code and realizes that one of Sam's classes has changed its behavior. Sam says he didn't make the change. How does Alice find when and why the change was made? How does Alice know who depends on the change?
A major customer has reported a bug in an older version of the program. This customer needs a fix and is important enough to warrant a patch. How do you add the code to the old zip file in a way that it also exists in the new files? Also, how do you record that there is a relationship between the two changes?
These are just three scenarios that a version control system handles well. Situation 1 is handled by development branches. Almost every version control system has a notion of branches that can be developed in parallel and merged as needed. Situation 2 is easily addressed by any source control system with a "blame" feature and less easily addressed by just searching commit logs. Situation 3 is a variant of situation 1, but when you merge branches most version control systems make a note. For example, you'd make a branch off of the old version, fix the bug, then merge that branch into the new code. Now when someone asks "Where did this change come from?" they see it was merged from the patch branch and the change was made to fix a bug.
By the way, I've been in each of these 3 situations and used both SVN and Perforce; both made finding a solution very easy.
These people already know all the arguments for SCM, there is nothing anyone can say to them that will sell them on it. These things must happen:
You install SCM on your local machine and use it. If you must, have it autogenerate these .zip files at every build, so no one outside your cube knows the difference.
Some kind of disaster occurs, like loss of work, show-stopper bug is re-introduced or some other worst-case scenario that is the real reason we all use SCM (the other features we learn to appreciate later).
You are unaffected by the disaster, and/or use your personal copy of the code in SCM to fix the problem/recover the lost work/whatever.
You are a hero and everyone wants to know how you did it.
Only by experiencing firsthand the pain of loss caused by poor SCM practices will your organization realize the benefits of SCM. You're smart enough to learn from the mistakes of others, but not everyone is. The rest of the time, you'll just be 2/3X more productive than the rest of the team and maybe, just maybe they'll wonder how.
By the way, this is how you get agile, continuous integration, unit testing, etc into the organization: lead by example.
The ZIP solution requires a pro-active step at the end of the development cycle when things tend to get dropped because no one outside the dev group notices when they doesn't happen. Sort of like that final code cleanup you always plan on doing when things slow down.
An SCM integrated into the dev environment pretty much enforces/encourages keeping a version history with a small amount of effort all the way through the process. This makes it more likely that a version history will actually be created.
On Using ZIP as a SCM
I'm not going to take as hard of a line as some of the others on the ZIP file solution. It is at least better than nothing. It is a perfectly valid way of keeping version histories, it is just a lot more labor intensive, error prone, and lacks a lot of useful features.
Know who you are selling to
Someone in the Dev Group: Focus your arguments on features like ease of troubleshooting by using change histories, safety to experiment with big code changes (because of rollback), and avoiding accidents where work is overwritten by other developers.
Non-Tech Managers/Bean-counters: There are free/low-cost tools that will reduce the labor cost of version control and give greater accountability/transparency into what each developer is doing/the source of coding mostakes.
I wrote a Version Control tool long ago for a company who did the authoring for DVD titles. Before that they had nothing, just a directory full of clips, icons, scripts etc. which anyone could hack away at, and no way to backtrack if it went wrong etc. HOWEVER these people were 'artists', not programmers, so they could not (would not???!) be trained to use a decent Version Control system. So as a bare-minimum, get-out-of-the-mud level tool I wrote a utility which zipped up the current state of the directory, gave the Zip a meaningful name (date + comment supplied by user) and stuck it in a Backups directory, and also allowed you to restore one of these backups.
So zips CAN provide minimum-level version control, and I speak as someone who endorsed that approach when it was right for the skill-level (in terms of programming, I don't want to imply that they couldn't manipulate pixels!) of the people using it.
But as a programmer, you should be thinking to use a tool which really helps you. As such you want to be able to compare differences for individual files, compare differences between complete milestone sets, and (if you are working on anything other than trivial programmes) handle branching and merging. If you want these features you need something BETTER than zip files.
I used to use ComponentSoftware RCS, and if it wasn't for its poor performance over a WAN we might still be using it: it is cheap (even free for single-developer use, in which form I used to use it at home) and simple to use. However nowadays I would suggest looking at SubVersion. It is very flexible, reasonably simple to understand, has a good set of Windows tools to make it even easier (e.g. Tortoise, Ankh), and ... best of all ... you can get it running for free.
It's not good as only creating a zip before a release means loosing a lot of power you get with version control.
Useually you should check in to the repository after you have added/removed/changed a functional aspekt. So that you can go back later when an error occurres that you think migth be because of this change. Or when you say "dammed this worked before the file format changed in someday in march." Naming revisions after changes makes it also easier to remember because you forgot what was done on 27 march 2009.
In general each developer is working
on their own program and has a
responsibility for it. But there are
times of course when other developers
are involved in work on that program.
In a normal development shop, this is not at all true. Different people work on the same source code all the time. XP makes it almost mandatory. Even if you separate the code into modules, there will still be interaction points with code that concerns at least two programmers.
Of course, it's almost impossible to collaborate without major problems if you don't use source control. But the scenario you describe is much more a way to adjust to this limitation than a sane project structure.
Having only a single person working on a module means that nothing will happen when that person is on vacation and you have a major problem when he leaves the company, gets sick for a long time, or dies.
How do you do a merge? How do you do an annotate? How do you bisect? Where are changelogs stored? Just go to wikipedia and look up "Version control" and go down the list: zip files can kind of sort of do about 2 things out of the whole page.
This is like asking "What arguments can be used against shorthand as a form of double-entry bookkeeping?". It's a completely different thing.
For arguments, there's Walter Tichy's original paper on RCS.
For missing features, among many others there's the ability to merge changes from different versions. This is especially well supported by tools like git and darcs, and to a lesser extent mercurial.
P.S. To Mercurial fans: the problem is that Mercurial delegates the merge process to external tools, and it's very difficult for the mercurial novice to know which tool to use, or to understand how they work—the mercurial model of merging seems far more powerful than others but correspondingly difficult to get a grip on.
I haven't seen an answer include Eric Sink's Source Control HOWTO, but it's a valuable reference. I haven't seen any formal white papers on version control, but I'm not sure the argument about "validity" is your strongest one. The problems you describe in your question indicate some pretty serious drawbacks with the current approach. If "the powers that be" in your environment aren't convinced by that, change the argument entirely.
If you make it a question of quality control, and point to continuous integration as a practice that encourages it, then the zip file approach to version control isn't a "not fully valid form of version control", but an obstacle to implementing continuous integration as a practice.
Your question doesn't indicate whether or not the end product "under control" is tested in any automated fashion (in addition to being reviewed). If the process you describe would prevent that from taking place as well, certainly add that to your argument too.
I think your best argument is showing a GOOD form of source control and showing how powerful it is. Don't trash what is currently being done (as someone is surely emotionally attached to that). You don't want to trash the "ZIP Source Control Method." Show the power of something like SVN. Make it very easy to explain. Show common use cases. (A solid demo would help.)
Let the source control version sell itself.

Is there any form of Version Control for LSL?

Is there any form of version control for Linden Scripting Language?
I can't see it being worth putting all the effort into programming something in Second Life if when a database goes down over there I lose all of my hard work.
Unfortunately there is no source control in-world. I would agree with giggy. I am currently moving my projects over to a Subversion (SVN) system to get them under control. Really should have done this a while ago.
There are many free & paid SVN services available on the net.
Just two free examples:
http://www.sourceforge.net
http://code.google.com
You also have the option to set one up locally so you have more control over it.
Do a search on here for 'subversion' or 'svn' to learn more about how to set one up.
[edit 5/18/09]
You added in a comment you want to backup entire objects. There are various programs to do that. One I came across in a quick Google search was: Second Inventory
I cannot recommend this or any other program as I have not used them. But that should give you a start.
[/edit]
-cb
You can use Meerkat viewer to backupt complete objects. or use some of the test programas of libopenmetaverse to backup in a text environment. I think you can backup scripts from the inventory with them.
Jon Brouchoud, an architect working in SL, developed an in-world collaborative versioning system called Wikitree. It's a visual SVN without the delta-differencing that occurs in typical source code control systems. He announced that it was being open sourced in http://archvirtual.com/2009/10/28/wiki-tree-goes-open-source/#.VQRqDeEyhzM
Check out the video in the blog post to see how it's used.
Can you save it to a file? If so then you can use just about anything, SVN, Git, VSS...
There is no good source control in game. I keep meticulous version information on the names of my scripts and I have a pile of old versions of things in folders.
I keep my source out of game for the most part and use SVN. LSLEditor is a decent app for working with the scripts and if you create a solution with objects, it can emulate alot of the in game environment. (Giving Objects, reading notecards etc.) link text
I personally keep any code snippets that I feel are worth keeping around on github.com (http://github.com/cylence/slscripts).
Git is a very good source code manager for LSL since its commits work line-by-line, unlike other SCM's such as Subversion or CVS. The reason this is so crucial is due to the fact that most Second Life scripts live in ONE FILE (since they can't call each other... grrr). So having the comparison done on the file level is not nearly as effective. Comparing line by line is perfect for LSL. With that said, it also (alike SourceForge and Google Code) allows you to make your code publicly viewable (if you so choose) and available for download in a compressed file for easier distribution.
Late reply, I know, but some things have changed in SecondLife, and some things, well, have not. Since the Third Party Viewer policy still keeps a hard wall up against saving and loading objects between viewer and system, I was thinking about another possibility so far completely overlooked: Bots!
Scripted agents, AKA Bots, have all usual avatar actions available to them. Although I have never seen one used as an object repository, there is no reason you couldn't create one. Logged in as a separate account the agent can be wherever you want automatically or by command, then collect any or all objects you are working on at set intervals or by command, and anything they have collected may be given to you or collaborators.
I won't say it's easy to script an agent, and couldn't even speak for making an extension to a scripted agent myself, but if you don't want to start from scratch there is an extensive open source framework to build on, Corrade. Other bot services don't seem to list 'object repository' among their abilities either but any that support CasperVend must already provide the ability to receive items on request.
Of course the lo-fi route, just regularly taking a copy and sending the objects to a backup avatar, may still be a simple backup solution for one user. Although that does necessitate logging in as the other account either in parallel or once every 20 or-so items to be sure they are being received and not capped by the server. This process cannot rename the items or sort them automatically like a bot may. Identically named items are listed in inventory as most recent at the top but this is a mess when working with multiples of various items.
Finally, there is a Coalesce feature for managing several items as one in inventory. This is currently not supported for sending or receiving objects, but in the absence of a bot, can make it easier to keep track of projects you don't wish to actually link as one item. (Caveat; don't rezz 'no-copy' coalesced items near 'no-build' land parcels, any that cannot be rezzed are completely lost)

Why should my team adopt source control? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I have the opportunity to give a formal presentation to my boss about anything that benefits the company. My idea is to adopt source control in my workplace. I have been using Mercurial to manage my own project at work, but the rest of the team does not have a formal source control system in place. Unfortunately, I'm not very good at presenting ideas.
So, can you guys tell me why developers MUST use source control? Additionally, why would you choose any tool except Visual SourceSafe? I don't have experience using VSS, but he is likely to ask why we wouldn't just use Microsoft's tools.
I want to hear opinions from the many smart programmers here! My preferred options are SVN or mercurial. Both seem to have good support for their Windows versions, and both are less archaic than CVS. Also, as a self-declared open source disciple, I would prefer to suggest an open-source tool. :)
Thank you!
Edit: To make it short, generally, current practice for other developers is copying folder, tag with date and maybe record on their own. You get the picture. What if my boss says "if it works, why fix it?"
Let's compare two examples, one development environment that uses source control, and one that doesn't.
A: Does Use
B: Does not Use
Scenario 1: A project is requested, completed, and rolled out
A + B) Programmers develop the project internally, when it's completed, push it out to testing, and then deliver to the client (whoever that may be)
Not much difference, in the big picture
Scenario 2: After a project is released, the client decides that they don't want feature X
A + B) Developers remove the code that the client doesn't want, test, and deliver.
Again, not much difference.
Scenario 3: Two weeks later, the client decides that they actually DO want feature X
A) Developers reintegrate the code they took out in 2 back into the normal development tree, test, and deliver.
B) The developers search for the old code on their personal machines, the file server, and backups. If they find the code, they must manually reinsert each file. If they do not, they probably have to recode that entire feature.
It's easy to get old code that you took out for one reason or another
Scenario 4: There's a strange bug in the system where a function is supposed to return a boolean result, but always returns false. It wasn't like that two weeks ago.
A) Developers examine all the old versions of the software, and figure out that a return false directive isn't in the proper scope - it's executing inside a loop instead of outside.
B) Developers spend weeks trying to figure out what the problem is. Eventually, they notice the return on the wrong line, and fix it. Not having source control means they had to examine each and every file that was executed, rather than finding the differences from when it was working and now.
Scenario 5: Someone breaks the build. It gets past testing and is only noticed weeks later.
A) The team examines the commit history, finds out who broke the build, makes that person fix it and buy the team dinner.
B) The team has to go back through the entire project to find the error, but can't figure out who put that code in. Developers blame each other, and the team dynamic fails.
It's easy to see who committed what, when, and why.
Use source control because neither you nor your team are perfect. The primary function of source control is to ensure that you have a complete historical record of your development process. Having this record, you have the ability to confidently branch out with "experimental" versions, knowing that if the experiment fails, you can back up to an earlier version.
In addition, a good source control system like svn will permit multiple developers to work on the same file and provide powerful tools for reconciling the differences that each introduces.
Simply - so you have a true history of the code - to investigate changes (reasons for bugs), revert to versions, audit, etc. Backup isn't enough - you simply have a copy of the current picture. Ever change a file and wish you could remember what you did?
You have to use Source Control for these reasons
1) You can rollback to any version
2) Different developers can work on the same files
3) All developers, will have access to the same code base
4) You can track changes
5) You can rollback changes that don't work
6) Source control is the basis of continuous integration and helps massively with TDD
7) If you don't use source control, you will slowly go mad as files get lost/overwritten and nothing works as it should
VSS is not the worst SCC application, I used it for years and grew to hate it, but it does work, is simple, and many people know it.
Here's a simple real-life example.
A few years ago, my boss says, "Feature XYZ used to work, and now it doesn't. No one knows what happened. Can you fix it?"
Now I've never worked with feature XYZ before. So fixing it would involve a lot of flailing around trying to figure out what it does.
But we have source control! So I do this:
Create a test script to test feature XYZ: "Click here, type this, click there, etc."
Get current version. Build. Test. Feature XYZ is broken.
Get version from a week ago. Build. Test. Feature XYZ works.
Get version halfway between those two. Build. Test. Feature XYZ works.
Get version halfway between previous one, and current one. Build. Test. Feature XYZ is broken.
I kept doing this binary search until eventually I hit the point of change: version 145 (we'll say) had the feature working, but version 146 had it broken. Then I just did a compare between those two versions to see what changed. Turns out our technical lead (sigh) had checked in code that changed functionality, but also introduced a side effect that broke feature XYZ.
So I removed the side effect, tested...and lo and behold, feature XYZ worked again.
Without source control, you can never do this. You'll have to flail around, changing one thing or another, hoping to magically hit on the thing that makes feature XYZ work again.
With source control, you just test your way through the versions, pinpoint the exact code that caused the problem, and fix it.
Microsoft (MSDN) has a good article on the benefits of source control.
http://msdn.microsoft.com/en-us/library/ms173539.aspx
There are also lots of good questions here on SO as to the pros and cons.
What are your pros and cons of git after having used it?
Subversion is very popular, but Git is going to be the "next big thing" in the source control world.
It seems to me that most people have covered the major feature of source control but one of the biggest positives is skipped over. These are:
Branches
Without a source code repository it is impossible to create branches (or copies/stream/etc.) of your code for particular purposes. Not being able to create and merge branches is one of the biggest things that disqualifies VSS from being a real source code control system. Some of the purposes of a branch include:
Bug Fix
Sometimes you need to resolve a bug and do it in a place away form the mainline or trunk version of your code. This may be to resolve a problem in the testing environment or any number of reasons. If you have a version control tool you should be able to easily make a new branch (something VSS sucks at) to fix the bug and be able to merge it back into the mainline code if necessary
Maintenance Release
This could be much the same as a bug fix but done after code has been released to production. Examples would be for fix packs, service releases, etc. Again, you want to be able to merge the changes back into the trunk if necessary
New Feature
Sometimes you need to start development of a new version while maintaining your current code. For example you release and maintain v1.0 but need to start work on v2.0 while maintaining v1.0. Branches help resolve this situation
Tagging/Labeling
Another thing source code control systems do is make snapshots of the source code at a particular point in time. These are called labels in VSS, tags in subversion, etc. By creating these on a regular basis and linking them to some substantial milestone in your project it then becomes possible to determine what exactly has changed in your code between releases. This can be important for auditors but also in tracking down the source/scope of an issue. VSS also gets a fail here because VSS only versions the files, not directories. This means it is impossible to re-create a previous version of the system if you rename/move/delete files or directories in the repository (something that happens a lot if you refactor). Good source code control systems like Subversion do just this.
I suggest using SVN, because:
Source control gives you excellent history. You can see where what changes have been made, thus providing a great way to see what's changed over time (even better if you fill out the submit summary each time)
To the developer, it provides an excellent fallback if something goes horribly wrong. You can revert changes to a file back to any point in its history, so you can try out that mod you wanted to make, and if it doesn't work, roll it right back easily.
It provides a central repository that is much easier to back up than running around to different developers' computers.
It allows you to branch a project off in a different direction - useful for specializations and customizations.
It enables more than one developer to work together on the same project, and the same source, by letting you merge and otherwise manipulate changes to one central copy.
I suggest NOT using VSS - see this page for reasons:
http://www.highprogrammer.com/alan/windev/sourcesafe.html for more reasons.
If the current process is copying a folder and giving it a date, isn't that so that you get some sort of development history, so isn't that basically a simple form of source control?
So to answer any criticisms about source control, you're already doing it. Now you just need to point out the weaknesses in the current system and suggest a better one.
Why do you need to re-invent the wheel when people have really thought about a lot of the complex scenarios which can occur during development and developed the tools which let them handle them.
What you're currently doing is very fragile and will fall over if any sort of complex scenario comes up, at which point you'll have to expend a lot of energy working out how to do something that the tools already do. VSS is better than what you're doing, but doesn't have the very useful conventions that SVN, git or mercurial has which allows multiple projects to live together in a well organised manner - I'm talking branches, tags and merging, both of which are fragile and basically a nightmare under vss.
SVN does have plugins for visual studio. Some are free. But I find that tortoise-svn just eclipses anything else. The only benefit I find with a plugin is that new files get added to svn automatically.
So, weaknesses of your current system:
If you have to make a change to a file, you are likely to overwrite or be overwritten by the other dev's changes. You may not even notice this.
If you have to remember which files you've changed to copy them over some 'master' copy, you're likely to miss one at some point.
Good luck ever finding any documentation about when you made a change and why.
How could you ever build a stable automated build system on your current system? Cruise control and hudson work really well, you're hobbling yourself
VSS doesn't group changes to multiple files together very well. Everything modern does this extremely well and with atomic consistency.
VSS branch and merge support is awful. When we used it we ended up bracketing every change with comments in source code and manually copying code around rather than relying on VSS merge.
It's going to be very hard, near impossible in your current system, to have some version of the code in live maintenance and some other, later version, in heavy development. Think about what's needed to keep two projects in sync like this, you'll need a good tool. SVN can do it, git can do it really well.
That might be enough to go on with, can do more.
Having some version control system helps in any, many cases:
Single developer, single branch
The most basic task that each version control system has to perform perfectly if it wants to call itself version control is to be able to go back to specified version of a project. If you made mess of things, you can got to previous version. You can examine some previous version to check how it was done then (for example how it was before refactoring, or before removing some code/file).
Version control systems take much less disk space compared to simply saving backup copies with specified date, because they use deltaification (storing only differences from previous version) and compression. Usually backup systems are means to store last N versions of a project, sometimes with N=1 (only previous version) while version control systems (VCS) store all the history of a project. Knowing Murphy a while after deleting Nth last version you would realize that was the version you want to examine.
Additionally going back to some last version is easy and automated. You can also examine how single file looked like at some past version, and you can get differences (in diff format) between current state and some past version. You can also tag (or 'label') versions, so you can refer to past version not only by date, or by being nth version from current one, but also by symbolic name, for example v1.2 or v1.2-rc0.
With version control system you can examine history to remind you why (and how) some piece of code (some part of a given file) arrived at current state. Most VCS allow to examine line-wise history of a file, i.e. annotating each line of a file when it was changed, in what commit, and by whom (the command is named annotate, blame or praise depending on VCS). In some VCS you can search history for a version (revision) which introduced given fragment of code (e.g. called 'pickaxe search' in Git, one of VCS).
For this feature to be really useful you have to maintain some discipline: you should describe each new version (each new revision / each new commit) writing down why the change was made. Such description (commit message) is very useful, but it doesn't have natural place in backup system.
This feature of course is even more useful if you are not the only developer...
Using version control system allows for alternate way to find bugs in the code, namely by searching history to find version which introduced bug: bisectiong history. When you find revision which introduced bug, you would have limited (in best case: very limited) area to search for bug, because bug has to be in the difference betwen last working version and first version with a bug. Also you would have description of a change (a commit message) to remind you what you wanted to do. This feature is also called sometimes diff debugging. Modern version control systems (VCS) have support for automated (or semi-automated) searching the history by bisecting it (dividing history in half, finding which part contains bug, repeat until single responsible version is found), in the form of bisect (or similar) command.
For this feature to be really useful you have to maintain some discipline: you should commit (save changes / put given state in version control system to remember) single change, dealing with only one feature, with only small difference from the previous version; i.e. commit often.
Most version control systems offer various hooks which allow for example for automated testing, or automated building of a product... or simply reminding you that you do not follow coding standard (coding guidelines).
Single developer, multiple branches
Version control systems allow to create multiple alternate parallel lines of development, called branches (or streams, or views). Common case is having development branches, i.e. having separate branch for unstable development (to test new features), separate branch for stable (main, trunk) version which is (or should be) current working version, and one on more separate maintenance (fixup) branches.
Having maintenance branches allow you to do bugfixes and generate service packs / minor version with corrections to some released version, without need to worry about interference from the new development. Later you can merge maintenace branch into stable, or pick bigfix from maintenance branch into stable and development branches (if further/other development didn't fix bug independently).
Modern VCS (here modern means that both branching and merging branches is easy) allow to go a bit further, i.e. generate separate branch for working on a separate feature (so called topic branches). This allow you to switch between working one one feature to working on other feature (and not only switch from eveloping new feature to working on urgent requested bugfix).
If you are developing your product based on source of some other (usually third party) product, you really should use vendor branches to be able to easy integrate new version of a product from vendor with the changes you made. Admittedly this is no longer purely "single developer" case.
Multiple developers
Using version control systems brings even further advantages if there are more than one developer working on the same project. VCS allow for concurent (parallel) development without worrying that somebody would overwrite your changes, or does not take your changes into account. Of course using version control system is no substitute for communication.
All of the above features are even more important in the multiple-developer case: examining who generated given change, who last changed the code (aka. who broke the build), finding a bug in code not written only by you.
Simple: If the code is not in source safe, it doesn't exist
Subversion is free and better than VSS but VSS is definitely better then nothing.
Before you say anything, find out why your company is not using source control.
Once you know why, it is easy to come up with scenarios where source control can help.
Long discussion on why you should absolutely have source control:
Is Version Control necessary for a small development group (1-2 programmers)?
My comments from that thread:
You always, always want to have some
sort of Source Control even if you are
working on a project by yourself.
Having a history of changes is vital
to being able to see the state of a
codebase at any given time. There are
a variety of reasons for looking back
in a project history which range from
just being able to rollback a bad
change to providing support for an old
release when the customer just wants a
patch to fix a bug rather than
upgrading to a newer version of the
software.
Not having some sort of source control
is pure insanity.
As far as VSS goes - it's certainly better than nothing. It's definitely not the best source control and it's very dated, but the fact it that it continues to do the job for an awful lot of companies out there.
If your boss is determined to stick with Microsoft tools, go for Team Foundation Server instead of VSS. It's a much better system than VSS and it has nice features like integrated bug tracking.
Take it from me, VSS blows. It's basic file storage w/ history. Anything is better than VSS and VSS is better than nothing :)
So, can you guys tell me why
developers MUST use source control?
It provides one method for an entire
team to use; everybody operates under
the same 'ground rules'.
Changes are
orderly vs. chaotic, saving
development time.
The ability to track
changes promotes accountability and
makes it easier to find the right
persom to solve problems in the
materials maintained.
A list of exact
changes made can be generated quickly
and easily, making it easier to
advise users of the information on
how it has changed from version to
version.
It is easy to 'roll back' to
an earlier version of the
information, if a serious mistake was
made during a change.
Source Control is like insurance! You hope you never need it, but are glad you have it when you do!
Why do a formal presentation?
Assuming the team size is at least two, do a real-world example: Let two (or more, the more the better) people get the code, make their changes and show what it takes to integrate all those changes using whatever non source control means you use.
Then do the same scenario using the source control.
The amount of time and pain you save by using source control will speak for itself.
Stick to the bottom line, explain how it relates to money and your boss will probably listen.
If you are only one programmer, I'd say the main argument is the reduced chance that you will waste time (and therefore money) fixing simple mistakes, trying to rollback code that turned to be the wrong idea etc.
If you are more than one programmer then the above goes twice plus it's the only sane way to be able to work together on the same codebase without wasting even more time waiting for eachother,
Visual Source safe is better than nothing but there are free options that are better in almost every respect. If your boss needs a presentation to understand why source control is essential he might not care what tool you use once he has been enlightened. That you have experience with other tools and not vss again relates to the bottom line so that might suffice.
Why shouldn't your team adopt source control?
Even as a solo developer, I use source control. In a modern software development environment, I can think of few if any reasons why you would not use source control. It is more surprising that you don't already have it. The question strikes me as something like house painters asking "Why should we adopt the use of ladders. You know, ladders don't get the house painted - brushes do."
I'm really sorry but if you actually have to argue for [the formalization of] source control in a development environment, you're in a hopeless situation. If your boss actually needs to be convinced that source control is a worthwhile endeavor, your boss is simply not suitable to be a manager of a group of software developers. In order for someone to effectively manage, they really need at the very least a basic understanding of the landscape. I can't even imagine what's going to happen when you actually need to argue for something that's worth having an argument and doing a presentation over.
Developing without source control is like driving a car without breaks. You lose the ability to do seamless concurrent development, you lose your code getting backed up in working copies, you lose the ability to do historic research via code annotations, you lose the benefit of seeing the context and comments that accompany discrete changes, you just lose, period. Using source control is so obvious and has so many benefits, it's shocking that you'd have to justify it.
At work, we use subversion, but some developers (myself included) use Git locally via the git-svn bridge. For personal work, I use Git.
Because:
It will reduce costs - Developers will have to spend less time checking an item in/out of a real VCS than their current ad-hoc approach.
It will protect the organization's intellectual property - this should be the most important consideration for any software company (other than data...). You are payed to create software - shouldn't it be accessible in its entirety?
It will provide quicker, more reliable and straightforward backup mechanisms - all VCSs have built in dumping capabilities. These tend to be more mature than a simple file copy.
It will act as a communication mechanism between developers - depending on the version control system you may use comments/labels/checkout status to determine if someone else has worked on a file, if it has been promoted to production, if it has a corresponding support ticket number etc.
It streamlines development - the ability to compare versions of files as well as the other mechanisms will be beneficial to your company period.
The main reason we use version control is consistentency.
If the projects are not consistent then problems are going to occur and code is going to be lost.
Make sure you have buy in for the rest of the team. Maybe you can test your presentation on them? Hopefully, they see the need as well.
Nice to see good practices being initiated from the bottom up. Maybe everyone will be more likely to adopt the practice if it comes from one of their own instead of some management mandate.
To avoid things like
"Hey! What happens ? It worked yesterday."
The easiest way to convince management to invest Time in a SCCS is focus on backup and archival. By utilizing something like Subversion (SVN), you can restore any project to any point in time instantly. There is no need to have someone look through backup tapes or worry about tracking multiple versions in an obtuse directory structure.
There are obviously many other advantages (i.e. 2 people working on the same file at the same time), but backups are what quickly sold my company many years ago.
Others have mentioned the specific benefits of source control elsewhere, but I wanted to explicitly address the "VSS" portion of the question.
If your boss wants to use a Microsoft tool, Team Foundation Server with Team Suite is a very nice combination. It also has other tools included, such as bug tracking, documents, and reporting capabilities, which makes a nice platform on which to later improve your process. We are quite happy with it where I work, and my coworkers have told me horror stories about VSS.
Keep TFS in mind as a response to the 'Microsoft Tools' question.