Can anyone explain in simple terms what the difference is between configuration management and version control? From the descriptions I've been able to find on various websites, it seems like configuration management is just a fancy term for putting your config files in a source control repository. But others lead me to believe there is a more involved explanation.
Version control is necessary but not sufficient for configuration management. Version control happens in some central or distributed repository, but says nothing about where any particular version is deployed or used.
Configuration management worries about how to take what is in version control and deploy that consistently to the appropriate places, primarily QA and production, but in a large enough development operation developers as well.
For example, you may keep all of your SQL queries in version control, including your table modification scripts, but that doesn't control when those scripts are deployed to the appropriate database server and kept in sync with the deployment of any other code that relies on that database structure.
Configuration management includes, but is not limited to, version control.
Configuration management is everything that you need to manage in terms of a project. This includes software, hardware, tests, documentation, release management, and more. It identifies every end-user component and tracks every proposed and approved change to it from Day 1 of the project to the day the project ends.
Version control is specifically applied to computer files. This includes documents, spreadsheets, emails, source code, and more.
Version control is saving files and keeping different versions of them, so you can see the change over time.
Configuration management is generally referred to as an overall process of which keeps track of what version of the code is on what server, how the servers are setup (and the install scripts to do so at many places). It is how process of what happens after the code goes into source control and how gets to deployed to the servers/desktops etc.
Configuration management is an ambigute term.
In software, it tends to be a superset of version control with emphasis on the entire process to produce a result in a repeatable and predictible manner.
In computing maintenance, it is related to the maintenance of the configuration settings and hardware/firmware/software versions of entire networks and set of attached computing machines (including servers, clients, routers...).
In hardware manufacturing, it represents even a superset of the two above, including the hardware pieces and software modules needed to obtain a product, with the description of the process to manufacture them, and even sometime the entire schemas and configurations of the production lines themselves.
In addition to everything said above I'd like to recommend Bob Aiello's book named "Configuration Management Best Practices" - http://www.amazon.com/dp/0321685865 .
It covers all aspects of Software Configuration Management including version control.
Version control is the control of deliverables whereas configuration management is managing the entire process leading to produce the deliverables. Configuration management involves change management, project management, etc., which generally are not managed by simple version control.
Roughly speaking, version control means you can check out the source for any particular version. Configuration management means you can build and deploy and probably test any particular version.
This can be helpful.
Versions and configurations
Versions:
Ability to maintain several versions of an object.
Commonly found in many software engineering and concurrent engineering environments.
Merging and reconciliation of various versions is left to the application program
Some systems maintain a version graph
Configuration:
A configuration is a collection compatible versions of modules of a software system (a version per module)
Version control is one of the features of a SCM system.
From the subversion user guide:
http://svnbook.red-bean.com/en/1.7/svn-book.html
"Some version control systems are also software configuration management (SCM) systems. These systems are specifically tailored to manage trees of source code and have many features that are specific to software development—such as natively understanding programming languages, or supplying tools for building software. Subversion, however, is not one of these systems. It is a general system that can be used to manage any collection of files. For you, those files might be source code—for others, anything from grocery shopping lists to digital video mixdowns and beyond."
Related
We are using a Variscite VAR-SOM-AM33 platform for our project, and software platform is based on OpenEmbedded/Yocto.
To ensure the hardware is running with the current software, the devices are connected to the internet. So far, we have been following the OE recipes and generating ipk and applying software updates via opkg.
However, the process is not satisfactory as some of the recipes are poorly written (fails to uninstall/install during the upgrade process). What robust technique/solution are available for OE/Yocto based systems?
Thanks in advance.
I'd like to add SWUpdate to the list of packages that you should consider. It was recommended in a 2016 paper by the Konsulko Group for Automotive-Grade Linux. That paper mentions a few other options, and provides an analysis of the various tools, so it's probably worth a read. Quoting from the paper:
It is our recommendation that the reference AGL software update strategy make use of SWUpdate in a dual copy configuration and integrate OSTree support. This allows recovery from a corrupt partition for the exception case, but also optimizes the common case where small, incremental updates can be quickly applied or rolled back as needed to [meet] OEM policy.
I don't completely agree with the paper. For example, they wrote off Mender.io because it lacks community support, but IMO the Automotive-Grade Linux group is influential enough to create popularity from scratch. Still, it's a good paper, and the fact that they settled on SWUpdate was interesting to me. I was already leaning toward it because the author, sbabic, is involved in U-Boot software development, and we use U-Boot to burn new images into our device.
At the moment I'm unsatisfied with all of the current options, but mostly because I want extra functionality. I'll probably settle on a custom system which incorporates one or more of the aforementioned packages. Unfortunately that's not the kind of definitive answer that SO prefers, but I hope that it was helpful.
I'm working on a metadata layer to integrate the Software Updater (swupd) from Clear Linux with the Yocto Project / OpenEmbedded Core.
swupd performs whole of OS updates, rather than package-based updates, using binary deltas to only update the files which change and to do so in an efficient manner.
I recently wrote some documentation (within the docs/Guide.md file in the meta-swupd repo) about adopting the "Clear Linux Way" to utilise meta-swupd from an OE/YP based distro. A wikified version of that guide, including a link to the layer git repository, are available on the Yocto Project wiki:
https://wiki.yoctoproject.org/wiki/Meta-swupd
I also have a sample layer on Github which demonstrates use of the layer (this is also the distro layer I test much of meta-swupd with):
https://github.com/incandescant/meta-myhouse
About mender.io, I have recently talked to them regarding their open-source update.
Currently, they already have their client side developed, and is working on the server side. They use HTTP and JSON. This is their git, it is only supporting Beaglebone and QEMU at the moment.
The way mender.io works is: they will have one persistence data and uboot. and 2 rootfs (active and backup) to update. So, when there is an update on the server, the users will be notified to pull it down. Give a mender -rootfs image update command. And if the upgrade is success, the user gives another mender -commit command. If there is no mender -commit, the rootfs will be rolled back to the previous rootfs in the next reboot. Mender currently only support update of kernel and rootfs.
The main role of mender.io is to ensure that the mass distributed image upgrade process is recoverable from errors. In the Server side, mender.io developed a management server to the mass distributed devices using UUID.
Not to advertise but please try out mender.io and give feedback so that the software could be more mature.
Mender Introduction pdf
Well, you can either use package based upgrades, like you do. In that case, you'll need to test and verify everything locally before you push any updates to the field. Obviously, you'll likely need to improve a number of recipes. (And I assume that you upstream those immprovements, right?)
The alternative is to use image-based upgrades. Either with full images, see for instance the discussion at Stackoverflow: Embedded Linux mechanism for deloying firmware updates or swupd
Note: I got distracted while writing this answer, so look at the answer from joshuagi; he explains a lot more of swupd.
I think they are two problems here. We (OpenEmbedded) do need to be careful that we do not break package based updates.
Also, there are image updates like swupd (mentioned above and swupdate, described at: https://sbabic.github.io/swupdate/swupdate.html
meta-updater provides support for OSTree-based updates to OE systems.
OSTree is interesting because it provides a half-way house between full image updates (which are large and tricky to handle correctly) and package based updates (which are tricky to make robust). It has a 'git-like' object representation of a root filesystem, and uses chroot and hard links to atomically switch between file system images.
(Disclosure: I'm a contributor to meta-updater)
Posts here were done years ago. Now also RAUC seems to be promising alternative to mender.io
This is not a "why should I use version control" question :-)
I have always used version control from the first line of code of every project I've written so far. However yesterday I came up with a question (maybe a stupid one) to which I find no answer: when should version control really start during the software development process? Should it start from the first line of code, as I've been doing all of my life, or should it start when you really have an operational version of your code? Put in other words: should version control be used before the first version of your software? (I mean version control, not source backup, of course!).
Pre-development you don't need version control; but what you do need is some form of collaboration mechanism to keep track of changes to the specifications and documentation.
Some teams deploy version control at this stage. Personally I don't find the value of it here, a wiki/trello or similar is more valuable and makes more sense; as you are tracking a lot of abstract ideas.
As soon as you start writing code - you should start the version control process; and through out the development phase before you have deployed you continue to use version control; this is where you start getting value out of it. Especially if you are developing with others. If you are a solo developer, version control may seem like extra work for no use; this is debatable, but when you are working in a team it is essential.
Once the project has deployed; revision control is critical and mandatory. You simply cannot afford to not have it - version control offers you lots of benefits for the type of work you undertake after deployment. Bugfixes, automated testing, deployment - these can easily be automated from your version control system. If you didn't use version control during development; now is the best time to deploy it since you have a solid codebase as your reference point.
Version control is so simple these days with mercurial/git and their online hosting services that it is costs nothing to get started; and the benefits far outweigh any drawbacks.
The question is quite abstract. So, an equally abstract answer.
I think you should use version control on a specific project as soon as it starts to add value.
If you can distinguish between two phases - proof of concept/prototype etc., and product code, I think you should separate the code bases for the two. And you can use version control tools for both (source backup first, then real version control), just avoiding cluttering the production repository with early stuff.
If you are using version control just for the code, you could ask that question.
But ideally, version control should help you reproduce a build, which means the configuration files and other settings can be as important as your first line of code.
See for instance ".classpath and .project - check into version control or not?.
That is the kind of data which will facilitate collaboration, as other developers will be up and running (ie able to build your program) very quickly.
Hi
Since I'm always transferring files, primarily plain text for coding, between home and office, I was wonder if other coders would share their preferred method for doing this. Is version control, checking in and out, really the best way to sync remote data, or is there some other solution (ie 3rd party cloud based tools like drop box or...)
Would luv to get other people's opinions on this subject...
When sharing files and source across more than one machine there is always the possibility to loose something when not using some version control. I personally use git (a distributed version control system, actually, just the stupid content tracker) for everything, especially for stuff that is live on many machines.
The benefits of a light and distributed version control system are:
Obviously, the versioning ;)
Consistency gets checked.
You retain a history of all your changes.
You can flexibly branch and merge as you like, w/o running into trouble when having too many parallel trees you sync manually.
You get to know version control, especially, you get to love distributed version control.
You can literally share a repository with others, who in turn can branch/merge at their own leisure.
Even when using version control, you are still free to have snapshots lying around w/o any version control ;).
So, I am strongly against manually managing synchronization - even in a one-man-show of a project.
We have an application at work which is web-based and comes with a bundled web server (Apache tomcat), and is for network monitoring/patch management. It allows for personalisation, all sorts of rules, custom UI design using proprietary components and definition language, and even custom code to fire on events (based on Java).
I am in a team of several developers, each of who will be customising this app to meet various requirements. As it's a server app, not a codebase, what's the best way to setup a dev environment for >1 user?
If there is one single shared VM with this app, I don't know how good source control like TFS would work with this sort of system? I think also, developers working on various parts of the project may even need the same file at the same time (though TFS does do multiple check-outs).
What is the best way to develop against this sort of product? Bare in mind, even with personal VMs and an instance of the app, changes have to be merged to one central instance. Something keeps making me think about how app-virtualisation could help with this?
Thanks
If it is just an instance of Tomcat (even though it was bundled) couldn't you put the whole Tomcat directory and all of its subdirectories under source control? You just need to check in the non-binary parts, so exclude all the .jar, .exe, .tar.gz and .dll files when you check in. That's what I would do, unless I misunderstood your question.
I'm not familiar with the source control software that you mentioned. I have been using SVN (which is free) and TortoiseSVN as a client (also free). Just an option if your software can't support what I've suggested.
I was overseeing branching and merging throughout the last release at my company, and a number of times had to modify our Subversion pre-commit hooks to enforce different requirements on check-in comments and such. I was a bit nervous every time I was editing those files, because (a) they're part of a live production system, albeit only used internally (and we're not a huge organization), and (b) they're not under version control themselves.
I'm curious what sort of fail-safes people have in place on their version control infrastructure. Daily backups? "Meta" version control? I suppose the former is in place here as part of the backup of the whole repository. And the latter would be useful as the complexity of check-in requirements grows...
Natch - the version-control and any other infrastructure code is also under version-control but I would use a separate project from any development project.
I prefer a searchable wiki or similar knowledge-base repository to clogging up your bug-tracking system with things like VCS config.
Most importantly, make sure that the documentation is kept up to date - in my experience, people are vastly better at keeping code docs up to date than admin docs. This may have been the individuals concerned . One thing that is often overlooked is, if systems are configured according to standard Unix Practices or similar philosophy, that implies a body of knowledge about locations that may not be familiar to an OS/X or Windows programmer, faced with suddenly fixing a broken script. Without being condescending, make sure basic assumptions about location and interdependency are documented.
You should document all "setup" configuration for all your tools and these documents should be checked into version control. For tools with text file configurations which allow comments, you could just checkin the config file. But for tools that require using the interface, you should have a full document with images of the dialog boxes showing what choices are chosen.
Most importantly though, these documents should say WHY you have set the values chosen (when not taking the default).
Second, as backup, the same documents should be included in your bug tracking software under a "How do I setup the version control software?" bug. (The bug tracking database is located on a different physical server, right?)
Third, all of this should be backed-up off-site. I'm sure there question on SO about backup strategies.
What's wrong with using the same version control repository for the commit hooks and other configuration files? That's how I've handled it in the past when I've been responsible for a project's configuration management.
You should also back up your svn repository. That way if the repository itself becomes corrupted or the server catches fire or something, you can recover both your project and the svn control files.
If you have build scripts that are doing this (such as Nant) then you could be checking in those.