I have minimal exposure to RPM, Windows installer mechanics, and WIX. That said, I'm interested in making a cross-platform installer tool (Linux, Windows) that supports upgrading and downgrading (versiona and patches) of my own product. I don't believe this is a topic to be approached lightly; I would like to learn the science of the art (or the art of the science). If I succeed, and build a minimally successful installer tool, it would have these features:
does not depend on a platform-specific tool (such as Windows Installer).
reads XML or a declarative syntax to fulfill installation requirements.
attempts to minimize steps to upgrade or downgrade one of my products (rather than requiring a complete uninstall and re-install).
does not require knowledge of interim product versions, in order to jump versions (i.e. can upgrade one of my products from version 1 to version 3, without passing through version 2).
I'm convinced that "the key" to achieving this goal is by seeing versions as a "point A to point B" problem, which implies that A and B are described by two XML "version" documents that hold info about all the parts and actions (files, or platform specifics such as registry entries). My installer tool would "join" or compare the two documents and determine a minimal set of changes to transform A into B. To some extent, I believe this is precisely what Windows Installer does.
Of course there are further complexities, but that is the point of this post. Where is "the bible" of information on this topic? Remember, I want to make my own installer - not use a platform-specific one. For those who care, my products are usually written in C++ or C#.
Or perhaps I should study something like Steam which is cross-platform and has "automated game updates" as part of its capabilities. In my case, the problem of online deployment is already handled. It is just the final installation step I'm examining. Does Steam use native installers (such as an MSI)? If yes, then that is not what I'm looking for.
In short, what path should I pursue to become somewhat competent on the science of this topic?
I'm not an expert and others can give you better answers but...
Don't declaratively list steps required to install your product - You'll end up making assumptions which will eventually prove wrong. Instead, you should be looking at defining the final state of the installation and let the installer worry about how to make that happen.
Another consideration is that being downgradable may involve huge complications depending on your product - Would it have to down-grade database schemas / file formats / ??? In short, every version of your app will need to be both fully forwards- and backwards-compatible (or at least fail gracefully). Also consider the scenario where V1 of your app stores settings in a file. V2 comes along and adds more settings. You downgrade to V1 - What should it do when changing settings? preserve the V2 settings? dump them? Do some of the V2 settings change the impact/meaning of the V1 settings? Are these decisions to be made by your app or your installer?
Anyway, all that aside, I'd say you need at the least:
A central server/farm with complete files for every version of your App and some API/Web Service which allows the installer to retrieve files/filesets/??? as appropriate (You may be able to tie this into a source control system like svn)
Some way of specifying the desired post-install state of the system in an environment-agnostic way (Think install paths - /usr/??? - should the map to C:\Users\??? or C:\Program Files on windows? Also don't forget it might be a 64-bit machine so it could be C:\Program Files (x86).
A very clever installer written for multiple platforms with as much code re-use as possible (Java, Mono, ???)
The installer should do (simply):
Determine the desired version of the product.
Download/read the appropriate manifest.
Compare the desired situation with the current situation (NB: What is currently on the local system, NOT what should be on the system according to the current version's manifest)
Generate a list of steps to reconcile the two, taking into account any dependencies (can't set file permissions before you copy the file). You can make use of checksums/hashing/similar to compare existing files with desired files - thus only downloading the files actually required.
Possibly take complete backups
Download/unpack required files.
Download/unpack 3rd party dependencies - Later .Net Framework Version/Similar
Perform install steps in atomic a manner as possible (at the very least keeping a record of steps taken so they can be undone)
Potentially apply any version-jump specific changes (up/down-grade db, config files, etc.)
verify installation as much as possible (checksums again)
None of this addresses the question of what to do when the installer itself needs upgrading.
A technique I've used on Windows is that the installer executable itself is little more than a wrapper with some interfaces which loads the actual installer dynamically at runtime - thus I can move files about/unload/reload assemblies, etc... from within a fixed process that almost never changes.
As I said above, I am definitely not an expert, just a novice who's done some of this myself. I sure you can get more complete answers from others but I hope this helped a little
Related
We've decided to create a custom bootstrapper for our deployment solution. We are currently re-writing and re-designing our deployment strategy for all of our products. Sadly, none of us are deployment experts.
Here's what we have so far:
A. The MSI packages will be authored in InstallShield. We will use whatever feature Installshield offers (IIS integration, COM registration, Registry, etc). The dialog's created by InstallShield will not be used (that is what the bootstrapper is for). The MSIs will be installed silently.
B. Whenever we need to write CA's for stuff that InstallShield can't handle, we will be writing them in managed code (C#) using DTF. We will be creating a "Custom Action Framework" that will "standardize" how we use custom actions.
C. We will create a custom bootstrapper (the "setup.exe") in C# to "handle" the installation.
We have decided to go with a multiple MSI approach and use MSI transaction to "chain" the installation from the boostrapper (inspired from Office 2007 installer)
The boostrapper that we are envisioning to create is inspired from Visual Studio's and SQL Server's bootstrapper. The boostrapper will be responsible for the following:
Prerequisite installation: Each application require a pre-requisite. These pre-requisites are listed in an XML file placed on the same folder as the MSI (inspired from Office 2007 installer) along with other metadata. Depending on current state of the system, the boostrapper will decide which pre-requisite to be installed or not.
Feature selection: We are planning to structure the "internal" MSI's feature in such a way that it will not be appropriate to be displayed right away to the end-user. We will have feature labeled as "Core_Files", or "Vista_Only" or "64bit_Only". Depending on the metadata on the XML file (on item 1) and the target system, the bootstrapper will be responsible in "populating" a "feature tree" that the user can customize (also inspired from Office 2007 bootstrapper).
Pre-installation Checks: The bootstrapper will be responsible in checking if the system is ready to receive the installation. For instance, if a machine needs to reboot prior to installation or if the user needs to manually install a service pack, patch or a windows component. Anything that needs to be done that needs user intervention should be displayed here. Think of it as a check list (a listbox) with checks and exes. (Inspired from SQL server's bootstrapper). The "rules" will be written in C#.
Application Configuration: For application that needs to be "configured" prior to installation. These "parameters" (user configuration) will be passed to the respective MSI via MSI Properties.
Actual Installation: The bootstrapper will then perform the installation. Proper "transaction" should be observed when necessary. All "products" that should be grouped together shall be displayed as one product in Add/Remove Programs (by messing with the ARP entries). Also, proper progress shall be reported by each MSI being installed.
-- That's what we have so far.
I think there are a couple of out-of-the-box solutions for creating a custom bootstrapper like dotNetInstaller and BMG. We've look into it but it's not as flexible as we've hoped. There's also BURN but we're not sure if it's ready for primetime.
So here we are... we've decided to create our own custom bootstrapper.
Question:
Are we crazy? Shouldn't we be creating our own bootstrapper? Which ideas listed above are not realistic? Is there a better approach?
Any input regarding our situation will be greatly appreciated. Also, if you have any questions, please don't hesitate to ask.
Frankly, Burn isn't going to be done for at least a year. You already have InstallShield and IMO it has the best off the shelf bootstrapper currently available. I'd scope your requirements back and make it fit the box. Pretty much everything I read from you can be done using InstallShield if you learn to push it to it's limits.
I would go for Burn anyway or some already existing solution.
I'm sure that after some time you'll face new problems that you can't now really imagine.
If you face them, that means that Burn's developers have already faced them and probably got them solved. If not, Burn has a large community that will fix the potential bug faster than you.
Focus on the software you're developing, not on writing installer/bootstrapper.
If I were in your shoes, I would give a burn a try. I'd get me a couple of days and see if it meets my requirements.
I thought this would be somewhere on the Web, but I couldn't Google it:
Given the complexity involved in creating an MSI package (compared to NSIS, InnoSetup, etc.), what would be a compelling reason to go through all the mess (using MSVS's crappy setup project wizard, learn a whole new langauge/ecosystem just to make the installer (WiX), or pay heavy license fees (InstallShield)) for the sake of making an MSI installer?
Would be nice to have real world opinions or experience (even to prove that MSI is really worthless) other than the obvious MSDN page, for instance :)
I don't think there is one prime advantage for all situations. Here are some things I like about it, vs other kinds of installers:
Install logic and code is contained in a database, which is in an accessible format.
I like this a lot when I'm debugging. Rather than rebuilding your installer, you can directly edit the database with a tool like Orca (free database editing download from MS), then run the install again to test your changes. Update your custom code, temporarily condition something out, change the order of operations, whatever you need to do.
Patching. The Installer service and its corresponding tools know how to create patches containing deltas of updated files, rather than complete files. It allows maintenance sizes to remain reasonable.
Administrative Images. The installer can create an administrative image. If you've generated patches, you can apply the patches to the administrative image, and new installations can then be run from the administrative image rather than the original installer. Like slipstreaming patches in OS images. If you're pushing your app out to a large number of machines, it's pretty cool to not need to push a bunch of patches out post-install.
Other interesting features include transforms, run from source, detect and repair, component sharing, and so on.
Take a look at this:
https://serverfault.com/questions/11670/advantages-of-using-msi-files
MSI (or ClickOnce) was required to obtain the Windows Vista Logo Program (Microsoft official certification). I believe this requirement was removed with Windows 7, but it's still easier to get certification with MSI (see here).
You don't need to buy any expensive 3rd party installer package though. If you're going for MSI, I suggest you use WIX and learn it. Once you're familiar with it, it works pretty well.
Another good read is:
Windows Installer: Benefits and Implementation for System Administrators
I've been a full time setup developer for 14 years. My first 7 years were InstallScript Setup.exe style projects and my last 7 years have been MSI based. At first I resisted MSI and then after 6 months of using it I became a true believer in how much better it is.
I'm pretty certain that there are enterprises that require MSI formats to remote bulk install an application on thousands of machines. However I don't deal with such organizations so don't know for certain.
We need to regularly synchronize many dozens of binary files (project executables and DLLs) between many developers at several different locations, so that every developer has an up to date environment to build and test at. Due to nature of the project, updates must be done often and on-demand (overnight updates are not sufficient). This is not pretty, but we are stuck with it for a time.
We settled on using a regular version (source) control system: put everything into it as binary files, get-latest before testing and check-in updated DLL after testing.
It works fine, but a version control client has a lot of features which don't make sense for us and people occasionally get confused.
Are there any tools better suited for the task? Or may be a completely different approach?
Update:
I need to clarify that it's not a tightly integrated project - more like extensible system with a heap of "plugins", including thrid-party ones. We need to make sure those modules-plugins works nicely with recent versions of each other and the core. Centralised build as was suggested was considered initially, but it's not an option.
I'd probably take a look at rsync.
Just create a .CMD file that contains the call to rsync with all the correct parameters and let people call that. rsync is very smart in deciding what part of files need to be transferred, so it'll be very fast even when large files are involved.
What rsync doesn't do though is conflict resolution (or even detection), but in the scenario you described it's more like reading from a central place which is what rsync is designed to handle.
Another option is unison
You should look into continuous integration and having some kind of centralised build process. I can only imagine the kind of hell you're going through with your current approach.
Obviously that doesn't help with the keeping your local files in sync, but I think you have bigger problems with your process.
Building the project should be a centralized process in order to allow for better control soon your solution will be caos in the long run. Anyway here is what I'd do.
Create the usual repositories for
source files, resources,
documentation, etc for each project.
Create a repository for resources.
There will be the latest binary
versions for each project as well as
any required resources, files, etc.
Keep a good folder structure for
each project so developers can
"reference" the files directly.
Create a repository for final buidls
which will hold the actual stable
release. This will get the stable
files, done in an automatic way (if
possible) from the checked in
sources. This will hold the real
product, the real version for
integration testing and so on.
While far from being perfect you'll be able to define well established protocols. Check in your latest dll here, generate the "real" versión from latest source here.
What about embedding a 'what' string in the executables and libraries. Then you can synchronise the desired list of versions with a manifest.
We tend to use CVS id strings as a part of the what string.
const char cvsid[] = "#(#)INETOPS_filter_ip_$Revision: 1.9 $";
Entering the command
what filter_ip | grep INETOPS
returns
INETOPS_filter_ip_$Revision: 1.9 $
We do this for all deliverables so we can see if the versions in a bundle of libraries and executables match the list in a associated manifest.
HTH.
cheers,
Rob
Subversion handles binary files really well, is pretty fast, and scriptable. VisualSVN and TortoiseSVN make dealing with Subversion very easy too.
You could set up a folder that's checked out from Subversion with all your binary files (that all developers can push and update to) then just type "svn update" at the command line, or use TortoiseSVN: right click on the folder, click "SVN Update" and it'll update all the files and tell you what's changed.
Using online interfaces to a version control system is a nice way to have a published location for the most recent versions of code. For example, I have a LaTeX package here (which is released to CTAN whenever changes are verified to actually work):
http://github.com/wspr/pstool/tree/master
The package itself is derived from a single file (in this case, pstool.tex) which, when processed, produces the documentation, the readme, the installer file, and the actual files that make up the package as it is used by LaTeX.
In order to make it easy for users who want to download this stuff, I include all of the derived files mentioned above in the repository itself as well as the master file pstool.tex. This means that I'll have double the number of changes every time I commit because the package file pstool.sty is a generated subset of the master file.
Is this a perversion of version control?
#Jon Limjap raised a good point:
Is there another way for you to publish your generated files elsewhere for download, instead of relying on your version control to be your download server?
That's really the crux of the matter in this case. Yes, released versions of the package can be obtained from elsewhere. So it does really make more sense to only version the non-generated files.
On the other hand, #Madir's comment that:
the convenience, which is real and repeated, outweighs cost, which is borne behind the scenes
is also rather pertinent in that if a user finds a bug and I fix it immediately, they can then head over to the repository and grab the file that's necessary for them to continue working without having to run any "installation" steps.
And this, I think, is the more important use case for my particular set of projects.
We don't version files that can be automatically generated using scripts included in the repository itself. The reason for this is that after a checkout, these files can be rebuild with a single click or command. In our projects we always try to make this as easy as possible, and thus preventing the need for versioning these files.
One scenario I can imagine where this could be useful if 'tagging' specific releases of a product, for use in a production environment (or any non-development environment) where tools required for generating the output might not be available.
We also use targets in our build scripts that can create and upload archives with a released version of our products. This can be uploaded to a production server, or a HTTP server for downloading by users of your products.
I am using Tortoise SVN for small system ASP.NET development. Most code is interpreted ASPX, but there are around a dozen binary DLLs generated by a manual compile step. Whilst it doesn't make a lot of sense to have these source-code versioned in theory, it certainly makes it convenient to ensure they are correctly mirrored from the development environment onto the production system (one click). Also - in case of disaster - the rollback to the previous step is again one click in SVN.
So I bit the bullet and included them in the SVN archive - the convenience, which is real and repeated, outweighs cost, which is borne behind the scenes.
Not necessarily, although best practices for source control advise that you do not include generated files, for obvious reasons.
Is there another way for you to publish your generated files elsewhere for download, instead of relying on your version control to be your download server?
Normally, derived files should not be stored in version control. In your case, you could build a release procedure that created a tarball that includes the derived files.
As you say, keeping the derived files in version control only increases the amount of noise you have to deal with.
In some cases we do, but it's more of a sysadmin type of use case, where the generated files (say, DNS zone files built from a script) have intrinsic interest in their own right, and the revision control is more linear audit trail than branching-and-tagging source control.
People,
We have 4 or 5 utilities that work in conjunction with our application. These utilities are either .bat files, or VB apps, PowerBuilder, etc. I am trying to manage these utils in source control, and am trying to figure out a better way to assign versions to them. Right now, the developers use the version control's meta-data -- specifically label -- to store the version number of the tool.
My goal is to have individual InstallShield packages for each utility, and an easy means to manage and assign version numbers to these packages.
Would you recommend a separate .ini file with the info, or store the info in InstallShield .ism file itself, or just use the meta-data info from version control tool?
UPDATE:
I like the idea Orion. I have one concern though. The script that increments the version number... it can not be intelligent enough to increment Major number etc. right. e.g. if one of the utils has version 1.2.3 and we are at a point where the new version is 2.0.0. The script may not be able to handle this.
I think this has to do a lot with our branching techniques -- we don't have any. The folks thought since the utils are so small, the source may not need branches.
PowerBuilder in particular has a nice trick you can do to incorporate the build number from an ini file into the compiled application.
Details here: http://www.pbdr.com/pbtips/ex/autorev.htm
We have ini file inside source control that stores the build number and its value is used in our build scripts to determine what label to apply to the source tree after a successful build. Works very nicely for our needs. When we branch, we do have to manually kick the file to increment the proper number though.
I managed our build system at my last job, which seemed to have some parallels to what you're asking.
There were ~30 C++ projects which needed compiling, and various .NET/Java things, and the odd perl script.
This was all built on our build machine using NAnt - If I were doing it today I'd use rake, but the idea is the same.
We basically had an auto-incrementing build number which was stored in a version.txt file in the root of the repository.
Each time we did a build (automatically done each night, or also on-demand if neccessary) the script would increment this number and check the file back into source control.
All the other apps referenced this file for their version number, or for things which didn't support working like this, the script would set environment variables or perform other workarounds
I'm pretty sure that our installshield programs referenced an environment variable for their version number, but we deprecated them in favour of wix as installshield really did suck
in the case of visual studio, grep/replace the number within the .csproj files, and check them back in
Hope this gives you some ideas
Using the meta data from your version control system should keep things simpler. It's how your developers already use the system. There is no additional file to maintain. My personal experience has taught me to version the satellite applications with the same as version as the main app. K.I.S.S