I am working on a project that depends on external programs, and needs to know the paths to them. I develop and use the project on several machines, using mercurial for version control. The paths are machine-dependent, so I keep them in a machine-specific config file. I would like the config file for each host to be version-controlled, but I need to ensure that the config file from one host would never overwrite the config file for another host when pushing or pulling between hosts. Is there any way to accomplish this?
In principle, Wim is right: machine specific configurations shouldn't be part of the project's source control. As long as you walk alone, this isn't a real problem, but once you want to provide generic releases of your project, you have to get rid of them. In that case you might not be happy about the fact, that the change history contains files with machine specific data.
Nevertheless, it may make sense to have machine specific data in version controlled files (personally I do this for my dot-rc files and shell scripts). In that case I would suggest to separate generic and specific configurations into different files and include/utilize the specific one at build- or runtime, depending on the currently used machine.
If it is not possible to detect the current machine automatically, you could still create an unversioned symbolic link on each machine, pointing to the appropriate specific configuration file. For instance, on the machine foo the file layout could look like this:
generic.conf version-controlled
specific-foo.conf version-controlled
specific-bar.conf version-controlled
specific.conf → specific-foo.conf unversioned symbolic link
An alternative to symbolic links is to use a hook which automatically creates specific.conf, e.g. on each invocation of hg update. As hooks are set in a repository's hgrc file, it can be defined individually on each machine. Here's an example of a corresponding hooks section in the .hg/hgrc file of a repository clone on the machine foo:
[hooks]
update = cp specific-foo.conf specific.conf
Machine specific configuration settings should not be version controlled in the same repository as the project code.
However, it is still a good idea to put an inactive sample configuration file in your code repository. And this sample could show a bunch of typical locations for the external program paths you mentioned as lines that are commented out. That way you make it easier to get your project running on new machines.
Related
I have already used some VCS like CVS, SVN and Git. One feature that I am missing cannot be found anywhere.
There are files which I would like to have in the repository but every user should have its own. So when you checkout you get a default of that file and that commit your changes only for yourself.
Why do I want this? There are some files like configuration where I would like to have a default version in the repository (e.g. for building releases or a starting base for new team members) but the changes to that file are only relevant to a certain developer (or working copy) because it will contain paths only valid for that developer/working copy.
Currently when I do not add this files:
- I miss them when creating a new working copy or exporting for a release build
- Have no history which changes I might have done for myself for experimenting
Currently when I add this files to the repository:
- I might never commit them so I have a default in the repository but my file is always flagged "changed". In SVN I can add it to the "ignore-on-commit" changelist to improve a bit.
- I might loose my very own changes of a difficult configuration file (data crash, laptop theft, etc.)
Is there a VCS capable of this? Do SNV or git support something regarding this I might have overseen?
If I understood and decomposed your task correctly
"Have a set of default templates of something, which are starting point of per-user customization and these customized versions must be stored separately and be accessible only by responsible person"
you can use this workflow (draft, subject of modifications and corrections), Subversion based for simplicity and transparent management (strong point of any CVCS really)
Subversion repository
Each user of repo have own predefined path inside repository-tree (with common path-pattern for manageability and easy automation of processes)
One special admin-only managed path also exist, not accessible by ordinary users
Our tree may seems like this (where Repository dir is a root of repository)
z:\>dir /s /B
z:\Repository
z:\Repository\Users
z:\Repository\Template
z:\Repository\Users\Alpha
z:\Repository\Users\Bravo
For every user-path we use Path-based Authentication, which provides access for every and each user only to own subtree in repository,
Template contains (as name assumes) templates stub for all user's documents
Adding new users to repo, obviously, becomes simple and easy automated task:
svn copy Template into new user's dir
add rw permissions for created location for user in authz-file
tell user URL of his personal tree in repo
I don't think the VCS is the problem here. It looks like if you have a file whose contents are dependent on the local environment, you should auto-generate it with a script. This way, you ignore the generated file, but version the script and each developer still gets a perfectly valid copy of the config files at run time. This is the same approach that is used, for example, with user specific IDE settings: .suo files on Visual Studio for example.
Update:
If you specifically need a set of defaults, then the solution is this:
Add the defaults to the repository.
Each dev works in their own branch. This way, they can version the
changes to the config files.
When re-basing onto master and/or merging, the devs simply never
merge their customized configs.
You can always set up a hook to check if the default config has been modified, and if so, maybe email the dev. You simply view such a commit the same as you would view a commit that does not compile.
Devs are smart. Sure, they make mistakes. But never under-estimate the power of some simple communication.
Of course, when the default configs do get overwritten with the customized ones of Dev X, then you use the powers of git to fix that commit immediately.
I have a setup as follows.
A private repository at bitbucket where I keep the 'master' repository.
A repository on my server which acts as the 'live' website.
A repository on my laptop which acts as my working copy.
My process is as follows. I make a change to a file in my local repository. I commit these locally. I push these changes to bitbucket. I then pull these changes from my bitbucket to the webserver.
The problem that I have however is that my local copy utilizes different configuration settings for databases, paths etc, ergo what I want is my 'config.php' file at bitbucket to contain the server settings, and the config.php on my local host to contain local settings.
I believe this can be achieved with .hgignore but i have had no success researching.
The problem i encounter is that i make my server settings file, push it to bitbucket, 'forget' the file in my local repository, create a .hgignore, and then recreate the file. However when i 'forget' the file TortoiseHG notices and asks me to commit the change to bitbucket....
Any ideas would be greatly appreciated.
Thanks
Additional Points.
Following the advice below I have developed a setup as follows:
I have my local repository on my laptop where i do my edits.
I have bitbucket which is essentially the 'main' repository - if any other developers join the team they clone this.
I have my live repository on my web host.
On my live repository I have a .hgignore file whichs ignores the respective config files.
As such when I do hg pull from my host, it pulls the repository as is with the localhost configuration files, but when i type hg update (to the live working copy), these files are ignored/not updated.
Could someone clarify as to if i have understood this correctly, and as to whether this is a suitable way of achieving what I want?
Thanks
.hgignore only ignores files if they are not versioned already, so I don’t think your idea in the question will work.
The common approach regarding local configuration is generally a variation on the same theme, like of one of the following:
Do not check in the config.php at all. You can check in a config.example.php with the most common settings, and document in the README that users have to copy it to config.php and then edit it.
Put any shared settings in config.php, and add an include statement to point to an unversioned file with settings specific to the machine, e.g. config.local.php. You can also provide an config.local.example.php-file for this.
Like 2, but the config.php contains all default settings and the local file has the ability to override them.
Check in a config.dev.php and config.server.php-file containing the settings for both environments, and then have an unversioned config.php which includes one of the above files. Advantage is that the configurations themselves are versioned and you can update them.
Which of these variations to pick, or whether you make another variation, depends on your environment.
The basic idea for working with version control and different configuration files is always the same, but I don't know enough PHP to give a detailed answer how you can do this in PHP.
I answered a similar question for .net/Visual Studio a few months ago, so I'll just give you the link to this answer and try to describe the basic idea again, but this time language-agnostic:
For your use case, the basic idea is to have two config files in the repository, one with your local data and one with your server data, for example like this:
config.local.php
config.server.php
The "real" config.php is not in the repository, and it should be in .hgignore, so it never will be in the repository either.
After pulling, you need to run something that copies one of these files (the "correct" one depending on the current environment, local or server) to config.php.
And exactly this last part is the part that I can not answer in detail, because I don't know how to do that in PHP and/or on a web server because I'm a .net/Windows guy.
As far as I know, deploying a PHP site is just copying the files on the web server, so there is no "build/compile" step where the copying/renaming of the config file could be done (where I would do it in .net). Correct me if I'm wrong...
EDIT:
Thomas, I'm not sure if I understood your edits correctly. Your "local" repository on your laptop and your "live" repository on your webserver are basically clones of your "main" repository on Bitbucket, correct?
If yes, are you saying that you have different .hgignore files in the different clones?? That's the part that confuses me.
No matter how you actually do it in the end (there are several possibilities to deal with configuration files, see below), the .hgignore file should be the same in all clones of your repository.
So all your repositories (no matter which clone on which machine) should all contain the same configuration file(s).
Then, you only need to make sure that different configurations are used in different environments. There's already an excellent list of different ways to achieve this in Lauren Holst's answer, so I'll just point you there.
As Laurens Holst already said, we can't tell which of these ways is the best for you - it depends on your environment.
You might want to check here. If both the config file and .hgignore are commited, the .hgignore will have no effect. You could also add a domain check conditional:
$domain = $_SERVER['HTTP_HOST'];
if ($domain=="localhost") {
//local copy config
}
else if ($domain=="yourdomain.com") {
//webserver config
}
I am looking at putting a code base that runs several website into version control. There are several instance of this code base running websites on different virtual servers.
The problem I'm grappling with is that each of these separate instances of more or less the same code have sub-directories with site-specific functions. But it seems that version control systems want to control the entire directory hierarchy.
For instance, each instance has the directory
/www/smarty/libs/plugins/
Where you'll find site-specific functions for smarty. When we are ready to put it into version control, the folder /www would be the root.
So one option is to have all the site-specific functions going out to all sites. I don't see a problem in and of itself, but it seems somehow architecturally 'wrong'. There would be a bunch of files that only belong to one deployment.
Another option is to have a separate repository for each site's specific files within the code base. But that sounds like it could quickly become a nightmare when trying to get new sites deployed properly.
What's the best way to do this? The version control system we're looking at is subversion.
Generally, source control systems should be used to control source. They are not at their best completely controlling file hierarchies, permissions, and other related things. These are best left to deployment configuration.
How about having each of the projects and directories you need represented once in the version control system. Then, in a separate directory (perhaps called /build/), have the various configuration layouts. You might have an ant file that builds each site, or maven. Or you can use tools like Capistrano or Fabric to have more control over each deployment.
The tools are made to be flexible (generally), so here are some suggestions:
Most VCS' allow you to ignore files and directories through some mechanism (e.g. Mercurial .hg ignore file), so you should be able to target what you want/should control versus what shouldn't be.
Separate the files/directories into common resource project and site-specific projects and then use a build system to integrate them to create a deployable package. The build system can be as simple as a shell script or a more sophisticated framework. If its a really simple integration, the VCS may have some basic features for merging bases (e.g. Mercurial subrespositories).
With subversion, you could have a bunch of repositories:
www be in a general repository
plugins each be in a site-specific repository
Then have nested working copies:
svn co http://www_repo www
cd www/smarty/libs
svn co http://foo_plugins_repo plugins
Tip: add plugins to svn:ignore property of www/smarty/libs
svn propset svn:ignore "plugins" www/smarty/libs
You could certainly do that with git too (through .gitignore), and probably with other version control systems but I don't know them.
(Alternatively you could skip the nested working copy part (which can freak some people out) and check out stuff side by side, but use a symlink in lieu of smarty/libs/plugins, while ignore still pertains)
You're missing a "build" step, which whould take the source in source control and create the deployment bundles for the different sites. Only one source package is needed, different build configurations create the different deployment packages. Don't try to directly put the deplyoment set into source control, it is not the source!
I believe the best thing to do would be to create a top level directory in your repository for each site (Site-01, Site-02, etc) and inside those directories put the source tree. Then you can checkout the projects separately. I think it's acceptable and somewhat standard to use the same repository for all the projects your company is involved with.
My terminology might be off kilter, but the fundamental idea is sound, I believe.
I have a file in mercurial that I want dev machines to pull the file, but I want the deployment server to NOT pull the file (it has special mods to it that the dev machines don't have). Is this possible, or should I just have a custom push to server solution instead of just doing an hg pull?
A typical way to do this would be to do the following:
You store a copy of each file in the repository, and name them correctly. For instance, if the file in question is web.config, you would store the following two in the repository:
web.server.config
web.dev.config
Then you would add a built step to ensure the right file was copied to the actual web.config file, you could use a batch file:
if "%COMPUTERNAME%" == "SERVER" copy web.server.config web.config
if not "%COMPUTERNAME%" == "SERVER" copy web.dev.config web.config
Then you would ignore web.config itself through .hgignore:
glob:web.config
A variant of Karlsen's answer that I always use.
I have /config or /etc directory in the project. That directory will often contain sample configs like:
dev.yaml
ci_server.yaml
Then my apps pull from /etc/app.yaml which is a symlink to the correct config depending on the host it's being run on.
The best think about doing it this way is you don't have variant code paths (the batch script branches) which eliminates a potential bug vector. This allows you to exercise the same code path that will be used in production (down to where to look for the override file).
Is this something that can be solved outside version control? For instance, include the file in all copies of the repository, but enable or disable its use with environment variables or something similar. This doesn't sound like something most version control systems are built to handle, unless you use nasty hacks to do things like add/remove/patch files after updating.
One option is to create a special hgignore file for the deployment server, that could or could not be included in the repository. Then in the server's hgrc file specify the path to the special hgignore file with the ignore variable.
This would ensure that updates to file would be ignored by the deployment server but could still be updated as usual for the development machines.
Suppose I have the following (desired) folder structure:
*CommonProject
*Project#1
----> CommonProject(link)
*Project#2
----> CommonProject(link)
Where the CommonProject is the location of the source belonging to that project, and CommonProject(link) is merely a soft link to the main location. If we imagine this as a tree-view in a visual client, if I expand Project#1 I will see CommonProject there as a subdirectory, even though the files are not actually stored there.
The purpose of this is to enable the following behaviour:
When I check out Project#1 I get the files associated with that project as well as a subfolder CommonProject containing all of its files (as if Project#1 contained the copy of the files in the Version Control repository). Now if I were to modify CommonProject's files inside of Project#1 and was to submit my changes to the repository, the changes would go into the CommonProject location (no file is actually stored locally under Project#1 in the repository). Now if I was to sync Project#2, as it also contains symlink to CommonProject, it will now get my updates.
Essentially the duplication of files only exists on my machine, but in the repository there is only one version of CommonProject.
I know Perforce can’t do this, without juggling 3 specs. This is very complicated and error prone, especially when a lot of people do it. Is there a source control repository out there that can do this? (a pointer to some docs on how it can be done is a plus)
Thank you.
Subversion can directly store symlinks in the repository. This only works for operating systems that support symlinks though, as svn just stores the symlink the same way it would with any other file.
I think what you really want is to link to separate projects though. Subversion supports this through externals and git through submodules. Another alternative is to manage this sort of thing with in your build process, so that some static resources are gathered when you initialize the build. Generally, updating a utilities library that changes often is going to cause stability problems, so you can do this manually (or with clever scripts) when you need to
You'd probably be much better off just storing the projects in a flat directory (1 directory per project, all at the same level), and using whatever you build system or IDE is to link all the stuff together.