Here's the problem: I'm developing a few projects that need to somehow store credentials used for external services - for example, an e-mail address and password. I figured it falls into "configuration" and decided to move the variable data into a separate file the program reads from. I don't want this file to be pushed to upstream though, because in some of my smaller programs, written for personal needs, I test on a production environment and the data are usually quite sensitive.
On the other hand, I don't want a basic .gitignore solution or its equivalent - instead of not uploading the file at all, I'd prefer to send another file in its place, an example configuration file, while keeping the "real" file on its place on my computer. Is there any simple way of achieving it?
If you need more details to answer the question, I'd prefer an answer regarding Git VCS, Python scripts and Linux OS.
One possible solution:
Commit and push a "sample" config file. Then, make the modifications you want to the local config file. It will now be marked as modified in Git.
Use git update-index --assume-unchanged config to permanently ignore future local modifications to the config file (use --no-assume-unchanged to resume tracking modifications).
This way, you will have a sample config in the upstream repo, a customized config in your repo, and you will not accidentally commit the changed config because Git will not mark it as modified.
Another thing you can do is use .gitattributes filters. It lets you specify certain files to pipe through a unix command. You could write a ruby script to scrub out your passwords and replace them with dummy values, or just use sed or awk. You could go a lot crazier than that and use it for some really dangerous things also, heh. See this other answer for some details.
Related
In our company, we just started using Mercurial and we are facing the following problem:
We have some files in the remote repository that are changed by each developer to add some local configuration but these files must never be changed in the remote repository.
Is there a way to tell Mercurial to stop tracking those files locally without making any change to the file on the remote repository?
We tried with hg forget <file> but as I understand, this will remove the file from the remote repository.
We also tried adding those files to .hgignore file, but somehow the files are not really being ignored, I guess Mercurial does this because the files are already being tracked.
So far, we are just ignoring the files when we perform a commit and we use shelve to maintain and restore our local changes after an Update, but it's starting to be a really tedious task.
Thanks in advance for any help.
EDIT: Although it didn't completely fix what we wanted to, accepted answer is the best approach. Our problem is probably a result of a bad design.
If the file you want unchanged is, for example, config.cfg, check in a config_template.cfg, forget config.cfg if it is already tracked, and add config.cfg to the ignore list. Then, a build rule can create config.cfg from the template if it does not already exist.
A user will then have a starting config.cfg that they can customize without checking it in.
You could use the configuration [defaults] section to add some "--exclude" options to usual commands (see my answer to Mercurial hg ignore does not work properly ) for more details.
But.. be careful that it is dangerous to silently ignore modifications to files and also that this [defaults] section has been marked as deprecated (it is still present in 2.9.2).
IMHO it's a wrong approach to have a file in the repository which every person needs changed anyway - it's an indication that you do not want to have it tracked at all.
Change the file to config.sample, and have your programme create a default config upon first start (thus when there's no existing config file) and have every developer use the config file as s/he needs.
And I see Mark Tolnen's answer only now :)
I have already used some VCS like CVS, SVN and Git. One feature that I am missing cannot be found anywhere.
There are files which I would like to have in the repository but every user should have its own. So when you checkout you get a default of that file and that commit your changes only for yourself.
Why do I want this? There are some files like configuration where I would like to have a default version in the repository (e.g. for building releases or a starting base for new team members) but the changes to that file are only relevant to a certain developer (or working copy) because it will contain paths only valid for that developer/working copy.
Currently when I do not add this files:
- I miss them when creating a new working copy or exporting for a release build
- Have no history which changes I might have done for myself for experimenting
Currently when I add this files to the repository:
- I might never commit them so I have a default in the repository but my file is always flagged "changed". In SVN I can add it to the "ignore-on-commit" changelist to improve a bit.
- I might loose my very own changes of a difficult configuration file (data crash, laptop theft, etc.)
Is there a VCS capable of this? Do SNV or git support something regarding this I might have overseen?
If I understood and decomposed your task correctly
"Have a set of default templates of something, which are starting point of per-user customization and these customized versions must be stored separately and be accessible only by responsible person"
you can use this workflow (draft, subject of modifications and corrections), Subversion based for simplicity and transparent management (strong point of any CVCS really)
Subversion repository
Each user of repo have own predefined path inside repository-tree (with common path-pattern for manageability and easy automation of processes)
One special admin-only managed path also exist, not accessible by ordinary users
Our tree may seems like this (where Repository dir is a root of repository)
z:\>dir /s /B
z:\Repository
z:\Repository\Users
z:\Repository\Template
z:\Repository\Users\Alpha
z:\Repository\Users\Bravo
For every user-path we use Path-based Authentication, which provides access for every and each user only to own subtree in repository,
Template contains (as name assumes) templates stub for all user's documents
Adding new users to repo, obviously, becomes simple and easy automated task:
svn copy Template into new user's dir
add rw permissions for created location for user in authz-file
tell user URL of his personal tree in repo
I don't think the VCS is the problem here. It looks like if you have a file whose contents are dependent on the local environment, you should auto-generate it with a script. This way, you ignore the generated file, but version the script and each developer still gets a perfectly valid copy of the config files at run time. This is the same approach that is used, for example, with user specific IDE settings: .suo files on Visual Studio for example.
Update:
If you specifically need a set of defaults, then the solution is this:
Add the defaults to the repository.
Each dev works in their own branch. This way, they can version the
changes to the config files.
When re-basing onto master and/or merging, the devs simply never
merge their customized configs.
You can always set up a hook to check if the default config has been modified, and if so, maybe email the dev. You simply view such a commit the same as you would view a commit that does not compile.
Devs are smart. Sure, they make mistakes. But never under-estimate the power of some simple communication.
Of course, when the default configs do get overwritten with the customized ones of Dev X, then you use the powers of git to fix that commit immediately.
I have a setup as follows.
A private repository at bitbucket where I keep the 'master' repository.
A repository on my server which acts as the 'live' website.
A repository on my laptop which acts as my working copy.
My process is as follows. I make a change to a file in my local repository. I commit these locally. I push these changes to bitbucket. I then pull these changes from my bitbucket to the webserver.
The problem that I have however is that my local copy utilizes different configuration settings for databases, paths etc, ergo what I want is my 'config.php' file at bitbucket to contain the server settings, and the config.php on my local host to contain local settings.
I believe this can be achieved with .hgignore but i have had no success researching.
The problem i encounter is that i make my server settings file, push it to bitbucket, 'forget' the file in my local repository, create a .hgignore, and then recreate the file. However when i 'forget' the file TortoiseHG notices and asks me to commit the change to bitbucket....
Any ideas would be greatly appreciated.
Thanks
Additional Points.
Following the advice below I have developed a setup as follows:
I have my local repository on my laptop where i do my edits.
I have bitbucket which is essentially the 'main' repository - if any other developers join the team they clone this.
I have my live repository on my web host.
On my live repository I have a .hgignore file whichs ignores the respective config files.
As such when I do hg pull from my host, it pulls the repository as is with the localhost configuration files, but when i type hg update (to the live working copy), these files are ignored/not updated.
Could someone clarify as to if i have understood this correctly, and as to whether this is a suitable way of achieving what I want?
Thanks
.hgignore only ignores files if they are not versioned already, so I don’t think your idea in the question will work.
The common approach regarding local configuration is generally a variation on the same theme, like of one of the following:
Do not check in the config.php at all. You can check in a config.example.php with the most common settings, and document in the README that users have to copy it to config.php and then edit it.
Put any shared settings in config.php, and add an include statement to point to an unversioned file with settings specific to the machine, e.g. config.local.php. You can also provide an config.local.example.php-file for this.
Like 2, but the config.php contains all default settings and the local file has the ability to override them.
Check in a config.dev.php and config.server.php-file containing the settings for both environments, and then have an unversioned config.php which includes one of the above files. Advantage is that the configurations themselves are versioned and you can update them.
Which of these variations to pick, or whether you make another variation, depends on your environment.
The basic idea for working with version control and different configuration files is always the same, but I don't know enough PHP to give a detailed answer how you can do this in PHP.
I answered a similar question for .net/Visual Studio a few months ago, so I'll just give you the link to this answer and try to describe the basic idea again, but this time language-agnostic:
For your use case, the basic idea is to have two config files in the repository, one with your local data and one with your server data, for example like this:
config.local.php
config.server.php
The "real" config.php is not in the repository, and it should be in .hgignore, so it never will be in the repository either.
After pulling, you need to run something that copies one of these files (the "correct" one depending on the current environment, local or server) to config.php.
And exactly this last part is the part that I can not answer in detail, because I don't know how to do that in PHP and/or on a web server because I'm a .net/Windows guy.
As far as I know, deploying a PHP site is just copying the files on the web server, so there is no "build/compile" step where the copying/renaming of the config file could be done (where I would do it in .net). Correct me if I'm wrong...
EDIT:
Thomas, I'm not sure if I understood your edits correctly. Your "local" repository on your laptop and your "live" repository on your webserver are basically clones of your "main" repository on Bitbucket, correct?
If yes, are you saying that you have different .hgignore files in the different clones?? That's the part that confuses me.
No matter how you actually do it in the end (there are several possibilities to deal with configuration files, see below), the .hgignore file should be the same in all clones of your repository.
So all your repositories (no matter which clone on which machine) should all contain the same configuration file(s).
Then, you only need to make sure that different configurations are used in different environments. There's already an excellent list of different ways to achieve this in Lauren Holst's answer, so I'll just point you there.
As Laurens Holst already said, we can't tell which of these ways is the best for you - it depends on your environment.
You might want to check here. If both the config file and .hgignore are commited, the .hgignore will have no effect. You could also add a domain check conditional:
$domain = $_SERVER['HTTP_HOST'];
if ($domain=="localhost") {
//local copy config
}
else if ($domain=="yourdomain.com") {
//webserver config
}
I am using Eclipse and Subversion for Java development, and I find myself wishing for a feature in version control systems (one that is not available in SVN, to the best of my knowledge).
I would like my project settings files to be half-ignored. To be more precise, I want them to be available in VCS, I want merge to occur when someone checks in changes, but. I want my own changes ignored unless I very explicitly tell the system to take them.
This would allow me to have my local paths (and other settings) in my local configuration w/o screwing up other people's configuration. But, when I have a substantial change, I can still check it in (very very carefully, may be temporarily removing my other local changes) and have it delivered to other people.
Now, the actual question: is there any VCS that supports this feature? Or may be I am missing something in SVN? How do other people solve this problem in Eclipse?
Yes, Git support that feature through filter driver (a clean script can run upon commit, allowing you to clean the content from any of your changes if you want).
But another way would be to never version that setting file, and only version:
a template file
a value file
a script able to replace variables in the template files with the values from the value file, in order to generate the actual (and "private", as in "not versioned") setting file.
That way, you can modifying it at your heart's content without ever committing your changes.
.gitignore for git, .hgignore for mercurial and file paths and patterns can be added that will not be committed. There similar in SVN but i never worked out how to use it myself but my sysop did set it up form me.
git supports this with
git update-index --assume-unchanged <file>
and the complementary
git update-index --no-assume-unchanged <file>
See http://blog.pagebakers.nl/2009/01/29/git-ignoring-changes-in-tracked-files and http://kernel.org/pub/software/scm/git/docs/git-update-index.html#_options for more details.
I have a file with database settings in my project which I have set to some defaults. The file is tracked by Mercurial and checked in. Since this file will be edited with different values various developer machines, is there a way I can tell Mercurial to ignore new changes to this file?
I tried adding the file to the .hgignore file, but since the file is tracked it isn't ignored. This is alright and good in other situations, but I am wondering if there is something I can do here?
Using a file template is definitely the best solution.
For example, if you have a database.ini file, commit a database.ini.template file and ignore database.ini in .hgignore
If you always want to ignore the file, you can add the -X option as a default for commit to your .hg/hgrc configuration file:
[defaults]
commit = -X program.conf
We wrote an extension for this called exclude. It will automatically add the -X options on the commands that support them -- so hg status and hg commit wont see the modified file. It works by reading a .hgexclude file from the root of your repository, just like the .hgignore file. You add the files that you want to exclude there:
syntax: glob
db.conf
The extension works quite well, but there is a known situation where it fails: merges and the commit that follows a merge (this is documented on the wiki). It would need to be improved so that it would save the modifications away to a temporary file and then restore them afterwards. Please get in contact if you need this feature.
There is no truly automated process, but you can try (as in this SO question) the -X option on hg commit:
% hg stat
M myfile
% hg commit -X 'myfile'
(other solutions might involve shelve or hq)
However, this is not the "right" solution. I would rather recommend versioning:
a file template
a script able to generate the final file (that you modify but can ignore altogether)
If you are using TortoiseHG, open the Settings for the repo, go to the Commit section (2nd icon down on the left) & add the file name(s) to the Auto Exclude list on the right (~ 3rd from the bottom in the list).
From https://tortoisehg.readthedocs.io/en/latest/settings.html#commit
Typically you would check in a reference copy of the file and track it then have the developers make a copy of that for local development, you wouldn't really want developers editing the source controlled file for their own environments.
If your configuration system supports it, it's even easier if you can use an override file that simply override values in the reference copy (e.g. the database connection string). That way devs only have to keep a very minimal local set of override values.
If the file is already being tracked, you can issue the Forget command to the file. If you're using TortoiseHg just right click the file during commit and select Forget. The file must also be already in the ignore list.
I had the same problem as yours, I file keeps on appearing on every commit even-though its already in the ignore list. I tried the Forget command and it did the trick.
You can try hg forget.
For more details, see the official manual about the same command.
It worked for me.
I think, something like this is closer to a correct answer to the original question Mercurial: How to ignore changes to a tracked file, rather than the others suggesting a template, etc.