Is there a way to keep Hudson / Jenkins configuration files in source control? - version-control

I am new to Hudson / Jenkins and was wondering if there is a way to check in Hudson's configuration files to source control.
Ideally I want to be able to click some button in the UI that says 'save configuration' and have the Hudson configuration files checked in to source control.

Most helpful Answer
There is a plugin called SCM Sync configuration plugin.
Original Answer
Have a look at my answer to a similar question. The basic idea is to use the filesystem-scm-plugin to detect changes to the xml-files. Your second part would be committing the changes to SVN.
EDIT: If you find a way to determine the user for a change, let us know.
EDIT 2011-01-10 Meanwhile there is a new plugin: SCM Sync configuration plugin. Currently it only works with subversion and git, but support for more repositories is planned. I am using it since version 0.0.3 and it worked good so far.

Note that Vogella has a recent (January 2014, compared to the OP's question January 2010) and different take on this.
Consider that the SCM Sync configuration plugin can generate a lot of commits.
So, instead of relying on a plugin and an automated process, he manages the same feature manually:
Storing the Job information of Jenkins in Git
I found the amount of commits a bit overwhelming, so I decided to control the commits manually and to save only the Job information and not the Jenkins configuration.
For this switch into your Jenkins jobs directory (Ubuntu: /var/lib/jenkins/jobs) and perform the “git init” command.
I created the following .gitignore file to store only the Git jobs information:
builds/
workspace/
lastStable
lastSuccessful
nextBuildNumber
modules/
*.log
Now you can add and commit changes at your own will.
And if you add another remote to your Git repository you can push your configuration to another server.
Alberto actually recommend to add as well (in $JENKINS_HOME):
jenkins own config (config.xml),
the jenkins plugins configs (hudson*.xml) and
the users configs (users/*/config.xml)

To manually manage your configuration with Git, the following .gitignore file may be helpful.
# Miscellaneous Hudson litter
*.log
*.tmp
*.old
*.bak
*.jar
*.json
# Generated Hudson state
/.owner
/secret.key
/queue.xml
/fingerprints/
/shelvedProjects/
/updates/
# Tools that Hudson manages
/tools/
# Extracted plugins
/plugins/*/
# Job state
builds/
workspace/
lastStable
lastSuccessful
nextBuildNumber
See this GitHub Gist and this blog post for more details.

There is a new SCM Sync Configuration plug-in which does exactly what you are looking for.
SCM Sync Configuration Hudson plugin
is aimed at 2 main features :
Keep sync'ed your config.xml (and other ressources) hudson files with a
SCM repository
Track changes (and author) made on every file with commit messages
I haven't actually tried this yet, but it looks promising.

You can find configuration files in Jenkins home folder (e.g. /var/lib/jenkins).
To keep them in VCS, first login as Jenkins (sudo su - jenkins) and create its git credentials:
git config --global user.name "Jenkins"
git config --global user.email "jenkins#example.com"
Then initialize, add and commit the basic files such as:
git init
git add config.xml jobs/ .gitconfig
git commit -m'Adds Jenkins config files' -a
also consider creating .gitignore with the following files to ignore (customize as needed):
# Git untracked files to ignore.
# Cache.
.cache/
# Fingerprint records.
fingerprints/
# Working directories.
workspace/
# Secret files.
secrets/
secret.*
*.enc
*.key
users/
id_rsa
# Plugins.
plugins/
# State files.
*.state
# Job state files.
builds/
lastStable
lastSuccessful
nextBuildNumber
# Updates.
updates/
# Hidden files.
.*
# Except git config files.
!.git*
!.ssh/
# User content.
userContent/
# Log files.
logs/
*.log
# Miscellaneous litter
*.tmp
*.old
*.bak
*.jar
*.json
*.lastExecVersion
Then add it: git add .gitignore.
When done, you can add job config files, e.g.
shopt -s globstar
git add **/config.xml
git commit -m'Added job config files' -a
Finally add and commit any other files if needed, then push it to the remote repository where you want to keep the config files.
When Jenkins files are updated, you need to reload them (Reload Configuration from Disk) or run reload-configuration from Jenkins CLI.

A more accurate .gitignore, inspired by the reply from nepa:
*
!.gitignore
!/jobs/
!/jobs/*/
/jobs/*/*
!/jobs/*/config.xml
!/users/
!/users/*/
/users/*/*
!/users/*/config.xml
!/*.xml
It ignores everything except for .xml config files and .gitignore itself.
(the difference to nepa's .gitignore is that it doesn't "unignore" all top-level directories (!*/) like logs/, cache/, etc.)

The way I prefer is to exclude everything in the Jenkins home folder except the configuration files you really want to be in your VCS. Here is the .gitignore file I use:
*
!.gitignore
!/jobs/*/*.xml
!/*.xml
!/users/*/config.xml
!*/
This ignores everything (*) except (!) .gitignore itself, the jobs/projects, the plugin and other important and user configuration files.
It's also worth considering to include the plugins folder. Annoyingly updated plugins should be included...
Basically this solution makes it easier for future Jenkins/Hudson updates because new files aren't automatically in scope. You just get on the screeen what you really want.

Answer from Mark (https://stackoverflow.com/a/4066654/142207) should work for SVN and Git (although Git configuration did not work for me).
But if you need it to work with Mercurial repo, create a job with following script:
hg remove -A || true
hg add ../../config.xml
hg add ../../*/config.xml
if [ ! -z "`hg status -admrn`" ]; then
hg commit -m "Scheduled commit" -u fill_in_the#blank.com
hg push
fi

I've written a plugin that lets you check your Jenkins instructions into source control. Just add a .jenkins.yml file with the contents:
script:
- make
- make test
and Jenkins will do it:

I checked in hudson entirely, you could use this as a starting point https://github.com/morkeleb/continuous-delivery-with-hudson
There are benefits to keeping entire hudson in git. All config changes are logged and you can test the testup quite easily on one machine and then update the other machine(s) using git pull.
We used this as a boilerplate for our hudson continuous delivery setup at work.
Regards
Morten

Related

stop git from tracking a file without removing the file from other developers machine

I have a file name settings.py. In my host machine settings.py has a different configuration with that of other developers. I want to stop git from tracking settings.py but not to delete this file from other developers machine when ever they pull changes in the project
Some people talk about including the file in .git/info/exclude folder.
But i have no idea as where to locate this file. I searched for it in my project folder( where git has been initialize). To no avail.
expected result
When ever i commit changes, all changes are push to master except my settings.py file, but not deleting this file from other developers machine when ever they pull my changes.
You already mentioned the correct answer. You can use .git/info/exclude as also documented here.
If you have not created this file yet, you can do so now. Simply place a text file called exclude in .git/info. The .git folder should be present already in the root of your repository.
The file uses the same syntax as a regular .gitignore file, but will only be applied to you.
You can also manually specify a core.excludesfile as described in this answer:
git config --global core.excludesfile ~/.gitignore

How to ignore eclipse metadata but preserve the template?

I've got an ARM project in Eclipse...Actually, I'm using the STM Workbench packaging of Base-CDT-Eclipse.
I'm working with a few other guys and we're using a git server to push and pull from.
However, everyone has a little bit different setup as far as where their toolchains are, OS's, etc.
This is causing trouble, because we're git dummies, and when we push changes after working locally, we do
git add .
git commit -m "some message"
git push origin master
And when we pull changes, we just do
git pull origin master
And pray that there no one else did anything in the meantime, because we're afraid of merging differences, but that's a different story.
Anyway, this whole project has a few sub directories that include things like datasheets, Word documents, and what-not...but, it also includes the metadata for the Eclipse project. So, the last person to commit also pushes their unique settings for things like tool-chain path, preferred builder, etc. This breaks the other guys' setup and after each pull, everyone else has to manually update their project settings to fix this.
So, what files are special to Eclipse for project settings and how can I tell git to ignore these files if they already exist? They need to be available for, say, a git clone but they need to be ignored for subsequent git push's and git pull's.
If you need the setting file and not rename it and it's ok forsetting file need not to do version control, so there is a way by .gitignore with below steps:
Create a .gitignore file. touch .gitignore
Edit and save the .gitignore file
.gitignore
filename
Remove the caches from version control. git rm --cached filename
Commit and push
You can ignore those files changes locally with:
git update-index --skip-worktree -- .project
git update-index --skip-worktree -- .classpath
See: "Difference Between 'assume-unchanged' and 'skip-worktree'", it should better resist to git pull.
Another option would be to a content filter driver which generates (automatically on git checkout) a .classpath if it does not yet exist.
That allows you to version a .classpath.tpl template, and you can keep your actual .classpath completely private (and in your .gitignore)
See this answer for more.

I've configured Composer to download HTMLPurifier locally, but Git won't push all the files to my OpenShift master repo. Why not?

I've got Composer installed and I've used it to download HTMLPurifier locally. Now I'd like to push that download to my OpenShift Git repo. So, in a Git Bash window, I run the following...
git add -A :/
git commit -a -m "Uploading HTML Purifier"
git push origin master
At this point Git reports that the push was successful but when I ls the directory through SSH, it shows that the HTMLPurifier directory is empty. Why is that? How do I get Git to push those files?
Additional Info: I noticed that the HTMLPurifier directory is indeed a Git repo itself and contains a .gitignore file in its root directory. I tried deleting it and re-running the above commands but to no avail...
You should try to avoid pushing downloaded dependencies into a repository. It is recommended to add the vendor directory into the .gitignore file at top level. But what you must do instead is commit and push both composer.json and composer.lock.
Here's why: The vendor directory is managed by Composer. Running Composer will probably do minor things during an update, but may also be doing heavy stuff if the Composer team decides to optimize things.
Also, if you require a branch of a package, and Composer knows the repository of that package, it will default to cloning a Git repo or do a SVN checkout instead of trying to grab a ZIP package of that branch (often there is no way to get such a package for branches, and even tagged versions in a plain Git repository do not have such download ability. Composer knows that Github offers such downloads, and detects Github by looking at the repo URL.)
So you can assume that Composer will put a lot of repository meta data into the vendor file, and if you blindly commit these, things will get ugly. First of all, you are committing way too many files, increasing your repository by an unnecessary amount, which will slow down things. Then, if cloning Git repositories, these will be treated as submodules, and that has another bunch of nastiness I am told. If you are just learning Git, it probably isn't a good idea to start with these. And if you are sufficiently known to the tools (Git and Composer), you probably won't need them either.
There really is only one reason why you'd try to commit a modified version of the vendor directory: If your release process is completely depending on all files being present in your one repository, without any way to run a composer install during the release to make these files appear on the target server.
In such cases, you'd install or update the packages with Composer, and then go through all created directories and delete any .git and .svn (and probably also .hg for Mercurial) folders you encounter. Only then you'd be able to commit the files into your own repository.
But note that this step might be a tedious step to do manually - you probably want to create an update script that does all that work for you. You also might run into issues when updating dependencies because Composer expects files to simply go away when deleted, and not be in the way when being written. I cannot tell you exactly what you'd be experiencing because it depends on how you'd do stuff, but I expect you stumbling upon random puzzling issues.
Bottom line: Avoid committing the dependencies into your own repository if possible.
Try using the -force option, you will also most likely need to delete the .git directory inside the HTMLPurifier directory too.

important files are not in list , all some files go in untract files list?

I am using a Git Repository to manage my project,
Now when i try to commit i see there are some impoertant file like content/image , scripts , etc files in untrack files
Why?
Is there any way to resolve this?
Regards,
vinit
You have to use "git add [wildcard or directory or file(s)]" first to add your files to the versioning system GIT. Manual page is here: https://www.kernel.org/pub/software/scm/git/docs/git-add.html
And the git book is worth a read.
Also I liked gitready very much.
Happy coding
If your Git repo has been initialized on GitHub (and then cloned), it is likely to come with a .gitignore (as well as a README.md, and even a LICENSE file).
Check if those files aren't ignored by the rules in the .gitignore.
You can do that with:
git check-ignore -v scripts
You can edit that file to remove the rules you don't want, then a git add . will add:
the modified .gitignore files
the files that were previously ignored.

Creating a new repo from a directory of old repo

I am very new to bazaar and I am exploring the features of it (and of version control system)
I have a bazaar repo, lets call it 'foo'. Under foo repo I have a directory, lets call it 'projects'.
so, I want to create a separate bazaar repo with only projects directory & I want to retain the log too. I mean to say, everything that is related to project folder present in log file, should be available with this new repo.
I tried export command, but I just got the directory without any log.
Any pointers where I should look ?
You can do this using the fastimport plugin:
bzr fast-export /path/to/orig/project | \
bzr fast-import-filter -i project1/ | \
bzr fast-import - /path/to/new/project1
(I broke the line for readability)
The first command dumps the revisions of the branch at the specified path to standard output
The second command filters the revisions, selecting only the ones that affect the project1/ directory. The trailing / is important.
The third command imports the revisions from the standard input to the specified branch. If the branch does not exist, bzr will create a shared repository with a branch named trunk in it.
For more details, see the help pages:
bzr help fast-export
bzr help fast-import-filter
bzr help fast-import
The fastimport plugin is included in the default installation on Windows and Mac OS X. If you have a more exotic setup, I recommend installing it with pip. I don't remember 100% the package name, maybe bzr-fastimport. You will also need the fastimport python library.