Jenkins: How do I check out two repository URLs into the same workspace without deleting the first one? - eclipse

our project structure has been split. On the one hand there is the /plugins folder containing all main plugin projects. On the other hand we have a /tests folder containing all the fragment projects that are the unit tests for their corresponding main plugin projects.
Jenkins lets me check out multiple repositories, and even to the same folder in the workspace.
The problem is that the first checkout is deleted when the second URL is checked out.
/plugins is placed into the workspace directory, then the workspace directory is "cleaned":
Cleaning local Directory .
Then the second directory /tests is checked out.
Of course I want that both folder contents are placed in the same workspace directory. How can I do that?

Assuming you're using Subversion to checkout your projects, you need to specify the "Local module directory" to something other than the default for each path you are checking out.
For example;
If you have svn://myrepo/myproject/plugins and svn://myrepo/myproject/tests, the configuration would be along the lines of;
Modules:
Repository URL : svn://myrepo/myproject/plugins
Local module directory (optional) : plugins
Repository URL : svn://myrepo/myproject/tests
Local module directory (optional) : tests
This would then inform Jenkins that it has two paths to checkout and into separate locations.
If you are trying to checkout a project into the subfolder of another working copy, you may have to use svn:externals on the parent directory.

In the new Jenkins 2.0 pipeline (previously named the Workflow Plugin), this is done differently for:
The main repository
Other additional repositories
Here I am specifically referring to the Multibranch Pipeline version 2.9.
Main repository
This is the repository that contains your Jenkinsfile.
In the Configure screen for your pipeline project, enter your repository name, etc.
Do not use Additional Behaviors > Check out to a sub-directory. This will put your Jenkinsfile in the sub-directory where Jenkins cannot find it.
In Jenkinsfile, check out the main repository in the subdirectory using dir():
dir('subDir') {
checkout scm
}
Additional repositories
If you want to check out more repositories, use the Pipeline Syntax generator to automatically generate a Groovy code snippet.
In the Configure screen for your pipeline project:
Select Pipeline Syntax. In the Sample
Step drop down menu, choose checkout: General SCM.
Select your SCM system, such as Git. Fill in the usual information
about your repository or depot.
Note that in the Multibranch Pipeline, environment variable
env.BRANCH_NAME contains the branch name of the main repository.
In the Additional Behaviors drop down menu, select
Check out to a sub-directory
Click Generate Groovy. Jenkins will display the Groovy code snippet
corresponding to the SCM checkout that you specified.
Copy this code into your pipeline script or Jenkinsfile.

Related

Workspace and repo paths arrangement for eclipse and git

I've seen similar questions but some are very old and some are not quite what I'm looking for or have no answers.
I have projects in Eclipse under /workspace/project1 and /workspace/project2. They are Gradle projects and are dependent on each other. I also have /workspace/project3 which is a Gradle project but independent.
I need to "upload" them to a git repo on Bitbucket. I installed EGit to help me with this. It asks me where my local repo is located and I don't know what to tell it. Should I create a repo per workspace or per group of dependent projects or what?
For example when I right click a project and press Team > Share > Git I get to this dialog and I don't know what I should fill in there.
Each (non-bare) git repository has a work directory in which the files of the currently checked out commit (referred to as HEAD) are stored.
By default, the work directory is the parent of the git metadata directory (named .git), but it can be detached through a config setting. For example: git config core.worktree /path/to/workdir
EGit cannot deal with repositories that have a detached work directory as answered here: EGit working directory not under the local repository
But irrespective of this limitation, you cannot split the work directory of a git repository among several workspaces.
Therefore, you should move the projects to the work directory to resemble this structure:
~/git/my-repo/.git
~/git/my-repo/project1
~/git/my-repo/project2
~/git/my-repo/project3
If project3 is independent of the others, it may remain where it is or be located somewhere else entirely.
A word of caution when planning to have interdependent projects spread across several git repositories. This makes the git repositories effectively interdependent as well, but without git having a dependency management tool.
To have EGit move the projects, you would enter project1 in the Path within repository input field.
Commonly used build files like the master pom.xml or corresponding Gradle files would be located in the root of the work directory. This also eases configuration of popular build services like Travis, Codeship and friends.
A few things to understand/keep in mind:
An Eclipse workspace is a logical container of projects, not necessarily a physical container. The folder in which a worksapce is located can also be a physical container of projects, meaning the project's files can be located in a folder under the workspace folder (this is the default location if you create a new project in Eclipse). But that is not a requirement; a project that is contained within a workspace can have its contents (its files) located in any location on your file system.
In most dev setups, the local git repos are located in a "standard" location in the user's home directory, namely %USER_HOME%/git/name-of-repo. Again, that's not a hard requirement, users have the option to clone or create repos in any location they choose.
So Eclipse eGit is showing you those 2 axes of flexibility. It's asking where you want to create your local git repo, which usually is different than your workspace folder. Use the Create... button to create it. I think at that point it will auto-populate the Path within repository field. If it does not, you can name that path anything you want (usually its the same as the project name, though not necessarily)
At the end of it all you'll have your workspace (logical container) as well as a local git repo which will be the physical location of the project contents.

Where to put Git repository in Eclipse?

If I try to put repository inside project folder
it warns it is not recommended to put git repository inside workspace.
If I try to create intermediate folder
it swears "overlaps the location of another project" at the stage of creating new project.
And if I try to create project inside workspace and repository outside, it moves the project also outside of workspace
So, the only way to use Git under Eclipse it totally abandon workspace?
If you choose the last solution you present, to deal with git and Eclipse, you will not abandon workspace, all your code will remain into your Project Workspace but you will get an additionnal folder somewhere else containing your same code, versioned, a local git repository. By choosing this solution, you can only version some parts of your project and you also avoid some possible conflicts that can arise when versioning your workspace project folder. You can also have a cleaner view of your code if you want it without eclipse metadata and do some complex git tasks on the commandline in this separate folder.

How to get src folder under project-root folder in github when staging from eclipse using egit

Current scenario :
In Eclipse when we create a project, say a PyDev project named ‘SimpleGit’, the folder structure created by eclipse will be like
EclipseWorkingDirectory/SimpleGit/src/
correct?
If we add this to git using Egit, by right clicking on the project folder SimpleGit then Team>Share, The folder structure in which repo is formed like this
Parent Folder/ SimpleGit/ .git
Parent Folder/ SimpleGit/SimpleGit/src/
And when we stage it to remote github everything will come under
Username/ SimpleGit / SimpleGit / src /
So when we look in GitHub in the root project folder (Username/ SimpleGit) we can’t see the source folder. It will be under another folder ‘SimpleGit’. How we can avoid this?
What I want is , I want to get my source forlder(src) listed under the Project Root directory, when staging from an already built project in eclipse using egit. How do I do this?
I want like this:
https://github.com/nicholasbishop/blender
Not like this:
https://github.com/afilash/SimpleGit
it cannot be done using eGit - it assumes the Eclipse project structure.
But it can be accomplished by using git itself. Just create a repository under SimpleGit folder executing 'git init'. From that point on you'd have to manage your git repository by either git itself (command line) or some other UI. SourceTree works really well for me.
It is a bit late, I know, but this method perfectly works and I hope it is useful for anyone who sees this post from now on.
The procedure is the following:
First step is creating a repository on Github (through github.com)
Move to Eclipse IDE (Git Repositories view) and click "Clone a Git Repository and add the clone to this view". Doing that the project will be somewhere located on the hard disk. Usually, it is stored in C:\Users\username\git\projectName but you are able to change it. This is our local repository.
Once done, press (in Eclipse IDE) File->Import->Projects from Git->Existing local repository(Select the one you cloned before)->Import using the new project wizard->Finish->Java->Java Project->Specify project name (just below the dialog box there is a checked checkbox that says use default location, uncheck it. Here is where you have to specify the directory you chose while cloning the repository previously).->Next->Finish
It should be enough for creating the right directory structure.

How to create Eclipse project with EGit clone

After spending a decade with SVN I've finally taken the plunge with Git. I have set up Git, Gitolite and GitLab on a server and have successfully added code to my local repository, committed, cloned repositories and pushed code back to repositories. So far, so good. Now enters EGit...
I have cloned a repository using the Git Repository Exploring view using the following syntax for the path:
ssh://dexter:vaultanalyser.git
(In GitLab, repositories are referred to as projects, so I assume that I am supposed to have one repository per Eclipse project? Rather than a SVN-style single parent repository that contains multiple projects?)
This imports the repository into:
/Users/mattpainter/git/vaultanalyser
I was expecting this step to automatically create an Eclipse project for me with all the source, but this isn't so. I tried fudging the target directory so it's in my workspace, but this isn't working either.
How do I get the cloned source available within Eclipse? This site implies that if you create a project with the same name as the repository, it all magically works, but this isn't the case (yes, I know the article is about Github, not GitLab, but I figured the two were close enough for the task at hand).
I've then tried creating a project in Eclipse and sharing it - but then the whole project appears as a sub-folder in the repository. If repositories are indeed analogous to projects, this isn't really what I want.
I've looked through other StackOverflow topics that look related, but I fear I'm still missing a key piece of understanding with how this is supposed to work and it's all looking remarkably convoluted thus far.
Help?
In GitLab, repositories are referred to as projects, so I assume that I am supposed to have one repository per Eclipse project?
Yes, but a GitLab "project" isn't necessarily an Eclipse one.
It doesn't have to follow an SVN structure, as illustrated in "Eclipse reference directory outside eclipse project directory but within repository".
All you need to do is to create an Eclipse project, specifying the source directory being not in the default path (Eclipse workspace), but wherever you cloned your repo (as described in "Getting started with Eclipse + EGit - confused").
That way, the eclipse project you just declared (and referenced in the Eclipse workspace) has its files (.project and .classpath) at the root of the Git repo.
And Egit can then manage that project just fine.
Or you can import it directly with Egit: "Eclipse + EGit: clone project into workspace".
As the OP nullpainter reports below:
The original issue was compounded by invalid permissions in the .git/objects folder on the server.
Running a chmod git:git -R * on the folder solved the issue
He details the right setup below.
To expand on #VonC's answer, the steps to get EGit and Eclipse to play nicely is:
Select 'Clone a Git repository' from EGit, accepting all defaults. This will create a folder in a git parent folder, somewhere outside your workspace.
Create a new Eclipse project. I'm using Java, but I assume there are similar steps for other languages. On the first dialog, untick 'Use default location' and instead select the repository folder created in step 1. Accept all defaults.
Select Team > Share Project... from your new project. Select Git, and tick the 'Use or create repository in parent folder of project'.
Now you can push your code and Eclipse dot files to your git repository from within Eclipse.
(My original issue was compounded by invalid permissions in the .git/objects folder on the server - running a chmod git:git -R * on the folder solved the issues)
I check it out with the command line - then build my projects on top of that. In fact, I end up doing most operations with command line git. EGit is useful for viewing the diffs but I find command line has more power and control. Git is mostly about giving devs lots of power and control.

Keeping custom build configuration files in a git repository

I've got a project in a git repository that uses some custom (and so far unversioned) setup scripts for the build environment etc. I'd like to put these under version control (hopefully git) but keep them versioned separate from the project itself, while still living in the base directory of the project - I've considered options like local branches but these seem to have the problem that switch back to master (or any other "real" branch) will throw away the working copies of the setup scripts.
I'm on Windows using msysgit so I've got a few tools to play with; does anyone have a recommendation or solution?
If you really need them separate from your main git repo while still living directly within it, you could try:
creating a new repo with those script within it
and:
adding that new repo as a submodule to your repo. Except:
a/ those scripts won't live directly in the base directory, but in a subfolder representing the submodule
b/ you need of course to not publish (push) that new repo, in order for other cloning your main repo to not get those setup files
or:
merging that new repo into your main repo (with the subtree project), but:
you need to split back your project to get rid of those files
for a project with a large history, and with frequent push, that step (the split) can be long and cumbersome.
I would consider a simpler solution, involving some evolution to your current setup files:
a private repo (as in "not pushed") with those setup files
environment variables with the path of your main git repo in order for your setup files (which would not be directly within the base directory of said main repo) to do their job in the right directory (like beginning for instance with a 'cd right_main_git_repo_dir').
I want to share an additional solution and some samples from which to start.
I've has a similar problem in attempting to build Mozilla Firefox with Buildbot -- I need to have some files in the root folder (namely the .mozconfig file and some helper scripts) and I wanted to version them separately.
My solution is as follow:
checkout the Firefox code from the Mercurial repository;
checkout an additional repository with the additional file I need;
before starting the build, I copy these file to the folder with the Firefox code.
This approach is implemented in the following repositories:
buildconfig-mozilla-central: it contains the Buildbot configuration, which
pulls both repositories
copies the files from the scripts repository
and start the build;
buildscripts-mozilla-central: the repository with the build configuration and helper scripts.
Please note that the code might not be well factored (for example the paths) but it should be a good starting point.
This procedure is tailored for Firefox, but it can be applied to any repository.