i am using jenkins for automation builds.
my issue is i want to download sources from svn and run the build steps and after running the builds steps once again i want to take latest sources from svn.
is there any plugin for it where my requirement satisfy.
Consider setting up two jobs (A and B) with a shared workspace (job > configure > Advanced Project Options ; click button Advanced...). check custom workspace and define a location). Once job A is finished it triggers job B and job B then performs a svn update plus whatever else you need. In order to avoid parallel execution of A and B, check Block build when upstream project is building and Block build when downstream project is building.
Maybe not a plugin, but you can always run manual SVN commands as part of the build step
Add a new build step to "Execute shell" (if on Linux) or "Execute Windows batch command"
(if on Windows).
Inside, write SVN commands, depending on your OS, for example:
svn up checkout_folder, note that path will be relative to Jenkin's workspace
Related
I have a Groovy repository which contains my Jenkins pipeline's Groovy code.
Currently, I am making changes in an IDE, commiting them to the repository, going to the Jenkins instance, manually triggering a Jenkins job, and checking to see if all of the changes all working. This is taking a lot of time.
Is there a way to do all of this from the IDE itself?
I would suggest to treat your pipeline code like some other code in IT. What are you doing now could be called "manual integration tests" because you are making your code changes and check how that code integrate with other components (like shell commands, jenkins plugins, etc.) on jenkins - this development loop is long and not efficient. So my proposition for you is to write simple unit tests using this framework:
https://github.com/jenkinsci/JenkinsPipelineUnit
So you can test your pipelines on your machine without any interaction with jenkins.
If you think that it's not proper way for you I would suggest to mix using this plugin for running jobs directly from IntelliJ: https://github.com/programisci/jenkins-control-plugin/
and of course IntelliJ git integration to commit your changes to repository.
For executing from the IDE, an option is to create some automation around using the Jenkins CLI. You should be able to see the CLI commands at http://your-jenkins-url/cli.
java -jar jenkins-cli.jar -s https://jenkins.physiq.zone/ replay-pipeline JOB [-n (--number) BUILD#] [-s (--script) SCRIPT]
Replay a Pipeline build with edited script taken from standard input
JOB : Name of the job to replay.
-n (--number) BUILD# : Build to replay, if not the last.
-s (--script) SCRIPT : Name of script to edit, such as Script3, if not the main Jenkinsfile.
For example, in IntelliJ you could use a Run Configuration that:
Downloads the CLI JAR
Executes it with the path to the local file with certain parameters
You can also write a script, Gradle build, or something else that wires into the IDE to pull the CLI JAR and execute a job with your local pipeline code.
For testing you may want to use https://github.com/jenkinsci/JenkinsPipelineUnit as already brought up, or a Gradle plugin that I maintain at https://github.com/mkobit/jenkins-pipeline-shared-libraries-gradle-plugin which uses the previously mentioned library for unit testing and the jenkinsci/jenkins-test-harness for integration testing.
I have a Gradle project that consists of a master project and 2 others that included using includeFlat directive. Each of these 3 projects has its own repo on GitHub. To build it I checkout all 3 projects into a common top folder then cd into the master project and run gradle build. And it works great!
Now I need to deploy the resulting app to AWS EB (Elastic Beanstalk) which is also works great when I produce the artifact locally and then deploy it manually. I want to automate the process so I'm trying to set it up using CodePipelines + Jenkins as described in this document adjusted for Gradle.
The problem is that if I specify 3 Sources in the pipe I end up with my projects extracted on top of each other creating a mess in Jenkins workspace. I need to somehow configure each project to be output to its own directory within Jenkins workspace and I just don't see a way to do it (at least in UI)
Then, of course even if I achieve what I want I need somehow to cd into the master directory to run gradle build and again I'm not sure how to do that
P.S. Great suggestions from #Phil but unfortunately is seems that CodePipeline does not currently support Git submodules or subtrees
I would start common build, when changes happened on any of 3 repos. With say 5 minutes delay, to have single build, even if changes are introduced to more then one repo.
I can't see good way to deal with deployment in other way than using eb deploy... old way... Please install aws tools at your jenkins machine. Create deployment job triggered on successful build. And put bash script doing deployment there. Please put more details about your deployment, that way I can help with deployment script.
I'm new to Jenkins CI.I'm trying to get SVN update (myFolder) inside a job as build steps. I want to explicitly copy some files to web root as I can't have them inside my solution.
Build Steps I need to perform.
Build Solution
Publish
Copy myFolder to web root
Sync
Up to Publish it works fine.Problem when trying to copy/update myFolder to web root.
MyFolder is located out of the project solution folder as I cant have it inside solution Folder.
Note: This myFolder has serialized items/object that I need to Sync in the next step.It should be copied to web root in-order to sync.
And this folder is committed to SVN.
In my local CMD following batch file works fine but when I try in Jenkins Execute Windows Batch Command it stops at
-- Updating source from SVN
-- Running update...
#echo off
cls
echo -- Initiating system instance variables...
echo. -- Setting the variables...
:: Here you need to make some changes to suit your system.
set SOURCE=C:\inetpub\wwwroot\Test\Website\App_Data\myFolder\
set SVN=C:\Program Files\TortoiseSVN\bin
:: Unless you want to modify the script, this is enough.
echo. %SOURCE%
echo. %SVN%
echo. ++ Done setting variables.
echo.
echo -- Updating source from SVN
echo. -- Running update...
"%SVN%\TortoiseProc.exe" /command:update /path:"%SOURCE%" /closeonend:1
echo. ++ Done.
echo. -- Cleaning up...
set SOURCE=
set SVN=
echo. ++ Done.
I have Subversion Plugin installed.Any solution for this problem.
And Also I tried using below Powershell Script
#Get checkout folder
TortoiseProc.exe /command:"update" /path:"C:\inetpub\wwwroot\Test\Website\App_Data\myFolder\"
It works in my local Windows Powershell but not in Jenkins Windows Powershell
In an effort to help answer your question, I will explain the configuration of a job which should accommodate what you are trying to achieve: building a project under version control after an svn update has been performed and moving the generated files to a separate directory.
Setup the Source Code Management section
Within this section in your job's configuration page, choose the appropriate version control system (ie, Subversion) and point the job to your project's URL, noted below. Also be mindful to select the appropriate check-out strategy. This is what Jenkins will use when your job runs (ie, svn update) as Jenkins will store a copy of your repository on the build-server in the job's workspace.
Without proceeding any further, this job will only pull down any changes from your repository through the appropriate check-out strategy configured above when this job runs.
However, you would like Jenkins job to actually do something meaningful when the job runs, such as build/publish your project. This is achieved through build steps, so let's configure build steps.
Configure the appropriate build step(s)
Build/Publish Website Locally
Assuming you have scripts already written to build/publish the website under version control (let's call it !Publish Website.bat as an example) which builds the project and publishes it locally, you can configure the step underneath the Build section as follows,
Note: %WORKSPACE% is a built-in environment variable which resolves to the current workspace of the job. There is a link under the build-step to list all the different environment variables exposed which can be used.
Without proceeding any further, the job will now pull down any changes and execute the batch file to publish/build a website locally within your workspace when this job runs.
Not quite done considering you wish to have these newly generated files to reside within your website's webroot folder so these changes are reflected on your website. For simplicity's sake we can go ahead and add another build-step to perform the copy.
Copy Contents to Webroot
Assuming you have scripts already written to copy the contents of the website under version control (let's call it !Copy Website.bat) which takes the published files and copies them to the appropriate directory on your webserver, you can configure the step underneath the Build section as follows,
Now when the job runs, it will perform an svn update against the repository on it's local workspace and execute the preceeding build-steps (ie, build/publish the solution and copy the contents to your webroot.)
I'm trying to split up a few Jenkins jobs using the Build Flow plugin so that instead of three monolithic jobs, we have three "starting points" that then use the DSL to trigger downstream jobs. I chose Build Flow over the Build Pipeline plugin because it seemed like it was a lot harder to share jobs between different pipelines ( ie, sharing the workspace of the multiple starting jobs with a single compile job ).
Previously, I had three jobs set up: Project-PR, Project-DEV, and Project-PROD.
Project-PR would build whenever a pull request happened in GitHub, and would just run a smaller subset of our unit tests, so that we could get quick verification that the PR is okay to merge.
Project-DEV would build whenever a feature branch was merged in GitHub into the main development branch, as well as having the ability to be manually triggered and given a different branch to pull. It would run the full suite of unit -- basically a sanity check that everything is still good. Then it would compile and minify, and push to a QA environment for testing, and then it would run the full suite of integration tests against that QA environment. This step was configured as a parametrized build, with the parameter being the name of the branch to pull, test, and push. It would push to and set up QA environment specific to that branch, so that we could QA multiple features without having to merge to development ( ie, feature-one.qa.example.com, feature-two.qa.example.com ).
Project-PROD would only ever be manually triggered, and would do the full unit and integration test suite, compile and minify the front-end code ( Less, JS, and CSS ), and push the built code into a special "release branch" in GitHub that can then be deployed -- we haven't quite reached the point of Jenkins being in charge of deployment.
Now, what I wanted to set up was to split the subtasks into their own jobs, so that it'd be easy to set up new jobs to without having to copy and paste all the build steps ( or copying the job and changing all the things that need to be unique ). This would let us do things like create a copy of the Project-DEV, but switch out the last job for one that deploys to a staging environment set up in the cloud. Or easily create a job that could report test results to a third party source, ie copy the results to a shared network folder or something. Or any number of things. The goal is basically to use these subtask jobs as building blocks to let us build more complicated jobs, while also making it easier to update how one portion of the build works ( for example, maybe we switch to a different technology for compiling, which might change how Jenkins would compile the code ).
For example, the Project-PR would be split into the following:
Project-PULL -> Project-SetupBuildEnv -> Project-PartialUnitTests
(BuildFlow) (Normal Job) (Normal Job)
The SetupBuildEnv would just pull down any NPM or Composer requirements, and set up the directories required for testing and building. PartialUnitTests then run, and report it's results back up to the
The Project-DEV could be split up like so:
Project-DEV -> Project->SetupBuildEnv -> Project-FullUnitTests -> Project-Compile -> Project-Minify -> Project-DeployQA -> Project-FullIntegrationTests
This way, the parts of the build process that are shared ( in this case, Project-SetupBuildEnv ) can be easily shared between jobs, reducing duplication, and making it easier to update a step in the build process without having to remember EVERY job that uses that step.
Right now, I'm using the Shared Workspace plugin so that all the steps use the same workspace. However, I'm running into an issue with that: it's not actually using one workspace. What's happening is that the Build Flow job will get a directory ( eg: /sharedspace/shared_one ), and download the code from GitHub into there. Then it will trigger the DSL, which starts up the 'SetupBuildEnv' job. But instead of working inside the same directory, it will get a directory with a name like "/sharedspace/shared_one#2", and run the build setup task in there. Then when it goes to do the third step ( unit testing ), it fails, because now it's got a third directory ( /sharedspace/shared_one#3 ), but that directory didn't have the setup run, so the required node and composer modules are missing. What's weird is that it looks like the Shared Workspace plugin is copying the first shared workspace to another directory and incrementing a counter ( the #N part of the directory name ) and giving that to the other jobs to work in.
So, question time:
is there a way to fix the Shared Workspace plugin so that it's actually only using one directory for each job?
if not, is it possible to have the Clone Workspace plugin take an argument, so I can specify what archived workspace to use instead of using the dropdown?
another possiblity: would using the shared workspace plugin, but use the "Local subdirectory for repo (optional)" option in the advanced git job options to specify the directory to use?
failing all that, is there some other way to set up a build pipeline that can share jobs with other pipelines that I've missed out on?
In my experience, even if you do get this working, this might not be a scalable way to go longer term. We've found the shared workspace plugin entirely a bad idea for long / complex builds (similar reasons to yours - but also: scaling across dozens of slaves becomes hard suddenly). Arguably the idea is slightly against the spirit of modern scalable CI.
I'd instead delegate more to your build tools, be they Maven / Gradle, Ant, even Grunt, whatever. If you want to keep these builds truly modular, but can't afford to rebuild at each step (we decided full independence was worth wasting a few minutes per build) then perhaps look at creating useful artefacts at key stages - in your case, minified assets TARs, library JARs, or maybe webjars, or whatever, and deploy them to a (Maven?) repo.
Later build steps in your pipeline can quickly, easily, and repeatably pull the latest (or named version) assets from this centralised repo, and continue with the build process.
An alternative (with similarities) is to build one or more assets, but only promote them after increasing numbers of tests are run, which can be done in separate builds coordinated by your build flow, using the Promoted Builds plugin etc.
I have a task to check compilation of code from one of our branches, lets call it "stable". Our team mainly working in "HEAD" branch and sometimes making some merges to "stable" branch. I've made a local copy of our main workspace and named it "main-workspace-stable" and replaced all of the code by code from "stable" branch. Now at the end of every work day I open that workspace, update the code to the most recent and wait for the compilation result.
This is pretty boring. Is there any ways to automate that task?
You might want to look at one of the Continuous Integration tools, for example Cruise Control or Hudson.
As mentioned by ChssPly76, Continuous Integration is the way to implement what you want to do.
Regarding Hudson, I can mention hudson-eclipse, in order for you to monitor a Hudson health icon, displayed at the bottom of the Eclipse window. The icon is red on build failure and green on success.
You can couple that with an automatic publication of your project as a maven snapshot artifact (also triggered by Hudson).
And have Maven compile your Eclipse project in that special Continuous Integration Environment.
The idea is not to rely on a pure Eclipse solution, because eclipse cannot be up and running every environment, especially those where you do not actively develop, but only compile and launch tests.