Publish Nunit Test Results in Post Always Section - powershell

I'm trying to run a pipeline that does some Pester Testing and publish the NUnit results.
New tests were introduced and for whatever the reason, Jenkins no longer publishes the test results and errors out immediately after the powershell script. Hence, it doesn't get to the nunit publish piece. I receive this:
ERROR: script returned exit code 128
Finished: FAILURE
I've been trying to include the publish in the always section of the post section of the Jenkinsfile, however, I'm running into problems on how to make that NUnit test file available.
I've tried establishing an agent and unstash the file (even though it probably won't stash if the powershell script cancels the whole pipeline). When I use agent I get the following exception:
java.lang.NoSuchMethodError: No such DSL method 'agent' found among steps
Here is the Jenkinsfile:
pipeline {
agent none
environment {
svcpath = 'D:\\svc\\'
unitTestFile = 'UnitTests.xml'
}
stages {
stage ('Checkout and Stash') {
agent {label 'Agent1'}
steps {
stash name: 'Modules', includes: 'Modules/*/**'
stash name: 'Tests', includes: 'Tests/*/**'
}
}
stage ('Unit Tests') {
agent {label 'Agent1'}
steps {
dir(svcpath + 'Modules\\'){deleteDir()}
dir(svcpath + 'Tests\\'){deleteDir()}
dir(svcpath){
unstash name: 'Modules'
unstash name: 'Tests'
}
dir(svcpath + 'Tests\\'){
powershell """
\$requiredCoverageThreshold = 0.90
\$modules = Get-ChildItem ../Modules/ -File -Recurse -Include *.psm1
\$result = Invoke-Pester -CodeCoverage \$modules -PassThru -OutputFile ${unitTestFile} -OutputFormat NUnitXml
\$codeCoverage = \$result.CodeCoverage.NumberOfCommandsExecuted / \$result.CodeCoverage.NumberOfCommandsAnalyzed
Write-Output \$codeCoverage
if (\$codeCoverage -lt \$requiredCoverageThreshold) {
Write-Output "Build failed: required code coverage threshold of \$(\$requiredCoverageThreshold * 100)% not met. Current coverage: \$(\$codeCoverage * 100)%."
exit 1
} else {
write-output "Required code coverage threshold of \$(\$requiredCoverageThreshold * 100)% met. Current coverage: \$(\$codeCoverage * 100)%."
}
"""
stash name: 'TestResults', includes: unitTestFile
nunit testResultsPattern: unitTestFile
}
}
post {
always {
echo 'This will always run'
agent {label 'Agent1'}
unstash name: 'TestResults'
nunit testResultsPattern: unitTestFile
}
success {
echo 'This will run only if successful'
}
failure {
echo 'This will run only if failed'
}
unstable {
echo 'This will run only if the run was marked as unstable'
}
changed {
echo 'This will run only if the state of the Pipeline has changed'
echo 'For example, if the Pipeline was previously failing but is now successful'
}
}
}
Any and all input is welcome! Thanks!

The exception you are getting is due to Jenkins' strict pipeline DSL. Documentation of allowable uses of agent are here.
Currently agent {...} is not allowed to be used in the post section. Maybe this will change in the future. If you require the whole job to run on the node that services label 'Agent1' the only way to currently do that is to
Put agent {label 'Agent1'} immediately under pipeline { to make it global
Remove all instances of agent {label 'Agent1'} in each stage
Remove the agent {label 'Agent1'} from the post section.

The post section acts more like traditional scripted DSL than the pipeline declarative DSL. So you have to use node() instead of agent.
I believe I've had this same question myself, and this SO post has the answer and some good context.
This Jenkins issue isn't exactly the same thing but shows the node syntax in the post stage.

Related

How to load variables from a powershell script and access the same in groovy jenkinsfile pipeline variable

I have a requirement where I have to load powershell variables from a powershell script and store the vairable value in a groovy jenkins pipeline variable and use it thereafter to edit the name of an artifact depending on that variable's value.
powershell script: Variables.ps1 (in real scenario this has number of variables but this is just for sample)
$Version = "22.4"
jenkinsfile:
pipeline {
agent any
stages {
stage('TestPowershell') {
steps {
script {
def path = "${env.WORKSPACE}\\Power\\Variables.ps1"
echo path
def versionFromPowershell = powershell(returnStdout: true, script: " . '${path}'; return $Version;")
echo versionFromPowershell
}
}
}
}
}
I get an error when I use this method as below:
groovy.lang.MissingPropertyException: No such property: Version for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:251)
at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:353)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:357)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at WorkflowScript.run(WorkflowScript:26)
In the vannila powershell the script works fine and does the job, not sure why the same syntax doesn't work when invoked via jenkins build. Any help is much appreciated!
Thanks
Shobhit
You cannot interpolate Powershell variables in a Groovy interpreter. Therefore, the script argument to the step method must contain escaped variable syntax characters such that the variable Version is interpreted by Powershell and not Groovy:
def versionFromPowershell = powershell(returnStdout: true, script: " . '${path}'; \$Version;")

Running executables with arguments from a Jenkinsfile (using bat - under Windows)

I have read multiple threads and I still cannot figure out how to get my Jenkinsfile to run some applications such as NuGet.exe or devenc.com.
Here is what I have so far as a Jenkins file:
pipeline {
agent any
options {
timestamps ()
skipStagesAfterUnstable()
}
environment {
solutionTarget = "${env.WORKSPACE}\\src\\MySolution.sln"
}
stages {
stage('Build Solution') {
steps {
dir ("${env.workspace}") {
script {
echo "Assumes nuget.exe was downloaded and placed under ${env.NUGET_EXE_PATH}"
echo 'Restore NuGet packages'
""%NUGET_EXE_PATH%" restore %solutionTarget%"
echo 'Build solution'
""%DEVENV_COM_PATH%" %solutionTarget% /build release|x86"
}
}
}
}
}
}
In this example, I get the following error:
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.lang.String.mod() is applicable for argument types: (java.lang.String) values: [C:\Program Files (x86)\NuGet\nuget.exe]
Note that a declarative checkout is in place, although not visible from the Jenkinsfile:
I have also tried to use a function to run those cmds, but without success either:
def cmd_exec(command) {
return bat(returnStdout: true, script: "${command}").trim()
}
Any tip would be highly appreciated.

Jenkinsfile with Docker PowerShell image is getting hanged

When the Docker powershell gets invoked from a jenkinsfile it keeps executing and the job doesn't gets terminated.
pipeline {
agent {
docker {
image 'mcr.microsoft.com/azure-powershell'
args "--mount type=bind,src=/opt,dst=/opt -i -t --entrypoint=''"
}
}
stages {
stage('PwShell') {
steps {
powershell(returnStdout: true, script: 'Write-Output "PowerShell is mighty!"')
}
}
}
}
jenkin-job-hang
As you have not provided any log details it's tough to pin point the error as why it is stuck. Generally, for Jenkins declarative pipeline like the following -
pipeline {
agent {
docker { image '...abc.com' }
}
stages {
stage('Test') {
steps {
sh 'node --version'
}
}
}
}
When the Pipeline executes, Jenkins will automatically start the specified container and execute the defined steps within it:
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
[guided-tour] Running shell script
+ node --version
v14.15.0
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
And since you are stuck in the PowerShell line in the steps, it means the steps is not correctly set.
If you check this Microsoft PowerShell Support for Pipeline document then you will find that by default returnStdout returns the standard output stream with a default encoding of UTF-8. And Write-Output cmdlet returns an output stream.
You should be able to solve your problem by following the changing your PowerShell line as the code snippet shown below.
steps {
def msg = powershell(returnStdout: true, script: 'Write-Output "PowerShell is mighty!"')
println msg
}
Check this example section of the same above mentioned document for more detailed operations on the PowerShell steps.
I would also suggest to read this Using Docker with Pipeline document for more information.

Not able to run an exe in jenkins pipeline using powershell

I am trying to execute a process which is written in c# through jenkins pipeline during the build and deployment process.
It is a simple executable which takes 3 arguments, when it gets called from jenkins pipeline using a powershell function it doesn't write any logs which are plenty within the code of this exe, also it does not show anything on the pipeline logs as to what happened to this process. Whereas the logs output is clean before and after the execution of this process i.e. "Started..." & "end" gets printed in the jenkins build log.
When i try to run the same exe on a server directly with the same powershel script it runs perfectly fine. Could you please let me know how can i determine whats going wrong here or how can i make the logs more verbose so i can figure out the root cause.
Here is the code snippet
build-utils.ps1
function disable-utility($workspace) {
#the code here fetches the executable and its supporting libraries from the artifactory location and unzip it on the build agent server.
#below is the call to the executable
Type xmlPath #this prints the whole contents of the xml file which is being used as an input to my exe.
echo "disable exe path exists : $(Test-Path ""C:\Jenkins\workspace\utils\disable.exe"")" // output is TRUE
echo "Started..."
Start-Process -NoNewWindow -Filepath "C:\Jenkins\workspace\utils\disable.exe" -ArgumentList "-f xmlPath 0" #xmlPath is a path to a xml file
echo "end."
}
jenkinsfile
library {
identifier: 'jenkins-library#0.2.14',
retriever: legacySCM{[
$class: 'GitSCM',
userRemoteConfigs: [[
credtialsId: 'BITBUCKET_RW'
url: <htps://gitRepoUrl>
]]
]}
}
def executeStep(String stepName) {
def butil = '.\\build\\build-utils.ps1'
if(fileExists(butil))
{
def status = powershell(returnStatus: true, script: "& { . '${butil}'; ${stepName}; }")
echo status
if(status != 0) {
currentBuild.Result = 'Failure'
error("$StepName failed")
}
}
else
{
error("failed to find the file")
}
}
pipeline {
agent {
docker {
image '<path to the docker image to pull a server with VS2017 build tools>'
lable '<image name>'
reuseNode true
}
}
environment {
#loading the env variables here
}
stages {
stage {
step {
executeStep("disable-utility ${env.workspace}")
}
}
}
}
Thanks a lot in advance !
Have you changed it ? go to Regedit [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System Set "EnableLUA"= 0

How to run continuous integration in parallel across multiple Pull Requests?

I am testing use of Jenkins with Github pull request builder plugin I have successfully set up a toy project on Github and dev installation of Jenkins so that raising a PR, or pushing changes to a PR branch triggers a build. Mostly this works as required - a few things don't match preferred workflow, but the freedom from having to write and maintain our own plugin is a big deal.
I have one potential showstopper. The plugin queues up all pushes in all PRs it sees, and only ever seems to run a single job at a time, even with spare executors available. In the real world project, we may have 10 active PRs, each may get a few pushed updates in a day in response to QC comments, and the full CI run takes > 30 mins. However, we do have enough build executors provisioned to run multiple jobs at the same time.
I cannot see any way to configure the PR request builder to process multiple jobs at once on the same trigger, but I may be missing something basic elsewhere in Jenkins. Is there a way to do this, without needing to customise the plugin?
I have installed Jenkins ver. 1.649 on a new Ubuntu 14.04 server (on a VirtualBox guest) and followed the README in the ghprb plugin (currently version 1.30.5), including setting up a jenkins "bot" account on Github as a collaborator to run all the integration API calls to Github.
I was wondering what the behaviour would be if I cloned the job (create new item and "Copy existing item"), and may try that next, but I expect that will result in the same job being run multiple times for no benefit as opposed to interacting smartly with other jobs polling the same pool of PRs.
I have found the config setting whilst exploring more for the question.
It is really easy when you know which config item it is, but Jenkins has a lot of configuration to work through, especially when you are exploring the plugins.
The key thing is that the option to serve queued jobs in parallel (available executors allowing) is core Jenkins config, and not part of the Github PR builder.
So, just check the option Execute concurrent builds if necessary. This option should be found at the bottom of the first, untitled section of config. It is a really basic Jenkins option, that a newbie like me missed due to the mountain of other options.
May be it is too late to answer this question, but after few days of researching I figured out a way to create multiple jobs per PR in github.
The code I am showing here applies to github enterprise, but it works well enough for the general github(bitbucket) as well with a few tweaks in url and git command.
The mainline repository against which the PRs are created needs to have a file, I call it PRJob.groovy and contains
import groovy.json.JsonSlurper
gitUrl = GIT_URL
repoRestUrl = "${GITHUB_WEB_URL}/repos/${project}/${repo}"
def getJSON(url) {
def conn = (HttpURLConnection) new URL(url).openConnection()
conn.setRequestProperty("Authorization", "token ${OAUTH_TOKEN}");
return new JsonSlurper().parse(new InputStreamReader(conn.getInputStream()))
}
def createPipeline(name, description, branch, prId) {
return pipelineJob(name) {
delegate.description description
if (ENABLE_TRIGGERS == 'true') {
triggers {
cron 'H H/8 * * *'
scm 'H/5 * * * *'
}
}
quietPeriod(60)
environmentVariables {
env 'BRANCH_NAME', branch
env 'PULL_REQUEST', prId
env 'GITHUB_WEB_URL', GITHUB_WEB_URL
env 'OAUTH_TOKEN', OAUTH_TOKEN
env 'PROJECT', project
env 'REPO', repo
}
definition {
cpsScm {
scriptPath "Jenkinsfile"
scm {
git {
remote {
credentials "jenkins-ssh-key"
delegate.url gitUrl
if (prId != "") {
refspec "+refs/pull/${prId}/*:refs/remotes/origin/pr/${prId}/*"
}
}
delegate.branch branch
}
}
}
}
}
}
def createPRJobs() {
def prs = getJSON("${repoRestUrl}/pulls?state=open")
if (prs.size() == 0) {
def mergedPrs = getJSON("${repoRestUrl}/pulls?state=closed")
if (mergedPrs.size() == 0) {
throw new RuntimeException("No pull-requests found; auth token has likely expired")
}
}
prs.each { pr ->
def id = pr.get("number")
def title = pr.get("title")
def fromRef = pr.get("head")
def fromBranchName = fromRef.get("ref")
def prRepo = fromRef.get("repo")
def repoName = prRepo.get("name")
def prHref = pr.get("url")
createPipeline("${repo}-PR-${id}-${fromBranchName}",
"${prHref} Pull Request ${id}: ${title}", "origin/pr/${id}/head", id)
}
}
createPRJobs()
This creates 1 jenkins job per PR.
This relies on the project having a Jenkinsfile which can be picked up for running a peipeline job. A sample Jenkinsfile will look like below:
//Jenkinsfile for building and creating jobs
commitId = null
repoRestUrl = "${GITHUB_WEB_URL}/repos/${PROJECT}/${REPO}"
try{
stage('Install and Tests') {
runTest("Hello")
}
notify_github 'success'
}catch (Exception e) {
notify_github 'failure'
print e
throw e
}
def runTest(String someDummyVariable) {
node {
checkout scm
sh 'git clean -qdf'
if (env.PULL_REQUEST == ""){
sh 'git rev-parse --verify HEAD > commit.txt'
} else {
// We check out PR after it is merged with master, but we need to report the result against the commit before merge
sh "git rev-parse refs/remotes/origin/pr/${env.PULL_REQUEST}/head^{commit} > commit.txt"
}
commitId = readFile 'commit.txt'
echo commitId
sh 'rm -f commit.txt'
//Here goes your code for doing anything
sh 'echo "Hello World!!!!!"'
}
}
def http_post(url, rawJson) {
def conn = (HttpURLConnection) new URL(url).openConnection()
conn.setRequestProperty("Authorization", "token ${OAUTH_TOKEN}");
conn.doOutput = true
conn.requestMethod = "POST"
conn.setRequestProperty("Content-Type", "application/json")
def wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(rawJson);
wr.close()
def code = conn.getResponseCode()
if (code < 200 || code >= 300){
println 'Failed to post to ' + url
def es = conn.getErrorStream();
if (es != null) {
println es.getText()
}
}
}
def notify_github(state) {
http_post(
"${repoRestUrl}/statuses/${commitId}",
"""
{ "state": "${state}",
"target_url": "${env.BUILD_URL}",
"description": "Build Pipeline",
"context": "Build Pipeline"
}
"""
)
}
Hope this helps someone.