I am trying to checkout code from SVN repo for which I am accepting the URL as argument. I have quoted the URL as shown below because it contains spaces. I also checked the parameter by redirecting the $svn_url in file (shown below). If I pick the URL from the file and pass it as is on the command line to the given script, it works fine but somehow when invoked from Puppet, it's not working.
Puppet manifests:
repo_checkout.pp:
define infra::svn::repo_checkout ($svn_url_params) {
$svn_url = $svn_url_params[svn_url]
include infra::params
$repo_checkout_ps = $infra::params::repo_checkout_ps
file { $repo_checkout_ps:
ensure => file,
source => 'puppet:///modules/infra/repo_checkout.ps1',
}
util::executeps { 'Checking out repo':
pspath => $repo_checkout_ps,
argument => "\'\"$svn_url\"\'",
}
}
params.pp:
$repo_checkout_ps = 'c:/scripts/infra/repo_checkout.ps1',
site.pp:
$svn_url_ad = {
svn_url => 'https:\\\\some_repo.abc.com\svn\dir with space\util',
}
infra::svn::repo_checkout { "Checking out code in C:\build":
svn_url_params => $svn_url_ad
}
executeps.pp:
define util::executeps ($pspath, $argument) {
$powershell = 'C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe -NoProfile -NoLogo -NonInteractive'
exec { "Executing PS file \"$pspath\" with argument \"$argument\"":
command => "$powershell -file $pspath $argument",
timeout => 900,
}
}
PowerShell code:
$svn_url = $args[0]
Set-Location C:\build
echo "svn co --username user --password xxx --non-interactive '$svn_url'" | Out-File c:\svn_url
svn co --username user --password xxx --non-interactive '$svn_url'
Puppet output on agent node:
Util::Executeps[Checking out repo]/Exec[Executing PS file "c:/scripts/infra/repo_checkout.ps1" with argument "'"https:\\some_repo.abc.com\svn\dir with space\util"'"]/returns: executed successfully
Notice: Applied catalog in 1.83 seconds
Content of c:\svn_url:
'https:\\\\some_repo.abc.com\svn\dir with space\util'
UPDATE: Sorry for the confusion but i was trying out several permutations and combinations and in doing that, i forgot to mention that when the $svn_url contains backslash (\), it does NOT work on the command line too if i copy the SVN URL from the text file where i am redirecting the echo output.
Based on #Ansgar's suggestion, i changed '$svn_url' to "$svn_url" in powershell code but the output in text file then contained ' quote twice around the URL. So i changed the argument parameter from "\'\"$svn_url\"\'" to "\"$svn_url\"". Now the output file had only single quote present around the URL. I copied only the URL (along with single quotes around it) from the output file and tried passing it to the powershell script. I now get the following error:
svn: E020024: Error resolving case of 'https:\\some_repo.abc.com\svn\dir with space\util'
Another thing to note is that if i change the back slashes in URL to forward slashes, it works fine on the command line. Invoking from Puppet still doesn't work.
Posting the final configuration that worked out for me based on #AnsgarWiechers' suggestion.
[tom#pe-server] cat repo_checkout.pp
define infra::svn::repo_checkout ($svn_url_params) {
$svn_url = $svn_url_params[svn_url]
...
...
util::executeps { 'Checking out repo':
pspath => $repo_checkout_ps,
argument => "\"$svn_url\"",
}
}
[tom#pe-server] cat repo_checkout.ps1
$svn_url = $args[0]
Set-Location C:\build
svn co --username user --password xxx --non-interactive "$svn_url"
[tom#pe-server] cat params.pp
$repo_checkout_ps = 'c:/scripts/infra/repo_checkout.ps1',
[tom#pe-server] cat site.pp
$svn_url_ad = {
svn_url => 'https://some_repo.abc.com/svn/dir with space/util',
}
infra::svn::repo_checkout { "Checking out code in C:\build":
svn_url_params => $svn_url_ad
}
Thanks a lot #AnsgarWiechers! :)
Note:
In site.pp: Used forwardslashes (/) when specifying svn_url
In repo_checkout.ps1: Changed '$svn_url' to "$svn_url"
In repo_checkout.pp: Changed double-nested (' and ") quoting in argument to single (") nested i.e., from "\'\"$svn_url\"\'" to "\"$svn_url\""
Related
I have a requirement where I have to load powershell variables from a powershell script and store the vairable value in a groovy jenkins pipeline variable and use it thereafter to edit the name of an artifact depending on that variable's value.
powershell script: Variables.ps1 (in real scenario this has number of variables but this is just for sample)
$Version = "22.4"
jenkinsfile:
pipeline {
agent any
stages {
stage('TestPowershell') {
steps {
script {
def path = "${env.WORKSPACE}\\Power\\Variables.ps1"
echo path
def versionFromPowershell = powershell(returnStdout: true, script: " . '${path}'; return $Version;")
echo versionFromPowershell
}
}
}
}
}
I get an error when I use this method as below:
groovy.lang.MissingPropertyException: No such property: Version for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:251)
at org.kohsuke.groovy.sandbox.impl.Checker$7.call(Checker.java:353)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:357)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:333)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at WorkflowScript.run(WorkflowScript:26)
In the vannila powershell the script works fine and does the job, not sure why the same syntax doesn't work when invoked via jenkins build. Any help is much appreciated!
Thanks
Shobhit
You cannot interpolate Powershell variables in a Groovy interpreter. Therefore, the script argument to the step method must contain escaped variable syntax characters such that the variable Version is interpreted by Powershell and not Groovy:
def versionFromPowershell = powershell(returnStdout: true, script: " . '${path}'; \$Version;")
I am trying to run MATLAB script inside powershell in one of the stages like this
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: "${myID}", usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']])
{
script {
env.VAL_RESULT = powershell(script: "matlab -wait -r \"JUsername='$USERNAME';JPassword='$PASSWORD';modelValidation;\"", returnStatus: true)
if(env.VAL_RESULT == "1" ) {
unstable("Failed")
}
}
}
Trying to do this will give a warning in console like
Warning: A secret was passed to "powershell" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [PASSWORD, USERNAME]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
In-order to solve the warning, I had to encapsulate powershell scripts within single quotes like
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: '${myID}', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']])
{
script {
env.VAL_RESULT = powershell(script: 'matlab -wait -r \"JUsername='$USERNAME';JPassword='$PASSWORD';modelValidation;\"', returnStatus: true)
if(env.VAL_RESULT == "1" ) {
unstable("Failed")
}
}
}
Now the script doesn't exit. I believe that there is something wrong with the way I have used single quotes in JPassword='$PASSWORD'. Can anyone tell me if there is a way to escape single quote?
I am trying to execute a process which is written in c# through jenkins pipeline during the build and deployment process.
It is a simple executable which takes 3 arguments, when it gets called from jenkins pipeline using a powershell function it doesn't write any logs which are plenty within the code of this exe, also it does not show anything on the pipeline logs as to what happened to this process. Whereas the logs output is clean before and after the execution of this process i.e. "Started..." & "end" gets printed in the jenkins build log.
When i try to run the same exe on a server directly with the same powershel script it runs perfectly fine. Could you please let me know how can i determine whats going wrong here or how can i make the logs more verbose so i can figure out the root cause.
Here is the code snippet
build-utils.ps1
function disable-utility($workspace) {
#the code here fetches the executable and its supporting libraries from the artifactory location and unzip it on the build agent server.
#below is the call to the executable
Type xmlPath #this prints the whole contents of the xml file which is being used as an input to my exe.
echo "disable exe path exists : $(Test-Path ""C:\Jenkins\workspace\utils\disable.exe"")" // output is TRUE
echo "Started..."
Start-Process -NoNewWindow -Filepath "C:\Jenkins\workspace\utils\disable.exe" -ArgumentList "-f xmlPath 0" #xmlPath is a path to a xml file
echo "end."
}
jenkinsfile
library {
identifier: 'jenkins-library#0.2.14',
retriever: legacySCM{[
$class: 'GitSCM',
userRemoteConfigs: [[
credtialsId: 'BITBUCKET_RW'
url: <htps://gitRepoUrl>
]]
]}
}
def executeStep(String stepName) {
def butil = '.\\build\\build-utils.ps1'
if(fileExists(butil))
{
def status = powershell(returnStatus: true, script: "& { . '${butil}'; ${stepName}; }")
echo status
if(status != 0) {
currentBuild.Result = 'Failure'
error("$StepName failed")
}
}
else
{
error("failed to find the file")
}
}
pipeline {
agent {
docker {
image '<path to the docker image to pull a server with VS2017 build tools>'
lable '<image name>'
reuseNode true
}
}
environment {
#loading the env variables here
}
stages {
stage {
step {
executeStep("disable-utility ${env.workspace}")
}
}
}
}
Thanks a lot in advance !
Have you changed it ? go to Regedit [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System Set "EnableLUA"= 0
been trying several trial and errors on where or how to cut the string in the variable and assign to a new variable to be used by jenkins stage. Normally just removing -TEST Jenkins pipeline indicated below:
properties([
[$class: 'RebuildSettings', autoRebuild: false, rebuildDisabled: false],
parameters([choice(choices: ['SQA-ENV-CLONE', 'DEV-ENV-CLONE'],
description: 'Select the ENV', name: 'ENV')])])
pipeline {
agent any
stages {
stage('VALIDATE ENVIRONMENT') {
def ACTIVE = sh(returnStdout: true, script: "echo $ENV | sed -e 's/-CLONE//g'")
steps {
echo 'Checking 1st the ${ACTIVE}'
}
}
}
}
Error I'm getting
Running in Durability level: MAX_SURVIVABILITY
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 6: Not a valid stage section definition: "def ACTIVE = sh(returnStdout: true, script: "echo $ENV | sed -e 's/-CLONE//g'")". Some extra configuration is required. # line 6, column 9.
stage('VALIDATE ENVIRONMENT') {
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:330)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
Lets say I choose "DEV-ENV-CLONE" as the value I'm expecting to have a successful build with this output:
Checking 1st the DEV-ENV
You need move the def ACTIVE = sh(...) into pipeline step script, They are Groovy script, only can be wrapped by script step.
stage('VALIDATE ENVIRONMENT') {
steps {
script {
ACTIVE = sh(
returnStdout: true,
script: "echo $ENV | sed -e 's/-CLONE//g'"
).trim()
}
echo "Checking 1st the ${ACTIVE}"
}
}
I am using Perl library File::RsyncP.
This moment script copies all files, but I need to use
extension and later pattern for example 2019*.xml
I don´t know how to filter files by extension.
I have tried following
I take a connection into localhost
I have
/home/raimo/A/SRC/srcDirectory/1.xml
/home/raimo/A/SRC/srcDirectory/2.xml
/home/raimo/A/SRC/srcDirectory/3.txt
here
I would like to copy only xml files into
I have tried:
rsyncCmd => "/bin/rsync -avz --include '*.xml' srcDirectory destDirectory ", and I have tired rsyncCmd => "/bin/rsync -avz --include '*.xml' destDirectory srcDirectory "
/home/raimo/A/SRC/destDirectory
my $rs = File::RsyncP->new({
logLevel => 5,
rsyncCmd => "/bin/rsync --include '*.xml'", #check if possible to filter
rsyncArgs => [
"--numeric-ids",
"--perms",
"--owner",
"--group",
"--devices",
"--links",
"--ignore-times",
"--block-size=700",
"--relative",
"--recursive",
"--verbose"
],
});
...
# Receive files from remote srcDirectory to local destDirectory
# # by running rsyncCmd with rsyncArgs.
$rs->remoteStart(1, srcDirectory);