How to get console output of downstream job in upstream job? - powershell

I'm trying to find a workaround because first question still is unanswered.
can't run Start-Job with credentials from Jenkins
I have job A. Job A starts powershell script at server and shows some output.
Also I have a pipeline B that runs multiple copy of job A against different servers.
Here is the groovy code
stage 'Copy sources from Git'
build job: 'DeploymentJobs/1_CopySourcesFromGit'
stage 'Deploy to servers'
def servers = env.SERVERLIST.split('\n')
def steps =[:]
for (int i=0; i<servers.size(); i++) {
def server = servers[i]
def stepName = "running ${server}"
steps[stepName] = {->
echo server
build job: 'DeploymentJobs/2_DeployToServer', parameters:
[booleanParam(name: 'REBOOTAFTER', value: Boolean.valueOf(REBOOTAFTER)),
string(name: 'SERVERNAME', value: server)]
}
}
parallel steps
In the output of pipeline I see only info than N copies of job A started but no their output.
I want to see only powershell output from all instances of job A in console output of pipeline B.
I have no iedea how to do this, Is it possible.

A shorter alternative:
def result = build job: 'job_name', wait: true
println result.getRawBuild().getLog()
It will be necessary to whitelist both methods.
Edit: since you don't want to wait for the build to run, you could add this at the end of your job (or at some point after all triggered builds should be finished) where number_of_builds in your case will be servers.size().
def job = Jenkins.getInstance().getItemByFullName('job_A_name')
def last_build = job.getLastBuild().getNumber()
def first_build = last_build - number_of_builds
(first_build..last_build).each {
println "Log of build $it"
def build = job.getBuildByNumber(it)
println build.log
}
If you really want to be sure the builds you're getting are the ones triggered by your job, you can get the build cause from the build object.

I solved my problem by adding this into the cycle. I could get log of downstrema jobs into upstream, display it and work with it.
def checkjob = build job: 'job name', parameters: [ any params here]
checklog = Jenkins.getInstance().getItemByFullName('job name').getBuildByNumber(checkjob.getNumber()).log
println checklog

Here's a silly example:
$myJobs = #()
$myJobs += Start-job -ScriptBlock { while (1) {Get-Item 'c:\*'; sleep 5}}
$myJobs += Start-job -ScriptBlock { while (1) {Get-Item 'c:\windows\*'; sleep 5}}
try {
while(1)
{
$myJobs | Get-Job -HasMoreData $true | Receive-Job
}
} finally {
$myJobs | Stop-Job
$myJobs | Remove-Job
}
Anything the job pipelines is queued. The -HasMoreData state indicates that the job has output that is available to read. The parent receives the output using Receive-Job. By default it displays in the console, but you can receive the output in the parent process and do further processing.
If this isn't what you're going for you'll have to be more specific in your question. Provide a little of the code you've tried so far.

def jobLog = Jenkins.getInstance().getItemByFullName('job-path/testjob').getLastSuccessfulBuild().log
println(jobLog)

Related

Celery chain - if any tasks fail, do x, else y

I'm just getting into Celery chains in my Django project. I have the following function:
def orchestrate_tasks_for_account(account_id):
# Get the account, set status to 'SYNC' until the chain is complete
account = Account.objects.get(id=account_id)
account.status = "SYNC"
account.save()
chain = task1.s(account_id) | task2.s() | task3.s()
chain()
# if any of the tasks in the chain failed, set account.status = 'ERROR'
# else set the account.status = 'OK'
The chain works as expected, but I'm not sure how to take feedback from the chain and update the account based on the results
In other words, I'd like to set the account status to 'ERROR' if any of the tasks in the chain fail, otherwise I'd like to set the account status to 'OK'
I'm confused by the Celery documentation on how to handle an error with an if/else like I've commented in the last two lines above.
Does anyone have experience with this?
Ok - here's what I came up with
I've leveraged the waiting library in this solution
from celery import chain
from waiting import wait
def orchestrate_tasks_for_account(account_id):
account = Account.objects.get(id=account_id)
account.status = "SYNC"
account.save()
job = chain(
task1.s(account_id),
task2.s(),
task3.s()
)
result = job.apply_async()
wait(
lambda: result.ready(), # when async job is completed...
timeout_seconds=1800, # wait 1800 seconds (30 minutes)
waiting_for="task orchestration to complete"
)
if result.successful():
account.status = 'OK'
else:
account.status = 'ERROR'
account.save()
I am open to suggestions to make this better!

Powershell script not working when included in Jenkins Pipeline Script

What I am trying to achieve with Powershell is as follows:
Increment the Build Number in the AssemblyInfo.cs file on the Build Server. My Script looks like below right now after over a 100 iterations of different variations I am still unable to get it to work. The script works well in the Powershell console but when included into the Jenkins Pipeline Script I get various errors that are proving hard to fix...
def getVersion (file) {
def result = powershell(script:"""Get-Content '${file}' |
Select-String '[0-9]+\\.[0-9]+\\.[0-9]+\\.' |
foreach-object{$_.Matches.Value}.${BUILD_NUMBER}""", returnStdout: true)
echo result
return result
}
...
powershell "(Get-Content ${files[0].path}).replace('[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+',
${getVersion(files[0].path)})) } | Set-Content ${files[0].path}"
...
How about a groovy approach (with Jenkins keywords) instead of PowerShell:
def updtaeAssemblyVersion() {
def files = findFiles(glob: '**/AssemblyInfo.cs')
files.each {
def content = readFile file: it.path
def modifedContent = content.repalceAll(/([0-9]+\\.[0-9]+\\.[0-9]+\\.)([0-9]+)/,"\$1${BUILD_NUMBER}")
writeFile file: it.path, text: modifedContent
}
}
It will read all relevant files and replace only the build section of the version for every occurrence that matches the version regex.

Timeout an MS Access ComObject from powershell

I have a powershell script that creates an MS Access ComObject and uses it to run a macro in an MS Access database as shown below:
$AccessDb= New-Object -ComObject Access.Application
try{
foreach($file in $Files)
{
...
$AccessDb.DoCmd.RunMacro('mcrScr')
...
}
}
catch { ... }
The issue is when there are runtime errors, MS Access throws the errors in an interactive window or dialog box and this causes the powershell to hang; just waiting for the window to close and thus macros for other .mdb files do not get to run.
I have been trying out options from online articles to timeout this piece of my code $AccessDb.DoCmd.RunMacro('mcrScr'), if it runs more than x number of seconds. I used jobs, runspace and [system.diagnostics.stopwatch] but have not been successful.
Is there any better approach to do this. I am kind of running out of options.
Edit:
#Paul, in response to your comment I am adding how i am using runspace to address the problem.
function Script-Timeout {
param([scriptblock]$Command,[int]$Timeout)
$Runspace = [runspacefactory]::CreateRunspace()
$Runspace.Open()
$PS = [powershell]::Create().AddScript($Command)
$PS.Runspace = $Runspace
$chk = $PS.BeginInvoke()
if($chk.AsyncWaitHandle.WaitOne($Timeout))
{
$PS.EndInvoke($chk)
}
else
{
throw "Command taking too long to run. Timeout exceeded."
$PS.EndInvoke($chk)
}
}
The Script-Timeout function is then used in the portion of my script that runs the MS Acess macro as shown below:
Try{
forEach($mdbfile in Files)
{
...
Script-Timeout -Command {
$AccessDb= New-Object -ComObject Access.Application
$AccessDb.OpenCurrentDatabase($mdbfile)
$AccessDb.DoCmd.RunMacro('mcrUpdate')
$AccessDb.CloseCurrentDatabase()
} -Timeout 20
...
}
}
catch
{
#catch exception
}
I have artificially created a runtime error in the VBA which throws an error dialog box. This way the RunMacro portion of the script, if it gets run, will hung the powershell. This is where i expect the runspace to timeout the macro run from powershell after x seconds.
The problem with runspace is that the MS Access Macro does not get run at all. In powershell debug mode, i see the if-block of the script-timeout function always execute successfully with or without the artificial runtime error

Update TFS Build Status via Powershell

I've a TFS Build Definition setup where, I execute a powershell script for security hardening, applying final patches etc.
Sometimes this powershell script can fail and I would like to change the status of build to be un-successfull or fail. Is there a way to change this TFS Build status via a powershell?
Try this:
$script:TFServerName = $ServerName
$script:TFServer = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($ServerName)
$script:TFBuildServer = $TFServer.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $TFBuildServer.CreateBuildDetailSpec($TeamProject, $DefinitionName)
$spec.Status = 'Failed'
Make sure you've Add-Type the right versions of the TFS assemblies. I'm assuming here that since the properties support setters, they communicate back to the server to affect the change. But I'm not 100% sure of that.
Here's what I pieced together from MSDN:
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
$tfsServerAddress = "http://{tfs hostname}:{port}/{CollectionName}"
$build = "{build definition name}_{build date}.{build count for the day}"
#update build status
$tfsProject = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($tfsServerAddress)
$tfsBuildServer = $tfsProject.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $tfsBuildServer.CreateBuildDetailSpec($teamProject)
$spec.BuildNumber = $build
$builds = $tfsBuildServer.QueryBuilds($spec).Builds
if ($builds[0].TestStatus -eq 'Succeeded'){
echo "updating test status:" $builds[0].TestStatus "to $testOutcome"
$builds[0].TestStatus = $testOutcome
}
if ($builds[0].Status -eq 'Succeeded'){
echo "updating build status:" $builds[0].Status "to $testOutcome"
$builds[0].Status = $testOutcome
}
$tfsBuildServer.SaveBuilds($builds)
Keep in mind that you need a special permission to update build status

How to get list of TFS builds running at the moment from command line?

I'm trying to automate the deployment process, and as part of it, I need to run my release build from command line. I can do it, using command like
.\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
It even returns some code for the queued build — Build queued. Queue position: 2, Queue ID: 11057.
What I don't know, is how to get info about currently running builds, or about the state of my running build from powershell command line? The final aim is to start publishing after that build completes.
I've already got all necessary powershell scripts to create the deployment package from the build results, zip it, copy to production and install there. All I need now — to know when my build succeedes.
This function will wait for a build with the Queue ID given by TFSBuild.exe:
function Wait-QueuedBuild {
param(
$QueueID
)
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Build.Client')
[void][Reflection.Assembly]::LoadWithPartialName('Microsoft.TeamFoundation.Client')
$uri = [URI]"http://server-name:8080/tfs/project-collection"
$projectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($uri)
$buildServer = $projectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$spec = $buildServer.CreateBuildQueueSpec('*','*')
do {
$build = $buildServer.QueryQueuedBuilds($spec).QueuedBuilds| where {$_.Id -eq $QueueID}
sleep 1
} while ($build)
}
You can get the id returned by TFSBuild.exe, then call the function.
$tfsBuild = .\TFSBuild start http://server-name:8080/tfs/project-collection project-name build-name priority:High /queue
Wait-QueuedBuild [regex]::Match($tfsBuild[-1],'Queue ID: (?<id>\d+)').Groups['id'].Value
Using the work by E.Hofman available here it is possible to write a C# console app that uses TFS SDK and reveals if any build agent is currently running as follows:
using System;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
namespace ListAgentStatus
{
class Program
{
static void Main()
{
TfsTeamProjectCollection teamProjectCollection = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://TFSServer:8080"));
var buildServer = teamProjectCollection.GetService<IBuildServer>();
foreach (IBuildController controller in buildServer.QueryBuildControllers(true))
{
foreach (IBuildAgent agent in controller.Agents)
{
Console.WriteLine(agent.Name+" is "+agent.IsReserved);
}
}
}
}
}
The parameter .IsReserved is what toggles to 'True' during execution of a build.
I 'm sorry my powershell skills are not good enough for providing with a PS variant of the above. Please take a look here, where the work by bwerks might help you do that.
# load classes for execution
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client") | Out-Null
# declare working variables
$Uri = New-Object System.Uri "http://example:8080/tfs"
# get reference to projection collection
$ProjectCollection = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($Uri)
# get reference to build server
$BuildServer = $ProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
# loop through the build servers
foreach($Controller in $BuildServer.QueryBuildControllers($true))
{
# loop through agents
foreach($BuildAgent in $Controller.Agents)
{
Write-Host "$($BuildAgent.Name) is $($BuildAgent.IsReserved)"
}
}