I am working on a perl script which will do the following:
Trigger a script in post build action when job fails.
Read the log file and try to match the errors with a consolidated error/solution file.
If error is matched with pattern file, then concatenate the error message with the solution at the end of log file.
I am facing following challenges:
All jobs are running in Slave but the error log file is stored in Master. How can I run the script in post-build action? The script path will be taken from slave but my script is located in master. Is there any workaround for this?
The path of the error log is - /home/jenkins/data/jobs//builds/BUILD_NUMBER/log
We have many jobs that have folders created by jenkins folder plugins…how do we set the common folder for these?
/home/jenkins/data/jobs/FOLDERX//builds/BUILD_NUMBER/log
Other questions -
Do you think that publishing the jenkins error log and displaying the solution is the right approach?
There is no information on how complex the pattern maching is, but if it is a simple line based regex match there is a plugin for that, called Build Failure Analyzer.
Related
I am fairly new to NUnit and I'm trying to see if NUnit support multiple logging, by that I mean that I want to capture the logs from an external device as well as the network traces. Since I don't want to pollute my result logs with all these logs, I would like to have them in different files so that I have something like this:
Test results logs file
Telnet logs file 1
Telnet logs file 2
Network trace file
Does Nunit support the addition of other logs or do I have to create my own logging system?
Strictly speaking, NUnit's result file is not a "log file." Generally, a log file is created incrementally as execution proceeds. The TestResult file is an XML representation of the entire test run and is only written at the end of the run.
NUnit does have a set of log files, called Internal Trace logs, which are produced by the console runner when you use the --trace option. As their name suggests, they trace the internal workings of NUnit rather than your tests.
Any other logging you perform is entirely up to you and is not captured by NUnit at all.
CamelServer..Sequential_File_2.DSLink3: ds_seqopen() - Win32 error in CreateFile - Access is denied.
It looks like a user rights issue, as if the user running the job does not have the necessary rights on the directory where you are reading/writing from a file. A little more context would help better understand the issue (i.e is the job run manually from the Director or is it scheduled, is the offending directory local on the server, etc..?).
The error message indicates that error occurs for the sequential file stage. It is unclear if this issue occurred on the first run of job or subsequent runs.
DataStage jobs can run under different userids unless you have all the users credential mapped to common id such as dsadm.
The most likely cause of above error are:
1. The target directory where you have selected to create sequential file has directory permissions that do not permit file create/write by the userid that is running the job.
2. OR, the job was previously run with different userid, creating a file owned by that userid, and new job run is with new userid that does not have permission to overwrite the original file.
Check the job log to see what userid is running the job...the userid is on every event message. Then confirm the file name and location sequential file is trying to write, and at OS level, confirm if that file already exists with different userid (if so, try updating file permission) and also confirm that directory permissions allow write by the userid now running the job.
In Informatica we have mapping source qualifiers connecting to Amazon Web Services—AWS.
We often and erratically get a failure that our s3 bucket names do not comply with naming standards. We restart the workflows again and they continue on every time successfully.
Is there a way to trap for this specifically and then maybe call a command object to restart the workflow command via PMCMD?
How are you starting the workflows in regular runs?
If you are using a shell script, you can add a logic to restart if you see a particular error. I have created a script a while ago to restart workflows for a particular error.
In a nut shell it works like this
start workflow (with pmcmd)
#in case of an error
check repository db and get the error
if the error is specific to s3 bucket name
restart the workflow
Well... It's possible for example to have workflow one (W1):
your_session --> cmd_touch_file_if_session_failed
and another workflow (W2), running continuously:
event_wait_for_W1_file --> pmcmd_restart_W1 --> delete_watch_file
Although it would be a lot better to nail down the cause for your failures and get it resolved.
In VSTS release management there is a nice remote Powershell task where we can run a script on the target machine. However I'd need a way to tell the release managment that only run this file if it exists, otherwise silently ignore that.
I know I can configure a task to not block the process in case of error, however in that case there still will be an exclamation mark in the log and the deployment will get the partial succeeded status. I'd like to avoid this and show success even if the file doesn't exist.
With this I need it to support kind of optional setup scripts for several deployed products.
There isn’t the setting or feature in VSTS to check whether the script file is existing or not.
The simple way is that, you can create another script to call target script.
Create another script (e.g. wapperScript.ps1) to call target script (can use parameter to accept the target script path) and add to source control
Add Windows Machine copy task to copy wapperScript.ps1 to target machine
Add Remote PowerShell task to run wapperScript.ps1
If you make your script more robust with a guard clause so that it can be called regardless of any given environmental condition. This keeps your pipeline less complicated. You can take action on the "file exists" leg and do a noop on the other. You can signal to the release process either way with log entries.
I'm trying to set timeout on workflow run in oozie, which fails after some time from start of workflow run.
For example, one can use sla:should-end within coordinator.xml for action/workflow, or specify sla:should-end within workflow.xml for entire workflow.
But the problem is SLA only sends email. What I want is just fail after some time (from start of run).
Is it possible? Any sample code would help.
I don't know of any straight solution for this in Oozie or in Yarn. There is a Yarn ticket that would provide a convenient solution.
Until it's implemented you can try something like this:
add a FS touchz action to the beginning of the workflow to create a file ( e.g. /tmp/WF_ID )
add a fork to the workflow after the file is created
one of the paths should be a shell action that checks for the existence of the file with hdfs dfs -ls /tmp/WF_ID until a timeout is reached ( see this post for some hints )
the other path is the original workflow logic and an FS delete action at the end to delete the file in /tmp
the shell action should kill the workflow if it times out before the file is deleted from HDFS. If the file is deleted before the timeout is reached, the shell script should terminate normally, letting the workflow to continue
This is quite an ugly workaround of the problem, but I can't think of a cleaner solution at the moment.