Rundeck pass common parameters/variable to the flow - rundeck

Consider I have one main flow execution job, I will have many more like this in the future.
I will have few backend flow jobs which will be used as reference jobs in the main flow execution job which I mentioned above.
Things I need to accomplish:
Pass a few command-line arguments in the main flow job for later reference jobs to use
Also, if reference jobs generate few more variables that should be available for later reference steps to use.
I did not find some very clear documents with the example on the Rundeck site, I would really appreciate it if someone could help on this.
#Update:
After making the text option with the list as plain text, which I able to use in any new step which I add like inline script or command.
But when I wanted to use the same options in Job reference from workflow steps, even after passing the command line argument I could able to get that in my python script.
I passed the argument both way but could not able to get the value of the option in the reference job. As mentioned like this: -source_details ${option.source_details}
and like this as well ${option.source_details}
The child job which I and referring to here is calling the one python script whereas I was checking the command line argument with sys.argv, which is not giving anything extra except the actual python filename.
Correct me if I am missing something here.

To do that, you need to pass options as arguments, for example, the child job uses its own options to use as arguments.
I leave an example:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='age' value='32' />
<option name='name' value='Alice' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</id>
<loglevel>INFO</loglevel>
<name>ChildJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<fileExtension>.sh</fileExtension>
<script><![CDATA[echo "Username: #option.name#";
echo "Age: #option.age#";]]></script>
<scriptargs />
<scriptinterpreter>/bin/bash</scriptinterpreter>
</command>
</sequence>
<uuid>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</uuid>
</job>
</joblist>
And the parent job can pass another options to override child job options:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='age' value='25' />
<option name='name' value='Bob' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>9fa68cd8-5bb0-4341-be32-f58c372cb765</id>
<loglevel>INFO</loglevel>
<name>Parent</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<jobref name='ChildJob' nodeStep='true'>
<arg line='-name ${option.name} -age ${option.age}' />
<uuid>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</uuid>
</jobref>
</command>
</sequence>
<uuid>9fa68cd8-5bb0-4341-be32-f58c372cb765</uuid>
</job>
</joblist>
In few words: you can use options in your jobs (to "receive" from parent job or to use itself), and keep your values across your workflows.
Update with a python3 example
The concept is the same, in the child job you can create some options to receive the values from the parent job, take a look.
Child Job:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='arg1' value='one' />
<option name='arg2' value='two' />
<option name='arg3' value='three' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>159e14d6-29e2-4fe9-b9b3-b1621d59843d</id>
<loglevel>INFO</loglevel>
<name>PythonChildJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<fileExtension>.py</fileExtension>
<script><![CDATA[#!/usr/bin/python3
import sys
print ('Number of arguments:', len(sys.argv), 'arguments.')
print ('Argument List:', str(sys.argv))]]></script>
<scriptargs>${option.arg1} ${option.arg2} ${option.arg3}</scriptargs>
<scriptinterpreter>/usr/bin/python3.8</scriptinterpreter>
</command>
</sequence>
<uuid>159e14d6-29e2-4fe9-b9b3-b1621d59843d</uuid>
</job>
</joblist>
And the parent job, you can pass options or just strings like -arg1 hello -arg2 from -arg3 mars
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='opt1' value='hello' />
<option name='opt2' value='entire' />
<option name='opt3' value='world' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>1442dbb7-55e3-45c0-af4b-a58ce5c07582</id>
<loglevel>INFO</loglevel>
<name>ParentJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<jobref name='PythonChildJob' nodeStep='true'>
<arg line='-arg1 ${option.opt1} -arg2 ${option.opt2} -arg3 ${option.opt3}' />
<uuid>159e14d6-29e2-4fe9-b9b3-b1621d59843d</uuid>
</jobref>
</command>
</sequence>
<uuid>1442dbb7-55e3-45c0-af4b-a58ce5c07582</uuid>
</job>
</joblist>

Related

Job Conditional Reports Previous Job "Never Run"

I have a Job that calls other jobs as sort of a main run-file. I execute a preliminary check, which sets some values on a management server that I have to ensure I don't get any erroneous alerts due to the updates being run.
The preliminary job executes fine, but the conditional that comes right after it, ALWAYS outputs the status of:
{{UUID}} is NOT RUNNING AND previously NEVER. Expected NOT RUNNING AND 'Succeeded'
No matter what I do, I can't figure out if either I'm doing something wrong or if I just have something misconfigured, or if the conditionals are just broken. This is a small install for my homelab of about 10 servers, so I'm running with the default database back end, not MySQL or something else, so I don't know if that's an issue.
My Rundeck detail
Rundeck version: Rundeck 3.2.8-20200608
Install type: deb
OS Name/version: Ubuntu 18.04
DB Type/version: H2? (whatever the default is)
Verify the job state conditional expectation, I leave an example on 3.2.8 that works. Basically Job Step Conditional expects another job last execution as successfully, anything different of that makes the condition fails.
HelloWorld job:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<group>test1</group>
<id>0bdd6fe5-addb-4051-b072-bb4430130a80</id>
<loglevel>INFO</loglevel>
<name>HelloWorld</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "hello world"</exec>
</command>
</sequence>
<uuid>0bdd6fe5-addb-4051-b072-bb4430130a80</uuid>
</job>
</joblist>
CheckerJob (With a Job Step Conditional step, expects that HelloWorld job with a successful execution):
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<group>test1</group>
<id>c6f2be75-81e8-4f8e-ba35-234d5c8a97d6</id>
<loglevel>INFO</loglevel>
<name>CheckerJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "starting..."</exec>
</command>
<command>
<step-plugin type='job-state-conditional'>
<configuration>
<entry key='condition' value='Equals' />
<entry key='executionState' value='Succeeded' />
<entry key='fail' value='true' />
<entry key='halt' value='true' />
<entry key='jobName' value='test1/HelloWorld' />
<entry key='jobProject' value='${job.project}' />
<entry key='jobUUID' value='0bdd6fe5-addb-4051-b072-bb4430130a80' />
<entry key='running' value='false' />
</configuration>
</step-plugin>
</command>
</sequence>
<uuid>c6f2be75-81e8-4f8e-ba35-234d5c8a97d6</uuid>
</job>
</joblist>
If the job never runs before.
If the job runs before (Job State Conditional expectation).
Another way
If you want to run Job State conditional in the same parent job, you need an individual execution of the "child" job, the way to do that is executing that job "externally" (using RD-CLI or API, I leave an example):
HelloWord:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>dd33434e-5b6e-40e1-88b0-e9b5b52a3801</id>
<loglevel>INFO</loglevel>
<name>HelloWorld</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "hello world!"</exec>
</command>
</sequence>
<uuid>dd33434e-5b6e-40e1-88b0-e9b5b52a3801</uuid>
</job>
</joblist>
CheckerFakeParentJob (Executes HelloWorld vi RD-CLI and evaluate that execution later):
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>a90ec790-b856-4f00-8b66-2f37da1d00cb</id>
<loglevel>INFO</loglevel>
<name>CheckerFakeParentJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "starting"</exec>
</command>
<command>
<fileExtension>.sh</fileExtension>
<script><![CDATA[# print a message
echo "running HelloWorld job"
# run the hello world job via RD-CLI to get the individual execution and evaluate later
rd run -j HelloWorld -p ProjectEXAMPLE
# some time depending of job
sleep 3
# print a message
echo "done"]]></script>
<scriptargs />
<scriptinterpreter>/bin/bash</scriptinterpreter>
</command>
<command>
<step-plugin type='job-state-conditional'>
<configuration>
<entry key='condition' value='Equals' />
<entry key='executionState' value='Succeeded' />
<entry key='fail' value='true' />
<entry key='halt' value='true' />
<entry key='jobUUID' value='dd33434e-5b6e-40e1-88b0-e9b5b52a3801' />
<entry key='running' value='false' />
</configuration>
</step-plugin>
</command>
</sequence>
<uuid>a90ec790-b856-4f00-8b66-2f37da1d00cb</uuid>
</job>
</joblist>
Result here.
You have more information here.
EDIT: If you're referencing a Job Reference Step based workflow, you must take the parent job status (even the child job is executed through parent job).

Rundeck - Run Steps dynamically based on input Options

I'd like to run number of steps based on different input Options.
For example, I've a Job that runs a ./script.sh with year as input param:
./script.sh <year>
the idea is to set a list of "years" as Options params (something like 2018, 2019, 2020) and run a script.sh for each year and each script.sh <year> should be run as a single Step.
step_1: ./script.sh 2018
step_2: ./script.sh 2019
step_3: ./script.sh 2020
step_n: ./script.sh n
Is there a way to do that?
Thank you.
UPDATE 2020-04-22
Expected on Rundeck where each Command has one year:
I did this example, using an option that passes it to an inline script that iterates using #option.myoption# (and referencing the "external" script step as argument).
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='theyear' value='2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>a72d4d32-d4fe-4774-b18c-723d18842c5e</id>
<loglevel>INFO</loglevel>
<name>WhatYearIs</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<fileExtension>.sh</fileExtension>
<script><![CDATA[#!/bin/bash
DataList=" #option.theyear# "
Field_Separator=$IFS
# set comma as internal field separator for the string list
IFS=,
for val in $DataList;
do
# echo $val
bash /path/to/your/script/year.sh $val
done
IFS=$Field_Separator]]></script>
<scriptargs />
<scriptinterpreter>/bin/bash</scriptinterpreter>
</command>
</sequence>
<uuid>a72d4d32-d4fe-4774-b18c-723d18842c5e</uuid>
</job>
</joblist>
The bash script to test:
#!/bin/bash
# print_args.sh
echo "Current year is:" "$#"
# You could pass all arguments to another program like this
# myProgram "$#"
UPDATE: You cannot add steps dynamically on a job meanwhile runs, but, another way to do that is basically a modification of the first suggestion, the difference is that we call another job through rd-cli tool on inline-script that calls (and pass the year as an option) to execute as an individual step.
WhatYearIs Job (As parent job).
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='theyear' value='2018,2019,2020' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<group>SO</group>
<id>a72d4d32-d4fe-4774-b18c-723d18842c5e</id>
<loglevel>INFO</loglevel>
<name>WhatYearIs</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<script><![CDATA[#!/bin/bash
DataList=" #option.theyear# "
Field_Separator=$IFS
# set comma as internal field separator for the string list
IFS=,
for val in $DataList;
do
# echo $val
sleep 3
rd run -p ProjectEXAMPLE -j WhatYearChild -- -childyear $val
done
IFS=$Field_Separator]]></script>
<scriptargs />
</command>
</sequence>
<uuid>a72d4d32-d4fe-4774-b18c-723d18842c5e</uuid>
</job>
</joblist>
WhatYearChild (As child job that executes the script).
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='childyear' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<group>SO</group>
<id>76c15c6d-be45-4540-8f44-8823ed2d2483</id>
<loglevel>INFO</loglevel>
<name>WhatYearChild</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<script><![CDATA[echo "Current year is: #option.childyear#"]]></script>
<scriptargs />
</command>
</sequence>
<uuid>76c15c6d-be45-4540-8f44-8823ed2d2483</uuid>
</job>
</joblist>
With this result (technically is an individual step of each iteration).

Is there any option to kill a job running for longer time on remote machine by rundeck itself?

I need to kill a rundeck job by itself when the job runs more than 2hours on remote machine,is there a possibility
You can set a timeout value in "Timeout" textbox (When creating or editing your job go to the "Other" tab an see that option, check this. You can use values in seconds (like 10s), minutes (like 5m), hours (like 2h), or days (like 2d), also, you can pass an option defined with that value.
I leave a job definition example to test:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='timeout' value='5s' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>13ba3f6b-f921-4ab5-95ba-56eb335eed4b</id>
<loglevel>INFO</loglevel>
<name>HugeDelay</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>name: node01</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<plugins />
<schedule>
<month month='*' />
<time hour='13' minute='19' seconds='0' />
<weekday day='*' />
<year year='*' />
</schedule>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>sleep 30; echo "hi"</exec>
</command>
</sequence>
<timeout>${option.timeout}</timeout>
<uuid>13ba3f6b-f921-4ab5-95ba-56eb335eed4b</uuid>
</job>
</joblist>

my rundeck job is not letting me pass all the parameters, just one, and I need all

I use this but only get a SINGLE PARAMETER, passed to the rundeck job:
... ${option.ticketnumber} ...
I want ALL the parameters (not just ONE) so that I can parse them within my code
free and CLEAR of Rundeck.
Anyone want to comment out there?
You can wrap your steps using key/value data with a regex to generate a set of data variables like this to use in your workflow (in this way you avoid to define individual options to pass later). Here a job definition example:
The job that generates the data values:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description>Regex and data passing demo.</description>
<executionEnabled>true</executionEnabled>
<id>a0daf1e3-e918-43dc-b232-bc46a7a287b6</id>
<loglevel>INFO</loglevel>
<name>Regex</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>env</exec>
</command>
<command>
<exec>echo ${data.SHELL}</exec>
</command>
<command>
<exec>echo ${data.USER}</exec>
</command>
<command>
<exec>echo ${data.PWD}</exec>
</command>
<command>
<jobref name='AnotherJob' nodeStep='true'>
<arg line='-option1 ${data.USER}' />
<uuid>4932f9c7-a435-4332-8b1a-5ade41da9edd</uuid>
</jobref>
</command>
<pluginConfig>
<LogFilter type='key-value-data'>
<config>
<logData>true</logData>
<regex>^(SHELL|USER|PWD)\s*=\s*(.+)$</regex>
</config>
</LogFilter>
</pluginConfig>
</sequence>
<uuid>a0daf1e3-e918-43dc-b232-bc46a7a287b6</uuid>
</job>
</joblist>
The job that gets the data values:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='option1' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>4932f9c7-a435-4332-8b1a-5ade41da9edd</id>
<loglevel>INFO</loglevel>
<name>AnotherJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "the user from Regex Job is ${option.option1}"</exec>
</command>
</sequence>
<uuid>4932f9c7-a435-4332-8b1a-5ade41da9edd</uuid>
</job>
</joblist>
But to get it (passing) from another job, you need to set individual options using a job reference step with option as arguments.
I programmed around all of this to avoid the restrictions of the interface.

rundeck pass output from job(step) one to job(step) two

In rundeck, I have two selections, selection1 and selection2 which will be executed on node1 and node2 respectively. Slection2 's input is from output of selections. How can I make it ? with two jobs or two steps? and How to pass the arguments.
You can pass options between jobs using Job Reference Step and arguments. I leave an example with two jobs: JobA and JobB, the JobA just print an option and the JobB that call the JobA and pass their option to JobA option through argument.
Job A:
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="opta" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>6d64bd33-71de-4d20-8bcd-c65785603b23</id>
<loglevel>INFO</loglevel>
<name>JobA</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>echo ${option.opta}</exec>
</command>
</sequence>
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</job>
</joblist>
Job B:
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="optb" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>22d0e6cc-e0f3-48e5-b8a5-3007e448ae18</id>
<loglevel>INFO</loglevel>
<name>JobB</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<jobref name="JobA" nodeStep="true">
<arg line="-opta ${option.optb}" />
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</jobref>
</command>
</sequence>
<uuid>22d0e6cc-e0f3-48e5-b8a5-3007e448ae18</uuid>
</job>
</joblist>
Also, you can capture some output and pass this data using arguments between jobs. I leave another Example:
Job A (just print an option):
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="opta" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>6d64bd33-71de-4d20-8bcd-c65785603b23</id>
<loglevel>INFO</loglevel>
<name>JobA</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>echo ${option.opta}</exec>
</command>
</sequence>
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</job>
</joblist>
Job B (it generates data, store at ${data.USER} variable and call the JobA to pass that variable as an argument using Job Reference Step):
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description>Regex and data passing demo.</description>
<executionEnabled>true</executionEnabled>
<id>a0daf1e3-e918-43dc-b232-bc46a7a287b6</id>
<loglevel>INFO</loglevel>
<name>Regex</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>env</exec>
</command>
<command>
<jobref name="JobA" nodeStep="true">
<arg line="-opta ${data.USER}" />
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</jobref>
</command>
<pluginConfig>
<LogFilter type="key-value-data">
<config>
<logData>true</logData>
<regex>^(USER)\s*=\s*(.+)$</regex>
</config>
</LogFilter>
</pluginConfig>
</sequence>
<uuid>a0daf1e3-e918-43dc-b232-bc46a7a287b6</uuid>
</job>
</joblist>
You can see a very useful example here.