Is there any option to kill a job running for longer time on remote machine by rundeck itself? - remote-server

I need to kill a rundeck job by itself when the job runs more than 2hours on remote machine,is there a possibility

You can set a timeout value in "Timeout" textbox (When creating or editing your job go to the "Other" tab an see that option, check this. You can use values in seconds (like 10s), minutes (like 5m), hours (like 2h), or days (like 2d), also, you can pass an option defined with that value.
I leave a job definition example to test:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='timeout' value='5s' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>13ba3f6b-f921-4ab5-95ba-56eb335eed4b</id>
<loglevel>INFO</loglevel>
<name>HugeDelay</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>name: node01</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<plugins />
<schedule>
<month month='*' />
<time hour='13' minute='19' seconds='0' />
<weekday day='*' />
<year year='*' />
</schedule>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>sleep 30; echo "hi"</exec>
</command>
</sequence>
<timeout>${option.timeout}</timeout>
<uuid>13ba3f6b-f921-4ab5-95ba-56eb335eed4b</uuid>
</job>
</joblist>

Related

Rundeck pass common parameters/variable to the flow

Consider I have one main flow execution job, I will have many more like this in the future.
I will have few backend flow jobs which will be used as reference jobs in the main flow execution job which I mentioned above.
Things I need to accomplish:
Pass a few command-line arguments in the main flow job for later reference jobs to use
Also, if reference jobs generate few more variables that should be available for later reference steps to use.
I did not find some very clear documents with the example on the Rundeck site, I would really appreciate it if someone could help on this.
#Update:
After making the text option with the list as plain text, which I able to use in any new step which I add like inline script or command.
But when I wanted to use the same options in Job reference from workflow steps, even after passing the command line argument I could able to get that in my python script.
I passed the argument both way but could not able to get the value of the option in the reference job. As mentioned like this: -source_details ${option.source_details}
and like this as well ${option.source_details}
The child job which I and referring to here is calling the one python script whereas I was checking the command line argument with sys.argv, which is not giving anything extra except the actual python filename.
Correct me if I am missing something here.
To do that, you need to pass options as arguments, for example, the child job uses its own options to use as arguments.
I leave an example:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='age' value='32' />
<option name='name' value='Alice' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</id>
<loglevel>INFO</loglevel>
<name>ChildJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<fileExtension>.sh</fileExtension>
<script><![CDATA[echo "Username: #option.name#";
echo "Age: #option.age#";]]></script>
<scriptargs />
<scriptinterpreter>/bin/bash</scriptinterpreter>
</command>
</sequence>
<uuid>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</uuid>
</job>
</joblist>
And the parent job can pass another options to override child job options:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='age' value='25' />
<option name='name' value='Bob' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>9fa68cd8-5bb0-4341-be32-f58c372cb765</id>
<loglevel>INFO</loglevel>
<name>Parent</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<jobref name='ChildJob' nodeStep='true'>
<arg line='-name ${option.name} -age ${option.age}' />
<uuid>bf45b9bd-f8f4-4f00-8aaf-86572b637e05</uuid>
</jobref>
</command>
</sequence>
<uuid>9fa68cd8-5bb0-4341-be32-f58c372cb765</uuid>
</job>
</joblist>
In few words: you can use options in your jobs (to "receive" from parent job or to use itself), and keep your values across your workflows.
Update with a python3 example
The concept is the same, in the child job you can create some options to receive the values from the parent job, take a look.
Child Job:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='arg1' value='one' />
<option name='arg2' value='two' />
<option name='arg3' value='three' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>159e14d6-29e2-4fe9-b9b3-b1621d59843d</id>
<loglevel>INFO</loglevel>
<name>PythonChildJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<fileExtension>.py</fileExtension>
<script><![CDATA[#!/usr/bin/python3
import sys
print ('Number of arguments:', len(sys.argv), 'arguments.')
print ('Argument List:', str(sys.argv))]]></script>
<scriptargs>${option.arg1} ${option.arg2} ${option.arg3}</scriptargs>
<scriptinterpreter>/usr/bin/python3.8</scriptinterpreter>
</command>
</sequence>
<uuid>159e14d6-29e2-4fe9-b9b3-b1621d59843d</uuid>
</job>
</joblist>
And the parent job, you can pass options or just strings like -arg1 hello -arg2 from -arg3 mars
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='opt1' value='hello' />
<option name='opt2' value='entire' />
<option name='opt3' value='world' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<executionEnabled>true</executionEnabled>
<id>1442dbb7-55e3-45c0-af4b-a58ce5c07582</id>
<loglevel>INFO</loglevel>
<name>ParentJob</name>
<nodeFilterEditable>false</nodeFilterEditable>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<jobref name='PythonChildJob' nodeStep='true'>
<arg line='-arg1 ${option.opt1} -arg2 ${option.opt2} -arg3 ${option.opt3}' />
<uuid>159e14d6-29e2-4fe9-b9b3-b1621d59843d</uuid>
</jobref>
</command>
</sequence>
<uuid>1442dbb7-55e3-45c0-af4b-a58ce5c07582</uuid>
</job>
</joblist>

Run pool of jobs in parallel, each job only executing once

I have a (powershell, if it matters) script I want to run once for each state. The job takes a variable amount of time per state (and not based on state population or anything like that, not predictable), so what I'd like to do is create a flow with the 50 jobs as steps, and then tell Rundeck to run those 50 jobs in 5 threads. Maybe 10 per thread, but probably not as some of them are faster than others; but let Rundeck manage that, so if it ends up being 8-9-12-10-11 that particular time, great. I understand how to use the Rundeck API to create those 50 steps and add them to a workflow.
I can't seem to see how Rundeck would manage this exactly how I want, however. The strategies for workflows seem to all assume you want to run all of the jobs on all of the nodes (meaning, if I have 50 steps and 5 nodes, it would run 250 total things, each state 5 times) - I don't want that; I want each job to run only on one node, but I don't care which node that is - whatever node is free. Just act as a load balancer, basically.
Does Rundeck support this, or would I have to build my own load balancing script?
I did an example using three nodes, three jobs, and a parent job to dispatch these three jobs to each node (once) via Job Reference Step.
First, the Parent job which calls the child jobs using a parallel strategy which means any step (each job reference) runs at the same time, orchestrator as "random subset" of a set of nodes (three on my example) and "count" set as "1" (each job reference step to a single node).
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>3</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>17aad193-1e2c-45dc-b736-901c1de00954</id>
<loglevel>INFO</loglevel>
<name>ParentJobRemoteParallel</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>3</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='parallel'>
<command>
<jobref name='HelloWorldONE' nodeStep='true'>
<uuid>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</uuid>
</jobref>
</command>
<command>
<jobref name='HelloWorldTWO' nodeStep='true'>
<uuid>a57e74f2-050d-4a13-841a-3a4552d8227b</uuid>
</jobref>
</command>
<command>
<jobref name='HelloWorldTHREE' nodeStep='true'>
<uuid>47067c03-da25-43e1-83cc-9fe0492f0797</uuid>
</jobref>
</command>
</sequence>
<uuid>17aad193-1e2c-45dc-b736-901c1de00954</uuid>
</job>
</joblist>
Now the Child Jobs, these jobs are configured pointed to a set of nodes with random subset orchestrator and "1", similar to Parent Job.
HelloWorldONE:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>true</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</id>
<loglevel>INFO</loglevel>
<name>HelloWorldONE</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "i am the first job"</exec>
</command>
</sequence>
<uuid>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</uuid>
</job>
</joblist>
HelloWorldTWO:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>a57e74f2-050d-4a13-841a-3a4552d8227b</id>
<loglevel>INFO</loglevel>
<name>HelloWorldTWO</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='sequential'>
<command>
<exec>echo "i am the second job"</exec>
</command>
</sequence>
<uuid>a57e74f2-050d-4a13-841a-3a4552d8227b</uuid>
</job>
</joblist>
HelloWorldTHREE:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>47067c03-da25-43e1-83cc-9fe0492f0797</id>
<loglevel>INFO</loglevel>
<name>HelloWorldTHREE</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "i am the third job"</exec>
</command>
</sequence>
<uuid>47067c03-da25-43e1-83cc-9fe0492f0797</uuid>
</job>
</joblist>
Here you can see the result.
This is also appliable to steps, job reference steps are steps like a command step or script-step.

Is there a way to use login/password of login account to reuse in a playbook?

I have created a job with 2 options for the user writes his login/password. This credentials will be used later in a playbook as extra-variables.
But all this credentials are readable in logs !!!
How to workaround that ?
You can use the "Mask Password" Log filter in your inline-playbook or your playbook steps (also, for any Rundeck step), you can check the documentation here. I did a Job Definition example that works:
<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='opt1' secure='true' value='12345' valueExposed='true' />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>2c9b3903-a545-4dbd-aeac-578889dbb611</id>
<loglevel>INFO</loglevel>
<name>HelloWorld</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>name: 192.168.33.20</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<node-step-plugin type='com.batix.rundeck.plugins.AnsiblePlaybookInlineWorkflowNodeStep'>
<configuration>
<entry key='ansible-base-dir-path' value='/home/user/' />
<entry key='ansible-become' value='false' />
<entry key='ansible-playbook-inline' value='---
# This playbook prints a simple debug message
- name: Echo
hosts: all
connection: local
tasks:
- name: Print debug message
debug:
msg: ${option.opt1}' />
<entry key='ansible-ssh-passphrase-option' value='option.password' />
<entry key='ansible-ssh-use-agent' value='false' />
</configuration>
</node-step-plugin>
<plugins>
<LogFilter type='LogFilter:mask-passwords'>
<config>
<color>red</color>
<replacement>[SECURE]</replacement>
</config>
</LogFilter>
</plugins>
</command>
</sequence>
<uuid>2c9b3903-a545-4dbd-aeac-578889dbb611</uuid>
</job>
</joblist>

Failed importing: Attribute "name" was already specified for element "option"

<joblist>
<job>
<context>
<options preserveOrder='true'>
<option name='PDATE' value='${DATE:yyyyMMdd}' />
<option name='TSTAMP' value='${DATE:yyyy-MM-dd&apos;T&apos;HH.mm.ssXX}' />
</options>
</context>
<description>Load Adjustment Transactions</description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<group>PointsUpdate</group>
<id>6475eb91-926f-476b-bf05-85f5e8b22eae</id>
<loglevel>INFO</loglevel>
<name>sglp-jbadjust</name>
<nodeFilterEditable>true</nodeFilterEditable>
<nodefilters>
<filter>.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<schedule>
<dayofmonth />
<month day='*' month='*' />
<time hour='00' minute='00' seconds='00' />
<year year='*' />
</schedule>
<scheduleEnabled>false</scheduleEnabled>
<sequence keepgoing='false' strategy='sequential'>
<command>
<exec>echo one ${option.PDATE} two ${option.TSTAMP} three</exec>
</command>
</sequence>
</job>
</joblist>
git-import importing this gives me errors, but I do not know why:
Failed importing: rundeck/jobs/PointsUpdate_0014.xml: rundeck.controllers.JobXMLException: Unable to parse xml: org.xml.sax.SAXParseException; lineNumber: 30; columnNumber: 110; Attribute "name" was already specified for element "option".
Importing into the Rundeck GUI works fine, no errors reported, and options are correct.
I just imported your job and also tested using Tomcat 9 + Rundeck 2.4.3 + Linux. I'm not able to reproduce this error. May you provide additional data about Rundeck version,installation method,OS?
Importing using GIT
Importing using GUI

rundeck pass output from job(step) one to job(step) two

In rundeck, I have two selections, selection1 and selection2 which will be executed on node1 and node2 respectively. Slection2 's input is from output of selections. How can I make it ? with two jobs or two steps? and How to pass the arguments.
You can pass options between jobs using Job Reference Step and arguments. I leave an example with two jobs: JobA and JobB, the JobA just print an option and the JobB that call the JobA and pass their option to JobA option through argument.
Job A:
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="opta" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>6d64bd33-71de-4d20-8bcd-c65785603b23</id>
<loglevel>INFO</loglevel>
<name>JobA</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>echo ${option.opta}</exec>
</command>
</sequence>
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</job>
</joblist>
Job B:
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="optb" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>22d0e6cc-e0f3-48e5-b8a5-3007e448ae18</id>
<loglevel>INFO</loglevel>
<name>JobB</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<jobref name="JobA" nodeStep="true">
<arg line="-opta ${option.optb}" />
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</jobref>
</command>
</sequence>
<uuid>22d0e6cc-e0f3-48e5-b8a5-3007e448ae18</uuid>
</job>
</joblist>
Also, you can capture some output and pass this data using arguments between jobs. I leave another Example:
Job A (just print an option):
<joblist>
<job>
<context>
<options preserveOrder="true">
<option name="opta" />
</options>
</context>
<defaultTab>nodes</defaultTab>
<description />
<executionEnabled>true</executionEnabled>
<id>6d64bd33-71de-4d20-8bcd-c65785603b23</id>
<loglevel>INFO</loglevel>
<name>JobA</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>echo ${option.opta}</exec>
</command>
</sequence>
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</job>
</joblist>
Job B (it generates data, store at ${data.USER} variable and call the JobA to pass that variable as an argument using Job Reference Step):
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description>Regex and data passing demo.</description>
<executionEnabled>true</executionEnabled>
<id>a0daf1e3-e918-43dc-b232-bc46a7a287b6</id>
<loglevel>INFO</loglevel>
<name>Regex</name>
<nodeFilterEditable>false</nodeFilterEditable>
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing="false" strategy="node-first">
<command>
<exec>env</exec>
</command>
<command>
<jobref name="JobA" nodeStep="true">
<arg line="-opta ${data.USER}" />
<uuid>6d64bd33-71de-4d20-8bcd-c65785603b23</uuid>
</jobref>
</command>
<pluginConfig>
<LogFilter type="key-value-data">
<config>
<logData>true</logData>
<regex>^(USER)\s*=\s*(.+)$</regex>
</config>
</LogFilter>
</pluginConfig>
</sequence>
<uuid>a0daf1e3-e918-43dc-b232-bc46a7a287b6</uuid>
</job>
</joblist>
You can see a very useful example here.