Run pool of jobs in parallel, each job only executing once - rundeck

Question

I have a (powershell, if it matters) script I want to run once for each state. The job takes a variable amount of time per state (and not based on state population or anything like that, not predictable), so what I'd like to do is create a flow with the 50 jobs as steps, and then tell Rundeck to run those 50 jobs in 5 threads. Maybe 10 per thread, but probably not as some of them are faster than others; but let Rundeck manage that, so if it ends up being 8-9-12-10-11 that particular time, great. I understand how to use the Rundeck API to create those 50 steps and add them to a workflow.
I can't seem to see how Rundeck would manage this exactly how I want, however. The strategies for workflows seem to all assume you want to run all of the jobs on all of the nodes (meaning, if I have 50 steps and 5 nodes, it would run 250 total things, each state 5 times) - I don't want that; I want each job to run only on one node, but I don't care which node that is - whatever node is free. Just act as a load balancer, basically.
Does Rundeck support this, or would I have to build my own load balancing script?

Answer 1

I did an example using three nodes, three jobs, and a parent job to dispatch these three jobs to each node (once) via Job Reference Step.
First, the Parent job which calls the child jobs using a parallel strategy which means any step (each job reference) runs at the same time, orchestrator as "random subset" of a set of nodes (three on my example) and "count" set as "1" (each job reference step to a single node).
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>3</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>17aad193-1e2c-45dc-b736-901c1de00954</id>
<loglevel>INFO</loglevel>
<name>ParentJobRemoteParallel</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>3</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='parallel'>
<command>
<jobref name='HelloWorldONE' nodeStep='true'>
<uuid>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</uuid>
</jobref>
</command>
<command>
<jobref name='HelloWorldTWO' nodeStep='true'>
<uuid>a57e74f2-050d-4a13-841a-3a4552d8227b</uuid>
</jobref>
</command>
<command>
<jobref name='HelloWorldTHREE' nodeStep='true'>
<uuid>47067c03-da25-43e1-83cc-9fe0492f0797</uuid>
</jobref>
</command>
</sequence>
<uuid>17aad193-1e2c-45dc-b736-901c1de00954</uuid>
</job>
</joblist>
Now the Child Jobs, these jobs are configured pointed to a set of nodes with random subset orchestrator and "1", similar to Parent Job.
HelloWorldONE:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>true</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</id>
<loglevel>INFO</loglevel>
<name>HelloWorldONE</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "i am the first job"</exec>
</command>
</sequence>
<uuid>e1ec6d2c-62d9-4de4-af6d-7e03f59cc4e5</uuid>
</job>
</joblist>
HelloWorldTWO:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>a57e74f2-050d-4a13-841a-3a4552d8227b</id>
<loglevel>INFO</loglevel>
<name>HelloWorldTWO</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='sequential'>
<command>
<exec>echo "i am the second job"</exec>
</command>
</sequence>
<uuid>a57e74f2-050d-4a13-841a-3a4552d8227b</uuid>
</job>
</joblist>
HelloWorldTHREE:
<joblist>
<job>
<defaultTab>nodes</defaultTab>
<description></description>
<dispatch>
<excludePrecedence>true</excludePrecedence>
<keepgoing>false</keepgoing>
<rankOrder>ascending</rankOrder>
<successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
<threadcount>1</threadcount>
</dispatch>
<executionEnabled>true</executionEnabled>
<id>47067c03-da25-43e1-83cc-9fe0492f0797</id>
<loglevel>INFO</loglevel>
<name>HelloWorldTHREE</name>
<nodeFilterEditable>false</nodeFilterEditable>
<nodefilters>
<filter>node.*</filter>
</nodefilters>
<nodesSelectedByDefault>true</nodesSelectedByDefault>
<orchestrator>
<configuration>
<count>1</count>
</configuration>
<type>subset</type>
</orchestrator>
<plugins />
<scheduleEnabled>true</scheduleEnabled>
<sequence keepgoing='false' strategy='node-first'>
<command>
<exec>echo "i am the third job"</exec>
</command>
</sequence>
<uuid>47067c03-da25-43e1-83cc-9fe0492f0797</uuid>
</job>
</joblist>
Here you can see the result.
This is also appliable to steps, job reference steps are steps like a command step or script-step.