Execute concurrent processes using PowerShell - powershell

Question

I would like to setup a new PowerShell script that invokes my Database Stored Procedure concurrently. I am currently having a Control table that has a Job_ID column and a Code column. There might be more than one Job_ID for a code value in the Control table. Based on the code value I pass in the PowerShell along with a date, I would like the PowerShell to trigger the Stored Procedure which is expecting "Job_ID" and "MyDate" as input parameters.
FYI, I am using PowerShell and SQL Server 2016.
PS C:\PowerShell> $PSVersionTable.PSVersion
Major Minor Build Revision
----- ----- ----- --------
2 0 -1 -1
Here is some sample data for your reference:
CREATE TABLE control_table(JOB_ID INT, CODE VARCHAR(5));
INSERT INTO control_table(1, 'ABC');
INSERT INTO control_table(2, 'ABC');
INSERT INTO control_table(3, 'ABC');
INSERT INTO control_table(1, 'DEF');
INSERT INTO control_table(1, 'GHI');
CREATE PROCEDURE myschema.run_job (#JOB_ID INT, #MyDate DATE)
AS
BEGIN
-- Do Something
END
When I run the PowerShell script by passing 'ABC" as code, it should execute all the three jobs concurrently by reading the control table.
Something like
.\test.ps1 –MyCode “ABC” –Dt “12/27/2018”

Answer 1

As an alternative to jobs you can use async methods of built-in SQL client. Below is the sample code. I assume you already has some "run_job" procedure that can execute other procedures (jobs) by id.
$code = "ABC"
$date = "2018-12-31"
$jobs = #{} # this will store results of async jobs
$str = "Server = YourServer; Database = YourDB; Integrated Security = True;"
#--------
function Async-Sql { param($connStr, $sql, [switch]$GetDataTable)
$conn = New-Object System.Data.SqlClient.SqlConnection $str
$cmd = $conn.CreateCommand()
$conn.Open()
$cmd.CommandText = $sql
if($GetDataTable) {
$dt = New-Object System.Data.DataTable "result"
$r = $cmd.ExecuteReader()
$dt.Load($r)
$conn.Close()
return #(,$dt)
} else {
$w = $cmd.ExecuteNonQueryAsync()
return [PSCustomObject]#{result=$w; conn = $conn} }
}
# ---------------------------------------------
# get a list of jobs from your control table, this will run synchronously
$jobList = Async-Sql -connStr $str -sql "select job_id, code from test.control_table where code = '$code'" -GetDataTable
# main loop. You should call your stored procedure here. Each iteration will create a new connection and execute command asynchronously
foreach($id in $jobList.job_id) {
$command = "EXEC run_job $id, $date"
$r = Async-Sql -connStr $str -sql $command
$jobs.Add( $id, $r )
}
# wait for all jobs to complete
while ($False -in $jobs.Values.result.isCompleted) { sleep -Milliseconds 500 }
# print results / close connections. If you see status as RanToCompletion the job is completed successfuly
foreach($j in $jobs.Keys) {
$res = $jobs[$j].result
[PSCustomObject]#{JobId=$j; isCompleted = $res.isCompleted; Status = $res.Status; result = $res.Result }
$jobs[$j].conn.close()
}

Answer 2

Since you have powershell V2, I'm adding a solution with PS Jobs. Save the code below as sqlExec.ps1:
param($connStr, $sql)
$conn = New-Object System.Data.SqlClient.SqlConnection $connStr
$cmd = $conn.CreateCommand()
$conn.Open()
$cmd.CommandText = $sql
$r = $cmd.ExecuteNonQuery()
$conn.Close()
return $r
Then use this code as a master script:
$str = "Server = YourServer; Database = YourDB; Integrated Security = True;"
$code = "ABC"
$date = (get-date).ToString("yyyy-MM-dd")
$execSript = "path\to\sqlExec.ps1"
# get a list of ids for code. You can achive the same with Invoke-sqlcmd or similar cmdlet.
$conn = New-Object System.Data.SqlClient.SqlConnection $str
$cmd = $conn.CreateCommand()
$conn.Open()
$cmd.CommandText = "select job_id, code from test.control_table where code = '$code'"
$dt = New-Object System.Data.DataTable "result"
$r = $cmd.ExecuteReader()
$dt.Load($r)
$conn.Close()
# main loop
$jobs = #{}
foreach($id in $dt.job_id) { $top = $id*3
$sql = "EXEC run_job $id, $date"
$jobs.Add($id,(Start-Job -FilePath $execSript -ArgumentList $str, $sql))
}
# Wait for jobs and get result
$jobs.Values | Wait-Job | Receive-Job
If you have Invoke-Sqlcmd module you can use it for sql code execution (instead of creating $conn,$cmd,etc)