I'm working on a script to extract data from BLOBs in a SQL database. The extraction process works great. I want to add some sort of progress indication to the script. I have a total record count from a SQL query, and an incremental counter that increases for each file exported. The incremental counter works, but the total record count - which I attempted to assign to a global variable - does not seem to hold its value. Am I declaring it incorrectly?
## Export of "larger" Sql Server Blob to file
## with GetBytes-Stream.
# Configuration data
$StartTime = Get-Date
$Server = "server";
$UserID = "user";
$Password = "password";
$Database = "db";
$Dest = "C:\Users\me\Desktop\Test\";
$bufferSize = 8192;
# Counts total rows
$CountSql = "SELECT Count(extension) as countall from
(
SELECT p.[people_id], right(pi.[file_name],4) as extension
FROM dbo.pictures as pi
INNER JOIN dbo.people AS p ON p.person_picture = pi.pictures_id
where left([image_type], 5) = 'image'
) as countall"
# Selects Data
$Sql = "SELECT p.[people_id], pi.[image_file], right(pi.[file_name],4), ROW_NUMBER() OVER (ORDER BY people_id) as count
FROM dbo.pictures as pi
INNER JOIN dbo.people AS p ON p.person_picture = pi.pictures_id
where left([image_type], 5) = 'image'";
# Open ADO.NET Connection
$con = New-Object Data.SqlClient.SqlConnection;
$con.ConnectionString = "Data Source=$Server;" +
"Integrated Security=False;" +
"User ID=$UserID;" +
"Password=$Password;" +
"Initial Catalog=$Database";
$con.Open();
# New Command and Reader for total row count
$CountCmd = New-Object Data.SqlClient.SqlCommand $CountSql, $con;
$crd = $CountCmd.ExecuteReader();
While ($crd.Read())
{
$crd.GetValue($global:1)
}
$crd.Close();
# New Command and Reader for rest of data
$cmd = New-Object Data.SqlClient.SqlCommand $Sql, $con;
$rd = $cmd.ExecuteReader();
# Create a byte array for the stream.
$out = [array]::CreateInstance('Byte', $bufferSize)
# Looping through records
While ($rd.Read())
{
$total = $global:1
$counter = ($rd.GetValue(3));
Write-Output ("Exporting $counter of $total`: {0}" -f $rd.GetGUID(0));
# New BinaryWriter
$fs = New-Object System.IO.FileStream ($Dest + $rd.GetGUID(0) + $rd.GetString(2)), Create, Write;
$bw = New-Object System.IO.BinaryWriter $fs;
$start = 0;
# Read first byte stream
$received = $rd.GetBytes(1, $start, $out, 0, $bufferSize - 1);
While ($received -gt 0)
{
$bw.Write($out, 0, $received);
$bw.Flush();
$start += $received;
# Read next byte stream
$received = $rd.GetBytes(1, $start, $out, 0, $bufferSize - 1);
}
$bw.Close();
$fs.Close();
}
# Closing & Disposing all objects
$fs.Dispose();
$rd.Close();
$cmd.Dispose();
$con.Close();
$EndTime = Get-Date
$TotalTime = $EndTime - $StartTime
Write-Host ("Finished in {0:g}" -f $TotalTime)
OUTPUT
PS C:\Users\me> C:\Scripts\ExportImagesFromNTST.ps1
21380
Exporting 1 of : 3089b464-e667-4bf4-80b3-0002d582d4fa
Exporting 2 of : 04cf7738-ae19-4771-92b8-0003c5f27947
Exporting 3 of : 94485b5d-fe71-438d-a097-000ad185c915
and so on. 21380 should be $1 which should also be $total.
I think PetSerAl hit the nail on the head here. You create a SqlCommand object ($CountCmd), and from that create a SqlDataReader ($crd), and then tell $crd to use the GetValue() method that accepts an integer as a parameter, so that it knows which column to return the value of, but you reference a global variable with a name of '1', which is never defined, so you effectively pass $null to that method, so it doesn't get any value. I'm honestly surprised that it doesn't throw errors at you right there. You would probably want to just pass the integer 1 as the argument for that method, and assign it to $Total in the While loop. I'm honestly guessing here, but from what I see I think it should be:
$crd = $CountCmd.ExecuteReader();
While ($crd.Read())
{
$Total = $crd.GetValue(0)
}
$crd.Close();
I'm pretty sure that will assign the value of the first column (which for that sql command should just be 1 row with 1 column, right? Just the total count?), anyway, assign the first column's value for the current row to $Total. Then later you can reference $Total just fine to update your progress.
You sir, need to look into the write-progress cmdlet if you want to track progress, it's perfect for your script.
Related
In my case i want to use PS script to build WinForm with some elements including DGV contains of 3 columns (#, Page_Name, shrt). First row need to be template row with default values(1;index;NDX)so i get it from csv-file.My code:
$DataGridView1 = New-Object system.Windows.Forms.DataGridView
$DataGridView1.location = New-Object System.Drawing.Point(20,121)
$DataGridView1.Name = "Page-List"
$DataGridView1.AllowUserToAddRowsChanged = $true
$DataGridView1.AllowUserToAddRows = $true
# $DataGridView1.DataBindings
$DataGridView1.width = 363
$DataGridView1.height = 150
$DataGridView1.ColumnCount = 3
$DataGridView1.ColumnHeadersVisible = $true
$DataGridView1.Columns[0].Name = '#'
$DataGridView1.Columns[0].Width = "40"
$DataGridView1.Columns[1].Name = "Page_Name"
$DataGridView1.Columns[1].Width = "205"
$DataGridView1.Columns[2].Name = "shrt"
$DataGridView1.Columns[2].Width = "75"
$DataGridView1.ReadOnly = $false
$DataGridView1.EditMode = "EditOnEnter"
$templateROW = #(Import-Csv -Delimiter ";" "C:\Users\vkons\OneDrive\Документы\PowerShell\Scripts\test\DGV\index.csv" -Header "#", "Page_Name", "shrt" )
$datatable = ($templateROW + $DataGridView1Rows)
$DataGridView1Data = $datatable
foreach ($Row in $DataGridView1Data){
$DataGridView1.Rows.Add($Row.'#', $Row.Page_Name, $Row.shrt)
}
If user will change Page_Name cells value in first row or will fill Page_Name cell`s in the next row (or rowS) - cells value in column "#" and column "shrt" in edited row(s) would get values programmly by this part code:
$DataGridView1.Add_CellValueChanged({autofill})
Function autofill{
$Numbr = $DataGridView1.CurrentRow.Index+1
$DataGridView1.CurrentRow.Cells[0].value = $Numbr
$Name_Page = $DataGridView1.CurrentRow.Cells[1].value
$preshrt = $Name_Page.ToString($Value) -ireplace "[aoueyi]"
$preshrt = $preshrt.ToUpper()
$shrt = $preshrt[0]+$preshrt[1]+$preshrt[2]
$DataGridView1.CurrentRow.Cells[2].value = $shrt
}
My main target is getting the values of all cells in a column Page_Name as a variable(or as array). So I tried to add next string to the function above.
$Pages = $Row.Page_Name+$DataGridView1.CurrentRow.Cells[1].value
But it returns nothing...(Either $Row.Page_Name)
I can get values of all cells in all rows by
$Page_NamesListRows = #($DataGridView1.Rows.Cells.Value)
(Unfortunately) it returns varriable, consist of all existing cells, not rows array.But when i try
$Page_Names = $DataGridView1.Rows.Cells[1].Value
or
$Page_Names = $DataGridView1.Columns[1].Cells.Value
to get only Names of the Pages, it returns error "cannot get value of a null-array" (either in case with #(...) for right part)
Could anybody answer... Is there any way to get values of all existing cells in "Page_Name" Column.Honestly it doesnt matter would the DGVData automaticly edit by changing cells value event or not.
I need to get only column "Page_Name" values.
In the end I want to apologize for my english language. It has rather poor level. And thank the moderator in advance for corrections my mistakes.
I'm afraid you will have to get the array of values by looping over the rows in the "Page_Name" column.
The last row in the DataGridView will always be the "New" row to create by the user, so you need to skip that one.
Either by doing this:
# -1 to skip the always present empty row at the bottom
$Page_Names = for($row = 0; $row -lt $DataGridView1.Rows.Count - 1; $row++) {
$DataGridView1.Rows[$row].Cells.Item("Page_Name").Value
}
Or something like:
$Page_Names = $DataGridView1.Rows | ForEach-Object {
$data = $_.Cells.Item("Page_Name").Value
if ($data) { $data }
}
Or:
$Page_Names = foreach ($row in $DataGridView1.Rows) {
$row.Cells.Item("Page_Name").Value
}
$Page_Names = $Page_Names[0..($Page_Names.Count - 2)]
The last alternative is costly, because it needs to recreate the entire array when removing the last item
P.S.1 Don't forget to call the Dispose() methods on both the $DataGridview1 object and the main form when done with the GUI
P.S.2 I don't see a property called AllowUserToAddRowsChanged on the DataGridView..
EDIT
To hopefully show better what I mean, here's a demo form with a DataGridView control on it.
The initial data comes from a dummy CSV file with this inside:
"1";"Page number 1";"PN1"
"2";"Page number 2";"PN2"
"3";"Page number 3";"PN3"
"4";"Page number 4";"PN4"
"5";"Page number 5";"PN5"
$form = New-Object System.Windows.Forms.Form
$form.ClientSize = New-Object Drawing.Size 580, 505
$form.text = "20/4/2020 v. 0.1 All Right reserved (c) "
$form.TopMost = $true
$DataGridView1 = New-Object system.Windows.Forms.DataGridView
$DataGridView1.Location = New-Object System.Drawing.Point 20,25
$DataGridView1.Width = 363
$DataGridView1.Height = 150
$DataGridView1.AllowUserToAddRows = $true
$DataGridView1.Name = "Page-List"
$DataGridView1.ColumnCount = 3
$DataGridView1.Columns[0].Name = '#'
$DataGridView1.Columns[0].Width = "40"
$DataGridView1.Columns[1].Name = 'Page_Name'
$DataGridView1.Columns[1].Width = "205"
$DataGridView1.Columns[2].Name = "shrt"
$DataGridView1.Columns[2].Width = "75"
$DataGridView1.AllowUserToAddRows = $true
$DataGridView1.ReadOnly = $false
# Populate the DGV with the data from the CSV
$CsvData = Import-Csv -Path 'D:\Test\TEMPLATE_ROW.csv' -Delimiter ";" -Header "#", "Page_Name", "shrt"
foreach ($row in $CsvData) {
[void]$DataGridView1.Rows.Add($row.'#', $row.Page_Name, $row.shrt)
}
# add the DGV to the form
$form.Controls.Add($DataGridView1)
# show the form and capture the result so you can check if the user cancelled or pressed OK
$result = $form.ShowDialog()
# at this point, you can read the data from the DataGridView column of interest
$Page_Names = for($row = 0; $row -lt $DataGridView1.Rows.Count - 1; $row++) {
$DataGridView1.Rows[$row].Cells.Item("Page_Name").Value
}
# cleanup memory by destroying the DGV and the from
$DataGridView1.Dispose()
$form.Dispose()
In variable $Page_Names you will now have the data from the "Page_Name" column in the DataGridView control.
# show on screen
$Page_Names
I am working on a program that has an unknown number of textboxes displayed.
These textboxes later will display data about different servers (ping data for ex.)
It is unknown because I don't know how many servers will be in work, they will be automatically selected from a list... it can be 2, 3 or 15.
The text boxes will appear in a new window that is sized according to the number of servers.
The problem is that I have difficulties referring to the textboxes later in the program.
My first attempt was this:
Created a function to display the textboxes:
function c_inputbox ($iname, $iposx, $iposy, $isizex, $isizey)
{
$iname = New-Object System.Windows.Forms.richTextBox
$iname.Location = New-Object System.Drawing.Size($iposx, $iposy)
$iname.Size = New-Object System.Drawing.Size($isizex, $isizey)
$iname
}
Then I generate the textboxes, $objform1 is the main window, $x and $y are variables to position the textboxes in colums:
foreach ($srv in $stringArray)
{
$name = "textbox" + $i
$objform1.Controls.Add((c_inputbox $name $x ($y + 20) 350 100))
$i ++
}
It displayes the textboxes as I want but referring to the .text property doesn't work anymore. I tried several ways to test it:
$textbox1.text = "test"
$name.text = "test"
My second attempt was to store the names in an array, I tried even filling up the array with names before declaring the texboxes ($length contains the number of servers):
$j = 1
for ($j; $j -le $length; $j++)
{
$textbox[$j] = "textbox" + $j
}
So now the array should contain "textbox1", "textbox2", etc. Then I try to call them again to define them as textbox objects:
$textbox[$i] = New-Object System.Windows.Forms.richTextBox
$textbox[$i].Location = New-Object System.Drawing.Size($positionx, $positiony)
$textbox[$i].Size = New-Object System.Drawing.Size(350, 100)
$objform1.Controls.Add($textbox[$i])
But again PowerShell returns an error:
Cannot index into a null array.
+ $objform1.Controls.Add($textbox[ <<<< $i])
+ CategoryInfo : InvalidOperation: (1:Int32) [], RuntimeException
+ FullyQualifiedErrorId : NullArray
Any idea how to make this happen, or if it is even possible to do in PowerShell?
Create and assign the control to a local variable, configure it as necessary, then add it to an array before adding it to Form.Controls:
$TextBoxes = #()
for($i = 0; $i -lt $stringArray.Count; $i++)
{
# Create the textbox
$name = "textbox$i"
$textBox = c_inputbox $name $x ($y + 20) 350 100
# Customise it
$textBox.Text = $stringArray[$i]
# Add to array
$TextBoxes += $textBox
# Add to Form Controls, index -1 is the last item in the array
$objform1.Controls.Add($TextBoxes[-1])
}
Now you can use $TextBoxes to refer to the boxes, or $TextBoxes[$index] to refer to a specific one
I have been wrestling with database connection to PostgreSQL from Powershell. I finally am able to connect to and insert into the database. Now I can't figure out how to extract data from a DB select into a variable.
I'm not including my insert for the sake of clarity but will tack it onto this thread later as I know it was super hard to find and may be helpful to someone.
so here's my code:
# use existing 64 bit ODBC System DSN that we set up manually
$DBconn = New-Object -comobject ADODB.Connection
$DBconn.Open("PostgreSQL35W")
$theQuery = "select * from test1"
$theObject = $DBconn.Execute($theQuery) # $theObject is a System.__ComObject
$numRecords = $theObject.RecordCount
write-host "found $numRecords records" # getting -1
$theObject.MoveFirst() # throws no error
# $theValue = $theObject.DataMember # throws no error, but gives no result
$theValue = $theObject.Index[1] # throws "Cannot index into a null array"
write-host($theValue)
try this
replace "#database#" with your database name in $cnString
replace "#server_ip#" with your server ip address in $cnString
replace "#user#" with a valid user in $cnString and $user
replace "#pass#" with a valid pass in $pass
replace "#table#" with a valid table name of your db
replace 5432 with your db port
$cnString = "DRIVER={PostgreSQL Unicode(x64)};DATABASE=#database#;SERVER=#server_ip#;PORT=5432;UID=#user#;"
$user="#user#"
$pass="#pass#"
$conn = New-Object -comobject ADODB.Connection
$conn.Open($cnString,$user,$pass)
$recordset = $conn.Execute("SELECT * FROM #table# limit 1;")
while ($recordset.EOF -ne $True)
{
foreach ($field in $recordset.Fields)
{
'{0,30} = {1,-30}' -f # this line sets up a nice pretty field format, but you don't really need it
$field.name, $field.value
}
'' # this line adds a line between records
$recordset.MoveNext()
}
$conn.Close();
Via psql, which comes with postgresql
$dburl="postgresql://exusername:expw#exhostname:5432/postgres"
$data="select * from extable" | psql --csv $dburl | ConvertFrom-Csv
You must have psql in your path or reference it, its within e.g. C:\Program Files\PostgreSQL\12\bin. Should be able to type "psql" and see output within powershell.
As a warning, expect strings. E.g $data[0].age.GetType() would be string, despite being stored in the database as an integer. You can immediately cast it, cast it later, or hope powershell infers type correctly.
If you want to add back in type information can do e.g.:
$data = $data | %{[pscustomobject]#{name=$_.name;age=[int]$_.age}}
I ended up figuring it out - here's what I did
$conn = New-Object -comobject ADODB.Connection
# use existing 64 bit ODBC System DSN that we set up manually
$conn.Open("PostgreSQL35W")
$recordset = $conn.Execute("SELECT * FROM JobHistory")
while ($recordset.EOF -ne $True)
{
foreach ($field in $recordset.Fields)
{
'{0,30} = {1,-30}' -f # this line sets up a nice pretty field format, but you don't really need it
$field.name, $field.value
}
'' # this line adds a line between records
$recordset.MoveNext()
}
$conn.Close();
Exit
use the dot notation. You don't need to split the data.
$list = New-Object Collections.Generic.List[OnlineCourse]
foreach($element in $results)
{
$tempObj= New-Object OnlineCourse($element.id,$element.courseName,$element.completedRatio,$element.completedRatio,$element.lastActivity, $element.provider)
$list.add($tempObj)
}
I have a slightly different approach to #dog, I couldn't get the --csv to work, so I resorted to tuple only rows returned, then parse them into a List of Classes (which happen to be called OnlineCourses):
class OnlineCourse
{
[int]$id
[string]$email
[string]$courseName
[int]$completedRatio
[datetime]$lastActivity
[String]$provider
OnlineCourse([int]$id,
[string]$email,
[string]$courseName,
[int]$completedPerc,
[datetime]$lastActivity,
[String]$provider) {
$this.id = $id
$this.email = $email.Trim()
$this.courseName = $courseName.Trim()
$this.completedRatio = $completedPerc
$this.lastActivity = $lastActivity
$this.provider = $provider.Trim()
}
}
$connstr="postgresql://exusername:expw#exhostname:5432/postgres"
$data = "select * from onlinecourses" | .\psql -t $connstr
$list = New-Object Collections.Generic.List[OnlineCourse]
foreach ($field in $data) {
$id, $email, $courseName, $completedratio, $lastactivity, $provider = $field.split('|')
$course = [OnlineCourse]::new($id, $email, $courseName, $completedratio, $lastactivity, $provider)
$list.Add($course)
}
This is slightly adapted from another answer and it worked for me.
$dburl="postgresql://postgres:secret_pwd#database-host:5432/dbname"
$psqlPath = 'C:\Program Files\PostgreSQL\11\bin\psql.exe'
function Query {
param($Sql)
Write-Host $Sql
$rows = $Sql `
| &$psqlPath "-A" $dburl | ConvertFrom-Csv -Delimiter '|'
$result = #($rows | Select-Object -SkipLast 1)
Write-Host "-> " (ConvertTo-Json $result)
$result
}
$rows = Query "select ... from ..."
I have the following function that I provide 3 arrays as variables
$columnHeaders = #('Ticket ID', 'Date Raised', 'Title', 'Status' )
$columnproperties = #('number', 'opened_at', 'short_description', 'state')
$contents
$contents has multiple rows of data matching the columns above, however sometimes may only have 1 row. When the $contents only has 1 row the below function errors out and doesnt print the data.
Using ISE I traced the issue to the $contents.count not showing a value, why is this? is there a way to get around it?
function TableOutput ($columnHeaders, $columnProperties, $contents){
# Number of columns
$columnCount = $columnHeaders.Count
# Create a new table
$docTable = $Word.ActiveDocument.Tables.Add($Word.Selection.Range,$contents.Count,$columnCount)
# Table style
$doctable.Style = "Adapt Table"
# Insert the column headers into the table
for ($col = 0; $col -lt $columnCount; $col++) {
$cell = $docTable.Cell(1,$col+1).Range
$cell.Font.Bold=$true
$cell.InsertAfter($columnHeaders[$col])
}
$doctable.Rows.Add() > Null
# Load the data into the table
$i = 1
$j = $contents.Count
for($row = 2; $row -lt ($contents.Count + 2); $row++){
if($row -gt 2){
}
for ($col = 1; $col -le $columnCount; $col++){
Write-Progress -Activity "Processing Table Information" -Status "Adding Row entry $i of $j" -PercentComplete (100*$i/$j)
$cell = $docTable.Cell($row,$col).Range
$cell.Font.Name="Calibri"
$cell.Font.Size="10"
$cell.Font.Bold=$FALSE
$cell.Text = $contents[$row-2].($columnProperties[$col-1])
}
$i++
}
$doctable.Columns.AutoFit()
}
any help is greatly appreciated.
Cast $content as an array of strings and see if that doesn't work better for you.
function TableOutput ($columnHeaders, $columnProperties, [string[]]$contents){
Edit: Sorry, my bad, you are passing objects with properties ad descripbed in $columnheaders, so you would need to cast it as an array of objects instead:
function TableOutput ($columnHeaders, $columnProperties, [object[]]$contents){
Tested on my end, it works fine with 1 object being passed to the function, as well as an array of two objects being passed to the function.
Extreme powershell newbie here. I appreciate any and all help.
I'm trying to put together a simple anti-flooding script to work with Sharepoint/Powershell. Need it to look at a datetime in a field and compare it to the current datetime then stop execution if within 5 seconds of the last submittal. The method im using now always seems to evaluate to true.
#get system datetime (output format - 06/12/2014 07:57:25)
$a = (Get-Date)
# Get current List Item
$ListItem = $List.GetItemById($ItemID)
$DateToCompare = $ListItem["baseline"].AddMilliseconds(5000)
if ($DateToCompare -gt $a)
{Break}
#set variable to field
$ListItem["baseline"] = $a
#write new item
$ListItem.Update()
Break
I don't have Sharepoint access so I cannot fully test.
Can you verify the datatype of "baseline" attribute?
($ListItem["baseline"]).getType().Name
Are you sure that 5000 millseconds is really being added?
Write-Output "NOW: $($curDate) BASELINE: $($DateToCompare) DIFF: $( ($curDate - $DateToCompare).TotalMilliseconds )"
Why use break rather than let the evaluation naturally end? Below is an alternative way you might restructure you code.
#The difference in Milliseconds acceptable
$threshold = 5000
#Get current date, the formatting depends on what you have defined for output.
$curDate = Get-Date
#Get current list item from SP
$listItem = $List.GetItemById($ItemID)
# Get current List Item's baseline
$DateToCompare = $listItem["baseline"]
Write-Output "NOW: $($curDate) BASELINE: $($DateToCompare) DIFF: $( ($curDate - $DateToCompare).TotalMilliseconds )"
if ( ($curDate - $DateToCompare).TotalMilliseconds -le $threshold ){
#set variable to field
$ListItem["baseline"] = $curDate
#write new item
$ListItem.Update()
} else {
#Outside of threshold
}
So it turns out the script as I gave it above was functional. The issue was the time I was pulling under the (Get-Date) function was the server time (central), rather than the local time (eastern).
#bring server time up to eastern time
$a = (Get-Date).AddMilliseconds(7200000)
# Get current List Item
$ListItem = $List.GetItemById($ItemID)
#take baseline time and add 5 seconds
$DateToCompare = $ListItem["baseline"].AddMilliseconds(5000)
#stop if script has run in the last 5 sec (loop prevention)
if ($DateToCompare -gt $a)
{Break}
#stop if the status hasnt changed
if ($ListItem["baselinestatus"] -eq $ListItem["Status"])
{Break}
#get current activity status
$currentstatus = $ListItem["Status"]
#get current contents of log
$log = $ListItem["Log"]
#append new entry to existing and write it to the log
$newentry = $log + "<br>" + $a + " - " + $currentstatus
#set variable to field
$ListItem["Log"] = $newentry
$ListItem["baseline"] = $a
$ListItem["baselinestatus"] = $currentstatus
#write new item
$ListItem.Update()