I'm running a stored procedure and am trying to filter out which columns are returned by GetSchemaTable()
$reader = $cmd.ExecuteReader()
$schemaTable = $reader.GetSchemaTable();
foreach ($row in $schemaTable.Rows)
{
foreach ($column in $schemaTable.Columns)
{
write-host $column;
}
}
...shows me a whole bunch of column names I don't care about and 2 that i do:
ColumnName and ColumnOrdinal
How do I go about restricting output to just those to fields?
thx
$reader = $cmd.ExecuteReader()
$reader.GetSchemaTable() | Select ColumnName, ColumnOrdinal
Related
How to remove duplicate rows from a DataTable?
I have a Datatable1 & Datatable2. Datatable1 has 5 rows and Datatable2 has 1 row.
When I called $Datatable1.Merge($Datatable2), it is resulting in 7 rows instead of 6 rows.
The duplicate row is from Datatable2.
How can I remove this duplicate one?
Posting the OP's own solution as an actual answer:
function RemoveDups {
Param($dt)
$distinct = New-Object "System.Data.DataTable"
$hash = New-Object "System.Collections.Generic.HashSet[string]"
foreach ($column in $dt.Columns) {
$distinct.Columns.Add($column.ToString()) | Out-Null
}
foreach ($row in $dt.rows) {
$columnsToCheck = [string]($row.COLUMNNAME)
$result = $hash.Add($columnsToCheck)
if ($result -eq $true) {
[void]$distinct.Rows.Add($row.ItemArray)
}
}
return ,$distinct
}
try this:
$Datatable1.Merge($Datatable2)
$Datatable1=$Datatable1.DefaultView.ToTable($true)
I have two powershell arrays of objects generated via Import-CSV, and I must match them by one of their properties. Specifically, it is a 1:n relationship so currently I'm following this pattern:
foreach ($line in $array1) {
$match=$array2 | where {$_.key -eq $line.key} # could be 1 or n results
...# process here the 1 to n lines
}
, which is not very efficient (both tables have many columns) and takes a time that is unacceptable for our needs. Is there a fastest way to perform this match?
Both data sources come from a csv file, so using something instead of Import-CSV would be also welcome.
Thanks
The standard method is to index the data using a hashtable (or dictionary/map in other languages).
function buildIndex($csv, [string]$keyName) {
$index = #{}
foreach ($row in $csv) {
$key = $row.($keyName)
$data = $index[$key]
if ($data -is [Collections.ArrayList]) {
$data.add($row) >$null
} elseif ($data) {
$index[$key] = [Collections.ArrayList]#($data, $row)
} else {
$index[$key] = $row
}
}
$index
}
$csv1 = Import-Csv 'r:\1.csv'
$csv2 = Import-Csv 'r:\2.csv'
$index2 = buildIndex $csv2, 'key'
foreach ($row in $csv1) {
$matchedInCsv2 = $index2[$row.key]
foreach ($row2 in $matchedInCsv2) {
# ........
}
}
Also, if you need speed and iterate a big collection, avoid | pipelining as it's many times slower than foreach/while/do statements. And don't use anything with a ScriptBlock like where {$_.key -eq $line.key} in your code because execution context creation adds a ridiculously big overhead compared to the simple code inside.
I have been wrestling with database connection to PostgreSQL from Powershell. I finally am able to connect to and insert into the database. Now I can't figure out how to extract data from a DB select into a variable.
I'm not including my insert for the sake of clarity but will tack it onto this thread later as I know it was super hard to find and may be helpful to someone.
so here's my code:
# use existing 64 bit ODBC System DSN that we set up manually
$DBconn = New-Object -comobject ADODB.Connection
$DBconn.Open("PostgreSQL35W")
$theQuery = "select * from test1"
$theObject = $DBconn.Execute($theQuery) # $theObject is a System.__ComObject
$numRecords = $theObject.RecordCount
write-host "found $numRecords records" # getting -1
$theObject.MoveFirst() # throws no error
# $theValue = $theObject.DataMember # throws no error, but gives no result
$theValue = $theObject.Index[1] # throws "Cannot index into a null array"
write-host($theValue)
try this
replace "#database#" with your database name in $cnString
replace "#server_ip#" with your server ip address in $cnString
replace "#user#" with a valid user in $cnString and $user
replace "#pass#" with a valid pass in $pass
replace "#table#" with a valid table name of your db
replace 5432 with your db port
$cnString = "DRIVER={PostgreSQL Unicode(x64)};DATABASE=#database#;SERVER=#server_ip#;PORT=5432;UID=#user#;"
$user="#user#"
$pass="#pass#"
$conn = New-Object -comobject ADODB.Connection
$conn.Open($cnString,$user,$pass)
$recordset = $conn.Execute("SELECT * FROM #table# limit 1;")
while ($recordset.EOF -ne $True)
{
foreach ($field in $recordset.Fields)
{
'{0,30} = {1,-30}' -f # this line sets up a nice pretty field format, but you don't really need it
$field.name, $field.value
}
'' # this line adds a line between records
$recordset.MoveNext()
}
$conn.Close();
Via psql, which comes with postgresql
$dburl="postgresql://exusername:expw#exhostname:5432/postgres"
$data="select * from extable" | psql --csv $dburl | ConvertFrom-Csv
You must have psql in your path or reference it, its within e.g. C:\Program Files\PostgreSQL\12\bin. Should be able to type "psql" and see output within powershell.
As a warning, expect strings. E.g $data[0].age.GetType() would be string, despite being stored in the database as an integer. You can immediately cast it, cast it later, or hope powershell infers type correctly.
If you want to add back in type information can do e.g.:
$data = $data | %{[pscustomobject]#{name=$_.name;age=[int]$_.age}}
I ended up figuring it out - here's what I did
$conn = New-Object -comobject ADODB.Connection
# use existing 64 bit ODBC System DSN that we set up manually
$conn.Open("PostgreSQL35W")
$recordset = $conn.Execute("SELECT * FROM JobHistory")
while ($recordset.EOF -ne $True)
{
foreach ($field in $recordset.Fields)
{
'{0,30} = {1,-30}' -f # this line sets up a nice pretty field format, but you don't really need it
$field.name, $field.value
}
'' # this line adds a line between records
$recordset.MoveNext()
}
$conn.Close();
Exit
use the dot notation. You don't need to split the data.
$list = New-Object Collections.Generic.List[OnlineCourse]
foreach($element in $results)
{
$tempObj= New-Object OnlineCourse($element.id,$element.courseName,$element.completedRatio,$element.completedRatio,$element.lastActivity, $element.provider)
$list.add($tempObj)
}
I have a slightly different approach to #dog, I couldn't get the --csv to work, so I resorted to tuple only rows returned, then parse them into a List of Classes (which happen to be called OnlineCourses):
class OnlineCourse
{
[int]$id
[string]$email
[string]$courseName
[int]$completedRatio
[datetime]$lastActivity
[String]$provider
OnlineCourse([int]$id,
[string]$email,
[string]$courseName,
[int]$completedPerc,
[datetime]$lastActivity,
[String]$provider) {
$this.id = $id
$this.email = $email.Trim()
$this.courseName = $courseName.Trim()
$this.completedRatio = $completedPerc
$this.lastActivity = $lastActivity
$this.provider = $provider.Trim()
}
}
$connstr="postgresql://exusername:expw#exhostname:5432/postgres"
$data = "select * from onlinecourses" | .\psql -t $connstr
$list = New-Object Collections.Generic.List[OnlineCourse]
foreach ($field in $data) {
$id, $email, $courseName, $completedratio, $lastactivity, $provider = $field.split('|')
$course = [OnlineCourse]::new($id, $email, $courseName, $completedratio, $lastactivity, $provider)
$list.Add($course)
}
This is slightly adapted from another answer and it worked for me.
$dburl="postgresql://postgres:secret_pwd#database-host:5432/dbname"
$psqlPath = 'C:\Program Files\PostgreSQL\11\bin\psql.exe'
function Query {
param($Sql)
Write-Host $Sql
$rows = $Sql `
| &$psqlPath "-A" $dburl | ConvertFrom-Csv -Delimiter '|'
$result = #($rows | Select-Object -SkipLast 1)
Write-Host "-> " (ConvertTo-Json $result)
$result
}
$rows = Query "select ... from ..."
I have the following function that I provide 3 arrays as variables
$columnHeaders = #('Ticket ID', 'Date Raised', 'Title', 'Status' )
$columnproperties = #('number', 'opened_at', 'short_description', 'state')
$contents
$contents has multiple rows of data matching the columns above, however sometimes may only have 1 row. When the $contents only has 1 row the below function errors out and doesnt print the data.
Using ISE I traced the issue to the $contents.count not showing a value, why is this? is there a way to get around it?
function TableOutput ($columnHeaders, $columnProperties, $contents){
# Number of columns
$columnCount = $columnHeaders.Count
# Create a new table
$docTable = $Word.ActiveDocument.Tables.Add($Word.Selection.Range,$contents.Count,$columnCount)
# Table style
$doctable.Style = "Adapt Table"
# Insert the column headers into the table
for ($col = 0; $col -lt $columnCount; $col++) {
$cell = $docTable.Cell(1,$col+1).Range
$cell.Font.Bold=$true
$cell.InsertAfter($columnHeaders[$col])
}
$doctable.Rows.Add() > Null
# Load the data into the table
$i = 1
$j = $contents.Count
for($row = 2; $row -lt ($contents.Count + 2); $row++){
if($row -gt 2){
}
for ($col = 1; $col -le $columnCount; $col++){
Write-Progress -Activity "Processing Table Information" -Status "Adding Row entry $i of $j" -PercentComplete (100*$i/$j)
$cell = $docTable.Cell($row,$col).Range
$cell.Font.Name="Calibri"
$cell.Font.Size="10"
$cell.Font.Bold=$FALSE
$cell.Text = $contents[$row-2].($columnProperties[$col-1])
}
$i++
}
$doctable.Columns.AutoFit()
}
any help is greatly appreciated.
Cast $content as an array of strings and see if that doesn't work better for you.
function TableOutput ($columnHeaders, $columnProperties, [string[]]$contents){
Edit: Sorry, my bad, you are passing objects with properties ad descripbed in $columnheaders, so you would need to cast it as an array of objects instead:
function TableOutput ($columnHeaders, $columnProperties, [object[]]$contents){
Tested on my end, it works fine with 1 object being passed to the function, as well as an array of two objects being passed to the function.
What I'm trying to do is create array variable names dynamically, and then with a loop, add the object to its relevant array based on the hash table value being equal to the counter variable.
$hshSite = #{} # Values like this CO,1 NE,2 IA,3
$counter = $hshSite.count
For($i = $counter; $i -gt 0; $i--) {
New-Variable -Name "arr$i" -Value #()
}
If $counter = 3, I would create arrays $arr1, $arr2, $arr3
$csv = Import-CSV....
ForEach ($x in $csv) {
#if $hshSite.Name = $x.location (ie CO), look up hash value (1),
and add the object to $arr1. If $hshSite.Name = NE, add to $arr2
I tried creating the dynamic arrays with New-Variable, but having issues trying to add to those arrays. Is it possible to concatenate 2 variables names into a single variable name? So taking $arr + $i to form $arr1 and $arr2 and $arr3, and then I can essentially just do $arr0 += $_
The end goal is to group things based on CO, NE, IA for further sorting/grouping/processing. And I'm open to other ideas of getting this accomplished. Thanks for your help!
Just make your hash table values the arrays, and accumulate the values to them directly:
$Sites = 'CO','NE','IA'
$hshSite = #{}
Foreach ($Site in $Sites){$hshSite[$Site] = #()}
ForEach ($x in $csv)
{
$hshSite[$x.location] += <whatever it is your adding>
}
If there's a lot of entries in the csv, you might consider creating those values as arraylists instead of arrays.
$Sites = 'CO','NE','IA'
$hshSite = #{}
Foreach ($Site in $Sites){ $hshSite[$Site] = New-Object Collections.Arraylist }
ForEach ($x in $csv)
{
$hshSite[$x.location].add('<whatever it is your adding>') > $nul
}
You could quite easily do add items to a dynamically named array variable using the Get-Variable cmdlet. Similar to the following:
$MyArrayVariable123 = #()
$VariableNamePrefix = "MyArrayVariable"
$VariableNameNumber = "123"
$DynamicallyRetrievedVariable = Get-Variable -Name ($VariableNamePrefix + $VariableNameNumber)
$DynamicallyRetrievedVariable.Value += "added item"
After running the above code the $MyArrayVariable123 variable would be an array holding the single string added item.