I am trying to work with DataTables in powershell, more specifically I am trying to check to see if a row with a certain value exists in the datatable.
Here is the method I am using to create the dataTable.
function GetPropertyIDs()
{
write-host "Retrieving all Property ID's from $DatabaseInstance.$DatabaseName..." -NoNewline
$cn = new-object system.data.SqlClient.SqlConnection("Data Source=$DatabaseInstance;Integrated Security=SSPI;Initial Catalog=$DatabaseName");
$cn.Open()
$cmd = new-Object System.Data.SqlClient.SqlCommand("Select Distinct CAST(ID as varchar(5)) as ID, Name, Code from TableName Order by ID", $cn)
$cmd.CommandType = [System.Data.CommandType]'Text'
$adapter = New-Object System.Data.SqlClient.SqlDataAdapter $cmd
$dataTable = New-Object System.Data.DataTable
$adapter.Fill($dataTable) | out-null
$cn.Close() | out-Null
$cn.Dispose()
$count = $dataTable.rows.count;
write-host "$count records retrieved."
#Set Primary Key
$pk = $dataTable.Columns["ID"]
$dataTable.PrimaryKey = $pk
return $dataTable
}
Here is the code where I am trying to check if a specific value exists in the datatable...
$PropertyIDs = GetPropertyIDs
foreach($PropertySite in $PropertySites)
{
$id = ([System.String] $PropertySite).Replace("$SiteCollectionURL/property/", "")
$found = $PropertyIDs.DefaultView.Find($id)
}
Currently I am getting an error:
"You cannot call a method on a null-valued expression."
I have also tried this with no luck either:
$found = $PropertyIDs.Rows.Contains($id)
It turns out that this was being caused by powershell unravelling the datatable into a collection of rows instead of a datatable like it was defined as, therefor the primary key which is required for .Rows.Contains(...) was not available and thus the error.
By adding the a ',' at the end of the GetPropertyIDs() method above I was able to get powershell to return a datatable with the primary keys intact. For Example:
return ,$dataTable
I ended up using the contains method because it returns a boolean value which is what I was looking for as a test...
Some References:
http://powershellcommunity.org/Forums/tabid/54/aft/6359/Default.aspx
Related
I'm getting some data from a SQL table, which I then store in a System.Data.DataSet object. I want to pass this data in this DataSet, as an Argument/Parameter, to a workflow, such that I can display all the data in this DataSet in a foreach -parallel style. But I'm at a loss for the correct syntax of passing data from a System.Data.DataSet object to a workflow. Currently I get an Error near the line "param([System.Data.DataSet]$pServiceDataSet)" as shown below.
Function GetSQLData
{
param ($TargetDBServer, $TargetDB, $SQLQuery)
# SQL Connection Object
$sqlConn = New-Object System.Data.SqlClient.SqlConnection
$sqlConn.ConnectionString = "Server=$TargetDBServer;Database=$TargetDB;User Id=SomeUser;Password=SomePassword;"
$sqlConn.Open()
# SQL Command
$sqlcmd = New-Object System.Data.SqlClient.SqlCommand
$sqlcmd.Connection = $sqlConn
$sqlcmd.CommandText = $SQLQuery
# SQL Adapter
$sqlAdp = New-Object System.Data.SqlClient.SqlDataAdapter ($sqlcmd)
# SQL DataSet
$ResultDataSet = New-Object System.Data.DataSet
$sqlAdp.Fill($ResultDataSet) | Out-Null
$sqlConn.Close()
return $ResultDataSet.Tables[0]
}
$CurrentComputerName = $env:COMPUTERNAME
# Export the Windows Services & it's config parameters from the "DatabaseABC..WindowsServicesConfig" Table.
$SQLQueryForService = "
SELECT [ServiceName], [StartUpParameter], [DBServerName], [DBName]
FROM [dbo].[WindowsServicesConfig] WITH (NOLOCK)
WHERE [HostServerName] = '$CurrentComputerName'
AND [ServiceName] LIKE '%MyService%' "
$ServicesDataSet = GetSQLData -TargetDBServer "ServerABC" -TargetDB "DatabaseABC" -SQLQuery $SQLQueryForService
$ServicesDataSet.GetType()
$ServicesDataSet | Format-Table
workflow DisplayAllServices
{
param([System.Data.DataSet]$pServiceDataSet) # <- I get an Error here
foreach -parallel ($Service in $pServiceDataSet)
{
$Service.ServiceName
$Service.StartUpParameter
$Service.DBServerName
$Service.DBName
}
}
DisplayAllServices -pServiceDataSet $ServicesDataSet
My final objective is to use the data in this DataSet to create Windows Services. But this is my most frustrating hurdle. I cannot get past the Error.
Figured out the Solution to my Problem. Replaced the "param([System.Data.DataSet]$pServiceDataSet)" with "param([PSObject]$pServiceDataSet)".
I am attempting to update data in an oledb datasource. I pull data from a table that who's schema has been written out as an access create table script (i could probably find a better way to get that schema information) and stored in the AccessCreateScript field of the lookup table.
I am able get the following things to work:
pull the data out of the source table on sql server
put it into a dataset
verify its good
create a second dataset based off the created table on the access file
copy the data from the source dataset to the destination dataset
verify that the data has been copied
create the update command using a command builder
verify the commands look correct
But, then when it comes to using the update method on the ole data adapter, it just doesnt do anything.
The data contained in the lookup table has been validated to be correct thoroughly.
There currently is only one record in the lookup table.
$LookupServer = "Redacted"
$LookupDatabase = "redacted"
$LookupSQL = "Select [DBServer]
,[SourceDB]
,[SourceTableOrScript]
,[DestinationFile]
,[DestinationTable]
,[AccessDropScript]
,[AccessCreateScript]
,[Active]
FROM [AccessDataExport_Lookup_Data]"
$LookupConnection = New-Object System.Data.SQLClient.SQLConnection
$LookupCSBuilder = New-Object System.Data.SqlClient.SqlConnectionStringBuilder
$LookupCSBuilder['server'] = $LookupServer
$LookupCSBuilder['trusted_connection'] = $True
$LookupCSBuilder['database'] = $LookupDatabase
$LookupConnection.ConnectionString = $LookupCSBuilder.ConnectionString
$LookupConnection.Open()
$LookupCommand = New-Object System.Data.SQLClient.SQLCommand
$LookupCommand.Connection = $LookupConnection
$LookupCommand.CommandText = $LookupSQL
$LookupDataAdapter = new-object System.Data.SqlClient.SqlDataAdapter $LookupCommand
$LookupDataset = new-object System.Data.Dataset
$LookupDataAdapter.Fill($LookupDataset)
$LookupConnection.Close()
foreach( $r in $LookupDataset.Tables[0].Rows){
$SourceConnection = New-Object System.Data.SQLClient.SQLConnection
$SourceCSBuilder = New-Object System.Data.SqlClient.SqlConnectionStringBuilder
$SourceCSBuilder['server'] = $r.DBServer
$SourceCSBuilder['trusted_connection'] = $true
$SourceCSBuilder['database'] = $r.SourceDB
$SourceConnection.ConnectionString = $SourceCSBuilder.ConnectionString
$SourceCmd = $SourceConnection.CreateCommand()
$DestConnection = New-Object System.Data.OleDb.OleDbConnection
$DestCSBuilder = New-Object System.Data.OleDb.OleDbConnectionStringBuilder
$DestCSBuilder['Provider'] = 'Microsoft.ACE.OLEDB.12.0'
$DestCSBuilder['Persist Security Info'] = $false
$DestCSBuilder['Data Source'] = $r.DestinationFile
$DestConnection.ConnectionString = $DestCSBuilder.ConnectionString
$DestCmd = $DestConnection.CreateCommand()
$DestConnection.Open()
$SourceConnection.Open()
$DestCmd.CommandText = $r.AccessDropScript
try
{
$DestCmd.ExecuteNonQuery()
}
catch
{
write-Output $_.Exception.Message
write-Output $_.InvocationInfo.ScriptLineNumber
}
$DestCmd.CommandText = $r.AccessCreateScript
try{
$DestCmd.ExecuteNonQuery()
$SourceCmd.CommandText = $r.SourceTableOrScript
$SourceDA = New-object System.Data.SqlClient.SqlDataAdapter $SourceCmd
$SourceData = new-object System.Data.Dataset
$DestCmd.CommandText = $r.SourceTableOrScript
$DestDA = New-Object System.Data.OleDb.OleDbDataAdapter $DestCmd
$DestData = new-object System.Data.Dataset
$result = $SourceDA.fill($SourceData)
Write-Output "Source filled $result rows"
$result = $DestDA.fill($DestData)
Write-Output "Dest filled $result rows"
$DestData.load($SourceData.CreateDataReader(),1,$SourceData.Tables[0].TableName)
$destRowCount = $DestData.Tables[0].Rows.Count
Write-Output "DestData Row Count $destrowcount"
$DestCb = New-Object System.Data.OleDb.OleDbCommandBuilder $DestDA
$DestDA.UpdateCommand = $DestCb.GetUpdateCommand()
$DestDA.InsertCommand = $DestCb.GetInsertCommand()
$DestDA.DeleteCommand = $DestCb.GetDeleteCommand()
$result = $DestDA.Update($DestData)
Write-Output "Copy data copied $result rows"
}
Catch
{
write-Output $_.Exception.Message
write-Output $_.InvocationInfo.ScriptLineNumber
}
finally{
$DestConnection.Close()
$SourceConnection.Close()
}
}
and the output is
PS C:\Users\****> & 'c:\Users\***\source\repos\Powershell Scripts\Modular-Export.ps1'
1
0
0
Source filled 1274 rows
Dest filled 0 rows
DestData Row Count 1274
Copy data copied 0 rows
Table data since this is pertinate, but with redacted sensitive information.
https://gist.github.com/KySoto/5a764f3fa3c0d139131056ed6d44529b
In the end a reddit user helped me figure it out, Their post was the thing that got everything rolling. What i needed to do was set the state of every datarow in my datatable to "Added" using the SetAdded method. Then i needed to set $DestCb.QuotePrefix = "[" and $DestCb.QuoteSuffix = "]" Then it properly filled.
I am trying to run a SQL from Power Shell(which is on my windows 7 64 bit desktop) and the remote database host is MS SQL Server 2012.
The code is:
$var1 = 'string';
function Get-ODBC-Data{
param(
[string]$query=$('
SELECT COUNT(*)
FROM [master].[sys].[table_name]
WHERE col2 = ''$var1''
;
'),
[string]$username='db_user_name',
[string]$password='db_password'
)
$conn = New-Object System.Data.Odbc.OdbcConnection
$conn.ConnectionString = "DRIVER={SQL Server};Server=123.456.78.90;Initial Catalog=master;Uid=$username;Pwd=$password;"
$conn.open()
$cmd = New-object System.Data.Odbc.OdbcCommand($query,$conn)
$ds = New-Object system.Data.DataSet
(New-Object system.Data.odbc.odbcDataAdapter($cmd)).fill($ds) | out-null
$conn.close()
$ds.Tables[0]
}
$result = Get-ODBC-Data
Write-Host "SQL_Output: " $result[0];
If I use 'string' in the SQL's where clause instead of $var1 then te script works fine and gives expected result.
Quetion
But I want to be able to pass any string as $var1 to the script as parameter. Then use it in the where clause of the SQL. How can I achieve this?
What I tried
I have tried to enclose $var1 in 1,2 or 3 single quotes in the where clause in attempt to escape the single quote. Also tried adding/removing single quote from 'string' when $var1 is assigned value. I did try [string]$var1 = 'string' as well but none of these worked and I keep getting error mostly related to SQL syntax.
Try this:
function Get-ODBC-Data{
param(
[string]$query=$("
SELECT COUNT(*)
FROM [master].[sys].[table_name]
WHERE col2 = '$($var1)'
;
"),
[string]$username='db_user_name',
[string]$password='db_password'
)
$conn = New-Object System.Data.Odbc.OdbcConnection
$conn.ConnectionString = "DRIVER={SQL Server};Server=123.456.78.90;Initial Catalog=master;Uid=$username;Pwd=$password;"
$conn.open()
$cmd = New-object System.Data.Odbc.OdbcCommand($query,$conn)
$ds = New-Object system.Data.DataSet
(New-Object system.Data.odbc.odbcDataAdapter($cmd)).fill($ds) | out-null
$conn.close()
$ds.Tables[0]
}
$result = Get-ODBC-Data
Write-Host "SQL_Output: " $result[0];
The following runs fine on my setup, and shows the correct results:
$var1 = "test22"
function Get-ODBC-Data{
param(
[string]$query=$("
SELECT COUNT(*)
FROM [master].[sys].[table_name]
WHERE col2 = '$($var1)'
;
"),
[string]$username='db_user_name',
[string]$password='db_password'
)
return $query
}
$result = Get-ODBC-Data
Write-Host " ################### Query ######################## "
Write-Host $result
However, you may have a much easier time just passing the entire query into the function as a parameter rather than just one variable part of the query.
Or setting it inside the function and passing $var1 as a mandatory parameter like so:
function Get-ODBC-Data{
param(
[parameter(Mandatory=$true)][string]$var1,
[string]$username='db_user_name',
[string]$password='db_password'
)
$query="
SELECT COUNT(*)
FROM [master].[sys].[table_name]
WHERE col2 = '$($var1)'
;"
return $query
}
$result = Get-ODBC-Data -var1 "working"
Write-Host " ################### Query ######################## "
Write-Host $result
I am attempting to query a CSV file using the Microsoft ACE OLEDB provider. When I add "PrctBusy > 60" to the where clause I receive the Error "Data type mismatch in criteria expression." I have searched StackOverFlow and used google to search for solutions, I see this is not an uncommon issue. From my readings it looks to be datatype issue. The data in the column PrctBusy is all numeric. I think I need to force it to be number but I have not found a solution.
Below is the code I am currently working with:
$ArrayNameUtil = "000198701258"
$CatNameUtil = "FE_DIR"
$sdLocalPath = "D:\Logs\SANData\Perf"
$InputCSV = "VMaxSANReportUtilFile.csv"
$csv = Join-Path $sdLocalPath $InputCSV
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$conn.Open()
#
$sql = "SELECT TimeStamp, count(PrctBusy) AS Above60 FROM [$tablename] WHERE array = '$ArrayNameUtil' and Category like '$CatNameUtil' and PrctBusy > 60 Group by TimeStamp "
$cmd = New-Object System.Data.OleDB.OleDBCommand
$cmd.Connection = $conn
$cmd.CommandText = $sql
$dtp = New-Object System.Data.DataTable
$dtp.Load($cmd.ExecuteReader())
Because of the pointer from TessellatingHeckler to Codeproject and some follow on queries, I was lead to http://aspdotnetcodes.com/Importing_CSV_Database_Schema.ini.aspx. I found that a schema.ini file in the same directory as the CSV file could specify the data type.
The schema.ini file ended up in the following format:
[VMaxSANReportUtilFile.csv]
ColNameHeader=True
Format=CSVDelimited
Col1=Array Text Width 20
Col2=TimeStamp Text Width 20
Col3=Category Text Width 20
Col4=Instance Text Width 20
Col5=PrctBusy Short
Col6=QueUtil Short
I went through several revisions to get the data type correct for an ACE OLE DB provider. If the columns are named the names need to be in the schema.ini file.
I have the following PS function:
function GetBuildData {
[System.Data.SqlClient.SqlConnection] $conn = New-Object System.Data.SqlClient.SqlConnection
[System.Data.SqlClient.SqlCommand] $cmd = New-Object System.Data.SqlClient.SqlCommand
[System.Data.SqlClient.SqlDataAdapter] $adapter = New-Object System.Data.SqlClient.SqlDataAdapter
[System.Data.DataTable] $dt = New-Object System.Data.DataTable
try {
[string] $connStr = "myConnectionString"
$conn.ConnectionString = $connStr
$cmd.Connection = $conn
$cmd.CommandText = "SELECT * FROM TestTable"
$conn.Open
$adapter.SelectCommand = $cmd
$adapter.Fill($dt)
$conn.Close
}
catch [system.exception]
{
Write-Host $_
}
finally {
$adapter.Dispose
$cmd.Dispose
$conn.Dispose
}
return $dt
}
Most of the function has been removed for brevity. The problem I have is when I call the function like so:
[System.Data.DataTable] $buildData = GetBuildData
I get the following error:
Cannot convert the "System.Object[]" value of type "System.Object[]"
to type "System.Data.DataTable".
I've already double-checked, and $dt does contain data. For example, the number of Rows.Count is 1 which is expected. Why does Powershell think that I'm wanting an object[] array returned when it's obvious that the $dt variable is a DataTable?
I don't know about the other issues people are bringing up, but I can think of two possible reasons why you're getting back an Object[].
It might be that you have uncaptured output somewhere else in the function. In your case, Fill returns an int. Try adding this to the end of your function calls:
| Out-Null
E.g.,
$adapter.Fill($dt) | Out-Null
If any statement returns a value and you're not capturing it, the output will be included in your return value, and since you'll have multiple return values at that point, PowerShell will stuff them all into an array.
The other possibility is that PowerShell converts returned collections of all sorts into Object[]s. Use the , operator to return the unmangled value:
return , $dt
The , creates an array containing only the value that follows. As near as I can guess, this causes PowerShell to automatically unroll the outermost array and leave the value alone, since the actual return value is now an array that only contains a single value.
The return is actually optional, since uncaptured values are included in the return value.
Just ran into this one myself yesterday. I can't for the life of me figure out why anyone would think silently converting to Object[] is useful.
You do have a couple of issues in what you posted. Since you explicitly create a DataTable object with New-Object System.Data.DataTable, there is no reason to also coerce it to the DataTable type with [System.Data.DataTable]. The same is true for when you call the function. You are returning a DataTable object, so that is what you will get back. Also if you did want to specify the variable type, there should not be a space between the type identifier and the variable name. Regardless, these issues would not cause the behavior that you are seeing. I ran this code:
function GetBuildData {
$dt = New-Object System.Data.DataTable
$column1 = New-Object system.Data.DataColumn 'Col1'
$column2 = New-Object system.Data.DataColumn 'Col2'
$dt.columns.add($column1)
$dt.columns.add($column2)
$row = $dt.NewRow()
$row.col1 = '1'
$row.col2 = '2'
$dt.rows.add($row)
return $dt
}
$buildData = GetBuildData
$buildData
Col1 Col2
---- ----
1 2
And it worked just fine. I suspect that the bit that is causing your problem is probably in the bit that you excluded for brevity. Shortcuts rarely are and you rarely get accurate answers from partial data.
A contributing factor is that you're not actually calling the open and close methods, but referring to them instead. PowerShell is outputting a reference to these methods as well as the datatable (which I imagine isn't actually populated due to the fact that the connection isn't open).
You need to include the parentheses on the open/close methods like this:
$conn.Open()
...
$conn.Close()
You also need to beware of methods which return values (.Add() methods are notorious for this) that you're not consuming either by assigning to a variable or piping to out-null.
You just have to void out all your dot commands that aren't assigned to variables
function GetBuildData {
[System.Data.SqlClient.SqlConnection] $conn = New-Object System.Data.SqlClient.SqlConnection
[System.Data.SqlClient.SqlCommand] $cmd = New-Object System.Data.SqlClient.SqlCommand
[System.Data.SqlClient.SqlDataAdapter] $adapter = New-Object System.Data.SqlClient.SqlDataAdapter
[System.Data.DataTable] $dt = New-Object System.Data.DataTable
try {
[string] $connStr = "myConnectionString"
$conn.ConnectionString = $connStr
$cmd.Connection = $conn
$cmd.CommandText = "SELECT * FROM TestTable"
[void]$conn.Open
$adapter.SelectCommand = $cmd
$adapter.Fill($dt)
[void]$conn.Close
}
catch [system.exception]
{
Write-Host $_
}
finally {
[void]$adapter.Dispose
[void]$cmd.Dispose
[void]$conn.Dispose
}
$dt
}
As for your question Why? ... Read Keith Hill's Blog: how does return work powershell functions