I'm getting some data from a SQL table, which I then store in a System.Data.DataSet object. I want to pass this data in this DataSet, as an Argument/Parameter, to a workflow, such that I can display all the data in this DataSet in a foreach -parallel style. But I'm at a loss for the correct syntax of passing data from a System.Data.DataSet object to a workflow. Currently I get an Error near the line "param([System.Data.DataSet]$pServiceDataSet)" as shown below.
Function GetSQLData
{
param ($TargetDBServer, $TargetDB, $SQLQuery)
# SQL Connection Object
$sqlConn = New-Object System.Data.SqlClient.SqlConnection
$sqlConn.ConnectionString = "Server=$TargetDBServer;Database=$TargetDB;User Id=SomeUser;Password=SomePassword;"
$sqlConn.Open()
# SQL Command
$sqlcmd = New-Object System.Data.SqlClient.SqlCommand
$sqlcmd.Connection = $sqlConn
$sqlcmd.CommandText = $SQLQuery
# SQL Adapter
$sqlAdp = New-Object System.Data.SqlClient.SqlDataAdapter ($sqlcmd)
# SQL DataSet
$ResultDataSet = New-Object System.Data.DataSet
$sqlAdp.Fill($ResultDataSet) | Out-Null
$sqlConn.Close()
return $ResultDataSet.Tables[0]
}
$CurrentComputerName = $env:COMPUTERNAME
# Export the Windows Services & it's config parameters from the "DatabaseABC..WindowsServicesConfig" Table.
$SQLQueryForService = "
SELECT [ServiceName], [StartUpParameter], [DBServerName], [DBName]
FROM [dbo].[WindowsServicesConfig] WITH (NOLOCK)
WHERE [HostServerName] = '$CurrentComputerName'
AND [ServiceName] LIKE '%MyService%' "
$ServicesDataSet = GetSQLData -TargetDBServer "ServerABC" -TargetDB "DatabaseABC" -SQLQuery $SQLQueryForService
$ServicesDataSet.GetType()
$ServicesDataSet | Format-Table
workflow DisplayAllServices
{
param([System.Data.DataSet]$pServiceDataSet) # <- I get an Error here
foreach -parallel ($Service in $pServiceDataSet)
{
$Service.ServiceName
$Service.StartUpParameter
$Service.DBServerName
$Service.DBName
}
}
DisplayAllServices -pServiceDataSet $ServicesDataSet
My final objective is to use the data in this DataSet to create Windows Services. But this is my most frustrating hurdle. I cannot get past the Error.
Figured out the Solution to my Problem. Replaced the "param([System.Data.DataSet]$pServiceDataSet)" with "param([PSObject]$pServiceDataSet)".
$sf = "\\\\domain\\dept\\dcgsi\\Extracts\\Tableau_Unlicensed_Users.csv"
if (Test-Path $sf){
Remove-Item $sf
}
$query = #"
\\copy (SELECT Name
FROM _users
WHERE licensing_role_name = 'Unlicensed')
TO $sf
WITH CSV DELIMITER ','
"#
$conn = New-Object -ComObject ADODB.Connection
# use existing 64 bit ODBC System DSN that we set up manually
$conn.Open('PostgreSQL30')
$conn.Execute($query)
$conn.Close()
I keep getting an error about "\" on the line with the $conn.Execute() when I try and do this. I assume it has to do with character escaping and maybe I am doing it wrong.
Is there a better way to do this with PowerShell if I just need to get the name field of any record from _users and output it to CSV?
Eventually I will be adding more to this to loop through each record in the CSV and execute a tabcmd to remove all the users that are unlicensed.
$sf = "\\domain\dept\dcgsi\Extracts\Tableau_Unlicensed_Users.csv"
if (Test-Path $sf){
Remove-Item $sf
}
$query = #"
SELECT Name
FROM _users
WHERE licensing_role_name = 'Unlicensed'
"#
function Get-ODBC-Data{
$conn = New-Object System.Data.Odbc.OdbcConnection
$conn.ConnectionString = "DSN=PostgreSQL30;"
$conn.Open()
$cmd = New-object System.Data.Odbc.OdbcCommand($query, $conn)
$ds = New-Object system.Data.DataSet
(New-Object System.Data.Odbc.OdbcDataAdapter($cmd)).Fill($ds) | Out-Null
$conn.Close()
$ds.Tables[0] | Export-Csv $sf
}
Get-ODBC-Data
This did like 99% of what I need; I just have to process the csv now and drop the first two lines. The first line is a type info message and the second is the column header.
I'm trying to create a script for querying a database, extract data and put it to an .csv file. This is the part of interest of the code:
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = $connection.CreateCommand()
$command.CommandText = $query
$result = $command.ExecuteReader()
###########################
##PRSENTAZIONE DEI RISULTATI##
$table = New-Object System.Data.DataSet
$table.Load($result)
$table | Export-Csv -Path $fileName -Delimiter "|" -NoTypeInformation
I have 2 problems:
On Windows 2003 PowerShell -Delimiter parameter is not present
If I delete that parameter the ouput file presents a wrong format date.
username, ipaddress, yyyy-mm-dd hh:mm:ss (running the same query in Management Studio), username, ipaddress, "dd/mm/yyyy hh.mm.ss" (apexes included ; printing the $table).
How can I solve this?
I am attempting to query a CSV file using the Microsoft ACE OLEDB provider. When I add "PrctBusy > 60" to the where clause I receive the Error "Data type mismatch in criteria expression." I have searched StackOverFlow and used google to search for solutions, I see this is not an uncommon issue. From my readings it looks to be datatype issue. The data in the column PrctBusy is all numeric. I think I need to force it to be number but I have not found a solution.
Below is the code I am currently working with:
$ArrayNameUtil = "000198701258"
$CatNameUtil = "FE_DIR"
$sdLocalPath = "D:\Logs\SANData\Perf"
$InputCSV = "VMaxSANReportUtilFile.csv"
$csv = Join-Path $sdLocalPath $InputCSV
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$conn.Open()
#
$sql = "SELECT TimeStamp, count(PrctBusy) AS Above60 FROM [$tablename] WHERE array = '$ArrayNameUtil' and Category like '$CatNameUtil' and PrctBusy > 60 Group by TimeStamp "
$cmd = New-Object System.Data.OleDB.OleDBCommand
$cmd.Connection = $conn
$cmd.CommandText = $sql
$dtp = New-Object System.Data.DataTable
$dtp.Load($cmd.ExecuteReader())
Because of the pointer from TessellatingHeckler to Codeproject and some follow on queries, I was lead to http://aspdotnetcodes.com/Importing_CSV_Database_Schema.ini.aspx. I found that a schema.ini file in the same directory as the CSV file could specify the data type.
The schema.ini file ended up in the following format:
[VMaxSANReportUtilFile.csv]
ColNameHeader=True
Format=CSVDelimited
Col1=Array Text Width 20
Col2=TimeStamp Text Width 20
Col3=Category Text Width 20
Col4=Instance Text Width 20
Col5=PrctBusy Short
Col6=QueUtil Short
I went through several revisions to get the data type correct for an ACE OLE DB provider. If the columns are named the names need to be in the schema.ini file.
Is there a way to execute an arbitrary query on a SQL Server using Powershell on my local machine?
For others who need to do this with just stock .NET and PowerShell (no additional SQL tools installed) here is the function that I use:
function Invoke-SQL {
param(
[string] $dataSource = ".\SQLEXPRESS",
[string] $database = "MasterData",
[string] $sqlCommand = $(throw "Please specify a query.")
)
$connectionString = "Data Source=$dataSource; " +
"Integrated Security=SSPI; " +
"Initial Catalog=$database"
$connection = new-object system.data.SqlClient.SQLConnection($connectionString)
$command = new-object system.data.sqlclient.sqlcommand($sqlCommand,$connection)
$connection.Open()
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataSet) | Out-Null
$connection.Close()
$dataSet.Tables
}
I have been using this so long I don't know who wrote which parts. This was distilled from others' examples, but simplified to be clear and just what is needed without extra dependencies or features.
I use and share this often enough that I have turned this into a script module on GitHub so that you can now go to your modules directory and execute git clone https://github.com/ChrisMagnuson/InvokeSQL and from that point forward invoke-sql will automatically be loaded when you go to use it (assuming your using PowerShell v3 or later).
You can use the Invoke-Sqlcmd cmdlet
Invoke-Sqlcmd -Query "SELECT GETDATE() AS TimeOfQuery;" -ServerInstance "MyComputer\MyInstance"
http://technet.microsoft.com/en-us/library/cc281720.aspx
This function will return the results of a query as an array of powershell objects so you can use them in filters and access columns easily:
function sql($sqlText, $database = "master", $server = ".")
{
$connection = new-object System.Data.SqlClient.SQLConnection("Data Source=$server;Integrated Security=SSPI;Initial Catalog=$database");
$cmd = new-object System.Data.SqlClient.SqlCommand($sqlText, $connection);
$connection.Open();
$reader = $cmd.ExecuteReader()
$results = #()
while ($reader.Read())
{
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
$results += new-object psobject -property $row
}
$connection.Close();
$results
}
Here's an example I found on this blog.
$cn2 = new-object system.data.SqlClient.SQLConnection("Data Source=machine1;Integrated Security=SSPI;Initial Catalog=master");
$cmd = new-object system.data.sqlclient.sqlcommand("dbcc freeproccache", $cn2);
$cn2.Open();
if ($cmd.ExecuteNonQuery() -ne -1)
{
echo "Failed";
}
$cn2.Close();
Presumably you could substitute a different TSQL statement where it says dbcc freeproccache.
If you want to do it on your local machine instead of in the context of SQL server then I would use the following. It is what we use at my company.
$ServerName = "_ServerName_"
$DatabaseName = "_DatabaseName_"
$Query = "SELECT * FROM Table WHERE Column = ''"
#Timeout parameters
$QueryTimeout = 120
$ConnectionTimeout = 30
#Action of connecting to the Database and executing the query and returning results if there were any.
$conn=New-Object System.Data.SqlClient.SQLConnection
$ConnectionString = "Server={0};Database={1};Integrated Security=True;Connect Timeout={2}" -f $ServerName,$DatabaseName,$ConnectionTimeout
$conn.ConnectionString=$ConnectionString
$conn.Open()
$cmd=New-Object system.Data.SqlClient.SqlCommand($Query,$conn)
$cmd.CommandTimeout=$QueryTimeout
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
[void]$da.fill($ds)
$conn.Close()
$ds.Tables
Just fill in the $ServerName, $DatabaseName and the $Query variables and you should be good to go.
I am not sure how we originally found this out, but there is something very similar here.
There isn't a built-in "PowerShell" way of running a SQL query. If you have the SQL Server tools installed, you'll get an Invoke-SqlCmd cmdlet.
Because PowerShell is built on .NET, you can use the ADO.NET API to run your queries.
Invoke-Sqlcmd -Query "sp_who" -ServerInstance . -QueryTimeout 3
To avoid SQL Injection with varchar parameters you could use
function sqlExecuteRead($connectionString, $sqlCommand, $pars) {
$connection = new-object system.data.SqlClient.SQLConnection($connectionString)
$connection.Open()
$command = new-object system.data.sqlclient.sqlcommand($sqlCommand, $connection)
if ($pars -and $pars.Keys) {
foreach($key in $pars.keys) {
# avoid injection in varchar parameters
$par = $command.Parameters.Add("#$key", [system.data.SqlDbType]::VarChar, 512);
$par.Value = $pars[$key];
}
}
$adapter = New-Object System.Data.sqlclient.sqlDataAdapter $command
$dataset = New-Object System.Data.DataSet
$adapter.Fill($dataset) | Out-Null
$connection.Close()
return $dataset.tables[0].rows
}
$connectionString = "connectionstringHere"
$sql = "select top 10 Message, TimeStamp, Level from dbo.log " +
"where Message = #MSG and Level like #LEVEL"
$pars = #{
MSG = 'this is a test from powershell'
LEVEL = 'aaa%'
};
sqlExecuteRead $connectionString $sql $pars
You can even format string and pass parameters as you want.
case "ADDSQLSERVERUSER":
//0 = coprorateName;
//1 = user password
//2 = servername
command = #"$sqlQuery = Use JazzUWS_'{0}'
Create login UWSUser_'{0}' with password='{1}';
Create user UWSUser_'{0}' for login UWSUser_'{0}';
Grant Execute to UWSUser_'{0}';
Use ReportSvrUWS_'{0}'
Create user UWSUser_'{0}' for login UWSUser_'{0}';
Grant Execute to UWSUser_'{0}';
Invoke-Sqlcmd -Query $sqlQuery -ServerInstance '{2}'";
break;
C# Code for remote execution(you can organize your way)
string script = PowershellDictionary.GetPowershellCommand("ADDSQLSERVERUSER");
script = String.Format(script, this.CorporateName, password, this.SQLServerName)
PowerShellExecution.RunScriptRemote(_credentials.Server, _credentials.Username, _credentials.Password, new List<string> { script });
You could use the best SQL Server module around: DBATOOLS. You would also benefit from running a query to multiple sql instances.
Install-Module dbatools -Scope CurrentUser
$sql = 'SQL1','SQL1\INSTANCE1','SQL2'
$query = "SELECT 'This query would run on all SQL instances'"
Invoke-DbaQuery -SqlInstance $sqlinstances -Query $query -AppendServerInstance