My desired result from the below PS script is to end up with the result in a CSV or XLS file, which every is the easiest..
I run the below Powershell command, which gives me the desired results - so far so good.
I'm stuck on how to get this information into Excel, so that I can then use this for reporting..
$OracleCommand = "SELECT user_info.directory_auth_id,
user_info.display_name,
chat_account.chat_system_id,
chat_account.login_id,
chat_account.account_id,
chat_account.is_deleted,
chat_system.server_address"
$OracleCommand += " FROM user_info"
$OracleCommand += " INNER JOIN chat_account ON chat_account.user_id = user_info.entity_id"
$OracleCommand += " INNER JOIN chat_system ON chat_system.system_id = chat_account.chat_system_id"
$OracleCommand += " WHERE user_info.directory_auth_id ="
$OracleCommand += "'" + $User + "'"
Invoke-OracleCommand -OracleConnectionString (Get-Content 'ConnectionString.txt') `
-QueryString $OracleCommand |
Do I need to loop through the results and export to csv....
Thanks,
boardman
I'd go ahead and export to CSV as that makes the rest of the process easier. This code will let you import the CSV:
$csv = "C:\myData.csv"
$excel = New-Object -ComObject Excel.Application
$excel.Visible = $Show
$wbObj = $excel.Workbooks.Add()
$dataSheet = $wbObj.Worksheets.Item(1)
$qryConn = ("TEXT;" + $csv)
$qryDest = $dataSheet.Range("A1")
$conn = $dataSheet.QueryTables.Add($qryConn,$qryDest)
$dataSheet.QueryTables.item($conn.name).TextFileCommaDelimiter = $true
$dataSheet.QueryTables.item($conn.name).TextFileParseType = 1
[void]$dataSheet.QueryTables.item($conn.name).Refresh()
Related
My output, will always contain the information from the second item (database), it seems that it is overwriting any values returned for the initial items? I can change the order of the items in to prove this. Please help...
$databaselist = Get-Content D:\AdvancedDB\Server2.txt
$servername = get-content D:\AdvancedDB\Server.txt
$dataSource = $servername
$myuserID = 'userid'
$mypassword = 'password'
$DatabaseIndexInfo =
"SELECT dbschemas.[name] as 'Schema',
dbtables.[name] as 'Table',
dbindexes.[name] as 'Index',
indexstats.avg_fragmentation_in_percent,
indexstats.page_count
FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS indexstats
INNER JOIN sys.tables dbtables on dbtables.[object_id] = indexstats.[object_id]
INNER JOIN sys.schemas dbschemas on dbtables.[schema_id] = dbschemas.[schema_id]
INNER JOIN sys.indexes AS dbindexes ON dbindexes.[object_id] = indexstats.[object_id]
AND indexstats.index_id = dbindexes.index_id
WHERE indexstats.database_id = DB_ID()
ORDER BY indexstats.avg_fragmentation_in_percent desc"
$connectionDetails = "Provider=sqloledb; " + "Server=$dataSource; " + "Database=$database; " +
"User
ID=$myuserID; " + " Password=$mypassword; "
$frag16 = #()
foreach ($database in $databaselist) {
##Connect to the data source using the connection details and T-SQL command we provided above, and
open the connection
$connection = New-Object System.Data.OleDb.OleDbConnection $connectionDetails
$command16 = New-Object System.Data.OleDb.OleDbCommand $DatabaseIndexInfo,$connection
$connection.Open()
##Get the results of our command into a DataSet object, and close the connection
$dataAdapter = New-Object System.Data.OleDb.OleDbDataAdapter $command16
$dataSet16 = New-Object System.Data.DataSet
$dataAdapter.Fill($dataSet16)
$connection.Close()
}
$frag16 += $dataset16.Tables | Out-File 'd:\advanceddb\test5.txt'
You will need to move your connection details within the foreach loop. If you need to work with different databases, then you need to update your connection string as well.
foreach ($database in $databaselist) {
$connectionDetails = "Provider=sqloledb; " + "Server=$dataSource; " + "Database=$database; " +
"User ID=$myuserID; " + " Password=$mypassword; "
...
Since connectionDetails is not updated within your loop, you keep seeing the same data.
I am using a combination of powershell scripts I've found online (shown below) to create an Excel document with pivot tables. But, I've only found examples where the datasource for the pivot tables was data already-imported into the excel document via CSV, or other similar methods. I can get those to work. But, my problem is that I would like to bypass the step of loading .csv data into excel and then creating the pivot tables...my whole issue is my dataset is way too large for excel...so I want to create the pivot table directly from a SQL connection. I cannot figure out the right syntax to to set the datasource. Below is my closest attempt...should use "xlExternal" for the datasource? But how do I set the datasource then? Micorsoft's documentation/APIs are somwhat helpful, but their examples are all written in VB and they also use other methods I'm not even familiar with in VB, so I'm having a hard time translating the QueryTables.Add method and some others.
I believe the only 1 or 2 lines that I'm stuck on are this one:
$qt = $ws.QueryTables.Add("ODBC;DSN=$connectionString", $ws.Range("A1"),
$SQL)
And/or this one:
$PivotTable =
$wb.PivotCaches().Create($xlExternal,$selection,$xlPivotTableVersion10)
But my code gives the exception on this line:
$PivotTable.CreatePivotTable("R1C1","Tables1") | Out-Null
and the error is saying:
Exception from HRESULT: 0x800A03EC
At C:\Users\me\Desktop\NEWpsCode.ps1:107 char:1
+ $PivotTable.CreatePivotTable("R1C1","Tables1") | Out-Null
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException
So, here is my code:
$DirectoryToSaveTo='C:\Users\me\Desktop\'
$Filename='myfile'
$ServerName = "BLAH1"
$DatabaseName = "BLAH2"
$userName = "BLAH3"
$password = "BLAH4"
$connectionString = "Server = $ServerName; Database = $DatabaseName; User ID
= $userName; Password = $password;"
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection
$connectionString
$sqlConnection.Open()
$xlCenter=-4108
$xlTop=-4160
$xlOpenXMLWorkbook=[int]51
$SQL=#"
SELECT Account1,Account2,Account3 from myTable
"#
$excel = New-Object -Com Excel.Application #open a new instance of Excel
$excel.Visible = $True
$wb = $Excel.Workbooks.Add()
$currentWorksheet=1
if ($currentWorksheet-lt 4)
{
$ws = $wb.Worksheets.Item($currentWorksheet)
}
else
{
$ws = $wb.Worksheets.Add()
}
$currentWorksheet += 1
$qt = $ws.QueryTables.Add("ODBC;DSN=$connectionString", $ws.Range("A1"),
$SQL)
$xlPivotTableVersion12 = 3
$xlPivotTableVersion10 = 1
$xlCount = -4112
$xlDescending = 2
$xlDatabase = 1
$xlExternal = 2
$xlHidden = 0
$xlRowField = 1
$xlColumnField = 2
$xlPageField = 3
$xlDataField = 4
$xlDirection = [Microsoft.Office.Interop.Excel.XLDirection]
$range1=$ws.range("A1")
$range1=$ws.range($range1,$range1.End($xlDirection::xlDown))
$range2=$ws.range("B1")
$range2=$ws.range($range2,$range2.End($xlDirection::xlDown))
$selection = $ws.range($range1, $range2)
$PivotTable =
$wb.PivotCaches().Create($xlExternal,$selection,$xlPivotTableVersion10)
$PivotTable.CreatePivotTable("R1C1","Tables1") | Out-Null
$filename = "$DirectoryToSaveTo$filename.xlsx"
if (test-path $filename ) { rm $filename }
$wb.SaveAs($filename, $xlOpenXMLWorkbook)
$wb.Saved = $True
$wb.Close()
$Excel.Quit()
$wb = $Null
$ws = $Null
$Excel=$Null
So, if anyone can please direct me to setting up a pivot table using SQL connection directly, that would be extremely helpful! Even a very simple example will help immensely.
I am attempting to query a CSV file using the Microsoft ACE OLEDB provider. When I add "PrctBusy > 60" to the where clause I receive the Error "Data type mismatch in criteria expression." I have searched StackOverFlow and used google to search for solutions, I see this is not an uncommon issue. From my readings it looks to be datatype issue. The data in the column PrctBusy is all numeric. I think I need to force it to be number but I have not found a solution.
Below is the code I am currently working with:
$ArrayNameUtil = "000198701258"
$CatNameUtil = "FE_DIR"
$sdLocalPath = "D:\Logs\SANData\Perf"
$InputCSV = "VMaxSANReportUtilFile.csv"
$csv = Join-Path $sdLocalPath $InputCSV
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$provider = (New-Object System.Data.OleDb.OleDbEnumerator).GetElements() | Where-Object { $_.SOURCES_NAME -like "Microsoft.ACE.OLEDB.*" }
if ($provider -is [system.array]) { $provider = $provider[0].SOURCES_NAME } else { $provider = $provider.SOURCES_NAME }
$connstring = "Provider=$provider;Data Source=$(Split-Path $csv);Extended Properties='text;HDR=$firstRowColumnNames;';"
$firstRowColumnNames = "Yes"
$delimiter = ","
$tablename = (Split-Path $csv -leaf).Replace(".","#")
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$conn.Open()
#
$sql = "SELECT TimeStamp, count(PrctBusy) AS Above60 FROM [$tablename] WHERE array = '$ArrayNameUtil' and Category like '$CatNameUtil' and PrctBusy > 60 Group by TimeStamp "
$cmd = New-Object System.Data.OleDB.OleDBCommand
$cmd.Connection = $conn
$cmd.CommandText = $sql
$dtp = New-Object System.Data.DataTable
$dtp.Load($cmd.ExecuteReader())
Because of the pointer from TessellatingHeckler to Codeproject and some follow on queries, I was lead to http://aspdotnetcodes.com/Importing_CSV_Database_Schema.ini.aspx. I found that a schema.ini file in the same directory as the CSV file could specify the data type.
The schema.ini file ended up in the following format:
[VMaxSANReportUtilFile.csv]
ColNameHeader=True
Format=CSVDelimited
Col1=Array Text Width 20
Col2=TimeStamp Text Width 20
Col3=Category Text Width 20
Col4=Instance Text Width 20
Col5=PrctBusy Short
Col6=QueUtil Short
I went through several revisions to get the data type correct for an ACE OLE DB provider. If the columns are named the names need to be in the schema.ini file.
I have come across a very strange situation and I have not a clue as to why this bug is occuring.
In short, I have a powershell script that if I run manually works 100%
I have set this as a scheduled task in Windows Task Manager and the event launches, runs the SQL element of the powershell script so I know that the issue doesn't exist there. However, within the powershell code I have it open a specific MS-Access database and this is not working.
shorter version, - Works when not automated, automation works but not MS-Access
I am confused, I have included the code can anyone see anything untoward?
$NewFile = '\\Operations Database\Compliance\CancelledLimitOrders\LIOAnalysis.csv'
$ComplianceSQL = #"
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
DECLARE #startdate CHAR(8)
DECLARE #finaldate CHAR(8)
SELECT #startdate=CONVERT(CHAR(8),GETDATE(),112), -- get date is editable in the negative figure to provide date range for SQL
#finaldate=CONVERT(CHAR(8),GETDATE()+7,112)
select LIOORDERREF,
lioclientcode,
liodatetimeinput,
liosecurityid,
CAST(LIOPRICE1 AS MONEY) AS PRICE1,
CAST(LIOPRICE2 AS MONEY) AS PRICE2,
CAST(lioexpiryperiod AS FLOAT) AS EXPIRYDAYS,
lioexpirydatetime,
liolastupdated,
liodatetimeoflaststatus,
liofailurecount,
liotransactiontype,
liobidprice1,
liobidprice2,
liobidprice3,
lioaskprice1,
lioaskprice2,
lioaskprice3,
CAST(lioordercashamount AS FLOAT) AS LIOCASHAMOUNT,
LIOSTOCKSYMBOL,
CASE WHEN liosecurityexchange = 'XLON' THEN LTRIM(RTRIM(liostocksymbol)) + '.L'
ELSE liostocksymbol
END AS STOCKAPI,
liocurrency,
lioquoteprovider,
liosecurityexchange,
CASE WHEN lioparenttype = 'NULL' THEN 'UNKNOWN' + '-' + LIOPARENTTYPE
WHEN lioparenttype = '' THEN 'UNKNOWN' + '-' + LIOPARENTTYPE
WHEN lioparenttype = 'BO' THEN 'BULK ORDER'
WHEN lioparenttype = 'DB' THEN 'BUY DEAL AT BEST'
WHEN lioparenttype = 'DS' THEN 'SELL DEAL AT BEST'
WHEN lioparenttype = 'LB' THEN 'LIMIT BUY'
WHEN lioparenttype = 'LS' THEN 'LIMIT SELL'
WHEN lioparenttype = 'PL' THEN 'UNKNOWN' + '-' + LIOPARENTTYPE
WHEN lioparenttype = 'RB' THEN 'RISING BUY'
WHEN lioparenttype = 'RT' THEN 'UNKNOWN' + '-' + LIOPARENTTYPE
WHEN lioparenttype = 'SL' THEN 'STOP LOSS'
WHEN LIOPARENTTYPE = 'SO' THEN 'SELL ORDER'
WHEN lioparenttype = 'TS' THEN 'UNKNOWN' + '-' + LIOPARENTTYPE
ELSE 'UNKNOWN' + '-' + LIOPARENTTYPE
END AS PARENTTYPE
from xtlimitorder
where left(liolastupdated,8) = #StartDate
and left(lioorderref,2) not like 'PP'
and liostatus = 'c'--
"#
##Development Notes##
## PS Script to run once every 5 minutes
## PS will download CSV of newly cancelled trades
## On download MS-Access Utilised to 'append data' to daily trades table
#####################
## - CREDENTIALS - ##
#####################
$MISA = 'xxxxxxxx'
$MISB = 'xxxxxxxx'
$userName = 'xxxxxxxxx'
$PassWord='xxxxxxxxxxx'
$DB = 'xxxxxxxxx'
$timeout=0
###### - StopWatch - ######
$timeout2 = new-timespan -Minutes 5
$sw = [diagnostics.stopwatch]::StartNew()
## CREATE MIS CREDENTIALS ##
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection("Connection Timeout=0")
$SqlConnection.ConnectionString = "Data Source=$MISA;Initial Catalog=$DB;
Initial Catalog=$DB;User ID=$userName;Password=$PassWord;"
## - Runs Script from Set Location
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand;
$SqlCmd.CommandTimeout=$timeout;
$SqlCMD.CommandText = $ComplianceSQL;
$SqlCmd.Connection = $SqlConnection;
## - Extract Data and build sql data object
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter;
$SqlAdapter.SelectCommand = $SqlCmd;
$DataSet = New-Object System.Data.DataSet;
$SqlAdapter.Fill($DataSet);
$DataSetTable = $DataSet.Tables["Table"];
$DSETCOUNT = $DataSetTable.Rows.Count
IF($DSETCOUNT -GT 0){
ECHO "AA"
}
else {
echo "NULL NULL NULL"
$SqlConnection.Close()
## CREATE MIS CREDENTIALS ##
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection("Connection Timeout=0")
$SqlConnection.ConnectionString = "Data Source=$MISb;Initial Catalog=$DB;
Initial Catalog=$DB;User ID=$userName;Password=$PassWord;"
## - Runs Script from Set Location
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand;
$SqlCmd.CommandTimeout=$timeout;
$SqlCMD.CommandText = $SECTABLEQUERY;
$SqlCmd.Connection = $SqlConnection;
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter;
$SqlAdapter.SelectCommand = $SqlCmd;
$DataSet = New-Object System.Data.DataSet;
$SqlAdapter.Fill($DataSet);
$DataSetTable = $DataSet.Tables["Table"];
$DSETCOUNT = $DataSetTable.Rows.Count
IF($DSETCOUNT -GT 0){
echo "A"
}
else {
echo "NULL NULL NULL"
$SqlConnection.Close()
}
$SqlConnection.Close()
}
#Finally Write the Data to the Table
$DataSet.Tables["Table"] | Export-Csv $NewFile -NoTypeINformation
#################################################
#### Data Download Concluded #############
################################################
### CALL ACCESS FOR FIVE MINUTE UPDATE
$CommandFinal = 'FiveMinuteDataUpdate'
$COMPDB = '\\operations database\compliance\cancelledlimitorders\DBCanxLIO.accdb'
$MSACCESS = New-Object -ComObject Access.Application
### this is not being done in automation, why?
$MSACCESS.OpenCurrentDatabase($COMPDB)
#$MSACCESS.AutomationSecurity = $False
$MSACCESS.Visible = $true
$MSACCESS.DoCmd.OpenForm($CommandFinal)
#$MSACCESS.RunCommand($CommandFinal)
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($MSACCESS)
#### Convert xlsx to csv file ####
$PricesToCSv = '\\Operations Database\Compliance\CancelledLimitOrders\GetCanxPriceTable.xlsx'
$PriceswbCSv = '\\Operations Database\Compliance\CancelledLimitOrders\GetCanxPriceTable.csv'
$Excel = New-Object -ComObject Excel.Application
$Excel.Visible = $true
$Excel.DisplayAlerts = $false
$PricesToCSVWB = $Excel.Workbooks.Open($PricesToCSv)
$xlcsv = 6
$WorkIt = $PricesToCSVWB.ActiveSheet
$WorkIt.SaveAs($priceswbcsv,$xlcsv)
$Excel.Quit()
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($WorkIt)
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($xlCsv)
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($PricesToCSVWB)
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($Excel)
######
### CSV Converted
##############################################
Remove-Item '\\operations database\compliance\cancelledlimitorders\array.csv'
#####
### Iterate through each line of the converted csv and obtain the quote from yahoo finance writing all output to the array.csv file
$GetCanxTradeImport = '\\operations database\compliance\cancelledlimitorders\GetCanxPriceTable.csv'
$GCTI = Import-Csv $GetCanxTradeImport -Header ('LIOORDERREF', 'lioclientcode', 'liodatetimeinput', 'liosecurityid', 'PRICE1', 'PRICE2', 'EXPIRYDAYS', 'lioexpirydatetime', 'liolastupdated', 'liodatetimeoflaststatus', 'liofailurecount', 'liotransactiontype', 'liobidprice1', 'liobidprice2', 'liobidprice3', 'lioaskprice1', 'lioaskprice2', 'lioaskprice3', 'LIOCASHAMOUNT', 'LIOSTOCKSYMBOL', 'STOCKAPI', 'liocurrency', 'lioquoteprovider', 'liosecurityexchange', 'PARENTTYPE') | SELECT -Skip 1
$IE = New-Object -ComObject internetexplorer.application
ForEach ($StockApi in $GCTI.STOCKAPI) {
$URLA = "http://download.finance.yahoo.com/d/quotes.csv?s="
$URLB = "&f=sc1ll1b2&e=.csv"
#echo $URLA, $StockApi, $URLB
$FUllURL = ($URLA+ $StockApi+ $URLB)
#$IE.navigate2($FUllURL)
#$IE.visible=$true
$OP = '\\operations database\compliance\cancelledlimitorders\array.csv'
$String = Invoke-RestMethod $FullUrl
$Data = #()
$Row = New-Object PSObject
$row | Add-Member -MemberType NoteProperty -Name "STRING" -Value $String
$Data += $Row
$Data | Export-CSv $OP -NoTypeInformation -Append
###############################################################
### Has now been written out to array.csv
###############################################################
### Array file requires importing into access to allow updating of files and percentages
}
#### this is not working when automated why??????????????
$Command3 = 'LogicUpdate'
$COMPDB = '\\operations database\compliance\cancelledlimitorders\DBCanxLIO.accdb'
$MSACCESS = New-Object -ComObject Access.Application
$MSACCESS.OpenCurrentDatabase($COMPDB)
#$MSACCESS.AutomationSecurity = $False
$MSACCESS.Visible = $true
$MSACCESS.DoCmd.OpenForm($Command3)
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($MSACCESS)
## Kill resident processes
Stop-Process -Name MSACCESS
Stop-Process -Name iexplore
The automated task is simply written as
Action = Start a program
Program/Script: = C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe
Add arguments (optional) = C:\scripts\ComplianceLimitOrderTestRunTime.ps1
I had wondered if this was an execution policy issue but it is set as unrestricted.
I cannot understand why this would work as a manual execution but not as an automated execution, any one shed any light on this?
I would like to be able to right click on a file(s) and "send-to" a local MSSQL database. The details are that I would like to store the file contents in "contents" column and the file name in the "filename" column ... how novel :)
*In most cases the file contents is HTML.
It seems like it should be possible through windows shell/SQL Shell using a shortcut to a command in the "shell:sendto" folder.
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | Out-Null
$Server1 = New-Object ("Microsoft.SqlServer.Management.Smo.Server") 'SQLSERVER'
$Server1.databases["DB"].tables["Table"].rowcount
$RowCount = $server1.databases["DB"].tables["Table"].rowcount.ToString()
$TotalRecords = [int]$RowCount
$wc = New-Object system.net.WebClient
$url = ""
$files = #(Get-ChildItem c:\test\*.*)
"Number of files $($files.length)"
# Errors out when no files are found
if($files.length -lt 1) { return }
foreach($file1 in $files) {
# $txt = Get-Content($file1)
# $txt = $txt.Replace("'", "''")
# Write-Host $file1.name + " - - " + $Txt
$url1 = $url + $file1
Write-Host("URL is " + $url1)
$webpage = $wc.DownloadData($url1)
$string = [System.Text.Encoding]::ASCII.GetString($webpage)
$string = $string.Replace("'", "''")
Invoke-SqlCmd -ServerInstance SERVER -Query "Insert into DATABASE.dbo.Table(text,filename) Values ('$string','$file1')"}