I have a PowerShell script that connects to a DB and loops over some data.
After the script finishes or throws an error, I need to append whatever the text displayed in the console to a log file.
I couldn't achieve that using Write-Output because I don't want to save specific values, I just need the whole console text to be appended to a file.
Thank you.
EDIT :
In fact, the final result that I'm looking for, is a log file with timestamps, here is my code :
$server = "USER\SQLEXPRESS"
$database = "database_test"
$tablequery = "SELECT name from sys.tables"
#Delcare Connection Variables
$connectionTemplate = "Data Source={0};Integrated Security=SSPI;Initial Catalog={1};"
$connectionString = [string]::Format($connectionTemplate, $server, $database)
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$command = New-Object System.Data.SqlClient.SqlCommand
$command.CommandText = $tablequery
$command.Connection = $connection
#Load up the Tables in a dataset
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$DriveName = (get-location).Drive.Name
$extractDir = md -Force "$($DriveName):\csv\files"
# Loop through all tables and export a CSV of the Table Data
foreach ($Row in $DataSet.Tables[0].Rows)
{
$connection.open();
#Specify the output location of your dump file
$command.CommandText = "SELECT * FROM [$($Row[0])]"
$command.Connection = $connection
(Get-Culture).NumberFormat.NumberDecimalSeparator = '.'
(Get-Culture).DateTimeFormat.ShortDatePattern = 'yyyy-MM-dd'
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $command
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$connection.Close()
$extractFile = "$($extractDir)\$($Row[0]).csv"
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation -Encoding UTF8
}
I need to print each filename exported to csv ($extractFile) with a timestamp in a log file, and then if an error occurs, I need to print that too with a timestamp, and so on till the script finishes.
You can do this with Start-Transcript, try/catch/finally, or writing your own code (to store the console output to a variable, and append a text file with the contents when required). Note the -Append parameter with Start-Transcript.
Without code, it's difficult to know which of these to recommend.
Expanded
Now that you've added some code, see some additional info on each method. I'm not familiar with SQL via PowerShell so not sure what kind of output/errors you will be getting (regarding errors, specifically if there are terminating or non-terminating)
Transcript
Start-Transcript should go at the beginning, and Stop-Transcript at the end. This will log whatever is normally displayed on the console. Running Start-Transcript while a transcript is already being recorded will lead to a nasty error.
Start-Transcript -Path "c\temp\mylogfile.txt"
$server = "USER\SQLEXPRESS"
$database = "database_test"
$tablequery = "SELECT name from sys.tables"
...
...
$extractFile = "$($extractDir)\$($Row[0]).csv"
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation -Encoding UTF8
}
Stop-Transcript
Terminating Errors
Add try/catch/finally as appropriate. You can be lazy and add this over the whole code, or do it properly and wrap the parts that could lead to terminating errors.
...
foreach ($Row in $DataSet.Tables[0].Rows)
{
try{
$connection.open();
#Specify the output location of your dump file
...
...
...
$extractFile = "$($extractDir)\$($Row[0]).csv"
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation -Encoding UTF8
}catch{
# what to do if there is a terminating error
}finally{
# what to do whether there is an error or not
if(Test-Path "$($extractDir)\$($Row[0]).csv"){
# simple check: if a file was created, no error... right?
"$(Get-Date -f 'yyyy-MM-dd hh:mm:ss') $($Error[0])" | Out-File "c:\temp\mylogfile.txt" -Append
}else{
"$(Get-Date -f 'yyyy-MM-dd hh:mm:ss') $extractFile" | Out-File "c:\temp\mylogfile.txt" -Append
}
}
}
...
No Terminating Errors
Just add a line to export errors. Ensure you clear the automatic variable $Error each loop
...
foreach ($Row in $DataSet.Tables[0].Rows)
{
$Error.Clear()
$connection.open();
#Specify the output location of your dump file
...
...
...
$extractFile = "$($extractDir)\$($Row[0]).csv"
$DataSet.Tables[0] | Export-Csv $extractFile -NoTypeInformation -Encoding UTF8
# if there are no errors, write filename. Otherwise write errors
if([string]::IsNullOrEmpty($Error){
"$(Get-Date -f 'yyyy-MM-dd hh:mm:ss') $extractFile" | Out-File "c:\temp\mylogfile.txt" -Append
}else{
"$(Get-Date -f 'yyyy-MM-dd hh:mm:ss') $Error" | Out-File "c:\temp\mylogfile.txt" -Append
}
}
...
You could use Start-Transcript for debugging purposes:
Start-Transcript -path "C:\temp\myTranscript.txt"
Add at the start of your script and get all the console output written into C:\temp\myTranscript.txt
Related
Currently I have a powershell that was written by someone else. It Connects to SQL and based on a date it will export the results against a Stored procedure...This Works fine Right now.
$FromDate = Read-Host "Enter Date of to Start (YYYY-MM-DD)"
$QueryText = "exec UDO_VW_ExportView_HISTORY #FromDate"
$SqlConnection = new-object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = $connString
$SqlCommand = $SqlConnection.CreateCommand()
$SqlCommand.CommandText = "EXEC UDO_VW_ExportView_HISTORY #FromDate"
# Add parameters to pass values to the stored procedure
$SqlCommand.Parameters.AddWithValue("#FromDate", $FromDate) | Out-Null
$DataAdapter = new-object System.Data.SqlClient.SqlDataAdapter $SqlCommand
$dataset = new-object System.Data.Dataset
Write-Host $DataAdapter.Fill($dataset) ' records have been exported.'
$dataset.Tables[0] | Export-CSV D:\_#Scripts\Daily_Export_HISTORY\Daily_Report_Txt\MyReport.csv -NoTypeInformation -Force
Write-Host 'New report D:\_#Scripts\Daily_Export_HISTORY\Daily_Report_Txt\MyReport.csv has been successfully generated'
(gc D:\_#Scripts\Daily_Export_HISTORY\Daily_Report_Txt\MyReport.csv) | % {$_ -replace '"""', '"'} | out-file D:\_#Scripts\Daily_Export_HISTORY\Daily_Report_Txt\Daily_Report_For_$FromDate.csv -Fo -En ascii
Remove-Item D:\_#Scripts\Daily_Export_HISTORY\Daily_Report_Txt\MyReport.csv
As you can see (i hope) the first line is:
$FromDate = Read-Host "Enter Date of to Start (YYYY-MM-DD)"
I just want it to get all transactions 7,6,5,4,3,2 days, and Yesterday as one export.
I also see in the code there are declared variables that may also be modified. Can you please look at this coed and provide an updated code that will achieve what I'm trying to do. I'm new to Powershell
Thank you so much!
You can replace :
$FromDate = Read-Host "Enter Date of to Start (YYYY-MM-DD)"
by
$FromDate = ([datetime]::Now).AddDays(-7).Date.ToString("yyyy-MM-dd")
It takes today date and time, remove 7 days, remove time and format it.
I am writing a function in PowerShell that will do the following when given some parameter:
Get Credetials to connect to remote SQL Server
Open SQL Connection with given parameters. If Opening connection takes longer than value in $sqlconnectionTimeout, it will report a log file with Timeout error
A query will be introduce in a specific DB from Remote Server. If executing command takes longer than value in $sqlCommandTimeout, it will report a log file with error.
The function will create a html file with connectivity result (True or False). If True, the script will check log reports within last 5 minutes. Connectivity will be True if no logs are found. If False, a log will be created and saved in a specific folder.
About the Timeout variables, No matter what I put in the variable, The Connection is opened and the command is executed. Even If I put 0 forcing it to output a timeout error. The connection is opened and the query executed.
My question is: What am I missing here so that Timeout takes effect when executing the function?
Thanks for your help
This is the Code:
# Editing Variables
$user = "userName"
$DatabaseName = "SampleDB"
$query = "INSERT INTO [SampleDB].[dbo].[HealthChecks]([DateTime],[Source],[User]) VALUES (CURRENT_TIMESTAMP, ##SERVERNAME, CURRENT_USER)"
#Set timeout when stablishing connection (Default = 15)
$sqlConnectionTimeout = 3
#Set timeout when executing sql command (Default = 30)
$sqlCommandTimeout = 3
# 1. R-etrieve the data
function Test-SqlConnection
{
param(
[Parameter(Mandatory)]
[string]$serverURL,
[Parameter(Mandatory)]
[string]$DatabaseName,
[Parameter(Mandatory)]
[string]$user,
[Parameter(Mandatory)]
[string]$query,
[Parameter(Mandatory=$False)]
[string]$sqlConnectionTimeout,
[Parameter(Mandatory=$False)]
[string]$sqlCommandTimeout
)
try
{
$logTime = (Get-Date -Format "hh-mm_dd-MM-yyyy")
$reportPath = "C:\Logs\DBLogs\"
$DBCheckPath = "C:\inetpub\wwwroot\UptimeRobot\"
# Obtain Credentials without revealing password in script
$userName = $user
$passwordFile = "C:\Utilities\keys\password.txt"
$keyFile = "C:\Utilities\keys\aes.key"
$key = Get-Content $keyFile
$Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $userName, (Get-Content $passwordFile | ConvertTo-SecureString -Key $key)
# Open Connection to DB
$userName = $Credential.UserName
$password = $Credential.GetNetworkCredential().Password
$connectionString = "Data Source={0};database={1};User ID={2};Password={3};Connection Timeout={4}" -f $serverURL,$DatabaseName,$userName,$password,$sqlConnectionTimeout
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection $ConnectionString
$sqlConnection.Open()
#3. P-repare the UpdateRequest
$sqlCommand = New-Object System.Data.SQLClient.SQLCommand
$sqlCommand.CommandTimeout = $sqlCommandTimeout
$sqlCommand.Connection = $sqlConnection
$sqlCommand.CommandText = $query
$res = $sqlCommand.ExecuteNonQuery()
# 4. S-end reports to log file in JSon format
$sqlLogs = Get-Childitem $reportPath -filter *.json | Where-Object {$_.LastWriteTime -gt [datetime]::Now.AddMinutes(-5)}
if($res -eq 1)
{
if($sqlLogs.count -gt 0)
{
$connectivityResult = #{
DBServer = $ServerFriendlyName
DataCenter = $dataCenter
Connectivity = $False
}
}else
{
$connectivityResult =#{
DBServer = $ServerFriendlyName
DataCenter = $dataCenter
Connectivity = $True
}
}
$connectivityResult | ConvertTo-Json -Compress | Out-File -Encoding utf8 -LiteralPath ($DBCheckPath + "DBCheck.html")
}
##TODO: Only report Connectivity Serviceable= $false when there are two logs with $false value within 5 minutes.
}catch
{
$errorMessage = $_.Exception
## Only return $false if the exception was thrown because it can't connect for some reason. Otherwise
## throw the general exception
if ($errorMessage -match 'The server was not found' -or 'timeout')
{
$connectivityResult = #{
DBServer = $ServerFriendlyName
DataCenter = $dataCenter
Connectivity = $False
ErrorMessage = $errorMessage.GetBaseException().message
}
$connectivityResult | ConvertTo-Json | Out-File -Encoding utf8 -LiteralPath ($reportPath + $logTime + ".json")
$connectivityResult | ConvertTo-Json | Out-File -Encoding utf8 -LiteralPath ($DBCheckPath + "DBCheck.html")
}
}
finally
{
$sqlConnection.Close()
}
}
$sf = "\\\\domain\\dept\\dcgsi\\Extracts\\Tableau_Unlicensed_Users.csv"
if (Test-Path $sf){
Remove-Item $sf
}
$query = #"
\\copy (SELECT Name
FROM _users
WHERE licensing_role_name = 'Unlicensed')
TO $sf
WITH CSV DELIMITER ','
"#
$conn = New-Object -ComObject ADODB.Connection
# use existing 64 bit ODBC System DSN that we set up manually
$conn.Open('PostgreSQL30')
$conn.Execute($query)
$conn.Close()
I keep getting an error about "\" on the line with the $conn.Execute() when I try and do this. I assume it has to do with character escaping and maybe I am doing it wrong.
Is there a better way to do this with PowerShell if I just need to get the name field of any record from _users and output it to CSV?
Eventually I will be adding more to this to loop through each record in the CSV and execute a tabcmd to remove all the users that are unlicensed.
$sf = "\\domain\dept\dcgsi\Extracts\Tableau_Unlicensed_Users.csv"
if (Test-Path $sf){
Remove-Item $sf
}
$query = #"
SELECT Name
FROM _users
WHERE licensing_role_name = 'Unlicensed'
"#
function Get-ODBC-Data{
$conn = New-Object System.Data.Odbc.OdbcConnection
$conn.ConnectionString = "DSN=PostgreSQL30;"
$conn.Open()
$cmd = New-object System.Data.Odbc.OdbcCommand($query, $conn)
$ds = New-Object system.Data.DataSet
(New-Object System.Data.Odbc.OdbcDataAdapter($cmd)).Fill($ds) | Out-Null
$conn.Close()
$ds.Tables[0] | Export-Csv $sf
}
Get-ODBC-Data
This did like 99% of what I need; I just have to process the csv now and drop the first two lines. The first line is a type info message and the second is the column header.
I'm trying to create a script for querying a database, extract data and put it to an .csv file. This is the part of interest of the code:
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$command = $connection.CreateCommand()
$command.CommandText = $query
$result = $command.ExecuteReader()
###########################
##PRSENTAZIONE DEI RISULTATI##
$table = New-Object System.Data.DataSet
$table.Load($result)
$table | Export-Csv -Path $fileName -Delimiter "|" -NoTypeInformation
I have 2 problems:
On Windows 2003 PowerShell -Delimiter parameter is not present
If I delete that parameter the ouput file presents a wrong format date.
username, ipaddress, yyyy-mm-dd hh:mm:ss (running the same query in Management Studio), username, ipaddress, "dd/mm/yyyy hh.mm.ss" (apexes included ; printing the $table).
How can I solve this?
I have a condition that kicks off a PowerShell script to append a short string to a text file. This condition can fire rapidly, so the file is being written multiple times by the same script. Additionally, a separate script is importing from that text file in batches (less frequently).
Whenever the condition fires very rapidly, I get the error: "The process can not access the file 'file_name' because it is being used by another process." When I do the same append in Python (my main language), I don't get the same error, but I could use some help fixing this in PowerShell.
$action = $args[0]
$output_filename = $args[1]
$item = $args[2]
if ($action -eq 'direct'){
$file_path = $output_filename
$sw = New-Object -typename System.IO.StreamWriter($file_path, "true")
$sw.WriteLine($item)
$sw.Close() }
I have also tried the following instead of StreamWriter, but apparently the performance is weak for Add-Content and Out-File (http://sqlblog.com/blogs/linchi_shea/archive/2010/01/04/add-content-and-out-file-are-not-for-performance.aspx):
out-file -Append -FilePath $file_path -InputObject $item }
Might try something like this:
while ($true)
{
Try {
[IO.File]::OpenWrite($file_path).close()
Add-Content -FilePath $file_path -InputObject $item
Break
}
Catch {}
}