Powershell adding numbers in a loop - powershell

I have the below code which is meant to total up the time offset as the loop rotates (I will then need to divide this by 10 to get the average but first I need to get this bit working).
I'm assuming I need to cast something as [INT] but I've tried multiple locations that would make sense to no avail. I just end up with O's.
$winTimeStripchart = w32tm /stripchart /computer:0.pool.ntp.org /dataonly /samples:10
$WinTimeOffset = $null
For($i=3; $i -le 12; $i++){
$Offset = $winTimeStripchart[$i].split("-")
$trimmedOffset = $Offset[1].trim("s")
$winTimeOffset = $winTimeOffset + $trimmedOffset
}
Write-Host "Total: $winTimeOffset"
# Now need to divide by 10.
sample data:
20:30:23, -00.0698082s
20:30:25, -00.0704645s
20:30:27, -00.0708694s
20:30:29, -00.0728990s
20:30:31, -00.0719226s
20:30:33, -00.0749031s
20:30:36, -00.0778656s
20:30:38, -00.0782183s
20:30:40, -00.0752974s
20:30:42, -00.0760958s

You can try this one line command :
$a = w32tm /stripchart /computer:0.fr.pool.ntp.org /dataonly /samples:10 | select -Skip 3 | % {[double]$_.Substring(11,10)} | Measure-Object -Average -sum
$a can also give you maximum and minimum adding Measure-Object params.

You'll want to cast to a double rather than int. The following should do it:
[System.Collections.ArrayList]$winTimeStripchart = w32tm /stripchart /computer:0.pool.ntp.org /dataonly /samples:10
$WinTimeOffset = $null
$winTimeStripchart.RemoveRange(0,3)
foreach($entry in $winTimeStripchart){
$Offset = $entry.split("-")
$trimmedOffset = $Offset[1].trim("s")
$winTimeOffset = [double]$winTimeOffset + [double]$trimmedOffset
}
Write-Host "Total: $winTimeOffset"
Write-Host "Average: $($winTimeOffset/$winTimeStripchart.count)"
Sample output:
Total: 6.1581437
Average: 0.61581437
I've make some other tweaks to the script as well to make it more scalable:
Foreach rather than a for loop
Using a list rather than array and stripping first 3 entries.
Dividing buy number of entries in list.
regards
Arcas

Related

Windows PowerShell: How to parse the log file?

I have an input file with below contents:
27/08/2020 02:47:37.365 (-0516) hostname12 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'session1' (from 'vmpms1\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT' - 1 licenses have been allocated by concurrent usage category 'Unlimited' (session module usage now 1, session category usage now 1, total module concurrent usage now 1, total category usage now 1)
27/08/2020 02:47:37.600 (-0516) hostname13 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'sssion2' (from 'vmpms2\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT-Read' - 1 licenses have been allocated by concurrent usage category 'Floating' (session module usage now 2, session category usage now 2, total module concurrent usage now 1, total category usage now 1)
27/08/2020 02:47:37.115 (-0516) hostname141 ult_licesrv CMN 5 Logging Housekee 00000 Deleting old log file 'C:\Program Files\PMCOM Global\License Server\diag_ult_licesrv_20200824_011130.log.gz' as it exceeds the purge threashold of 72 hours
27/08/2020 02:47:37.115 (-0516) hostname141 ult_licesrv CMN 5 Logging Housekee 00000 Deleting old log file 'C:\Program Files\PMCOM Global\License Server\diag_ult_licesrv_20200824_021310.log.gz' as it exceeds the purge threashold of 72 hours
27/08/2020 02:47:37.625 (-0516) hostname150 ult_licesrv ULT 5 LiceSrv Main[108 00000 Session 'session1' (from 'vmpms1\app1#pmc21app20.pm.com') request for 1 additional licenses for module 'SA-XT' - 1 licenses have been allocated by concurrent usage category 'Unlimited' (session module usage now 2, session category usage now 1, total module concurrent usage now 2, total category usage now 1)
I need to generate and output file like below:
Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage
27/08/2020,02:47:37.365 (-0516),hostname12,1,1,1,1
27/08/2020,02:47:37.600 (-0516),hostname13,2,2,1,1
27/08/2020,02:47:37.115 (-0516),hostname141,0,0,0,0
27/08/2020,02:47:37.115 (-0516),hostname141,0,0,0,0
27/08/2020,02:47:37.625 (-0516),hostname150,2,1,2,1
The output data order is: Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage.
Put 0,0,0,0 if no entry for session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage
I need to get content from the input file and write the output to another file.
Update
I have created a file input.txt in F drive and pasted the log details into it.
Then I form an array by splitting the file content when a new line occurs like below.
$myList = (Get-Content -Path F:\input.txt) -split '\n'
Now I got 5 items in my array myList. Then I replace the multiple blank spaces with a single blank space and formed a new array by splitting each element by blank space. Then I print the 0 to 3 array elements. Now I need to add the end values (session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage).
PS C:\Users\user> $myList = (Get-Content -Path F:\input.txt) -split '\n'
PS C:\Users\user> $myList.Length
5
PS C:\Users\user> $myList = (Get-Content -Path F:\input.txt) -split '\n'
PS C:\Users\user> $myList.Length
5
PS C:\Users\user> for ($i = 0; $i -le ($myList.length - 1); $i += 1) {
>> $newList = ($myList[$i] -replace '\s+', ' ') -split ' '
>> $newList[0]+','+$newList[1]+' '+$newList[2]+','+$newList[3]
>> }
27/08/2020,02:47:37.365 (-0516),hostname12
27/08/2020,02:47:37.600 (-0516),hostname13
27/08/2020,02:47:37.115 (-0516),hostname141
27/08/2020,02:47:37.115 (-0516),hostname141
27/08/2020,02:47:37.625 (-0516),hostname150
If you really need to filter on the granularity that you're looking for, then you may need to use regex to filter the lines.
This would assume that the rows have similarly labeled lines before the values you're looking for, so keep that in mind.
[System.Collections.ArrayList]$filteredRows = #()
$log = Get-Content -Path C:\logfile.log
foreach ($row in $log) {
$rowIndex = $log.IndexOf($row)
$date = ([regex]::Match($log[$rowIndex],'^\d+\/\d+\/\d+')).value
$time = ([regex]::Match($log[$rowIndex],'\d+:\d+:\d+\.\d+\s\(\S+\)')).value
$hostname = ([regex]::Match($log[$rowIndex],'(?<=\d\d\d\d\) )\w+')).value
$sessionModuleUsage = ([regex]::Match($log[$rowIndex],'(?<=session module usage now )\d')).value
if (!$sessionModuleUsage) {
$sessionModuleUsage = 0
}
$sessionCategoryUsage = ([regex]::Match($log[$rowIndex],'(?<=session category usage now )\d')).value
if (!$sessionCategoryUsage) {
$sessionCategoryUsage = 0
}
$moduleConcurrentUsage = ([regex]::Match($log[$rowIndex],'(?<=total module concurrent usage now )\d')).value
if (!$moduleConcurrentUsage) {
$moduleConcurrentUsage = 0
}
$totalCategoryUsage = ([regex]::Match($log[$rowIndex],'(?<=total category usage now )\d')).value
if (!$totalCategoryUsage) {
$totalCategoryUsage = 0
}
$hash = [ordered]#{
Date = $date
time = $time
hostname = $hostname
session_module_usage = $sessionModuleUsage
session_category_usage = $sessionCategoryUsage
module_concurrent_usage = $moduleConcurrentUsage
total_category_usage = $totalCategoryUsage
}
$rowData = New-Object -TypeName 'psobject' -Property $hash
$filteredRows.Add($rowData) > $null
}
$csv = $filteredRows | convertto-csv -NoTypeInformation -Delimiter "," | foreach {$_ -replace '"',''}
$csv | Out-File C:\results.csv
What essentially needs to happen is that we need to get-content of the log, which returns an array with each item terminated on a newline.
Once we have the rows, we need to grab the values via regex
Since you want zeroes in some of the items if those values don't exist, I have if statements that assign '0' if the regex returns nothing
Finally, we add each filtered item to a PSObject and append that object to an array of objects in each iteration.
Then export to a CSV.
You can probably pick apart the lines with a regex and substrings easily enough. Basically something like the following:
# Iterate over the lines of the input file
Get-Content F:\input.txt |
ForEach-Object {
# Extract the individual fields
$Date = $_.Substring(0, 10)
$Time = $_.Substring(12, $_.IndexOf(')') - 11)
$Hostname = $_.Substring(34, $_.IndexOf(' ', 34) - 34)
$session_module_usage = 0
$session_category_usage = 0
$module_concurrent_usage = 0
$total_category_usage = 0
if ($_ -match 'session module usage now (\d+), session category usage now (\d+), total module concurrent usage now (\d+), total category usage now (\d+)') {
$session_module_usage = $Matches[1]
$session_category_usage = $Matches[2]
$module_concurrent_usage = $Matches[3]
$total_category_usage = $Matches[4]
}
# Create custom object with those properties
New-Object PSObject -Property #{
Date = $Date
time = $Time
hostname = $Hostname
session_module_usage = $session_module_usage
session_category_usage = $session_category_usage
module_concurrent_usage = $module_concurrent_usage
total_category_usage = $total_category_usage
}
} |
# Ensure column order in output
Select-Object Date,time,hostname,session_module_usage,session_category_usage,module_concurrent_usage,total_category_usage |
# Write as CSV - without quotes
ConvertTo-Csv -NoTypeInformation |
ForEach-Object { $_ -replace '"' } |
Out-File F:\output.csv
Whether to pull the date, time, and host name from the line with substrings or regex is probably a matter of taste. Same goes for how strict the format must be matched, but that to me mostly depends on how rigid the format is. For more free-form things where different lines would match different regexes, or multiple lines makes up a single record, I also quite like switch -Regex to iterate over the lines.

Efficient way to find and replace many strings in a large text file

The Text file contains a software output on a time domain analysis. 10800 seconds simulation and 50 nodes being considered. We have 540,000 strings to be replaced in 540 MB text file with 4.5 million lines.
Which is currently projected to take more than 4 days. Something is going wrong. Don't know what. Please suggest me a better efficient approach.
Below is the function which does the find and replace.
To replace the string the script goes through the original text file line by line at the same time it generates a duplicate file with replaced strings. So another 540 MB file with 4.5 million lines will be generated at the end of the script.
Function ReplaceStringsInTextFile
{
$OutputfilebyLine = New-Object -typename System.IO.StreamReader $inputFilePathFull
$uPreviousValue = 0
$time = 60
$u = 0; $LastStringWithoutFindResult = 0
$lineNumber = 0
while ($null -ne ($line = $OutputfilebyLine.ReadLine())) {
$lineNumber = $lineNumber + 1
if ($time -le $SimulationTimeSeconds) # time simulation start and end checks
{
# 10800 strings corresponds to one node
# there are 50 nodes.. Thus 540,000 values
# $StringsToFindFileContent contains strings to find 540,000 strings
# $StringsToReplaceFileContent contains strings to replace 540,000 strings
$StringToFindLineSplit = -split $StringsToFindFileContent[$time-60]
$StringToReplaceLineSplit = -split $StringsToReplaceFileContent[$time-60]
if($u -le $NumberofNodes-1)
{
$theNode = $Nodes_Ar[$u]
$StringToFindvalue = $StringToFindLineSplit[$u]
$StringToReplacevalue = $StringToReplaceLineSplit[$u]
if (($line -match $theNode) -And ($line -match $StringToFindvalue)){
$replacedLine = $line.replace($StringToFindvalue,$StringToReplacevalue)
add-content -path $WriteOutputfilePathFull -value "$replacedLine"
$uPreviousValue = $u
$checkLineMatched = 1
if (($line -match $LastNodeInArray)) {
$time = $time + 1
$LastStringWithoutFindResult = 0
}
} elseIf (($line -match $LastNodeInArray) -And ($checkLineMatched -eq 0)) {
$LastStringWithoutFindResult = $LastStringWithoutFindResult + 1
} else {
#"Printing lines without match"
add-content -path $WriteOutputfilePathFull -value "$line"
$checkLineMatched = 0
}
}
if ($checkLineMatched -eq 1) {
# incrementing the value of node index to next one in case the last node is found
$u = $uPreviousValue + 1
if ($u -eq $Nodes_Ar.count) {
$u = 0
$timeElapsed = (get-date -displayhint time) - $startTime
"$($timeElapsed.Hours) Hours $($timeElapsed.Minutes) Minutes $($timeElapsed.Seconds) Seconds"
}
}
}
# Checking if the search has failed for more than three cycles
if ($LastStringWithoutFindResult -ge 5) { # showing error dialog in case of search error
[System.Windows.Forms.MessageBox]::Show("StringToFind Search Fail. Please correct StringToFind values. Aborting now" , "Status" , 0)
$OutputfilebyLine.close()
}
}
$OutputfilebyLine.close()
}
The above function is the last part of the script. Which is taking the most time.
I had run the script in under 10 hours 1 year ago.
Update The script sped up running after 4 hours and suddenly time to complete projection reduced from 4 days to under 3 hours. The script finished running in 7 hours and 9 minutes. However i am not sure what made the sudden change in speed other than asking the question on stack overflow :)
As per the suggestion by https://stackoverflow.com/users/478656/tessellatingheckler
I have avoided writing one line at a time using
add-content -path $WriteOutputfilePathFull -value "$replacedLine"
Instead i am now writing ten thousand lines at a time using add-content
$tenThousandLines = $tenThousandLines + "`n" + $replacedLine
And at the appropriate time I am using add-content to write 10,000 lines at one go like below. The if block follows my methods logic
if ($lineNumber/10000 -gt $tenThousandCounter){
clear-host
add-content -path $WriteOffpipeOutputfilePathFull -value "$tenThousandLines"
$tenThousandLines = ""
$tenThousandCounter = $tenThousandCounter + 1
}
I have encountered system out of memmory exception error when trying to add 15,000 or 25,000 lines at a time. After using this the time required for the operation has reduced from 7 hours to 5 hours. And at another time to 2 hours and 36 minutes.

Passing a path directory to a function in PowerShell

I am new to PS and I am trying to write a function which takes in parameters from a global variable. I want to pass a path name read from a .txt file into a function in the same script.
function GetCorrectChildren ([string] $homepath,$min,$max,$row)
{
#Testpoint 2
write-host "homepath = $homepath"
$ColItem = (Get-ChildItem $homepath |? {$_.PSIsContainer} | sort-object)
}
foreach ($homepath in (Get-Content $PSScriptRoot\homepath_short.txt))
{
$freeSpace = [win32api]::GetDiskFreeSpace("$homepath").FreeBytesAvailable / 1073741824
$totalSpace = [win32api]::GetDiskFreeSpace("$homepath").TotalNumberOfBytes / 1073741824
$percentageFreeSpace = $freeSpace / $totalSpace * 100
if($freeSpace -lt $threshold)
{
#Testpoint 1
write-host "homepath = $homepath"
GetCorrectChildren ("$homepath",$min,$max,$OriRow)
}
For #Testpoint 1, it returns the path name correctly which is \\C:\test1\test_a. However in #Testpoint 2 it returns \\C:\test1\test_a 20 30 System.Object.
I don't understand what does the 20 30 System.Object mean and where does it come from? Can some one shine some light on this? Thanks
Change the last line
GetCorrectChildren ("$homepath",$min,$max,$OriRow)
to
GetCorrectChildren $homepath $min $max $OriRow
as ("$homepath",$min,$max,$OriRow) creates a single array with the four values and passes it to the function GetCorrectChildren as its first parameter so that write-host "homepath = $homepath" in it will print all 4 values

Simulating `ls` in Powershell

I'm trying to get something that looks like UNIX ls output in PowerShell. This is getting there:
Get-ChildItem | Format-Wide -AutoSize -Property Name
but it's still outputting the items in row-major instead of column-major order:
PS C:\Users\Mark Reed> Get-ChildItem | Format-Wide -AutoSize -Property Name
Contacts Desktop Documents Downloads Favorites
Links Music Pictures Saved Games
Searches Videos
Desired output:
PS C:\Users\Mark Reed> My-List-Files
Contacts Downloads Music Searches
Desktop Favorites Pictures Videos
Documents Links Saved Games
The difference is in the sorting: 1 2 3 4 5/6 7 8 9 reading across the lines, vs 1/2/3 4/5/6 7/8/9 reading down the columns.
I already have a script that will take an array and print it out in column-major order using Write-Host, though I found a lot of PowerShellish idiomatic improvements to it by reading Keith's and Roman's takes. But my impression from reading around is that's the wrong way to go about this. Instead of calling Write-Host, a script should output objects, and let the formatters and outputters take care of getting the right stuff written to the user's console.
When a script uses Write-Host, its output is not capturable; if I assign the result to a variable, I get a null variable and the output is written to the screen anyway. It's like a command in the middle of a UNIX pipeline writing directly to /dev/tty instead of standard output or even standard error.
Admittedly, I may not be able to do much with the array of Microsoft.PowerShell.Commands.Internal.Format.* objects I get back from e.g. Format-Wide, but at least it contains the output, which doesn't show up on my screen in rogue fashion, and which I can recreate at any time by passing the array to another formatter or outputter.
This is a simple-ish function that formats column major. You can do this all in PowerShell Script:
function Format-WideColMajor {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[AllowNull()]
[AllowEmptyString()]
[PSObject]
$InputObject,
[Parameter()]
$Property
)
begin {
$list = new-object System.Collections.Generic.List[PSObject]
}
process {
$list.Add($InputObject)
}
end {
if ($Property) {
$output = $list | Foreach {"$($_.$Property)"}
}
else {
$output = $list | Foreach {"$_"}
}
$conWidth = $Host.UI.RawUI.BufferSize.Width - 1
$maxLen = ($output | Measure-Object -Property Length -Maximum).Maximum
$colWidth = $maxLen + 1
$numCols = [Math]::Floor($conWidth / $colWidth)
$numRows = [Math]::Ceiling($output.Count / $numCols)
for ($i=0; $i -lt $numRows; $i++) {
$line = ""
for ($j = 0; $j -lt $numCols; $j++) {
$item = $output[$i + ($j * $numRows)]
$line += "$item$(' ' * ($colWidth - $item.Length))"
}
$line
}
}
}

Compare more than two strings

Here is what I am trying to achieve...
I have to view the ADAM db in VMWARE to see the replication times. My question is how would I compare more than two strings using the compare-object command. I cannot find any articles on more than two values.
This is what I started writing. I am trying to make this as dynamic as possible...
#PORT FOR LDAP
$ldap = 389;
#PATH
$path = 'DC=vdi,DC=vmware,DC=int';
#SERVERS
$vm = #("fqdn" , "fqdn" , "fqdn");
#ARRAY FOR LOOP
$comp = #();
#LOOP FOR ARRAY COMPARE
for($i = 1; $i -le $vm.count; $i++)
{
$comp += repadmin.exe /showrepl $svr":"$ldap $path | Select-String "Last attempt";
}
#CREATE DYNAMIC VARIABLES
for($i = 0; $i -le ($comp.count - 1); $i++)
{
New-Variable -name repl$i -Value $comp[$i];
}
Thank you in advanced!!!
As I mentioned in my comment, your question is too vague for us to provide a good answer for your situation, so I'll focus on "compare more than two strings". To do this, I wuold recommend Group-Object. Ex.
$data = #"
==== INBOUND NEIGHBORS ======================================
CN=Configuration,CN={B59C1E29-972F-455A-BDD5-1FA7C1B7D60D}
....
Last attempt # 2010-05-28 07:29:34 was successful.
CN=Schema,CN=Configuration,CN={B59C1E29-972F-455A-BDD5-1FA7C1B7D60D}
....
Last attempt # 2010-05-28 07:29:34 was successful.
OU=WSFG,DC=COM
....
Last attempt # 2010-05-28 07:29:35 failed, result -2146893008
(0x8009033
0):
"# -split [environment]::NewLine
$comp = $data | Select-String "Last attempt"
$comp | Group-Object
Count Name Group
----- ---- -----
2 Last attempt # 2010-05-28 07:29:34 was successful. { Last atte...
1 Last attempt # 2010-05-28 07:29:35 failed, result -2146893008 { Last atte...
Group-Object and PowerShell is very flexible, so you could customize this to ex. display the servernames and status for the servers that wasn't equal to the rest (ex. count = 1 or not in any of the biggest groups) etc., but I won't spend more time going into details because I have no idea of what you are trying to achieve, so I'll probably just waste both of ours time.
Summary: What I can tell you is the I would proabably (99% sure) use Group-Object to "compare more than two strings".