I'm trying to write this PS script but if someone beats me to the punch I'm sure they will get free karma.
Anyway here is what I want to take a file setup like this
foo.bar=Some random text is stored here
foo.bar=Lazy maintainers make me angry
bar.foo=Hello World!
bar.foo=Hello World!
The main goal is to remove any duplicated entries, which I have several of . . . This seems easy enough with
Get-Content c:\list.txt | Select-Object -Unique
But I would also like to store any conflicts that have the same key identifiers into a separate file for so I can review which ones I should keep.
I'm still a PS novice and haven't found a good way to do this yet.
You can use Group-Object to group together items with the same key. Then look for groups with more than one element in them (indicating duplicate entries). Finally, print those out to a file somewhere:
# raw content
$lines = Get-Content C:\data.txt
# package each line into a little object with properties Key and Val
$data = $lines |%{ $key,$val = $_.Split('='); new-object psobject -prop #{Key = $key; Val = $val} }
# group the objects by key, only keep groups with more than 1 element
$duplicates = $data | group Key |?{$_.Count -gt 1}
# print out each key and the different values it has been given
$duplicates |%{ "--- [$($_.Name)] ---"; $_.Group | select -expand Val }
Result:
--- [foo.bar] ---
Some random text is stored here
Lazy maintainers make me angry
--- [bar.foo] ---
Hello World!
Hello World!
You can pipe that to Out-File if you want to store in a log.
Related
I want to create a PowerShell script to extract a text file with a list of words specified in an Exchange Server 2016 transport rule. I'm stuck with handling the obtained list.
To get the list, I do this within Exchange Management Shell:
$SubjectOrBodyContainsWords = Get-TransportRule "My rule name" | Select-Object -Property SubjectOrBodyContainsWords
I verify that the list is correct using this:
$FormatEnumerationLimit = 10000
$SubjectOrBodyContainsWords | Format-Table -HideTableHeaders | Out-String -width 10000
The output looks like (Just an example, actual list is much much bigger):
{unsubscribe, mailing, blabla}
Now I want to iterate the list to do something with each item. I tried something like this (Just a simple example):
$I = 10;
foreach ($A in $SubjectOrBodyContainsWords)
{
$I++;
$I;
$A;
}
The problem is that it doesn't loop all the items. It looks like there is only one item.
What am I doing wrong?
Thanks.
I don't have access to my Exchange server at the moment but just try the following suggestions. Just ask for and expand the property.
(Get-TransportRule "My rule name").SubjectOrBodyContainsWords
# Or
Get-TransportRule "My rule name" |
Select-Object -ExpandProperty SubjectOrBodyContainsWords
Since this returns an array, you need to expand that listing.
Or you can do this to turn into a list to work with...
"{unsubscribe, mailing, blabla}" -replace '\s|{|}' -split ',' | foreach {
# Code to do something with each item
$PSItem }
# Results
unsubscribe
mailing
blabla
... and
Potential duplicate of this use case
Exchange Shell - SubjectOrBodyContainsWords
I am trying to find a way to sort a CSV by two fields and retrieve only the latest item.
CSV fields: time, computer, type, domain.
Item that works is below but is slow due to scale of CSV and I feel like there is a better way.
$sorted = $csv | Group-Object {$_.computer} | ForEach {$_.Group | Sort-Object Time -Descending | Select-Object -First 1}
As Lee_Dailey suggests, you'll probably have better luck with a hashtable instead, Group-Object (unless used with the -NoElement parameter) is fairly slow and memory-hungry.
The fastest way off the top of my head would be something like this:
# use the call operator & instead of ForEach-Object to avoid overhead from pipeline parameter binding
$csv |&{
begin{
# create a hashtable to hold the newest object per computer
$newest = #{}
}
process{
# test if the object in the pipeline is newer that the one we have
if(-not $newest.ContainsKey($_.Computer) -or $newest[$_.Computer].Time -lt $_.Time){
# update our hashtable with the newest object
$newest[$_.Computer] = $_
}
}
end{
# return the newest-per-computer object
$newest.Values
}
}
I am trying to automate few of the daily health check tasks using PowerShell.
I would like to achieve the following (Step by Step), though I have a partial success in few,
Extract content of text (Log) file located at shared location (I have succeeded) by defining $Path = Set-Location ... etc.,
Send email (succeeded) to mail box, by defining
Real help I need is here,
I want Headings to be appended in Email, along with original extracted text from Step 1,
For ex..
Original Text looks like this (extracted from text file at shared location):
01-01-2018 Number of Successful object - 1
I would like to add the header for this on email, like
date Description Number of Objects
01-01-2018 Successful objects 1
Assuming the content from the log file gathered in Step 1 is a string array of log entries where every string has a format similar to '01-01-2018 Number of Successful object - 1 '
In this example I call that array $logEntries
# create an array to store the results objects in
$result = #()
# loop through this array of log entries
$logEntries | ForEach-Object {
# Here every text line is represented by the special variable $_
# From your question I gather they all have this format:
# '01-01-2018 Number of Successful object - 1 '
if ($_ -match '(?<date>\d{2}-\d{2}-\d{4})\s+(?<description>[^\-\d]+)[\s\-]+(?<number>\d+)\s*$') {
# Try to get the 'Succesful' or 'Failed' (??) text part out of the description
$description = ($matches['description'] -replace 'Number of|object[s]?', '').Trim() + ' object'
if ([int]$matches['number'] -ne 1) { $description += 's' }
# add to the result array
$result += New-Object -TypeName PSObject -Property ([ordered]#{
'Date' = $matches['date']
'Description' = $description
'Number of Objects' = $matches['number']
})
}
}
# now decide on the format of this result
# 1) as plain text in tabular form.
# This looks best when used with a MonoSpaced font like Courier or Consolas.
$result | Format-Table -AutoSize | Out-String
# or 2) as HTML table. You will have to style this table in your email.
# You may include a stylesheet using the '-CssUri' parameter, but inserting
# it in a nicely drawn-up HTML template will give you more creative freedom.
$result | ConvertTo-Html -As Table -Fragment
p.s. because the PSObject has [ordered] properties, this needs PowerShell version 3.0 or better.
I have a lot of csv files with values arranged like so:
X1,Y1
X2,Y2
...,...
Xn,Yn
I find it very tedious processing these with excel, so I want to setup a batch script to process these files such that they appear like this:
#where N is a specified value like 65536
X1,N-Y1,1
X2,N-Y2,2
...,...,...
Xn,N-Yn,n
I have only recently started using powershell for image processing (really simple scripts) and file name appending, so I am not certain how to go about this. A lot of the scripts I have encountered looking to answer this question use csv files with titles per column whereas my files are just arrays of values without object titles in the first row. I would like to avoid running multiple scripts to add titles.
My bonus question is something I have yet to find a good answer to at all, and is the most tedious part of processing. Using excels sort function, I usually change the order of the Yn values in Col2 such that they are sorted in the exported csv like so:
X1,N-Yn,n
...,...,...
Xn-1,N-Y2,2
Xn,N-Y1,1
Using the Col3 values as the sorting order (largest to smallest), then I delete this column so that the final saved csv only contains the first two columns (crucial step). Any help at all would be greatly appreciated, I apologize for the long-winded-ness of this question.
I have encountered looking to answer this question use csv files with titles per column whereas my files are just arrays of values without object titles in the first row.
The -Header parameter of Import-Csv is for adding column headers when the file does not contain them. It takes an array of strings, of however many columns there are.
I would like to avoid running multiple scripts to add titles.
If you couldn't use -Header, you could read the lines with Get-Content into memory, add a header in memory, and then use ConvertFrom-CSV all in one script.
That said, if I'm reading it rightly, you want:
No headers in the input file, and I imagine no headers in the output file
The whole point of adding the third column and sorting and removing it is just to reverse the lines?
The only column you keep is column 1?
I wouldn't use Import-Csv for this, it won't make it much nicer.
$n = 65536
# Read lines into a list, and reverse it
$lines = [Collections.Generic.List[String]](Get-Content -LiteralPath 'c:\test\test.csv')
$lines.Reverse()
# Split each line into two, create a new line with X and N-Y
# write new lines to an output file
$lines | ForEach-Object {
$x, $y = $_.split(',')
"$x,$($n - [int]$y)"
} | Set-Content -LiteralPath 'c:\test\output.csv' -Encoding Ascii
If you do want to use CSV handling, then:
$n = 65536
$counter = 1
Import-Csv -LiteralPath 'C:\test\test.csv' -Header 'ColX', 'ColY' |
Add-Member -MemberType ScriptProperty -Name 'ColN-Y' -Value {$n - $_.ColY} -PassThru |
Add-Member -MemberType ScriptProperty -Name 'N' -Value {$script:counter++} -PassThru |
Sort-Object -Property 'N' -Descending |
Select-Object -Property 'ColX', 'ColN-Y' |
Export-Csv -LiteralPath 'c:\test\output.csv' -NoTypeInformation
But the output will have CSV headers and double-quoted values.
I would try something like, by extending the original table with a calculatable script-property as a new column:
#Your N number
$N = 65536
# Import CSV file without header columns
$table = Import-Csv -Header #("colX","colY") `
-Delimiter ',' `
-Path './numbers.csv'
Write-Host "Original table"
$table | Format-Table
# Manipulate table
$newtable = $table |
Add-Member -MemberType ScriptProperty -Name colNX -Value { $N-$this.colX } - PassThru
Write-Host "New table"
$newtable | Format-Table
I've been using a PowerShell script that reads a file and extracts error codes. It quick, simple and does the job that I want it to but I've now been asked to share it with a wide audience so I need to make it a bit more robust.
The problem I've got is that I need to take the output from my script and use it to lookup against a CSV file so that I get a user friendly table at the end that lists:
A count of the how many time each error occurred (in descending order)
The Error code
The corresponding error message that it displays to the end user
This is the line format in the source file, there's normally upwards on 2000 lines
17-12-2016,10:17:44:487{String=ERROR->(12345678)<inData:{device=printer, formName=blah.frm, eject=FALSE, operation=readForm}><outData:{fields=0, CODE=, field1=}> <outError:{CODE=Error103102, extendedErrorCode=-1, VARS=0}>}
This is my current script:
$WS = Read-Host "Enter computer name"
$date = Read-host "Enter Date"
# Search pattern for select-string (date always at the beginning of the line and the error code somewhere further in)
$Pattern = $date+".*<outError:{CODE="
# This find the lines in the files that contain the search pattern
$lines = select-string -path "\\$WS\c$\folder\folder\file.dat" -pattern $Pattern
# This is the specific Error code pattern that I'm looking for in each line
$regex = [regex] 'Error\d{1,6}'
$Var = #()
# Loops through each line and extracts Error code
foreach ($line in $lines) { $a = $line -match $regex
# Adds each match to variable
$Var += $matches.Values
}
# Groups and sorts results in to easy to read format
$Var | group | Sort-Object -Property count -descending
And this is the result it gives me:
Count Name Group
----- ---- -----
24 Error106013 {Error106013, Error106013, Error106013, Error106013...}
14 Error106109 {Error106109, Error106109, Error106109, Error106109...}
12 Error203002 {Error203002, Error203002, Error203002, Error203002...}
The CSV that I need to lookup against is as simple as it gets, with just 2 values per line in the format:
Code,Error message
What I need to get to is something like this:
Count Name Error Message
----- ---- -----
24 Error106013 Error:blah
14 Error106109 Error:blah,blah
12 Error203002 Error:blah,blah,balh
Google has failed me so I'm hoping that there is someone out there that can at the least point me in the right direction.
Not tested but it should work with a simple calculated property - just replace the last line with:
$errorMap = Import-Csv 'your_errorcode.csv'
$Var | Group-Object | Sort-Object -Property count -descending |
Select-Object Count, Name, #{l='Error Message'; e={($errorMap | Where-Object Code -eq $_.Name)."Error message"}}
Note: You also have to replace path to your CSV.