I am trying to work with a directory full of files.
I want to find specific rows within the file,
from those rows, extract a numeric value
and them sum up all these values, for all values, in a directory.
It would look like this...
File1.txt
bread:123
ham:456
eggs:789
File2.txt
bread:999
mayo:789
eggs:123
and so on...
I want to find the row with eggs, extract the number, and sum these numbers together across files.
I found this script from other posts but it's only segements, I still have trouble understanding how to use and pipe/ variables /braces.
dir . -filter "*.txt" -Recurse -name | foreach{(GC $_).Count} | measure-object -sum
#?
Get-Content | Select-String -Pattern "eggs*"
#?
$record -split ":"
I want the script to say "eggs = 912" which would be 123 + 789 = 912
Here is a possible solution:
$pattern = 'eggs'
$sum = Get-ChildItem . -File -Recurse -Filter *.txt |
Get-Content |
Where-Object { $_ -match $pattern } |
ForEach-Object { ($_ -split ':')[1] } |
Measure-Object -Sum |
ForEach-Object Sum
"$pattern = $sum"
Output:
eggs = 912
Get-ChildItem finds all files recursively that match the filter
Get-Content reads each line of every file and passes that on in the pipeline
Where-Object includes only lines that match the given RegEx pattern
The ForEach-Object line splits the line at : and extracts the sub string, which is at array index [1].
Measure-Object accumulates all numbers (it converts strings to double, if necessary). Internally, it creates a variable in its begin block, accumulates the pipeline input to this variable in its process block and outputs the variable value in its end block.
The last ForEach-Object line is necessary because Measure-Object actually outputs an object with a Sum property, but we only want the value of that property, not the entire object. If you'd remove that line you'd have to write "$pattern = $($sum.Sum)" instead, to access the Sum property of the sum object.
You can treat the as csv files. import-csv doesn't take wildcards for the filename.
import-csv file1.txt,file2.txt -Delimiter : -Header item,amount |
where item -eq eggs | measure -sum amount
Count : 2
Average :
Sum : 912
Maximum :
Minimum :
StandardDeviation :
Property : amount
I've created the following small script to remove 2++ strings from a CSV.
Each row is a log of a given person and a answer they give.
The CSV has X columns.
The column named FIRST identifies the person.
What I need to do is when I delete a row matching the answer, I also need to delete the person from the whole CSV if it had one of the two strings.
What I've made so far, removes the row of people having the answers but the person is still left in the overall CSV with other answers. I want to remove the person fully if the questions have been answered.
Can somebody help me out with making the addition or changes to make this happen?
INPUT File
FIRST,LAST,ADDR,ADDR2,GENDER,HOME,WORK
1,N/A,N/A,N/A,N/A,BAF,N/A
10005,JAS,AA,N/A,,ZAV,N/A
10007,JADE,BB,N/A,OMA,N/A,N/A
10007,JADE,N/A,RAV,N/A,N/A,N/A
10011,KIAH,N/A,N/A,BALI,BB,N/A
SCRIPT
$CSVfile = "C:\Temp\Test\Test.csv"
$CSVfile_filtered = "C:\Temp\Test\Test.csv"
$regex001 = "AA"
$regex002 = "BB"
$filterArray = #($regex001,$regex002)
Get-Content $CSVfile | Select-String -pattern $filterArray -notmatch | Set-Content $CSVfile_filtered
The file should then remove 10005, 10011 and both lines of 10007. But my version only removes one of the 10007 since it only matches one of the two patterns.
Using more of PowerShell's built-in cmdlets can make this a little easier to manage.
# Assuming searching only properties ADDR and ADDR2
$filter = 'AA','BB'
# Grouping by First and Last values to easily remove duplicates
# -match uses regex so | is needed for an OR of multiple items
Import-Csv Test.csv | Group-Object First,Last |
Where {!($_.Group.ADDR,$_.Group.ADDR2 -match ($filter -join '|'))} |
Foreach-Object Group |
Export-Csv output.csv -NoType
You would think strictly using text manipulation would be simpler, but it adds other scenarios to consider:
You will need to track users that have duplicate entries and potentially back track to remove them (if not grouping). This could require reading the file contents twice.
Your header row could match the string you want to filter so you will need to add it to the output if filtering removes it.
Keeping the scenarios above in mind, you can still use a grouping concept:
$filter = 'AA','BB'
$file = Get-Content Test.csv
# $file[0] is the header row
# -split string uses regex and splits at the second comma
# -split results' [0] element is First,Last values
$file[0],($file |
Select-Object -Skip 1 |
Group-Object {($_ -split '(?<=^[^,]*,[^,]*),')[0]} |
where {!($_.Group -match ($filter -join '|'))} |
Foreach-Object Group) | Set-Content output.csv
If I got it right you could do something like this:
$SearchPattern = 'AA', 'BB'
$INPUTCSV = #'
FIRST,LAST,ADDR,ADDR2,GENDER,HOME,WORK
1,N/A,N/A,N/A,N/A,BAF,N/A
10005,JAS,AA,N/A,,ZAV,N/A
10007,JADE,BB,N/A,OMA,N/A,N/A
10007,JADE,N/A,RAV,N/A,N/A,N/A
10011,KIAH,N/A,N/A,BALI,BB,N/A
'# | ConvertFrom-Csv
$ActualSearchPattern =
$INPUTCSV |
Where-Object {
$_.LAST -in $SearchPattern -or
$_.ADDR -in $SearchPattern -or
$_.ADDR2 -in $SearchPattern -or
$_.GENDER -in $SearchPattern -or
$_.HOME -in $SearchPattern -or
$_.Work -in $SearchPattern
} |
Select-Object -ExpandProperty FIRST
$INPUTCSV |
Where-Object -Property FIRST -NotIn -Value $ActualSearchPattern |
Format-Table -AutoSize
There might be more sophisticated or more elegant ways but I cannot think about one at the moment. ;-)
There is a nice PowerShell module you can use to manipulate the content of a csv or xlsx file: ImportExcel
This give you a lot of options to manipulate the sheets, columns etc.
I'm trying to get the output of two separate files although I'm stuck on the wild card or contains select-string search from file A (Names) in file B (name-rank).
The contents of file A is:
adam
george
william
assa
kate
mark
The contents of file B is:
12-march-2020,Mark-1
12-march-2020,Mark-2
12-march-2020,Mark-3
12-march-2020,william-4
12-march-2020,william-2
12-march-2020,william-7
12-march-2020,kate-54
12-march-2020,kate-12
12-march-2020,kate-44
And I need to match on every occurrence of the names after the '-' so my ordered output should look like this which is a combination of both files as the output:
mark
Mark-1
Mark-2
Mark-3
william
william-2
william-4
william-7
Kate
kate-12
kate-44
kate-54
So far I only have the following and I'd be grateful for any pointers or assistance please.
import-csv (c:\temp\names.csv) |
select-string -simplematch (import-csv c:\temp\names-rank.csv -header "Date", "RankedName" | select RankedName) |
set-content c:\temp\names-and-ranks.csv
I imagine the select-string isn't going to be enough and I need to write a loop instead.
The data you give in the example does not give you much to work with, and the desired output is not that intuitive, most of the time with Powershell you would like to combine the data in to a much richer output at the end.
But anyway, with what is given here and what you want, the code bellow will get what you need, I have left comments in the code for you
$pathDir='C:\Users\myUser\Downloads\trash'
$names="$pathDir\names.csv"
$namesRank="$pathDir\names-rank.csv"
$nameImport = Import-Csv -Path $names -Header names
$nameRankImport= Import-Csv -Path $namesRank -Header date,rankName
#create an empty array to collect the result
$list=#()
foreach($name in $nameImport){
#get all the match names
$match=$nameRankImport.RankName -like "$($name.names)*"
#add the name from the First list
$list+=($name.names)
#if there are any matches, add them too
if($match){
$list+=$match
}
}
#Because its a one column string, Export-CSV will now show us what we want
$list | Set-Content -Path "$pathDir\names-and-ranks.csv" -Force
For this I would use a combination of Group-Object and Where-Object to first group all "RankedName" items by the name before the dash, then filter on those names to be part of the names we got from the 'names.csv' file and output the properties you need.
# read the names from the file as string array
$names = Get-Content -Path 'c:\temp\names.csv' # just a list of names, so really not a CSV
# import the CSV file and loop through
Import-Csv -Path 'c:\temp\names-rank.csv' -Header "Date", "RankedName" |
Group-Object { ($_.RankedName -split '-')[0] } | # group on the name before the dash in the 'RankedName' property
Where-Object { $_.Name -in $names } | # use only the groups that have a name that can be found in the $names array
ForEach-Object {
$_.Name # output the group name (which is one of the $names)
$_.Group.RankedName -join [environment]::NewLine # output the group's 'RankedName' property joined with a newline
} |
Set-Content -Path 'c:\temp\names-and-ranks.csv'
Output:
Mark
Mark-1
Mark-2
Mark-3
william
william-4
william-2
william-7
kate
kate-54
kate-12
kate-44
How do I get a specific string from txt file using select-string for example, what I tried so far is:
$path = "\\serverpath\servername.txt"
$list = select-string -path $path -pattern "node"
write-host $list
servername.txt contains:
servername is node1 and it is development server, it has problem
servername is node2 and it is production server, it is good
So I need to list only the words node1, node2 ...from the .txt file.
When you want to use Select-String for this you can just expand your Pattern to match the digit following node and extract the matched Value like this:
$path = "\\serverpath\servername.txt"
$list = Select-String -Path $path -Pattern "node\d+" -AllMatches | % {$_.Matches.Value}
Write-Host $list
Explanation:
"node\d+" -> \d+ matches the word node + x-digits after the word (x > 0)
% -> alias for ForEach-Object Value given from the Pipe
$_.Matches.Value -> Gives the matched Value
You can use a regex pattern and then acceess the Matches attribute of the resulting select-string object.
$path = "\\serverpath\servername.txt"
$list = (select-string -Path $path -pattern "(node\d)").Matches
write-host $list
The regex pattern matches node plus a single digit between 0-9 following it.
https://regex101.com/r/vg4LIz/5
Parse .rtf file, output email addresses in .csv file?
I have an .rtf file containing a bunch of email addresses, I need this parsed so that I can compare a .csv file to active users in Active Directory.
Basically I want what is to the left of "#my.domain.com"
$finds = Select-String -Path "path\to\my.rtf" -Pattern "#my.domain.com" | ForEach-Object {$_.Matches}
$finds | Select-Object -First 1 | ft *
This of course gives me one result so that I don't have alot of output.
I only manage to get matches or the complete line.
I've tried adding something along the line of
$finds = Select-String -Path "path\to\my.rtf" -Pattern "\w.#my.domain.com"
This gives me the very two last letters in the addresses.
If I keep adding dots to the "wildcard"
-Pattern "\w.....#my.domain.com"
I also get a ton of numbers/characters (.rtf formatting) for addresses that contains fewer characters.
How do I do this?
EDIT: I will update the question as soon as I've found a solution. As of now I'm trying with regular expressions.
Example:
-Pattern "\w*?#my.domain.com"
$mPattern = "[a-zA-Z0-9._%+-]+#[a-zA-Z0-9.-]+(\.[a-zA-Z]{2,4})"
$lines = get-content "path\to\your.rtf"
foreach($line in $lines){
([regex]::MAtch($rtfInput, $mpattern, "IgnoreCase ")).value }
This code worked for me. My inital code but with a new search pattern.
$finds = Select-String -Path "path\to\my.rtf" -Pattern "[a-zA-Z0-9._%+-]+#[a-zA-Z0-9.-]+(\.[a-zA-Z]{2,4})" | ForEach-Object {$_.Matches}
$finds | Select-Object -First 10 | ft *
Thanks!