I have an directory full of Log´s
My current code is
(Get-ChildItem -Path |Select-string -Pattern -AllMatches).matches.count
That will get me an 8 as output. But i want a list of all Logs with an hit and how many was in them.
Like This: "File2020.09.02 17"
How can i do that?
try something like this:
$files = Get-Childitem -Path 'C:\xyz' -File
$out = foreach ($file in $files){
$content = Get-Content $file
$matchCount = ($content | Select-String -Pattern 'xyz' -AllMatches).matches.count
[PSCustomObject]#{
Filename = $file
Matches = $matchCount
}
}
$out
Since Select-String returns MatchInfo objects with a Filename property, you can use Group-Object to group objects by Filename. Group-Object inherently counts the occurrences of any passed properties.
Select-String -Path $path -Pattern $pattern -AllMatches |
Group-Object -Property FileName | Foreach-Objecet {
"{0} {1}" -f $_.Name,$_.Count
}
Group-Object's Name property contains the value of the grouped properties. Its Count property lists the number of occurrences where those grouped properties match among the piped in objects.
You can use a hash table to track the number of hits in your files.
$tracker = #{}
Select-String -Path $path -Pattern $pattern -AllMatches | Foreach-Object {
if ($tracker.Contains($_.Filename)) {
$tracker[$_.Filename]++
} else {
$tracker[$_.Filename] = 1
}
}
$tracker.GetEnumerator() | Foreach-Object {
"{0} {1}"-f $_.Key,$_.Value
}
Related
In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach
I'm tying to automate gci in order to work on each row in a config file, where for each row I have as first column the path, and following it a list of files. Something like this:
C:\Users\*\AppData\Roaming\* *.dll
C:\Test file.txt,file2.txt
This means that gci will search for:
*.dll in C:\Users*\AppData\Roaming*
file.txt in C:\Test
file2.txt in C:\Test
In order to do this I'm creating dynamically the where condition in the script below. Here the ps script I'm using
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$filter = $files -join(" -or `$_.Name` -like ")
$filter = "`$_.Name` -like " + $filter
echo "Searching Path: $path, Pattern: $filter" | out-file -append -encoding ASCII -filepath .\result.txt
if ($path.Contains("*"))
{
gci -Path $path -Recurse | Where {$filter} | Select -ExpandProperty FullName | Out-String -Width 2048 | out-file -append -encoding UTF8 -filepath .\result.txt
}
else
{
gci -Path $path | Where {$filter} | Select -ExpandProperty FullName | Out-String -Width 2048 | out-file -append -encoding UTF8 -filepath .\result.txt
}
}
The problem is that the where filter is not considered. All files are returned
First attempt, suggested by
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$filter = $files -join(" -or `$_.Name -like ")
$filter = "`$_.Name -like " + $filter
$gciParams = #{
Path = $Path
Recurse = $Path.Contains('*')
}
"Searching Path: $path, Pattern(s): [$($files -join ',')]" | Add-Content -Path .\result.txt -Encoding ASCII
Get-ChildItem #gciParams | Where $filter | Select -ExpandProperty FullName | Add-Content -Path .\result.txt -Encoding UTF8
}
If you want to create a piece of code and defer execution of it until later, you need a Script Block.
A Script Block literal in PowerShell is just {}, so for constructing script block to filter based on a single comparison, you'd want to define $filter like this:
$filter = {$_.Name -like $filter}
At which point you can pass it directly as an argument to Where-Object:
Get-ChildItem $path |Where-Object $filter
... but since you want to test against multiple wildcard patterns, we'll need to write a slightly different filtering routine:
$filter = {
# Store file name of file we're filtering
$FileName = $_.Name
# Test ALL the patterns in $files and see if at least 1 matches
$files.Where({$FileName -like $_}, 'First').Count -eq 1
}
Since the $filter block now references $files to get the patterns, we can simplify your loop as:
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$gciParams = #{
Path = $Path
Recurse = $Path.Contains('*')
}
"Searching Path: $path, Pattern(s): [$($files -join ',')]" | Add-Content -Path .\result.txt -Encoding ASCII
Get-ChildItem #gciParams | Where $filter | Select -ExpandProperty FullName | Add-Content -Path .\result.txt -Encoding UTF8
}
Note that we no longer need to re-define $filter everytime the loop runs - the condition is based on the value of $files at runtime, so you can define $filter once before entering the loop and then reuse $filter every time.
The "trick" with using #gciParams (which allows us to remove the big if/else block) is known as splatting, but you could achieve the same result with Get-ChildItem -Path:$Path -Recurse:$Path.Contains('*') :)
I want to filter lines according to specific word from file in powershell.
For example: the files animal1.txt and animal2.txt. Every file contain lines
dog
cat
dog
dog
bird
Then I want to create two derived files:
animal1_bak.txt that stores lines which contains the word 'dog' from animal1.txt
animal2_bak.txt that stores lines which contains the word 'dog' from animal2.txt
What I found on web is:
Select-String -Path "*.*" -Pattern "dog"
But the instruction to create the derived word is missing.
What can I do?
You can first get-content and use set-content like below
Get-Content -Path E:\KTDocs\Scripts\animal1.txt | where {
$_ -like '*dog*'} |Set-Content e:\animalbak.txt
try Something like this
select-string -Path "c:\temp\animal*.txt" -Pattern "dog" | Group Path | %{
$FileName="{0}_bak.txt" -f $_.Name
$_.Group.Line | select -unique | Out-File $FileName -Append
}
$folderpath = "D:\AnimalFolder" # your folder path here
$Allfiles = Get-ChildItem -Path $folderpath -Recurse -File -Force -ErrorAction SilentlyContinue |where{$_.Name -match ".txt"} |Select-Object -expandproperty FullName
foreach($filepath in $allfiles)
{
$Data = get-content $filepath
foreach($line in $data)
{
if($line -match "dog")
{
$newpath = $filepath.split('.')[0]
$getfullpath = $newpath + "_bak.txt"
$line | out-file $getfullpath -append
}
}
}
I need to take a slew of csv files from a directory and get them into an array in Powershell (to eventually manipulate and write back to a CSV).
The problem is there are 5 file types. I need around 8 columns from each. The columns are essentially the same, but have different headings.
Is there an easy way to do this? I started creating a custom object with my 8 fields, looping through the files importing each one, looking at the filename (which tells me the column names I need) and then a bunch of ifs to add it to my custom object array.
I was wondering if there is a simpler way...like with a template saying which columns from each file.
wound up doing this. It may have not been the most efficient, but works. I wound up writing out each file separately and combining at the end as PS really got bogged down (over a million rows combined).
$Newcsv = #()
$path = "c:\scrap\BWFILES\"
$files = gci -path $path -recurse -filter *.csv | Where-Object { ! ($_.psiscontainer) }
$counter=1
foreach($file in $files)
{
$csv = Import-Csv $file.FullName
if ($file.Name -like '*SAV*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"SV"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*TIME*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TM"}},TMBRCH,TMACCT,TMSHRT
}
if ($file.Name -like '*TRAN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"TR"}},DMBRCH,DMACCT,DMSHRT
}
if ($file.Name -like '*LN*')
{
$Newcsv = $csv | Select-Object #{Name="PRODUCT";Expression={"LN"}},LNBRCH,LNNOTE,LNSHRT
}
$Newcsv | Export-Csv "C:\scrap\$file.name$counter.csv" -force -notypeinformation
$counter++
}
get-childItem "c:\scrap\*.csv" | foreach {
$filePath = $_
$lines = $lines = Get-Content $filePath
$linesToWrite = switch($getFirstLine) {
$true {$lines}
$false {$lines | Select -Skip 1}
}
$getFirstLine = $false
Add-Content "c:\scrap\combined.csv" $linesToWrite
}
With a hashtable for reference, a little RegEx matching, and using the automatic variable $Matches in a ForEach-Object loop (alias % used) that could all be shortened to:
$path = "c:\scrap\BWFILES\"
$Reference = #{
'SAV' = 'SV'
'TIME' = 'TM'
'TRAN' = 'TR'
'LN'='LN'
}
Set-Content -Value "PRODUCT,BRCH,ACCT,SHRT" -Path 'c:\scrap\combined.csv'
gci -path $path -recurse -filter *.csv | Where-Object { !($_.psiscontainer) -and $_.Name -match ".*(SAV|TIME|TRAN|LN).*"}|%{
$Product = $Reference[($Matches[1])]
Import-CSV $_.FullName | Select-Object #{Name="PRODUCT";Expression={$Product}},*BRCH,#{l='Acct';e={$_.LNNOTE, $_.DMACCT, $_.TMACCT|?{$_}}},*SHRT | ConvertTo-Csv -NoTypeInformation | Select -Skip 1 | Add-Content 'c:\scrap\combined.csv'
}
That should produce the exact same file. Only kind of tricky part was the LNNOTE/TMACCT/DMACCT field since obviously you can't just do the same as like *SHRT.
In order, I have to:
1) grab all links from txt file
http://example1.htm http://example2.htm http://example3.htm ...
2) get source from each link
3) get my strings from source
4) export strings to csv
It works with one link. Example:
$topic1 = "kh_header.><b>((?<=)[^<]+(?=</b>))"
$topic2 = "<b>Numer ogłoszenia:\s([^;]+(?=;))"
Select-String -Path strona1.htm -pattern $topic1 | foreach-object {
$_.line -match $topic1 > $nul
$out1 = $matches[1]
}
Select-String -Path strona1.htm -pattern $topic2 | foreach-object {
$_.line -match $topic2 > $nul
$out2 = $matches[1]
}
echo $out1';'$out2';' | Set-content out.csv -force
, But I cant get it with many links in txt file. I try it:
$topic = "kh_header.><b>((?<=)[^<]+(?=</b>))"
$topic2 = "<b>Numer ogłoszenia:\s([^;]+(?=;))"
$folder = Get-ChildItem e:\sk\html
ForEach ($htmfile in $folder){
If ($_.extension -eq ".htm"){
$htmfile = ForEach-Object {
$WC = New-Object net.webclient
$HTMLCode = $WC.Downloadstring($_.fullname)
}
Select-String -Path $HTMLCode -pattern $topic | foreach-object {
$_.line -match $topic > $nul
$out1 = $matches[1]
}
Select-String -Path $HTMLCode -pattern $topic2 | foreach-object {
$_.line -match $topic2 > $nul
$out2 = $matches[1]
}
echo $out1';'$out2';' | Set-content out.csv -force
}
}
How can I get it?
When you use Select-String by default it only finds the first match on any particular line. You can use the AllMatches parameter to fix that e.g.:
foo.txt contains: "static void Main(string[] args)"
Select-String foo.txt -pattern '\W([sS]..)' -AllMatches |
Foreach {$_.Matches} |
Foreach {$_.Groups[1].Value}
Also, Select-String is line oriented so it won't find pattern matches across lines. In order to find those, you need to read in the file as a string string e.g.:
$text = [io.file]::readalltext("$pwd\foo.txt")
And then use some special regex directives e.g.:
$text | Select-String -pattern '(?si)\W([sS]..)' -AllMatches |
Foreach {$_.Matches} |
Foreach {$_.Groups[1].Value}