I have a set of .txt files in a folder. For example:
1.txt
2.txt
3.txt
These contain a date following by a filename. For example, 1.txt may contain:
06/11/2017 randomdocument.txt
06/12/2017 otherdocument.txt
07/11/2017 yadocument.txt
01/02/2017 randomdocument.txt
I want to:
get the line that matches a particular date regex pattern
write the line, and the path of the file it is in, to a new document.
My code does the first part. I've tried various iterations with no cigar with the second part. Any help would be appreciated.
Code
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$line = Get-ChildItem -recurse | Get-Content | Select-String $SearchPattern
$d = $line | Select-Object file.FullName
$d | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
Desired Output:
06/12/2017 randomdocument.txt in c:\users\john.smith\desktop\FORREPORT\1.txt
With minimal changes to the code:
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$lines = Get-Content $file | Select-String $SearchPattern
if($lines){
foreach($line in $lines){
"$line`t$($file.FullName)" | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
}
}
This prints each line on a new line in the new document. If you want all matching lines on a single line, remove the foreach:
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$lines = Get-Content $file | Select-String $SearchPattern
if($lines){
"$lines`t$($file.FullName)" | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
}
if will also check see that matched lines were actually found so that you don't get ouput of just $file.Fullname with $lines is empty.
PS Your question and code are the best examples I can think of to use this quote. heed it! ;-p
Related
cd 'A:\P\E\D'
$files = Get-ChildItem . *.CSV -rec
ForEach ($file in $files) {
(Get-Content $file -Raw) | ForEach-Object {
*some simple code*
} | Set-Content $file
}
How to modify this powershell script to locate only files starting with letters A/a to O/o and ending with .csv in specified directory cd?
I thought the solution below would work, but the test file M_K_O_X.CSV stored in the cd directory was not found and modified. The solution above will find and modify the file. It's possible that I have the regex expression wrong or the problem is somewhere else? I tried also this regex -- "[A-O]..CSV"
cd 'A:\P\E\D'
$files = Get-ChildItem . -rec | Where-Object { $_.Name -like "[a-oA-O]*.*.CSV" }
ForEach ($file in $files) {
(Get-Content $file -Raw) | ForEach-Object {
*some simple code*
} | Set-Content $file
}
Looking at your wildcard pattern, seems like you have an extra *. that shouldn't be there:
'M_K_O_X.CSV' -like '[a-oA-O]*.*.CSV' # False
'M_K_O_X.CSV' -like '[a-oA-O]*.CSV' # True
In this case you could simply use the -Include Parameter which supports character ranges. Also PowerShell is case insensitive by default, [a-oA-O]*.CSV can be reduced to [a-o]*.CSV:
Get-ChildItem 'A:\P\E\D' -Recurse -Include '[a-o]*.csv' | ForEach-Object {
($_ | Get-Content -Raw) | ForEach-Object {
# *some simple code*
} | Set-Content -LiteralPath $_.FullName
}
As commented, I would use the standard wildcard -Filter to filter for all files with a .csv extension.
Then pipe to a Where-Object clause in which you can use regex -match
$files = Get-ChildItem -Path 'A:\P\E\D' -Filter '*.csv' -File -Recurse |
Where-Object { $_.Name -match '^[a-o]' }
foreach ($file in $files) {
# switch `-Raw` makes Get-Content return a single multiline string, so no need for a loop
$content = Get-Content -Path $file.FullName -Raw
# *some simple code manipulating $content*
$content | Set-Content -Path $file.FullName
}
However, if these are valid csv files, I would not recommend using a pure textual manipulation on them, instead use Import-Csv -Path $file.FullName and work on the properties on each of the objects returned.
In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach
I try to read big data log file, in folder C: \ log \ 1 \ i put 2 txt files, i need open-> read all file .txt and find with filter some text like whis: [text]
# Filename: script.ps1
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "[Error]"
if ($StringMatch) {out-file -filepath C:\log\outputlog.txt -inputobject $StringMatch}
}
# end of script
not work
Would doing something like a select-string work?
Select-String C:\Scripts\*.txt -pattern "SEARCH STRING HERE" | Format-List
Or if there are multiple files you are wanting to parse maybe use the same select-string but within a loop and output the results.
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "SEARCH STRING HERE"
if ($StringMatch) {out-file -filepath c:\outputlog.txt -inputobject $StringMatch}
}
This will print out the file name along with the line number in the file. I hope this is what you are looking for.
Remove-Item -Path C:\log\outlog.txt
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$lineNumber = 0
$content = Get-Content -Path "C:\log\1\$File"
foreach($line in $content)
{
if($line.Contains('[Error]'))
{
Add-Content -Path C:\log\outlog.txt -Value "$File -> $lineNumber"
}
$lineNumber++
}
}
Code below works
It selects strings in txt files in your folder based on -SimpleMatch and then appends it to new.txt file.
Though i do not know how to put two simple matches in one line. Maybe someone does and can post it here
Select-String -Path C:\log\1\*.txt -SimpleMatch "[Error]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Select-String -Path C:\log\1\*.txt -SimpleMatch "[File]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Regards
-----edit-----
If you want to you may not append it anywhere just display - simply dont pipe it to out-file
use index then check it :
New-Item C:\log\outputlog.txt
$Files = Get-ChildItem "C:\log\1\" -Include "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = Get-Content $File
if($StringMatch.IndexOf("[Error]") -ne -1)
{
Add-Content -Path C:\log\outputlog.txt -Value ($StringMatch+"
-------------------------------------------------------------
")
}
}
# end of script
Hello I have this code I've been working on to loop through a folder add file name to command line parameter of an executeable then output the results to a text file..
The code works for one interation but does not seem to loop through all the files and append to the text file.
Can you take a look at my structure and see why it is not looping through all the files and appending.
Regards.
$Path = "C:\rawfiles"
$files = Get-ChildItem C:\rawfiles\*.001
ForEach ($file in $files) {
c:\outputfiles\ldump.exe $file.fullName > c:\outputfiles\test9.txt -Append
"=======End of Batch========" | Out-File c:\outputfiles\test9.txt -Append
}
You can't mix > with -Append. Try this instead:
$Path = "C:\rawfiles"
$files = Get-ChildItem C:\rawfiles\*.001
ForEach ($file in $files) {
c:\outputfiles\ldump.exe $file.fullName | Out-File c:\outputfiles\test9.txt -Append
"=======End of Batch========" | Out-File c:\outputfiles\test9.txt -Append
}
Or:
$Path = "C:\rawfiles"
$files = Get-ChildItem C:\rawfiles\*.001
ForEach ($file in $files) {
c:\outputfiles\ldump.exe $file.fullName >> c:\outputfiles\test9.txt
"=======End of Batch========" >> c:\outputfiles\test9.txt
}
You may want to add a line at the very beginning to delete or empty test9.txt.
How can I change the following code to look at all the .log files in the directory and not just the one file?
I need to loop through all the files and delete all lines that do not contain "step4" or "step9". Currently this will create a new file, but I'm not sure how to use the for each loop here (newbie).
The actual files are named like this: 2013 09 03 00_01_29.log. I'd like the output files to either overwrite them, or to have the SAME name, appended with "out".
$In = "C:\Users\gerhardl\Documents\My Received Files\Test_In.log"
$Out = "C:\Users\gerhardl\Documents\My Received Files\Test_Out.log"
$Files = "C:\Users\gerhardl\Documents\My Received Files\"
Get-Content $In | Where-Object {$_ -match 'step4' -or $_ -match 'step9'} | `
Set-Content $Out
Give this a try:
Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files" -Filter *.log |
Foreach-Object {
$content = Get-Content $_.FullName
#filter and save content to the original file
$content | Where-Object {$_ -match 'step[49]'} | Set-Content $_.FullName
#filter and save content to a new file
$content | Where-Object {$_ -match 'step[49]'} | Set-Content ($_.BaseName + '_out.log')
}
To get the content of a directory you can use
$files = Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files\"
Then you can loop over this variable as well:
for ($i=0; $i -lt $files.Count; $i++) {
$outfile = $files[$i].FullName + "out"
Get-Content $files[$i].FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}
An even easier way to put this is the foreach loop (thanks to #Soapy and #MarkSchultheiss):
foreach ($f in $files){
$outfile = $f.FullName + "out"
Get-Content $f.FullName | Where-Object { ($_ -match 'step4' -or $_ -match 'step9') } | Set-Content $outfile
}
If you need to loop inside a directory recursively for a particular kind of file, use the below command, which filters all the files of doc file type
$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc
If you need to do the filteration on multiple types, use the below command.
$fileNames = Get-ChildItem -Path $scriptPath -Recurse -Include *.doc,*.pdf
Now $fileNames variable act as an array from which you can loop and apply your business logic.
Other answers are great, I just want to add... a different approach usable in PowerShell:
Install GNUWin32 utils and use grep to view the lines / redirect the output to file http://gnuwin32.sourceforge.net/
This overwrites the new file every time:
grep "step[49]" logIn.log > logOut.log
This appends the log output, in case you overwrite the logIn file and want to keep the data:
grep "step[49]" logIn.log >> logOut.log
Note: to be able to use GNUWin32 utils globally you have to add the bin folder to your system path.