I try to read big data log file, in folder C: \ log \ 1 \ i put 2 txt files, i need open-> read all file .txt and find with filter some text like whis: [text]
# Filename: script.ps1
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "[Error]"
if ($StringMatch) {out-file -filepath C:\log\outputlog.txt -inputobject $StringMatch}
}
# end of script
not work
Would doing something like a select-string work?
Select-String C:\Scripts\*.txt -pattern "SEARCH STRING HERE" | Format-List
Or if there are multiple files you are wanting to parse maybe use the same select-string but within a loop and output the results.
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = select-string $File -pattern "SEARCH STRING HERE"
if ($StringMatch) {out-file -filepath c:\outputlog.txt -inputobject $StringMatch}
}
This will print out the file name along with the line number in the file. I hope this is what you are looking for.
Remove-Item -Path C:\log\outlog.txt
$Files = Get-ChildItem "C:\log\1\" -Filter "*.txt"
foreach ($File in $Files)
{
$lineNumber = 0
$content = Get-Content -Path "C:\log\1\$File"
foreach($line in $content)
{
if($line.Contains('[Error]'))
{
Add-Content -Path C:\log\outlog.txt -Value "$File -> $lineNumber"
}
$lineNumber++
}
}
Code below works
It selects strings in txt files in your folder based on -SimpleMatch and then appends it to new.txt file.
Though i do not know how to put two simple matches in one line. Maybe someone does and can post it here
Select-String -Path C:\log\1\*.txt -SimpleMatch "[Error]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Select-String -Path C:\log\1\*.txt -SimpleMatch "[File]" -ca | select -exp line | out-file C:\log\1\new.txt -Append
Regards
-----edit-----
If you want to you may not append it anywhere just display - simply dont pipe it to out-file
use index then check it :
New-Item C:\log\outputlog.txt
$Files = Get-ChildItem "C:\log\1\" -Include "*.txt"
foreach ($File in $Files)
{
$StringMatch = $null
$StringMatch = Get-Content $File
if($StringMatch.IndexOf("[Error]") -ne -1)
{
Add-Content -Path C:\log\outputlog.txt -Value ($StringMatch+"
-------------------------------------------------------------
")
}
}
# end of script
Related
In a directory, there are files with the following filenames:
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
I want to iterate through the directory, and IF there is a filename that contains the string '_pn.mp3', I want to test if there is a similarly named file without the '_pn.mp3' in the same directory. If that file exists, I want to remove it.
In the above example, I'd want to remove:
ExampleFile.mp3
ExampleFile2.mp3
and I'd want to keep ExampleFile3.mp3
Here's what I have so far:
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path '$path' | Where-Object {! $_.PSIsContainer}
Foreach ($file in $files) {
If($file.Name -match $pattern){
# filename with _pn.mp3 exists
Write-Host $file.Name
# search in the current directory for the same filename without _pn
<# If(Test-Path $currentdir $filename without _pn.mp3) {
Remove-Item -Force}
#>
}
enter code here
You could use Group-Object to group all files by their BaseName (with the pattern removed), and then loop over the groups where there are more than one file. The result of grouping the files and filtering by count would look like this:
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1
Count Name Group
----- ---- -----
2 ExampleFile {ExampleFile.mp3, ExampleFile_pn.mp3}
2 ExampleFile2 {ExampleFile2.mp3, ExampleFile2_pn.mp3}
Then if we loop over these groups we can search for the files that do not end with the $pattern:
#'
ExampleFile.mp3
ExampleFile_pn.mp3
ExampleFile2.mp3
ExampleFile2_pn.mp3
ExampleFile3.mp3
'# -split '\r?\n' -as [System.IO.FileInfo[]] | Set-Variable files
$pattern = "_pn"
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$_.Group.Where({-not $_.BaseName.Endswith($pattern)})
}
This is how your code would look like, remove the -WhatIf switch if you consider the code is doing what you wanted.
$pattern = "_pn.mp3"
$files = Get-ChildItem -Path -Filter *.mp3 -File
$files | Group-Object { $_.BaseName.Replace($pattern,'') } |
Where-Object Count -GT 1 | ForEach-Object {
$toRemove = $_.Group.Where({-not $_.BaseName.Endswith($pattern)})
Remove-Item $toRemove -WhatIf
}
I think you can get by here by adding file names into a hash map as you go. If you encounter a file with the ending you are interested in, check if a similar file name was added. If so, remove both the file and the similar match.
$ending = "_pn.mp3"
$files = Get-ChildItem -Path $path -File | Where-Object { ! $_.PSIsContainer }
$hash = #{}
Foreach ($file in $files) {
# Check if file has an ending we are interested in
If ($file.Name.EndsWith($ending)) {
$similar = $file.Name.Split($ending)[0] + ".mp3"
# Check if we have seen the similar file in the hashmap
If ($hash.Contains($similar)) {
Write-Host $file.Name
Write-Host $similar
Remove-Item -Force $file
Remove-Item -Force $hash[$similar]
# Remove similar from hashmap as it is removed and no longer of interest
$hash.Remove($similar)
}
}
else {
# Add entry for file name and reference to the file
$hash.Add($file.Name, $file)
}
}
Just get a list of the files with the _pn then process against the rest.
$pattern = "*_pn.mp3"
$files = Get-ChildItem -Path "$path" -File -filter "$pattern"
Foreach ($file in $files) {
$TestFN = $file.name -replace("_pn","")
If (Test-Path -Path $(Join-Path -Path $Path -ChildPath $TestFN)) {
$file | Remove-Item -force
}
} #End Foreach
I'm tying to automate gci in order to work on each row in a config file, where for each row I have as first column the path, and following it a list of files. Something like this:
C:\Users\*\AppData\Roaming\* *.dll
C:\Test file.txt,file2.txt
This means that gci will search for:
*.dll in C:\Users*\AppData\Roaming*
file.txt in C:\Test
file2.txt in C:\Test
In order to do this I'm creating dynamically the where condition in the script below. Here the ps script I'm using
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$filter = $files -join(" -or `$_.Name` -like ")
$filter = "`$_.Name` -like " + $filter
echo "Searching Path: $path, Pattern: $filter" | out-file -append -encoding ASCII -filepath .\result.txt
if ($path.Contains("*"))
{
gci -Path $path -Recurse | Where {$filter} | Select -ExpandProperty FullName | Out-String -Width 2048 | out-file -append -encoding UTF8 -filepath .\result.txt
}
else
{
gci -Path $path | Where {$filter} | Select -ExpandProperty FullName | Out-String -Width 2048 | out-file -append -encoding UTF8 -filepath .\result.txt
}
}
The problem is that the where filter is not considered. All files are returned
First attempt, suggested by
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$filter = $files -join(" -or `$_.Name -like ")
$filter = "`$_.Name -like " + $filter
$gciParams = #{
Path = $Path
Recurse = $Path.Contains('*')
}
"Searching Path: $path, Pattern(s): [$($files -join ',')]" | Add-Content -Path .\result.txt -Encoding ASCII
Get-ChildItem #gciParams | Where $filter | Select -ExpandProperty FullName | Add-Content -Path .\result.txt -Encoding UTF8
}
If you want to create a piece of code and defer execution of it until later, you need a Script Block.
A Script Block literal in PowerShell is just {}, so for constructing script block to filter based on a single comparison, you'd want to define $filter like this:
$filter = {$_.Name -like $filter}
At which point you can pass it directly as an argument to Where-Object:
Get-ChildItem $path |Where-Object $filter
... but since you want to test against multiple wildcard patterns, we'll need to write a slightly different filtering routine:
$filter = {
# Store file name of file we're filtering
$FileName = $_.Name
# Test ALL the patterns in $files and see if at least 1 matches
$files.Where({$FileName -like $_}, 'First').Count -eq 1
}
Since the $filter block now references $files to get the patterns, we can simplify your loop as:
foreach($line in Get-Content .\List.txt) {
try {
$path,$files = $line.split(' ')
$files = $files.split(',')
}
catch {
$path = $line
$files = "*.*"
}
if([string]::IsNullOrEmpty($files)){
$files = "*.*"
}
$gciParams = #{
Path = $Path
Recurse = $Path.Contains('*')
}
"Searching Path: $path, Pattern(s): [$($files -join ',')]" | Add-Content -Path .\result.txt -Encoding ASCII
Get-ChildItem #gciParams | Where $filter | Select -ExpandProperty FullName | Add-Content -Path .\result.txt -Encoding UTF8
}
Note that we no longer need to re-define $filter everytime the loop runs - the condition is based on the value of $files at runtime, so you can define $filter once before entering the loop and then reuse $filter every time.
The "trick" with using #gciParams (which allows us to remove the big if/else block) is known as splatting, but you could achieve the same result with Get-ChildItem -Path:$Path -Recurse:$Path.Contains('*') :)
I want to filter lines according to specific word from file in powershell.
For example: the files animal1.txt and animal2.txt. Every file contain lines
dog
cat
dog
dog
bird
Then I want to create two derived files:
animal1_bak.txt that stores lines which contains the word 'dog' from animal1.txt
animal2_bak.txt that stores lines which contains the word 'dog' from animal2.txt
What I found on web is:
Select-String -Path "*.*" -Pattern "dog"
But the instruction to create the derived word is missing.
What can I do?
You can first get-content and use set-content like below
Get-Content -Path E:\KTDocs\Scripts\animal1.txt | where {
$_ -like '*dog*'} |Set-Content e:\animalbak.txt
try Something like this
select-string -Path "c:\temp\animal*.txt" -Pattern "dog" | Group Path | %{
$FileName="{0}_bak.txt" -f $_.Name
$_.Group.Line | select -unique | Out-File $FileName -Append
}
$folderpath = "D:\AnimalFolder" # your folder path here
$Allfiles = Get-ChildItem -Path $folderpath -Recurse -File -Force -ErrorAction SilentlyContinue |where{$_.Name -match ".txt"} |Select-Object -expandproperty FullName
foreach($filepath in $allfiles)
{
$Data = get-content $filepath
foreach($line in $data)
{
if($line -match "dog")
{
$newpath = $filepath.split('.')[0]
$getfullpath = $newpath + "_bak.txt"
$line | out-file $getfullpath -append
}
}
}
I have a set of .txt files in a folder. For example:
1.txt
2.txt
3.txt
These contain a date following by a filename. For example, 1.txt may contain:
06/11/2017 randomdocument.txt
06/12/2017 otherdocument.txt
07/11/2017 yadocument.txt
01/02/2017 randomdocument.txt
I want to:
get the line that matches a particular date regex pattern
write the line, and the path of the file it is in, to a new document.
My code does the first part. I've tried various iterations with no cigar with the second part. Any help would be appreciated.
Code
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$line = Get-ChildItem -recurse | Get-Content | Select-String $SearchPattern
$d = $line | Select-Object file.FullName
$d | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
Desired Output:
06/12/2017 randomdocument.txt in c:\users\john.smith\desktop\FORREPORT\1.txt
With minimal changes to the code:
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$lines = Get-Content $file | Select-String $SearchPattern
if($lines){
foreach($line in $lines){
"$line`t$($file.FullName)" | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
}
}
This prints each line on a new line in the new document. If you want all matching lines on a single line, remove the foreach:
Set-Location c:\users\john.smith\desktop\FORREPORT
$files = Get-ChildItem -recurse
$SearchPattern = "0[5,6]{1}/[0-9]\w{1,2}/2017"
Foreach ($file in $files) {
$lines = Get-Content $file | Select-String $SearchPattern
if($lines){
"$lines`t$($file.FullName)" | Add-Content -path c:\users\john.smith\desktop\ohsnap.txt
}
}
if will also check see that matched lines were actually found so that you don't get ouput of just $file.Fullname with $lines is empty.
PS Your question and code are the best examples I can think of to use this quote. heed it! ;-p
I have a lot of .csv Files and I want to write the filename into the same file, at the end of the last position.
For example in C:\CSV I have:
P0_0050569F52981EE39CEF8C857147E850.csv
P0_0050569F52981EE39CEF8D4825092850.csv
P0_0050569F52981EE39CEF8EE13B954850.csv
...and another thousand more of these files
In every one of this files I have some content:
P0_0050569F52981EE39CEF8C857147E850.csv = 365013;253;9001
I want to transform this: 365013;253;9001 to this:
365013;253;9001;
P0_0050569F52981EE39CEF8C857147E850.csv
I cannot find the error...
Get-ChildItem -Filter "*.csv" -Path "C:\CSV" -Recurse | % {
#Open file
$reader = New-Object System.IO.StreamReader $_.FullName
#Ignore first line
$reader.ReadLine() | out-null
#Get name
$filename = $filename
#Write
Add-Content ";" $filename
#Close stream
$reader.Close()
}
MickyB's option should work. Here's another way to do it:
Get-ChildItem -Filter "*.csv" -Path "C:\CSV" -Recurse | foreach {$contents = get-content $_.fullname; $contents +=";"+"`r`n"+$_.name; Set-Content $_.fullname -Value $contents }
This will happen the filename at the end of each CSV.
Get-ChildItem -Filter "*.csv" -Path "C:\CSV" -Recurse | % {
(";"+"`r`n"+$_.name) | Out-File -FilePath $_.fullname -Append
}