I've tried searching in text in parallel and no. Runtime is the same
To determine the execution time, I used a Measure-Command -Expression
$listlogtoday = Get-ChildItem -Path $folder\*.log | Where-Object Length -ne 0
$arraylist = $listlogtoday.Name
$tmp="code "
workflow Test-ParallelForeach
{
param(
[array[]]$arraylist
)
foreach -parallel ($namefile in $arraylist )
{
$text = ((Select-String -Path "C:\test_ps1\$namefile" -Pattern "text_text") -replace "(?s)^.*$tmp", '') -replace "text2_text2.",""
$allkey += $text
}
}
Test-ParallelForeach -arraylist $arraylist
$allkey = $allkey | select -Unique
write-host ($allkey -join "`n")
Related
I've got the below Powershell script, and I'm struggling to finish it off.
Get-ChildItem -Path 'C:\temp\xml' -Include '*.xml' |
ForEach-Object
{
$FileName = $_.Fullname
$Pattern = "</Date>"
$FileOriginal = Get-Content $FileName
$date = Get-Date
$DateStr = $date.ToString("yyyyMMdd")
[String[]] $FileModified = #()
Foreach ($Line in $FileOriginal)
{
$FileModified += $Line
if ($Line -match $pattern)
{
$FileModified += "<CDate>$DateStr</CDate>"
}
}
$FileModified = $FileModified -replace "CUR","PIT"
$FileModified = $FileModified -replace "Current","Time"
Set-Content $fileName $FileModified
}
When I attempt to run it, I get the following messages:
cmdlet ForEach-Object at command pipeline position 2
Supply values for the following parameters:
Process[0]:
Can anyone see what I'm doing wrong?
Put the curly brace on the same line as the foreach-object:
echo hi | foreach-object {
$_
}
This would fail, even in a script. It's a cmdlet, not a statement like "if", so it needs its -Process parameter on the same line.
echo hi | foreach-object
{
$_
}
I have a script that seems to work correctly only it works to good.
I have files that contain multiple lines with the string "PROCEDURE DIVISION.", with the period at the end.
What I need to do...
ONLY remove the [2nd occurrence] of the string "PROCEDURE DIVISION." if it's in the text file twice and bypass the file if it is only found once. I need to preserve the 1st occurrence and change/remove the 2nd occurrence.
I can find and replace all the occurrences easily, I have no clue how to replace only 1 of 2.
Is this possible using Powershell?
Here is my code so far...
Get-ChildItem 'C:\Temp\*.cbl' -Recurse | ForEach {#
(Get-Content $_ | ForEach { $_ -replace "PROCEDURE DIVISION\.", " "}) | Set-Content $_
}
UPDATE
I got this to work and it's not pretty.
The only problem is is is capturing the string in the comments section.
What I need to do is only count the string as a hit when it's found starting in position 8 on each line.
Is that possible?
Get-ChildItem 'C:\Thrivent\COBOL_For_EvolveWare\COBOL\COBOL\*.*' -Recurse | ForEach {
($cnt=(Get-Content $_ | select-string -pattern "PROCEDURE DIVISION").length)
if ($cnt -gt "1") {
(Get-Content $_ | ForEach { $_ -replace "PROCEDURE DIVISION\.", " "}) | Set-Content $_
$FileName = $_.FullName
Write-Host "$FileName = $cnt" -foregroundcolor green
}
There are potential issues with all of the provided answers. Reading a file using switch statement is likely going to be the fastest method. But it needs to take into account PROCEDURE DIVISION. appearing multiple times on the same line. The method below will be more memory intensive than using switch but will consider the multi-match, single line condition. Note that you can use -cmatch for case- sensitive matching.
# Matches second occurrence of match when starting in position 7 on a line
Get-ChildItem 'C:\Temp\*.cbl' -Recurse -File | ForEach-Object {
$text = Get-Content -LiteralPath $_.Fullname -Raw
if ($text -match '(?sm)(\A.*?^.{6}PROCEDURE DIVISION\..*?^.{6})PROCEDURE DIVISION\.(.*)\Z') {
Write-Host "Changing file $($_.FullName)"
$matches.1+$matches.2 | Set-Content $_.FullName
}
}
This maybe a bit of a hack, but it works. $myMatches = $pattern.Matches in the case below gives us 3 matches, $myMatches[1].Index is the position of the second occurrence of the string you want to replace.
$text = "Hello foo, where are you foo? I'm here foo."
[regex]$pattern = "foo"
$myMatches = $pattern.Matches($text)
if ($myMatches.count -gt 1)
{
$newtext = $text.Substring(0,$myMatches[1].Index) + "bar" + $text.Substring($myMatches[1].Index + "foo".Length)
$newtext
}
try this:
$Founded=Get-ChildItem 'C:\Temp\' -Recurse -file -Filter "*.cbl" | Select-String -Pattern 'PROCEDURE DIVISION.' -SimpleMatch | where LineNumber -GT 1 | select Path -Unique
$Founded | %{
$Nb=0
$FilePath=$_.Path
$Content=Get-Content $FilePath | %{
if($_ -like '*PROCEDURE DIVISION.*')
{
$Nb++
if ($Nb -gt 1)
{
$_.replace('PROCEDURE DIVISION.', '')
}
else
{
$_
}
}
else
{
$_
}
}
$Content | Set-Content -Path $FilePath
}
You could use switch for this:
Get-ChildItem -Path 'C:\Temp' -Filter '*.cbl' -File -Recurse | ForEach-Object {
$occurrence = 0
$contentChanged = $false
$newContent = switch -Regex -File $_.FullName {
'PROCEDURE DIVISION\.' {
$occurrence++
if ($occurrence -eq 2) {
$_ -replace 'PROCEDURE DIVISION\.', " "
$contentChanged = $true
}
else { $_ }
}
default { $_ }
}
# only rewrite the file if a change has been made
if ($contentChanged) {
Write-Host "Updating file '$($_.FullName)'"
$newContent | Set-Content -Path $_.FullName -Force
}
}
Here's a method for deleting a line that I thought would work
#earlier in the script
$inFile = Get-Content -Path ".\input.txt"
# ...Later... #
$inFile = Get-Content -path ".\input.txt" | where-object {$_ -notmatch $line}
set-content -path ".\input.txt" -Value $inFile
The problem is that the -notmatch parameter doesn't seem to work. The Get-Content cmdlet just copies all the content from input.txt, including $line. I've also tried changing the code to clear $inFile completley and create a temporary holder, but no dice.
Clear-Variable -name "inFile"
$holder = Get-Content -path ".\input.txt" | where-object {$_ -notmatch $line}
set-content -path ".\input.txt" -Value $holder
$inFile = Get-Content -path ".\input.txt"
Am I using -notmatch incorrectly? Here's the full text script for context.
Write-Host "Starting"
[bool] $keepRunning = 1
[bool] $everFound = 0
[bool] $searchComplete = 0
:main while($keepRunning)
{
$inFile = Get-Content -path ".\input.txt"
$completed = Get-Content -Path ".\output.txt"
$line = $inFile[0]
$holder
if($inFile.count -eq 1)
{
$line = $inFile
}
# create condition to check if $line matches any line in completed.txt
# if it does, skip this line and move on to the next line
:search while($everFound -eq 0 -and $searchComplete -eq 0)
{
#Write-Host "Outer loop"
foreach($url in $completed)
{
#Write-Host $line
#write-host $url
if ($line -eq $url)
{
Write-Host "`nThis file was already downloaded --Skipping to the next line"
$inFile = Get-Content -path ".\input.txt" | where-object {$_ -notmatch $line}
set-content -path ".\input.txt" -Value $inFile
$inFile = Get-Content -path ".\input.txt"
$line = $inFile[0]
$everFound = 1
break
}
}
if ($everFound -eq 1)
{
break
}
$searchComplete = 1
Write-Host "Search Complete`n"
}
Write-Host "Before the download--------"
Write-Host $everFound
Write-Host $searchComplete
if ($everFound -eq 0 -and $searchComplete -eq 1)
{
#download the files
$downloadCommand = "youtube-dl.exe --verbose --cookies .\cookies.txt `"$line`""
get-date
invoke-Expression $downloadCommand
#delete the url
add-content -Path ".\output.txt" -Value $line
$inFile = Get-Content -path ".\input.txt" | where-object {$_ -notmatch $line}
set-content -path ".\input.txt" -Value $inFile
write-host "`n"
get-date
Write-Host "Sleeping for 45mins"
#start-sleep -s 2700
}
$everFound = 0
$searchComplete = 0
Write-Host "-------------After the download!!"
Write-Host $everFound
Write-Host $searchComplete
# check if the file is empty. If it is, set the keepRunning flag to false and exit the main while loop
if($Null -eq $inFile)
{
$keepRunning = 0
}
}
Write-Host "Done"
Read-Host "Press the Enter Key to Exit"
EDIT:
$inFile contains a list of youtube URLs on each line. $line is assigned the value of the first line of $inFile
$line = $inFile[0]
Here is a youtube URL: https://www.youtube.com/watch?v=sB5zlHMsM7k
I also added some statements to output the values of $line right before the file. Someone please point me to the right direction.
Am I using -notmatch incorrectly?
You're using it incorrectly, if $line contains a substring to search for literally (as-is, verbatim) in the input file's lines, and that substring happens to contain regex metacharacters, such as . and $.
To use -match / -notmatch for literal substring matching, you must escape the substring:
$_ -notmatch [regex]::Escape($line)
If you want to match lines only in full, you must anchor the regex:
$_ -notmatch ('^' + [regex]::Escape($line) + '$')
Note that PowerShell has no operator for literal substring matching.
However, the System.String ([string]) type has a .Contains() method for literal substring matching, but not that, unlike PowerShell's operators, it is case-sensitive by default (there are overloads for case-insensitive matching, but only in PowerShell (Core) 7+):
-not $_.Contains($line) # case-sensitive, literal substring matching
# PS 7+ only: case-INsensitive, literal substring matching
-not $_.Contains($line, 'CurrentCultureIgnoreCase')
For full-line matching:
-not ($_.Length -eq $line.Length -and $_.Contains($line))
or:
-not $_.Equals($line, 'CurrentCultureIgnoreCase')
The advantage of using .Contains() is that it performs better than -match, though the latter offers much more flexibility.
Hi i have powershell where i am using a foreach-object and would like to skip the first iteration always. And i am using continue statement as well. But the current behaviour of continue is like break. Please suggest if something i am doing wrong here.
Below is the sample code.
$xmlfile = 'D:\testdirecotry\sample.xml'
[xml]$xmlcontent = (Get-Content $xmlfile)
$folderprefix = 'plm_z'
$regex = '<!--__AMAZONSITE id="(.+?)" instance="(.+?)"__-->'
$i=0
(Get-Content $xmlfile) | select-string -Pattern $regex | ForEach-Object {
write-host "Test Iterartion"
if($i -eq 0)
{
write-host "entering if loop"
write-host $i
$i++
write-host $i
continue
}
else
{
write-host "entering else loop"
write-host $_
$pscustomobject=#(
# write-host $_
$id = $_.Matches.Groups[1].Value
$instance = $_.Matches.Groups[2].Value
write-host "Do Something"
)
}
}
An easier way to skip the first object would potentially be to use Select-Object in the pipeline:
Get-Content $xmlfile |
Select-string -Pattern $regex |
Select-Object -Skip 1 |
ForEach-Object {
...
I am trying to create a log reader. The data looks like so:
2017-11-27 13:24:41,791 [8] INFO CTSipEndpoint.CLogger.provider.gsiplib [(null)] - -00001 [Info] Info | 4744 | REGISTERdialog[1] 2-e:5;t:1-3 (dn:85188)
2017-11-27 13:24:41,791 [8] INFO CTSipEndpoint.CLogger.provider.gsiplib [(null)] - -00001 [Info] Info | 4744 | REGISTERdialog[1] event 2 REG/accepted
I am trying to do the following:
Return only lines in the last 48 hours to query further.
From above return any lines that contain the following phrases: "error"
"device","does not exist", "Could not identify speaker!","warn"
So far i have only been able to get this to work in an inefficient way, which runs against the file for each phrase and appends an array. Unfortunately this means that the date time becomes non-sequential. I need to now sort the content object at the end of the script to it be in sequence, or find a way to run this query smarter. Here is my script for reference:
$logfile = "C:\users\test\desktop\programlogs.log"
$content = ""
cat $logfile |
Select-String "ERROR" -SimpleMatch |
select -expand line |
foreach {
$_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null
$error_time = [datetime]($matches[1]).split(",")[0]
if ($error_time -gt (Get-Date).AddDays(-2)) {
$content += $_ + "`n"
}
}
cat $logfile |
Select-String "device" -SimpleMatch |
select -expand line |
foreach {
$_ -match '(.+)\s\[(device)\]\s(.+)'| Out-Null
$error_time = [datetime]($matches[1]).split(",")[0]
if ($error_time -gt (Get-Date).AddDays(-2)) {
$content += $_ + "`n"
}
}
cat $logfile |
Select-String "does not exist" -SimpleMatch |
select -expand line |
foreach {
$_ -match '(.+)\s\[(does not exist)\]\s(.+)'| Out-Null
$error_time = [datetime]($matches[1]).split(",")[0]
if ($error_time -gt (Get-Date).AddDays(-2)) {
$content += $_ + "`n"
}
}
cat $logfile |
Select-String "Could not identify speaker!" -SimpleMatch |
select -expand line |
foreach {
$_ -match '(.+)\s\[(Could not identify speaker!)\]\s(.+)'| Out-Null
$error_time = [datetime]($matches[1]).split(",")[0]
if ($error_time -gt (Get-Date).AddDays(-2)) {
$content += $_ + "`n"
}
}
cat $logfile |
Select-String "Warn" -SimpleMatch |
select -expand line |
foreach {
$_ -match '(.+)\s\[(Warn)\]\s(.+)'| Out-Null
$error_time = [datetime]($matches[1]).split(",")[0]
if ($error_time -gt (Get-Date).AddDays(-2)) {
$content += $_ + "`n"
}
}
$content = $content | select -uniq
$file = "c:\temp\shortenedlog.txt"
$content| Add-Content -Path $file
Here's a short take, let me know if this helps or if needs tweaking:
$newlog=#()
$logfile = get-content C:\temp\programlogs.log
$searchFor="error","device","does not exist","Could not identify speaker!","warn"
foreach($line in $logfile){
if($line.Length -gt 18){
$datetime=date $line.substring(0,$line.indexof(","))
if(((date)-$datetime).TotalHours -lt 49){
$keep=$false
foreach($item in $searchFor){
if($line.contains($item)){ $keep=$true }
}
if ($keep){$newlog+=$line}
}
}
}
$newlog | sort | % {add-content C:\temp\NewLog.log $_}
Came up with a solution which I have fed in the days i am interested in, followed by the search criteria. Used this solution as works quickly.
#Set parameters.
$File = "c:\temp\RefinedLogs.txt"
$DateParam = (Get-Date).AddDays(-1).ToString('yyyy-MM-dd')
$DateParam1 = (Get-Date).ToString('yyyy-MM-dd')
$SearchForDate = #("$dateparam", "$dateparam1")
$SearchFor=#("error","device","does not exist","Could not identify speaker!","warn")
#Filter file with dates set in $SearchForDate.
$DateFiltered = Get-Content '.\MyAPP.log' | Select-String -Pattern $SearchForDate -SimpleMatch
#Filter variable for phrases set in $SearchFor.
$Content = $DateFiltered | Select-String -Pattern $SearchFor -SimpleMatch
#Make results readable
ForEach($line in $content){
$Object = "$line" + "`n"
$FinalResult += $Object
}
#Output results.
write-host $FinalResult