I've been given the duty of validating installation of an update across many hosts. This validation is performed by querying for a string of an error code that signifies success. I would like this output to both appear in the shell and also be written to a file.
$computerList = #($userInput)
foreach ($_ in $computerList){
get-content -tail 20 ("filepath") `
| where {$_| select-string "All steps complete!"} `
| where {$_| select-string "Output Error = 0 "} `
| out-file C:\users\me\Desktop\validation_log.txt -append
}
I based the multiple string "grep"-ing off of an online article,
However, this doesn't write the desired strings to the out-file path, nor does it display in console.
Can anyone please explain the best method for querying multiple strings and then outputting those to a file?
Your example is more complex than necessary.
You could just chain the Select-String. And Tee-Object is the way to go if you want to output something to both file and down the pipeline:
PS C:\temp> Get-Content -LiteralPath ".\input.txt"
All steps complete!
All steps complete! Output Error = 0
asdf
PS C:\temp> Get-Content -LiteralPath ".\input.txt" | Select-String -Pattern "All steps" | Select-String -Pattern "Output Error" | ForEach-Object {$_.ToString()} | Tee-Object -FilePath ".\output.txt" -Append
All steps complete! Output Error = 0
PS C:\temp> Get-Content -LiteralPath ".\output.txt"
All steps complete! Output Error = 0
The above behaves like an logical "and" for each pattern. If you want to "or" the patterns you could use the fact that the pattern is a regular expression:
PS C:\temp> Get-Content -LiteralPath ".\input.txt" | Select-String -Pattern "All steps|Output Error" | ForEach-Object {$_.ToString()} | Tee-Object -FilePath ".\output.txt" -Append
All steps complete!
All steps complete! Output Error = 0
Also notice that Select-String outputs Microsoft.PowerShell.Commands.MatchInfo objects and not strings. You will probably get unwanted newlines in your output if you pipe these directly to Tee-Object. I therefore convert these to strings in the Foreach-Object
Related
Data mapping project, in house system to new vendor system. First step is find all the occurrences of current database field names (or column names to be precise) in the C# .cs source files. Trying to use Powershell. Have recently created PS searches with Get-ChildItem and Select-String that work well but the search string array was small and easily hard coded inline. But the application being ported has a couple hundred column names and significant amounts of code. So armed with a text file of all the column names Pipleline would seem like a god tool to create a the basic cross ref for further analysis. However, I was not able to get the Pipeline to work with an external variable anyplace other than first step. Trying using -PipelineVariable, $_. and global variable. Did not find anything specific after lots of searching. P.S. This is my first question to StackoOverflow, be kind please.
Here is what I hoped would work but do dice so far.
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr = [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String $s | Select-Object Path, LineNumber, line | Export-csv $outputfile
}
Did find that this will print the list one time but not twice. In fact it seems using the variable in this way results in processing simply skipping any further pipeline steps.
foreach ($s in $arr) {Write-Host $s | Write $s}
If it isn't possible to do this in Powershell easily my fallback is to do with C# although would much rather get the level up with PowerShell if anyone can point me to the correct understanding of how to do things in the Pipepline, or alternatively construct an equivalent function. Seems like such a natural fit for Powershell.
Thanks.
You're calling Export-csv $outputfile in a loop, which rewrites the whole file in every iteration, so that only the last iteration's output will end up in the file.
While you could use -Append to iteratively append to the output file, it is worth aking a step back: Select-String can accept an array of patterns, causing a line that matches any of them to be considered a match.
Therefore, your code can be simplified as follows:
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
-Pattern (Get-Content $inputFile) passes the lines of input file $inputFile as an array of patterns to match.
By default, these lines are interpreted as regexes (regular expressions); to ensure that they're treated as literals, add -SimpleMatch to the Select-String call.
This answer to a follow-up question shows how to include the specific pattern among the multiple ones passed to -Pattern that matched on each line in the output.
I think you want to append each occurrence to the csv file. And you need to get the content of the file. Try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | Export-csv -Append -Path "$outputfile"
}
}
-Append was not introduced before powershell v3.0 (Windows 8) then try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -Path "$outputfile"
}
}
I have a log file which I am reading and getting what I need. But I have an additional trouble.
I want to get the line which is before the line which has keyword "Error"
I tried to use
Get-Content -Path $File -Tail 2 | Select-String -Pattern $Patterns -Context 1,0 -SimpleMatch
This is also giving me an output which I need. But after doing that I want to extract data from the two lines. Can I use a foreach loop on the output?
$Patterns = #('execution ended')
$File= "C:\GProcess\log.txt"
#-Context 1,0
Get-Content -Path $File -Tail 2 | Select-String -Pattern $Patterns -Context 1,0 -SimpleMatch
Current Output is
Process failed as there was an error
> Process execution ended
Maybe I just want Process failed as there was an error but search for keyword "execution ended"
I hope I have explained my query properly.
You can get just the line(s) of the context directly.
$match = Get-Content -Path $File | Select-String -Pattern $Patterns -Context 1,0 -SimpleMatch
$match.Context.PreContext
The below code will give you the line number of your string search, rather than using -tail.
If you have that then you can code the extra bit you need to grab the line above or below quite easily.
(Get-Content -Path 'C:\GProcess\log.txt' | Select-String -Pattern 'error').LineNumber
Hello my question is is there a faster way to search for a phrase in a file other than select-string. I need to find a certain phrase in the first line of about 60k files but the current way i am doing it is too slow for what i need to have done.
I have tried doing
(Select-String "Phrase I am looking for" (cat mylist1)).Filename > mylist2
which gave me a result of 2 minutes and 30 seconds and then i tried
cat mylist1| %{ if ((cat $_ -first 1) -match "Phrase I am looking for") {echo $_}} > mylist2
which gave me a result of 2 minute and 57 seconds. Is there another method of searching for a string through a large amount of files that can bring the search time down?
Since you have at least PowerShell 3.0 then you could use .Where with Get-Content's -TotalCount and that should help some. -TotalCount defines how many lines of the file are being read in. I see that you are already using its alias -First though so there won't be any big changes here for that.
$path = "d:\temp"
$matchingPattern = "function"
(Get-ChildItem $path -File).Where{(Get-Content $_ -TotalCount 1) -match $matchingPattern }
I will try and test this against 60K of files and see what I can get in the mean htim. The above would return file objects where the first line contains "function". My test ran against 60K of files but my lines were likely shorter. Still did it in 44 seconds so perhaps that will help you
StreamReader will usually beat out Get-Content as well but since we are only getting one line I don't think it will be more efficient. This uses a streamreader in the where clause and reads the first line.
(Get-ChildItem $path -File).Where{([System.IO.File]::OpenText($_.Fullname).ReadLine()) -match $matchingPattern }
Note that the above code could contain a memory leak but it finished in 8 seconds compared to my first test. Writing to file added a second or two. Your mileage will vary.
Note that -match supports regex so you would need to escape regex meta characters if present.
You can do simply it:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}}
or A short version for non-purists:
gc "c:\temp\*.*" -To 1 | ? { $_ -ilike "*$yoursearch*"} | select PSPath, {$_}
If you want export your result:
$yoursearch = "PowerShell is cool!"
get-content "c:\temp\*.*" -TotalCount 1 | where { $_ -ilike "*$yoursearch*"} | select PSPath, #{N="Founded";E={$_}} |
export-csv "c:\temp\yourresult.csv" -notype
If you want a better filter for files input :
Get-ChildItem "c:\temp" -File |
Where {$firstrow= (Get-Content $_.FullName -TotalCount 1); $firstrow -ilike "*$yoursearch*"} |
Select fullName, #{N="Founded";E={$firstrow}} |
Export-Csv "c:\temp\yourresult.csv" -notype
or A short version for non-purists:
gci "c:\temp" -File | ? {$r= (gc $_.FullName -TotalCount 1); $r -ilike "*$yoursearch*"} |
Select f*, #{N="Founded";E={$r}} |
epcsv "c:\temp\yourresult.csv" -notype
Note: -file option exist only in PowerShell V5 (or +), else use psiscontainer propertie into where instruction
Note2: You can use option -list of select-string, seach all in file but stop when 1 row is founded
$yoursearch = "PowerShell where are you"
Select-String -Path "c:\temp\*.*" -Pattern $yoursearch -list | select Path, Line | export-csv "C:\temp\result.csv" -NoTypeInformation
A quick way to write to a file is to use the StreamWriter object. Assuming the files are in a folder:
$writer = [System.IO.StreamWriter] "selection.txt"
$files = gci -Path $path
$pattern ="Phrase"
$files | %{gc -Path $_.FullName | select -First 1 | ?{$_ -match $pattern}} | %{$writer.WriteLine($_)}
An example on how I would do it would be something like
Get-ChildItem -Path $path | Where-Object{$_.Name -contains "My String"}
This is generally a pretty fast way of achieving this however be advised if you -recurse through the entire C:\ drive then regardless you will be sitting for a minute unless you multi-thread
Example Text
DESKTOP HARD DRIVE (S/N:9VMJ31W0)
I would like to find the text in the file
9VMJ31W0
Additional Example Text
SERVER HARD DRIVE (S/N:3NM2Y5HB)
SERVER HARD DRIVE (S/N:3NM2YXBD)
SERVER HARD DRIVE (S/N:6SD1MZFE)
SERVER HARD DRIVE (S/N:3NM2YX1Q)
SERVER HARD DRIVE (S/N:6SD1E8SA)
SERVER HARD DRIVE (S/N:3NM305ZQ)
SERVER HARD DRIVE (S/N:B365P760VG2F)
SERVER HARD DRIVE (S/N:B365P760VG54)
I would like the output file to read something like this
3NM2Y5HB
3NM2YXBD
6SD1MZFE
3NM2YX1Q
6SD1E8SA
3NM305ZQ
B365P760VG2F
B365P760VG54
then output this to a file in PowerShell preferably.
The files will be located in a specific folder, searching sub folders would be awesome but not necessary.
The output would be a single multiple lined .txt file.
Does anyone have an example file I could use to perform this? I found lots of things similar but nothing I was able to actually complete the whole task with.
#Clear output variable
$Output = #()
#Get your files
$Files = Get-ChildItem -Recurse -Path "*" -Exclude "Output.txt"
#Loop through files
$Files | ForEach-Object {
#Use Regular expression to match the desired serial number string
$Matched = Get-Content $_.FullName | Select-String -AllMatches 'S\/N:([A-Za-z0-9]*)'
#Loop through the matched strings
$Matched | ForEach-Object {
#Save to $Output the grouped (inner) string i.e. you want "9VMJ31W0" not "S/N:9VMJ31W0"
$Output += $_.Matches.Groups.Value
}
}
#Write output to file
$Output | Out-File Output.txt
A little searching pointed me to powershell's SubString method when dealing with strings. See this page for more information about strings.
PS C:\Scripts\updates> $f = gc C:\Scripts\p.txt
PS C:\Scripts\updates> $f
DESKTOP HARD DRIVE (S/N:9VMJ31W0)
DESKTOP HARD DRIVE (S/N:9VMJ31W1)
DESKTOP HARD DRIVE (S/N:9VMJ31W2)
PS C:\Scripts\updates> $f | GM
(Truncated)
Substring Method string Substring(int startIndex), str
PS C:\Scripts\updates> $f.substring(24,8) | out-file C:\Temp\HDDSerials.txt
PS C:\Scripts\updates> Get-Content C:\Temp\HDDSerials.txt
9VMJ31W0
9VMJ31W1
9VMJ31W2
For starters, this will output all the lines that contain "DESKTOP HARD DRIVE" in all the files in "D:\MyFolder" and subfolders (-Recurse) and append them to "D:\MyFolder\Output.txt".
Get-ChildItem -Recurse -Path "D:\MyFolder" -Exclude "Output.txt" |
% {Get-Content $_.FullName | Where-Object {$_ -like '*DESKTOP HARD DRIVE*'} |
Select-Object} | Out-File "D:\MyFolder\Output.txt"
Best to send output to a separate folder, or use the -Exclude to exclude it from being processed.
Regular Expressions are the way to do this as it doesn't matter where in the file the Serial number is, it will find it:
#Clear output variable
$Output = #()
#Get your files
$Files = Get-ChildItem -Recurse -Path "*" -Exclude "Output.txt"
#Loop through files
$Files | ForEach-Object {
#Use Regular expression to match the desired serial number string
$Matched = Get-Content $_.FullName | Select-String -AllMatches 'S\/N:([A-Za-z0-9]*)'
#Loop through the matched strings
$Matched | ForEach-Object {
#Save to $Output the grouped (inner) string i.e. you want "9VMJ31W0" not "S/N:9VMJ31W0"
$Output += $_.Matches.Groups[1].Value
}
}
#Write output to file
$Output | Out-File Output.txt
If you want to more specifically match "DESKTOP HARD DRIVE (S/N:9VMJ31W0)" and not just "S/N:9VMJ31W0" then you change the matches to:
Select-String -AllMatches 'DESKTOP HARD DRIVE \(S\/N:([A-Za-z0-9]*)\)'
Here is a one-liner:
Select-String -Path C:\temp\files\*.txt -Exclude output.txt -Pattern '(?<=S/N:)\w+(?=\))' -AllMatches |
Select-Object -ExpandProperty matches |
Select-Object -ExpandProperty Value |
Out-File -FilePath C:\temp\files\output.txt -Append
uses a lookbehind to find text after S\N: and a lookahead for the end brace )
Note: this assumes that your text is stored in text files *.txt
I run the following code using PowerShell to get a list of add/remove programs from the registry:
Get-ChildItem -path hklm:\software\microsoft\windows\currentversion\uninstall `
| ForEach-Object -Process { Write-Output $_.GetValue("DisplayName") } `
| Out-File addrem.txt
I want the list to be separated by newlines per each program. I've tried:
Get-ChildItem -path hklm:\software\microsoft\windows\currentversion\uninstall `
| ForEach-Object -Process { Write-Output $_.GetValue("DisplayName") `n } `
| out-file test.txt
Get-ChildItem -path hklm:\software\microsoft\windows\currentversion\uninstall `
| ForEach-Object {$_.GetValue("DisplayName") } `
| Write-Host -Separator `n
Get-ChildItem -path hklm:\software\microsoft\windows\currentversion\uninstall `
| ForEach-Object -Process { $_.GetValue("DisplayName") } `
| foreach($_) { echo $_ `n }
But all result in weird formatting when output to the console, and with three square characters after each line when output to a file. I tried Format-List, Format-Table, and Format-Wide with no luck. Originally, I thought something like this would work:
Get-ChildItem -path hklm:\software\microsoft\windows\currentversion\uninstall `
| ForEach-Object -Process { "$_.GetValue("DisplayName") `n" }
But that just gave me an error.
Or, just set the output field separator (OFS) to double newlines, and then make sure you get a string when you send it to file:
$OFS = "`r`n`r`n"
"$( gci -path hklm:\software\microsoft\windows\currentversion\uninstall |
ForEach-Object -Process { write-output $_.GetValue('DisplayName') } )" |
out-file addrem.txt
Beware to use the ` and not the '. On my keyboard (US-English Qwerty layout) it's located left of the 1.
(Moved here from the comments - Thanks Koen Zomers)
Give this a try:
PS> $nl = [Environment]::NewLine
PS> gci hklm:\software\microsoft\windows\currentversion\uninstall |
ForEach { $_.GetValue("DisplayName") } | Where {$_} | Sort |
Foreach {"$_$nl"} | Out-File addrem.txt -Enc ascii
It yields the following text in my addrem.txt file:
Adobe AIR
Adobe Flash Player 10 ActiveX
...
Note: on my system, GetValue("DisplayName") returns null for some entries, so I filter those out. BTW, you were close with this:
ForEach-Object -Process { "$_.GetValue("DisplayName") `n" }
Except that within a string, if you need to access a property of a variable, that is, "evaluate an expression", then you need to use subexpression syntax like so:
Foreach-Object -Process { "$($_.GetValue('DisplayName'))`r`n" }
Essentially within a double quoted string PowerShell will expand variables like $_, but it won't evaluate expressions unless you put the expression within a subexpression using this syntax:
$(`<Multiple statements can go in here`>).
I think you had the correct idea with your last example. You only got an error because you were trying to put quotes inside an already quoted string. This will fix it:
gci -path hklm:\software\microsoft\windows\currentversion\uninstall | ForEach-Object -Process { write-output ($_.GetValue("DisplayName") + "`n") }
Edit: Keith's $() operator actually creates a better syntax (I always forget about this one). You can also escape quotes inside quotes as so:
gci -path hklm:\software\microsoft\windows\currentversion\uninstall | ForEach-Object -Process { write-output "$($_.GetValue(`"DisplayName`"))`n" }
Ultimately, what you're trying to do with the EXTRA blank lines between each one is a little confusing :)
I think what you really want to do is use Get-ItemProperty. You'll get errors when values are missing, but you can suppress them with -ErrorAction 0 or just leave them as reminders. Because the Registry provider returns extra properties, you'll want to stick in a Select-Object that uses the same properties as the Get-Properties.
Then if you want each property on a line with a blank line between, use Format-List (otherwise, use Format-Table to get one per line).
gci -path hklm:\software\microsoft\windows\currentversion\uninstall |
gp -Name DisplayName, InstallDate |
select DisplayName, InstallDate |
fl | out-file addrem.txt
The option that I tend to use, mostly because it's simple and I don't have to think, is using Write-Output as below. Write-Output will put an EOL marker in the string for you and you can simply output the finished string.
Write-Output $stringThatNeedsEOLMarker | Out-File -FilePath PathToFile -Append
Alternatively, you could also just build the entire string using Write-Output and then push the finished string into Out-File.