I am trying to extract several strings from log files. So far so good. But I don't know how to extract only the required details from that result.
Example
A log entry looks like this:
(DIAG:XMLRPC)(11:07:01 15/04/2019)(MEM:130590)(TID:3632)(USER:Administrator)(REMOTE:10.67.125.177:59032)(XmlRpc: called method 'QueryCreativeFilterInfoList'.31)
I can parse it with the the following code:
$output_file = 'C:\Copy-Test\logins.txt'
$regex = 'QueryCreativeFilterInfoList'
$files = Get-ChildItem "C:\Copy-Test\Logs"
foreach ($file in $files)
{
gc $file.FullName | select-string -Pattern $regex | Select-String -Pattern "Administrator" | Out-File -FilePath $output_file -Append
}
Now I would like to only output the data, time username but I don't know how. I saw some crazy regex stuff but that was way above of my skill level.
I would appreciate if someone could guide me on this
The easiest way would be to split the log entry string by )( delimiter and then print members of a resulting array. Something like this:
gc $file.FullName | select-string -Pattern $regex | Select-String -Pattern "Administrator"| foreach {
$a=$_ -split "\)\("
"$(($a[1] -split " ")[1,0] -join ' ') $($a[4].substring(5))"
}
Related
I have several single line xml files out of which I want to extract the contents of a particular tag <LIFNR>. I managed doing that using the following two lines of power shell:
get-content P:\Webservice\21*_*CR_d*.xml | foreach-object { $_ -replace '>', ">`r`n" } > xmls_newline.txt
get-content .\xmls_newline.txt | Select-String -Pattern '.*<\/LIFNR>'
However if I want to skip the step of creating an intermediary text file I cannot achieve the same result.
$xml = get-content P:\Webservice\21*_*CR_d*.xml | foreach-object { $_ -replace '>', ">`r`n" }
$xml | Select-String -Pattern '.*<\/LIFNR>'
or
get-content P:\Webservice\21*_*CR_d*.xml | foreach-object { $_ -replace '>', ">`r`n" } | Select-String -Pattern '.*<\/LIFNR>'
each just print the result of the string splitting in the foreach-object statement, the Select-String command is ignored.
Can somebody explain to me what is different about the latter two attempts compared to the working solution?
Get-Content outputs each line as a separate string. To emulate the same behavior, split the string after > instead of appending a linebreak:
Get-Content P:\Webservice\21*_*CR_d*.xml |ForEach-Object { $_ -split '(?<=>)' } | Select-String -Pattern '.*<\/LIFNR>'
The construct (?<=...) is a lookbehind assertion - this way, PowerShell will split on a zero-length string immediately after >
I am looping through all files matching the pattern maint-*.js.
These files contains tokens in the form of __MyTokenA__, __MyTokenB__, etc...so I would like to find all of these tokens. I've tried the regex below, but it doesn't find anything.
I'd like to store the tokens in an array. What would be the correct way ?
$files = Get-ChildItem dist/main-*.js
Foreach ($file in $files)
{
$matches = $file | Select-String ', "(?:\(__\))(.*?)(?:\(__\))"' -AllMatches
echo $matches
}
I think you can simplyfy your regex, then enumerate the matches property.
$file | Select-String '__(.*?)__' -AllMatches | ForEach-Object {
$_
$_.Matches | Select-Object Value
}
Data mapping project, in house system to new vendor system. First step is find all the occurrences of current database field names (or column names to be precise) in the C# .cs source files. Trying to use Powershell. Have recently created PS searches with Get-ChildItem and Select-String that work well but the search string array was small and easily hard coded inline. But the application being ported has a couple hundred column names and significant amounts of code. So armed with a text file of all the column names Pipleline would seem like a god tool to create a the basic cross ref for further analysis. However, I was not able to get the Pipeline to work with an external variable anyplace other than first step. Trying using -PipelineVariable, $_. and global variable. Did not find anything specific after lots of searching. P.S. This is my first question to StackoOverflow, be kind please.
Here is what I hoped would work but do dice so far.
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr = [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force |
Select-String $s | Select-Object Path, LineNumber, line | Export-csv $outputfile
}
Did find that this will print the list one time but not twice. In fact it seems using the variable in this way results in processing simply skipping any further pipeline steps.
foreach ($s in $arr) {Write-Host $s | Write $s}
If it isn't possible to do this in Powershell easily my fallback is to do with C# although would much rather get the level up with PowerShell if anyone can point me to the correct understanding of how to do things in the Pipepline, or alternatively construct an equivalent function. Seems like such a natural fit for Powershell.
Thanks.
You're calling Export-csv $outputfile in a loop, which rewrites the whole file in every iteration, so that only the last iteration's output will end up in the file.
While you could use -Append to iteratively append to the output file, it is worth aking a step back: Select-String can accept an array of patterns, causing a line that matches any of them to be considered a match.
Therefore, your code can be simplified as follows:
$inputFile = 'C:\DataColumnsNames.txt'
$outputFile = 'C:\DataColumnsUsages.txt'
Get-ChildItem C:\ProjectFolder -Filter *.cs -Recurse -Force -ea SilentlyContinue |
Select-String -Pattern (Get-Content $inputFile) |
Select-Object Path, LineNumber, line |
Export-csv $outputfile
-Pattern (Get-Content $inputFile) passes the lines of input file $inputFile as an array of patterns to match.
By default, these lines are interpreted as regexes (regular expressions); to ensure that they're treated as literals, add -SimpleMatch to the Select-String call.
This answer to a follow-up question shows how to include the specific pattern among the multiple ones passed to -Pattern that matched on each line in the output.
I think you want to append each occurrence to the csv file. And you need to get the content of the file. Try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | Export-csv -Append -Path "$outputfile"
}
}
-Append was not introduced before powershell v3.0 (Windows 8) then try this:
$inputFile = "C:\DataColumnsNames.txt"
$outputFile = "C:\DataColumnsUsages.txt"
$arr [string[]](Get-Content $inputfile)
foreach ($s in $arr) {
Get-ChildItem -Path "C:ProjectFolder\*" -Filter *.cs -Recurse -ErrorAction SilentlyContinue -Force | Foreach {
Get-Content "$_.Fullname" | Select-String $s | Select-Object Path, LineNumber, line | ConvertTo-CSV -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -Path "$outputfile"
}
}
I have a text file that looks like this:
Data I'm NOT looking for
More data that doesn't matter
Even more data that I don't
&Start/Finally the data I'm looking for
&Data/More data that I need
&Stop/I need this too
&Start/Second batch of data I need
&Data/I need this too
&Stop/Okay now I'm done
Ending that I don't need
Here is what the output needs to be:
File1.txt
&Start/Finally the data I'm looking for
&Data/More data that I need
&Stop/I need this too
File2.txt
&Start/Second batch of data I need
&Data/I need this too
&Stop/Okay now I'm done
I need to do this for every file in a folder (sometimes there will be multiple files that will need to be filtered.) The files names can be incrementing: ex. File1.txt, File2.txt, File3.txt.
This is what I have tried with no luck:
ForEach-Object{
$text -join "`n" -split '(?ms)(?=^&START)' -match '^&START' |
Out-File B:\PowerShell\$filename}
Thanks!
Looks like you were pretty close: your code correctly extracted the paragraphs of interest, but intra-paragraph out-filtering of non-&-starting lines was missing, and you needed to write to paragraph-specific output files:
$text -join "`n" -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object { $ndx=0 } { $_ -split '\n' -match '^&' | Out-File "File$((++$ndx)).txt" }
This creates sequentially numbered files starting with File1.txt for every paragraph of interest.
To do it for every file in a folder, with output filenames using fixed naming scheme File<n> across all input files (and thus cumulative numbering):
Get-ChildItem -File . | ForEach-Object -Begin { $ndx=0 } -Process {
(Get-Content -Raw $_) -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object { $_ -split '\n' -match '^&' | Out-File "File$((++$ndx)).txt" }
}
To do it for every file in a folder, with output filenames based on the input filenames and numbering per input file (PSv4+, due to use of -PipelineVariable):
Get-ChildItem -File . -PipelineVariable File | ForEach-Object {
(Get-Content -Raw $_) -split '(?m)(?=^&Start)' -match '^&Start' |
ForEach-Object {$ndx=0} { $_ -split '\n' -match '^&' | Out-File "$($File.Name)$((++$ndx)).txt" }
}
You post a second question (against the rules) and it was deleted but here is my quick answer for it. I hope it will help you and give you more sense how PS works:
$InputFile = "C:\temp\test\New folder (3)\File1.txt"
# get file content
$a=Get-Content $InputFile
# loop for every line in range 2 to last but one
for ($i=1; $i -lt ($a.count-1); $i++)
{
#geting string part between & and / , and construct output file name
$OutFile = "$(Split-Path $InputFile)\$(($a[$i] -split '/')[0] -replace '&','').txt"
$a[0]| Out-File $OutFile #creating output file and write first line in it
$a[$i]| Out-File $OutFile -Append #write info line
$a[-1]| Out-File $OutFile -Append #write last line
}
Something like this?
$i=0
gci -path "C:\temp\ExplodeDir" -file | %{ (get-content -path $_.FullName -Raw).Replace("`r`n`r`n", ";").Replace("`r`n", "~").Split(";") | %{if ($_ -like "*Start*") {$i++; ($_ -split "~") | out-file "C:\temp\ResultFile\File$i.txt" }} }
I am trying to find all files, which does not contains a selected string. Find files which contains is easy:
gci | select-string "something"
but I do not have an idea how to negate this statement.
You can use Where-Object;
gci | Where-Object { !( $_ | Select-String "something" -quiet) }
I'm not sure if it can be done without the foreach-object but this works:
gci |foreach-object{if (-not (select-string -inputobject $_ -Pattern "something")){$_}}
As mentionend in How do I output lines that do not match 'this_string' using Get-Content and Select-String in PowerShell?
Select-String has the NotMatch parameter.
So you could use it:
gci | Select-String -notmatch "something"
foreach($line in Get-Content .\file.txt)
{
if(findstr $line "dummyText.txt" ){
# Work here or skip here
}
else {
echo $line
}