Select host names from log files - powershell

I have an application that generates 100's of text log files which are like
DaemonReruns=2|
Phase=|
Log=false|
DS=LOG_4|
Schema=LOLYY|
DBMS=mssql|
Host=abc.XYz.com|
IDs=xxxxx,xxxx
I need to select Host from these
I tried
GC C:\log_5.txt |
Select-String -Pattern 'Host=\"([^\"]*)\"'
Gives no results, any help ?

There aren't any quotes in your example input. Try this regex:
get-content C:\log_5.txt | foreach {
if ($_ -match 'Host=([^|]+)') {
$Matches.1
}
}
Note: This actually returns the host names, not just the line.

marsze's helpful answer fixes the problem with your regex and uses a ForEach-Object (foreach) call to extract and return matches via the -match operator and the automatic $Matches variable.
Here's a concise (and better-performing) alternative using the switch statement:
PS> switch -Regex -File C:\log_5.txt { 'Host=([^|]+)' { $Matches[1] } }
abc.XYz.com
Note that -File doesn't accept wildcard-based paths, however, so in order to process multiple file, you'll have to loop over them via Get-ChildItem or Convert-Path.

((Get-Content -Path .\log_5.txt) -match 'Host=') -replace 'Host=',''
returns all the lines starting with Host=

Just for fun ... the super-fast solution:
$regex = [Regex]::new('Host=([^|]+)', 'Compiled, IgnoreCase, CultureInvariant')
& {foreach ($line in [IO.File]::ReadLines("C:\log_5.txt")) {
$m = $regex.Match($line)
if ($m.Success) {
$m.Groups[1].Value
}
}}

If your logs are huge, it could be worth the overhead of Add-Type, and the rest would be much faster:
Add-Type '
using System.IO;
using System.Collections.Generic;
using System.Text.RegularExpressions;
namespace PowerShell
{
public class Tools
{
static Regex regex = new Regex(#"Host=([^|]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
public static IEnumerable<string> GetHosts(string path)
{
foreach(var line in File.ReadLines(path))
{
var matches = regex.Match(line);
if (matches.Success)
{
yield return matches.Groups[1].Value;
}
}
}
}
}'
# call this for each log file (very fast)
[PowerShell.Tools]::GetHosts("C:\log_5.txt")

Other answers have the regex side covered well enough. Whenever I see little logs like this I always think about ConvertFrom-StringData which
converts a string that contains one or more key and value pairs into a hash table.
From: help ConvertFrom-StringData
In its basic form we just do something like this:
$pairs = Get-Content -Raw -File $pathtofile | ConvertFrom-StringData
[pscustomobject]$pairs
Which would give you a PowerShell object that you can interact with easily!
DS : LOG_4|
Schema : LOLYY|
IDs : xxxxx,xxxx
Log : false|
DBMS : mssql|
Host : abc.XYz.com|
Phase : |
DaemonReruns : 2|
Doubtful that you need the trailing pipes. You can remove those with some regex or simpler string methods.
[pscustomobject](Get-Content -File $pathToFile | ForEach-Object{$_.trimend("|")} | Out-string | ConvertFrom-StringData)
[pscustomobject]((Get-Content -Raw -File $pathToFile) -replace "(?m)\|$" | ConvertFrom-StringData)
In any case this gives you more options as to how you need to deal with your data.

Related

How do I change foreach to for in PowerShell?

I want to print the word exist in a text file and print "match" and "not match". My 1st text file is: xxaavv6J, my 2nd file is 6J6SCa.yB.
If it is match, it return like this:
Match found:
Match found:
Match found:
Match found:
Match found:
Match found: 6J
Match found:
Match found:
Match found:
My expectation is just print match and not match.
$X = Get-Content "C:\Users\2.txt"
$Data = Get-Content "C:\Users\d.txt"
$Split = $Data -split '(..)'
$Y = $X.Substring(0, 6)
$Z = $Y -split '(..)'
foreach ($i in $Z) {
foreach ($j in $Split) {
if ($i -like $j) {
Write-Host ("Match found: {0}" -f $i, $j)
}
}
}
The operation -split '(..)' does not produce the result you think it does. If you take a look at the output of the following command you'll see that you're getting a lot of empty results:
PS C:\> 'xxaavv6J' -split '(..)' | % { "-$_-" }
--
-xx-
--
-aa-
--
-vv-
--
-6J-
--
Those empty values are the additional matches you're getting from $i -like $j.
I'm not quite sure why -split '(..)' gives you any non-empty values in the first place, because I would have expected it to produce 5 empty strings for an input string "xxaavv6J". Apparently it has to do with the grouping parentheses, since -split '..' (without the grouping parentheses) actually does behave as expected. Looks like with the capturing group the captured matches are returned on top of the results of the split operation.
Anyway, to get the behavior you want replace
... -split '(..)'
with
... |
Select-String '..' -AllMatches |
Select-Object -Expand Matches |
Select-Object -Expand Value
You can also replace the nested loop with something like this:
foreach ($i in $Z) {
if (if $Split -contains $i) {
Write-Host "Match found: ${i}"
}
}
A slightly different approach using regex '.Match()' should also do it.
I have added a lot of explaining comments for you:
$Test = Get-Content "C:\Users\2.txt" -Raw # Read as single string. Contains "xxaavv6J"
$Data = (Get-Content "C:\Users\d.txt") -join '' # Read as array and join the lines with an empty string.
# This will remove Newlines. Contains "6J6SCa.yB"
# Split the data and make sure every substring has two characters
# In each substring, the regex special characters need to be Escaped.
# When this is done, we join the substrings together using the pipe symbol.
$Data = ($Data -split '(.{2})' | # split on every two characters
Where-Object { $_.Length -eq 2 } | # don't care about any left over character
ForEach-Object { [Regex]::Escape($_) } ) -join '|' # join with the '|' which is an OR in regular expression
# $Data is now a string to use with regular expression: "6J|6S|Ca|\.y"
# Using '.Match()' works Case-Sensitive. To have it compare Case-Insensitive, we do this:
$Data = '(?i)' + $Data
# See if we can find one or more matches
$regex = [regex]$Data
$match = $regex.Match($Test)
# If we have found at least one match:
if ($match.Groups.Count) {
while ($match.Success) {
# matched text: $match.Value
# match start: $match.Index
# match length: $match.Length
Write-Host ("Match found: {0}" -f $match.Value)
$match = $match.NextMatch()
}
}
else {
Write-Host "Not Found"
}
Result:
Match found: 6J
Further to the excellent Ansgar Wiechers' answer: if you are running (above) Windows PowerShell 4.0 then you could apply the .Where() method described in Kirk Munro's exhaustive article ForEach and Where magic methods:
With the release of Windows PowerShell 4.0, two new “magic” methods
were introduced for collection types that provide a new syntax for
accessing ForEach and Where capabilities in Windows PowerShell.
These methods are aptly named ForEach and Where. I call
these methods “magic” because they are quite magical in how they work
in PowerShell. They don’t show up in Get-Member output, even if you
apply -Force and request -MemberType All. If you roll up your
sleeves and dig in with reflection, you can find them; however, it
requires a broad search because they are private extension methods
implemented on a private class. Yet even though they are not
discoverable without peeking under the covers, they are there when you
need them, they are faster than their older counterparts, and they
include functionality that was not available in their older
counterparts, hence the “magic” feeling they leave you with when you
use them in PowerShell. Unfortunately, these methods remain
undocumented even today, almost a year since they were publicly
released, so many people don’t realize the power that is available in
these methods.
…
The Where method
Where is a method that allows you to filter a collection of objects.
This is very much like the Where-Object cmdlet, but the Where
method is also like Select-Object and Group-Object as well,
includes several additional features that the Where-Object cmdlet
does not natively support by itself. This method provides faster
performance than Where-Object in a simple, elegant command. Like
the ForEach method, any objects that are output by this method are
returned in a generic collection of type
System.Collections.ObjectModel.Collection1[psobject].
There is only one version of this method, which can be described as
follows:
Where(scriptblock expression[, WhereOperatorSelectionMode mode[, int numberToReturn]])
As indicated by the square brackets, the expression script block is
required and the mode enumeration and the numberToReturn integer
argument are optional, so you can invoke this method using 1, 2, or 3
arguments. If you want to use a particular argument, you must provide
all arguments to the left of that argument (i.e. if you want to
provide a value for numberToReturn, you must provide values for
mode and expression as well).
Applied to your case (using the simplest variant Where(scriptblock expression) of the .Where() method):
$X = '6J6SCa.yB' # Get-Content "C:\Users\2.txt"
$Data = 'xxaavv6J' # Get-Content "C:\Users\d.txt"
$Split = ($Data -split '(..)').Where({$_ -ne ''})
$Y = $X.Substring(0, 6)
$Z = ($Y -split '(..)').Where{$_ -ne ''} # without parentheses
For instance, Ansgar's example changes as follows:
PS > ('xxaavv6J' -split '(..)').Where{$_ -ne ''} | % { "-$_-" }
-xx-
-aa-
-vv-
-6J-

PowerShell format text file

I get the below output from a PowerShell query. I don't have access to the server to run the query, so I have no option to influence the output format.
Example:
Name : folderC
FullName : D:\folderA\folderB\folderC
Length :
CreationTime : 2/8/2014 11:12:58 AM
LastAccessTime: 2/8/2014 11:12:58 AM
Name : filename.txt
FullName : D:\folderA\folderB\filename.txt
Length : 71560192
CreationTime : 11/25/2015 3:10:43 PM
LastAccessTime: 11/25/2015 3:10:43 PM
How can I format above content to get something more usable, maybe like a table format like so:
Name|FullName|Length|CreationTime|LastAccessTime
I think you need to split the text into records, replace the colons with equals so that you can use the ConvertFrom-StringData to turn each record into a hash which you can then feed into New-Object to convert into an object. Outputting the the object into pipe separated data can then be done with the ConvertTo-Csv. Something like so:
$x = #"
Name : folderC
FullName : D:\folderA\folderB\folderC
Length : 0
CreationTime : 2/8/2014 11:12:58 AM
LastAccessTime : 2/8/2014 11:12:58 AM
Name : filename.txt
FullName : D:\folderA\folderB\filename.txt
Length : 71560192
CreationTime : 11/25/2015 3:10:43 PM
LastAccessTime : 11/25/2015 3:10:43 PM
"#
($x -split '[\r\n]+(?=Name)') | % {
$_ -replace '\s+:\s+', '='
} | % {
$_ | ConvertFrom-StringData
} | % {
New-Object psobject -Property $_
} | ConvertTo-Csv -Delimiter '|' -NoTypeInformation
As #alroc notes in a comment on the question, it is possible that objects are available to the OP, given that they state that the output is "from a Powershell query" - if so, simple reformatting of the object array using the usual cmdlets is an option.
By contrast, this answer assumes that only a text representation, as printed in the question, is available.
Dave Sexton's answer is a simpler and more elegant choice, if:
the input has no empty values (the OP's sample input does).
the input file is small enough to be read into memory as a whole.
Consider the approach below to avoid the issues above and/or if you want more control over how the input is converted into custom objects, notably with respect to creating properties with types other than [string]: extend the toObj() function below (as written, all properties are also just strings).
Get-Content File | % `
-begin {
function toObj([string[]] $lines) {
$keysAndValues = $lines -split '(?<=^[^ :]+)\s*: '
$htProps = #{}
for ($i = 0; $i -lt $keysAndValues.Count; $i += 2) {
$htProps.($keysAndValues[$i]) = $keysAndValues[$i+1]
}
return [PSCustomObject] $htProps
}
$lines = #()
} `
-process {
if ($_.trim() -ne '') {
$lines += $_
} else {
if ($lines) { toObj $lines }
$lines = #()
}
} `
-end {
if ($lines) { toObj $lines }
} | Format-Table
Explanation:
Uses ForEach-Object (%) with separate begin, process, and end blocks.
The -begin, executed once at the beginning:
Defines helper function toObj() that converts a block of contiguous nonempty input lines to a single custom object.
toObj() splits an array of lines into an array of contiguous key-value elements, converts that array to a hashtable, which is then converted to a custom object.
Initializes array $lines, which will store the lines of a single block of contiguous nonempty input lines
The -process block, executed for each input line:
If the input line at hand is nonempty: Adds it to the current block of contiguous nonempty input lines stored in array $lines.
Otherwise: Submits the current block to toObj() for conversion to a custom object, and then resets the $lines array to start the next block. In effect, toObj() is invoked for each paragraph (run of nonempty lines).
The -end block, executed once at the end:
Submits the last paragraph to toObj() for conversion to a custom object.
Finally, the resulting array of custom objects is passed to Format-Table.

Powershell select filenames that match

List all filenames that match a pattern but can't seem to get it to list only the files I need I am eventually going to replace the filter with user input
$src = $env:ININ_TRACE_ROOT
$cmp = $env:COMPUTERNAME
$dst = $env:USERPROFILE + "\Desktop\" + $cmp
$lDate = Read-Host "Which Date?"
$s2 = $src + "\$ldate\"
$filter = "ip"
Get-ChildItem -Path $s2 | Where-Object { $_.Name -match $filter } | Select Name
Tried the above problem is it returns
accserver.zip
acdserver.zip
adminserver.zip
adminserver_1.zip
caasbillingserver.zip
caasproxyserver.zip
CallLog.zip
clientservices.zip
ClientStatsWkgQDataLog.zip
compressormanager.zip
datamanager.zip
dsserver.zip
fbmc.zip
hostserver.zip
httppluginhost.zip
i3runcrreport.zip
i3runcrreport_1.zip
i3runcrreport_2.zip
i3runcrreport_3.zip
imapconnector.zip
ininfaxserver.zip
interactionclient.zip
interactionrecoveryu.zip
ip.ininlog_journal
ip.zip
ipdbserver.ininlog_journal
ipdbserver.zip
ipserver.ininlog_journal
ipserver.zip
ip_1.zip
ip_10.zip
ip_11.zip
ip_12.zip
ip_13.zip
ip_14.zip
ip_2.zip
ip_3.zip
ip_4.zip
ip_5.zip
ip_6.zip
ip_7.zip
ip_8.zip
ip_9.zip
iwpserver.zip
LineGroupStatsDataLog.zip
mail account monitor.zip
mrcpsubsystem.zip
notifier.zip
notifierserver.zip
notifier_1.zip
notifier_2.zip
notifier_3.zip
optimizer server.zip
OutOfProcCustomDLL.zip
postofficeserver.zip
processautomationserver.zip
promptserver.zip
provisionserver.zip
QueuePeriodAgentStatsDataLog.zip
QueuePeriodWorkgroupStatsDataLog.zip
queuestatprovider.zip
recorder server.zip
RecoSubsystem.zip
remocoserver.zip
rstrapmonitor.zip
sessionmanager.zip
SIPEngine-mrcp.ininlog_journal
SIPEngine-mrcp.zip
SIPEngine.ininlog_journal
SIPEngine.zip
smsserver.zip
smtpconnector.zip
SNMPAgent.zip
statalertserver.zip
statserveragent.zip
statserveragent_1.zip
statserveragent_2.zip
statserverworkgroup.zip
statserverworkgroup_1.zip
statserverworkgroup_2.zip
surveyservice.zip
switchover.zip
switchoverfilemonitor.zip
tracker server.zip
tracker server_1.zip
transactionserver.zip
transactionserver_1.zip
transactionserver_2.zip
transactionserver_3.zip
transactionserver_4.zip
tsserver.zip
tsserver_1.zip
tsserver_2.zip
tsserver_3.zip
voicexml host server.zip
Problem is I need it to only return
ip.zip
ip_1.zip
ip_10.zip
ip_11.zip
ip_12.zip
ip_13.zip
ip_14.zip
ip_2.zip
ip_3.zip
ip_4.zip
ip_5.zip
ip_6.zip
ip_7.zip
ip_8.zip
ip_9.zip
Any ideas on how to achieve this
Updated using this now returns the list but is there a better way to do it?
$filter = "^ip[^server][^db][^ininlog_journal]"
Also this works but missing the non ip.zip
$filter = "^ip_[0-9]"
Since -match uses a regular expression, you should be able to write something like this:
get-childitem $s2 | where-object { $_.Name -match '^ip' }
(i.e., match when the Name property starts with ip).
See the about_Regular_Expressions help topic for more information.

Passing variables to subscripts

I was wondering if this was possible. I am trying to make a script we will refer to as a master script. This script queries a DB to get a list of servers we will call $svrs. Simple stuff.
The thing I don't know how to do or if it is possible is to run a series of subscripts from the master script using the $srvrs.Name variable as a parameter on those scripts.
$svrs = "get list sql stuff"
$scrpath = 'D:\test'
$scripts = Get-ChildItem $scrpath
$scripts.Name | ForEach-Object {
Invoke-Expression $_ {I have no idea how to get server name variable here}
}
Based on the comments, you do need a nested loop which won't be too complicated.
$Scripts | Select-object Name | % {$curScript = $_
$Servers | % {.\$_ $CurScript}
}
I ended up resolving this myself with #JNK 's assistance...
Here is how I got the result I needed.
$allServers | ForEach-Object {
$currentServer = $_
$scripts.Name | ForEach-Object {
Invoke-Expression ".\$_ $currentServer"
}
}

How to use a string value in foreach?

How to use a string value in foreach?
The following works.
$printString='$_.name+","+$_.name'
Get-ChildItem|foreach {$_.name+','+$_.name}
But the following doesn't work
Get-ChildItem|foreach {$printString}
But I need it to work: because I have a task to print each column in a table, I can use table dictionary to get all the columns, so all are dynamic, and then when I try to print the result, I also use a string like above to print the result. Any solution
There are several solutions. Some of them that came on my mid are:
$printString='$($_.name),$($_.name)'
Get-ChildItem | % { $ExecutionContext.InvokeCommand.ExpandString($printString) }
$formatString='{0},{0}'
Get-ChildItem | % { $formatString -f $_.Name }
$s = {param($file) $file.Name + "," + $file.Name }
Get-ChildItem | % { & $s $_ }
The first one expands string and that's probably what you wanted. Note that composed variables have to be enclosed in $(..). The second just formats some input. The third uses scriptblock there you can create any string you want (the most powerfull)
One possible solution:
$printString={$_.name+","+$_.name}
Get-ChildItem |foreach {.$printString}
Another possible solution:
$printString='$_.name+","+$_.name'
Get-ChildItem|foreach { Invoke-Expression $printString }
Interesting possibility:
dir | select #{Name='Name'; Expression={$_.Name, $_.Name}}