I need to match the a pattern i.e. "Commodity Name" and get the string in the next line between the patterns "<dd>" "</dd>".
Sample Input file:
C:\Users\rpm\Desktop\sample.txt:133: <dt>Commodity Name</dt>
C:\Users\rpm\Desktop\sample.txt:134: <dd>Grocery</dd>
C:\Users\rpm\Desktop\sample.txt:136: <dt>IP address</dt>
C:\Users\rpm\Desktop\sample.txt:137: <dd>XXX.XXX.XXX.XXX port 8000</dd>
C:\Users\rpm\Desktop\sample.txt:144: <dt>Commodity Serial #</dt>
C:\Users\rpm\Desktop\sample.txt:145: <dd>0055500000</dd>
C:\Users\rpm\Desktop\sample.txt:147: <dt>Client IP</dt>
C:\Users\rpm\Desktop\sample.txt:148: <dd>xxx.xxx.xxx.xxx</dd>
C:\Users\rpm\Desktop\sample.txt:150: <dt>Client Logged In As</dt>
C:\Users\rpm\Desktop\sample.txt:151: <dd>rpm123</dd>
C:\Users\rpm\Desktop\sample.txt:153: <dt>User is member of</dt>
C:\Users\rpm\Desktop\sample.txt:154: <dd>BP-RPM\COMD_CSO_ITM-AVAI_Def,BP-RPM\user</dd>
Need to match patterns such as
Commodity Name
IP address
Commodity Serial #
Client IP
Client Logged In As
User is member of
and get the values in the next line of the matched patterns between the tags <dd> & </dd>.
Desired output:
Grocery | XXX.XXX.XXX.XXX port 8000 | 0055500000 | xxx.xxx.xxx.xxx | rpm123 | BP-RPM\COMD_CSO_ITM-AVAI_Def,BP-RPM\user
I would start to create an array defining your keywords:
$keywords = #(
'<dt>Commodity Name</dt>'
'<dt>IP address</dt>'
'<dt>Commodity Serial #</dt>'
'<dt>Client IP</dt>'
'<dt>Client Logged In As</dt>'
'<dt>User is member of</dt>'
)
Now you can join the keywords by an | to use it with the Select-String cmdlet:
$file = 'C:\Users\rpm\Desktop\sample.txt'
$content = Get-Content $file
$content | Select-String -Pattern ($keywords -join '|')
This will give you the line number of each matched keyword. Now you can iterate over the result, access the next line by index and crop the <dd> pre and </dd> postifx:
ForEach-Object {
[regex]::Match($content[$_.LineNumber], '<dd>(.+)</dd>').Groups[1].Value
}
Regex:
Output:
Grocery
XXX.XXX.XXX.XXX port 8000
0055500000
xxx.xxx.xxx.xxx
rpm123
BP-RPM\COMD_CSO_ITM-AVAI_Def,BP-RPM\user
Finally you have to join the result by | to get the desired output. Here is the whole script:
$keywords = #(
'<dt>Commodity Name</dt>'
'<dt>IP address</dt>'
'<dt>Commodity Serial #</dt>'
'<dt>Client IP</dt>'
'<dt>Client Logged In As</dt>'
'<dt>User is member of</dt>'
)
$file = 'C:\Users\rpm\Desktop\sample.txt'
$content = Get-Content $file
($content | Select-String -Pattern ($keywords -join '|') |
ForEach-Object {
[regex]::Match($content[$_.LineNumber], '<dd>(.+)</dd>').Groups[1].Value
}) -join ' | '
Output:
Grocery | XXX.XXX.XXX.XXX port 8000 | 0055500000 | xxx.xxx.xxx.xxx | rpm123 | BP-RPM\COMD_CSO_ITM-AVAI_Def,BP-RPM\user
Related
This question is more about my understanding Powershell's objects rather than solving this practical example. I know there are other ways of separating out a page number from a string.
In my example I want to do this by accessing the object-match-value of the piped pattern match.
# data
$headerString = 'BARTLETT-BEDGGOOD__PAGE_5 BEECH-BEST__PAGE_6'
# require the number of page only
$regexPageNum = '([0-9]$)'
# split the header string into two separate strings to access page numbers
[string[]]$pages = $null
$pages = $headerString -split ' '
# access page numbers using regex pattern
$pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object {$_.Matches.Value}
The output is:
$_.Matches.Value
----------------
5
Okay. So far so good. I see the page number of array member pages[0] But how do I take this value from the object? The following does not work.
$x = $pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object {$_.Matches.Value}
Write-Host "Here it is:"$x
Output:
Here it is: #{$_.Matches.Value=5}
Instead of assigning the value 5 to the variable $x Powershell assigns, what looks to me: a hash table with an object description as its only member?
But if I try to access my variable using "Brackets for Access" Reference: hashtables Powershell indicates that variable $x is in fact an array.
x = $pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object {$_.Matches.Value}
Write-Host "Here it is:"$x
$y = $x[$_.Matches.Value]
Write-Host "What about now:"$y
Output:
Here it is: #{$_.Matches.Value=5}
InvalidOperation:
Line |
33 | $y = $x[$_.Matches.Value]
| ~~~~~~~~~~~~~~~~~~~~~~~~~
| Index operation failed; the array index evaluated to null.
What about now:
Okay. At this stage I know I'm being silly. But the point I'm trying to make is: How can I retrieve the value I want when I'm done with the Powershell object?
You can use $x.{ $_.Matches.Value } to access the value.
$x = $pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object { $_.Matches.Value }
$x.{ $_.Matches.Value } # This will print 5
ie, You would have to wrap the property name inside {} since the property name contains "."
Instead of this way, I would suggest you to create a calculated property using Select-Object which makes the code more readable.
$x = $pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object #{Name = 'PageNumber'; Expression = {$_.Matches.Value}}
$x.PageNumber
#Access matches in case of single match
$x = "red blue yellow green" | select-string -Pattern 'blue'
$x.matches.value
#Output
blue
#Access matches in case of multi match
$x = "red blue yellow green blue" | select-string -Pattern 'blue' -AllMatches
$x.matches.value
#Output
blue
blue
When you use a scriptblock as a parameter to Select-Object the return value will contain a property whose name matches the source code of the script block...
PS> #{ "aaa" = "bbb" } | select-object { $_.aaa; <# xxx #> }
$_.aaa; <# xxx #>
-------------------
bbb
In this pathological case, if I want to access the property I can't use the name in the default "dotted" notation because it contains reserved characters, but you can access it if you quote the property name:
PS> $x = #{ "aaa" = "bbb" } | select-object { $_.aaa; <# xxx #> }
# note the leading and trailing spaces in the string because the
# the original scriptblock source contains spaces between the "{" and "}"
PS> $x.' $_.aaa; <# xxx #> '
bbb
In your case you'd do this:
PS> $x = $pages[0] | Select-String -AllMatches -Pattern $regexPageNum | Select-Object {$_.Matches.Value}
PS> $x.'$_.Matches.Value'
Other options work too:
$x = $pages[0] `
| Select-String -AllMatches -Pattern $regexPageNum `
| Select-Object {$_.Matches.Value}
# get the property whose name is contained in the $name variable
PS> $name = '$_.Matches.Value'
PS> $x.$name
5
# the scriptblock gets converted into a string, and then that string
# is used as a property name
PS> $x.{$_.Matches.Value}
5
# note the whitespace in both scriptblocks has to match *exactly* otherwise the property name won't be found
PS> $x.{ $_.Matches.Value }
ParentContainsErrorRecordException: The property ' $_.Matches.Value ' cannot be found on this object. Verify that the property exists.
but...
There's an easier way - if you pass a hashtable to Select-Object instead of a scriptblock you can specify the name of the property - e.g.
PS> $x = $pages[0] `
| Select-String -AllMatches -Pattern $regexPageNum `
| Select-Object #{ "l"="Count"; "e"={$_.Matches.Value} }
PS> $x
Count
-----
5
PS> $x.Count
5
References:
about_Calculated_Properties - Hashtable key definitions
Need to take my $sw CSV file and use foreach to compare that against a hash translation table $swtranslation, Key field, then output matches including the hash table's values that match into a text file.
Problem I have is it runs the search for a few minutes and returns the sw_names.txt output file with nothing in it. It should have well over 1074+ matches. My guess is my syntax or something is not right.
See code for what I have going so far.
# This is the CSV file listing all the network switches I need to run against the translation table.
$sw = Import-Csv .\AllDeviceForExport.csv -Header Host_Name, IP_Address
# Compile the switch translation table for processing and convert to hash //
$swtranslation = #{};
Import-Csv .\sw_translation.csv -Header Host_Name, DataSpace_ID | % {
$swhash[$_.Host_Name] = $_.DataSpace_ID
}
# Run the Switch listing $sw against the translation table $swtranslation
# matching the DataSpace_ID and merging DataSpace_ID and Host name and
# all other switch fields together in output //
foreach ($key in $swhash.Keys) {
$sw | Select-Object #{n="Name";e={$outputhash[$swhash.Keys($_.Host_Name).Value]}},* |
Where-Object { $_.Name -ne $null } |
Foreach { $_ -replace '--' } |
Out-File ./sw_names.txt -Force
}
Expected results:
Host_Name DataSpace_ID
ABC-123-3750-SW1 1
DEF-234-2950-SW1 5
DEF-234-2950-SW2 5
GHI-567-4510-SW1 6
GHI-567-4510-SW2 6
It's unclear what you are after.
You have two csv files without headers,
.\AllDeviceForExport.csv -Header Host_Name, IP_Address
.\sw_translation.csv -Header Host_Name, DataSpace_ID
Usually one builds a hash table from one file and iterates the other to check if there are matching properties or not.
What your code tries to do is building the hash table, iterate the keys of it and then (very inefficiently) on each key search the whole other file thwarting the whole idea.
Not knowing which files Host_Name property should be checked I suggest a different approach:
Use Compare-Object
## Q:\Test\2019\08\15\SO_57515952.ps1
# simulate $swtrans = Import-Csv .\sw_translation.csv -Header Host_Name, DataSpace_ID
$swtrans = #"
ABC-123-3750-SW1,1
DEF-234-2950-SW1,5
DEF-234-2950-SW2,5
GHI-567-4510-SW1,6
GHI-567-4510-SW2,6
"# -split '\r?\n' | ConvertFrom-Csv -Header Host_Name, DataSpace_ID
# simulate $sw = Import-Csv .\AllDeviceForExport.csv -Header Host_Name, IP_Address
$sw = #"
DEF-234-2950-SW1,192.168.234.1
DEF-234-2950-SW2,192.168.234.2
GHI-567-4510-SW1,192.168.567.1
GHI-567-4510-SW2,192.168.567.2
GHI-567-4510-SW3,192.168.567.3
"# -split '\r?\n' | ConvertFrom-Csv -Header Host_Name, IP_Address
Compare-Object -Ref $swtrans -Diff $sw -Property Host_Name -PassThru -IncludeEqual
This yields:
> Q:\Test\2019\08\15\SO_57515952.ps1
Host_Name DataSpace_ID SideIndicator
--------- ------------ -------------
DEF-234-2950-SW1 5 ==
DEF-234-2950-SW2 5 ==
GHI-567-4510-SW1 6 ==
GHI-567-4510-SW2 6 ==
GHI-567-4510-SW3 =>
ABC-123-3750-SW1 1 <=
The SideIndicator Property can be used to specify which lines to output and itself suppressed.
I am attempting to use netstat -bano and collect the output in PowerShell for some very specific reporting requirements.
I have working regex that should be able to parse this output no problem, but since the output appears on multiple lines, the regex isn't being processed correctly
here's a screenshot of how it comes out of netstat
desired output is something like this (all on one line):
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 1092 RpcSs [svchost.exe]
TCP 0.0.0.0:445 0.0.0.0:0 LISTENING 4 Can not obtain ownership information
TCP 0.0.0.0:623 0.0.0.0:0 LISTENING 7404 [LMS.exe]
TCP 0.0.0.0:3389 0.0.0.0:0 LISTENING 1224 TermService [svchost.exe]
the use of tools outside of Windows isn't possible, so I'm confined to common tools.
Using Get-Process matching on PID also won't work, as it hides sub process information under svchost and lsass. netstat with a -b is perfect because it shows both svchost.exe and the process that utilizes the port
I've scoured the internet to find a viable solution but most end in a different resolution
EDIT**here is my final script using input from you guys
$data = (netstat -bano |select -skip 4 | Out-String) -replace '(?m)^ (TCP|UDP)', '$1' -replace '\r?\n\s+([^\[])', "`t`$1" -replace '\r?\n\s+\[', "`t[" -split "`n"
[regex]$regex = '(?<protocol>TCP|UDP)\s+(?<address>\d+.\d+.\d+.\d+|\[::\]|\[::1\]):(?<port>\d+).+(?<state>LISTENING|\*:\*)\s+(?<pid>\d+)\s+(?<service>Can not obtain ownership information|\[\w+.exe\]|\w+\s+\[\w+.exe\])'
$output = #()
$data | foreach {
$_ -match $regex
$outputobj = #{
protocol = [string]$matches.protocol
address = [string]$matches.address -replace '\[::\]','[..]' -replace '\[::1\]','[..1]'
port = [int]$matches.port
state = [string]$matches.state -replace "\*:\*",'NA'
pid = [int]$matches.pid
service = ([string]$matches.service -replace 'Can not obtain ownership information','[System' -split '.*\[')[1] -replace '\]',''
subservice = ([string]$matches.service -replace 'Can not obtain ownership information','' -split '\[.*\]')[0]
}
$output += New-Object -TypeName PSobject -Property $outputobj
}
$output |select address,port,protocol,pid,state,service,subservice
I would probably do something like this:
mangle the output into a single string:
netstat -bano | Out-String
remove indention of the lines beginning with UDP or TCP to make them distinguishable from the other lines:
-replace '(?m)^ (TCP|UDP)', '$1'
join all indented lines that don't begin with a square bracket to the line preceding them:
-replace '\r?\n\s+([^\[])', "`t`$1"
join all indented lines that do begin with a square bracket to the line preceding them:
-replace '\r?\n\s+\[', "`t["
Complete statement:
(netstat -bano | Out-String) -replace '(?m)^ (TCP|UDP)', '$1' -replace '\r?\n\s+([^\[])', "`t`$1" -replace '\r?\n\s+\[', "`t["
Building off what TheMadTechnician had, I found a few cases that broke his output. Specifically: netstat sometimes skips columns; when a friendly app name and an executable are both output on separate lines, it breaks CSV formatting. Here's an updated version of his/her code to take those pieces into account:
$netstat = (netstat -abn | Select -skip 2) -replace "Address\s{2,}State", "Address,State,Application" -join "`n" `
-split "(?= [TU][CD]P\s+(?:\d+\.|\[\w*:\w*:))" | % {
$_.trim() -replace "`n",' ' `
-replace '\*\:\*', '*:*,' `
-replace '\s{2,}', ',' `
-replace '(.*?),(.*?),(.*?),(.*?),(.*?),', '$1,$2,$3,$4,$5 '
} | `
ConvertFrom-Csv
You can join the output of netstat so that it becomes one large multi-line string, then split it on lines that start with whitespace followed by TCP or UDP, followed by an IP address (to remove false positives of an application having a name of 'TCP tracker' or something). Then trim any whitespace from the beginning or end of the line, replace anywhere that there's two or more spaces with a comma, and push the results to ConvertFrom-Csv to create objects. From that you could filter, group, or just simply pipe to Format-Table to see the results.
$Netstat = (netstat -bano | Select -skip 2) -join "`n" -split "(?= [TU][CD]P\s+(?:\d+\.|\[\w*:\w*:))" |
ForEach-Object {$_.trim() -replace "`n",' ' -replace '\s{2,}',','} |
ConvertFrom-Csv
# Filter the results for TCP connections and pipe the results to Format-Table
$Netstat | Where {$_.Proto -eq 'TCP'} | Format-Table
I am trying to get a list of IP addresses from several servers defined in server.txt.
Each server has 2 IP addresses and 2 FQDN.
Example:
servername (Production lan):server1 IPaddress:147.111.111.16
servername (backup lan):server1-bck IPaddress:10.0.4.12
Here is the code I'm using:
$servers = Get-Content server.txt
$server2 = "$servers-bck"
$zai = ""
foreach ($server in $servers)
{
$zai = $zai + $server + "`t" +
([System.Net.Dns]::GetHostAddresses($server) | foreach {echo $_.IPAddressToString}) +
"`t" +
([System.Net.Dns]::GetHostAddresses($server2) | foreach {echo $_.IPAddressToString}) +
"`n"
}
$zai > IP-address.csv
Unfortunately only the IP for Production lan is correct. The IP for backup lan only shows the IP of the last server in server.txt. I assume the problem is in: "foreach { echo $._IPAddressToString". I don't know how to fix it.
Any idea or advice will be helpful.
You seem to assume that
$server2="$servers-bck"
would append -bck to the name of each element in the array $servers. That is not the case. Instead, the array is expanded by joining its elements using the output field separator, so that -bck ends up after the last array element:
PS C:\> $a = 'a', 'b', 'c'
PS C:\> $a
a
b
c
PS C:\> "$a-bck"
a b c-bck
To get the backup server for each server from your list you need to append -bck inside the loop:
...([System.Net.Dns]::GetHostAddresses("$server-bck"))...
As a side-note, you shouldn't build CSVs manually. Let PowerShell do that for you:
$servers | % {
New-Object -Type PSObject -Property #{
'Name' = $_
'Production' = ([Net.Dns]::GetHostAddresses($_) | select -Expand IPAddressToString) -join ' '
'Backup' = ([Net.Dns]::GetHostAddresses("$_-bck") | select -Expand IPAddressToString) -join ' '
}
} | Export-Csv 'IP-address.csv' -NoType -Delimiter "`t"
I need to extract values of specific column. I need to parse output generated with cmd :
netstat -an |
Select-String "TCP\s+.+\:.+\s+(.+)\:(\d+)\s+(\w+)" |
ForEach-Object {
$key = $_.matches[0].Groups[1].value
$Status = $_.matches[0].Groups[3].value.
Above 2 strings when printed gives me Foreign IP and connection state. I need a column with port no of local IP to which a foreign IP is connected.
If you are running on Windows 8 or Windows Server 2012, you can use the following command examples in Powershell V3:
Pipelining with Select-Object
Get-NetTCPConnection | Select LocalAddress,LocalPort,RemoteAddress,RemotePort
Selecting each property individually
(Get-NetTCPConnection).LocalAddress
(Get-NetTCPConnection).LocalPort
(Get-NetTCPConnection).RemoteAddress
(Get-NetTCPConnection).RemoteAddress
Creating variables using each property
$LocalAddress = (Get-NetTCPConnection).LocalAddress
$LocalPort = (Get-NetTCPConnection).LocalPort
$Remote Address = (Get-NetTCPConnection).RemoteAddress
$RemotePort = (Get-NetTCPConnection).RemoteAddress
These should all come out as lists.
Hope this helps :)
I'm not sure what you mean by a column but I've tweaked your regex and this gets the local and foreign addresses and ports:
netstat -an |
Select-String "TCP\s+(.+)\:(.+)\s+(.+)\:(\d+)\s+(\w+)" |
ForEach-Object {
$LocalAddress = $_.matches[0].Groups[1].value
$LocalPort = $_.matches[0].Groups[2].value
$ForeignAddress = $_.matches[0].Groups[3].value
$ForeignPort = $_.matches[0].Groups[4].value
Write-Output "ForeignAddress: $ForeignAddress `t ForeignPort: $ForeignPort `t LocalAddress: $LocalAddress `t LocalPort: $LocalPort"
}