I have a bunch of IIS logs and powershell 2.0.
Currently i'm using the following command to find some info out about them (where 'crossdomain' occurs in them):
dir -r -i *.log | select-string "crossdomain" | Select-Object | export-csv test.csv
This then gives me some data like so:
TRUE 1132740 2011-06-09 11:13:49 W3SVC322653822 myserver-WEB1 1.1.1.1 GET /crossdomain.xml - 80 - 1.1.1.1 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+6.0;+Trident/4.0;+GTB6.5;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30618) WT_FPC=id=82.40.25.58-3980883472.30062468:lv=1307614413232:ss=1307614405277;+__utma=151217894.932228880.1307618019.1307618019.1307618019.1;+__utmz=151217894.1307618019.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);+myserverVISITOR=eyJzaVZpc2l0VHJhY2tpbmcxIjpbeyJWSVNJVERBVEUiOiJNYXksIDEzIDIwMTEgMDY6NTc6NTAiLCJTVEFOREFSRElURU1JRCI6NjQ0MTkzLjB9LHsiVklTSVREQVRFIjoiTWF5LCAxMyAyMDExIDE1OjU0OjMyIiwiU1RBTkRBUkRJVEVNSUQiOjYwMzc4OC4wfSx7IlZJU0lUREFURSI6Ik1heSwgMTUgMjAxMSAxMzo0MDoxNCIsIlNUQU5EQVJESVRFTUlEIjo2NDQxOTUuMH0seyJWSVNJVERBVEUiOiJNYXksIDE1IDIwMTEgMTQ6MDE6NDEiLCJTVEFOREFSRElURU1JRCI6NjQ0MTkyLjB9LHsiVklTSVREQVRFIjoiTWF5LCAxNSAyMDExIDE0OjAzOjIyIiwiU1RBTkRBUkRJVEVNSUQiOjY0NDIxMC4wfSx7IlZJU0lUREFURSI6Ik1heSwgMTUgMjAxMSAxNDoyMTozMiIsIlNUQU5EQVJESVRFTUlEIjo2NDQ2MjAuMH0seyJWSVNJVERBVEUiOiJNYXksIDIyIDIwMTEgMDk6MTM6NTIiLCJTVEFOREFSRElURU1JRCI6NjI5NzYyLjB9LHsiVklTSVREQVRFIjoiSnVuZSwgMDcgMjAxMSAxMTo1MjoxMiIsIlNUQU5EQVJESVRFTUlEIjo2NDUxMjMuMH1dLCJ2aXNpdG9ySWQiOiI1QkFGNzg4NjNERDBENjQ3MUU4NkZENTYwQzU4NTFEMCJ9;+myserverGFSurvey=1;+ebNewBandWidth_.myserver.co.uk=251%3A1303383702897;+__utmb=151217894.1.10.1307618019;+__utmc=151217894 - myserver.co.uk 200 0 0 601 1506 0 W3SVC322653822_ex110609.log.log E:\w3\W3SVC322653822_ex110609.log.log crossdomain System.Text.RegularExpressions.Match[]
which is fine and dandy, but not dandy enough.
What I really want to do is get an export of the 7th column from the end where crossdomain occurs in the file. So this part in here:
**myserver.co.uk** 200 0 0 601 1506 0
(the myserver.co.uk)
any tips on this?
Cheers
Similar to Mjolinors' answer, but I'd try to keep the regexp as simple as possible. And since you've already selected lines with the word "crossdomain" you don't have to search for just that:
Get-Content test.csv | Foreach-Object
{
if ($_ -match '(\w+\.\w+\.\w+ \d+ \d+ \d+ \d+ \d+ \d+)')
{
$matches[1]
}
}
You won't get any 'Unexpected token' error, but you might have to tweak the regexp to get the result you want (I'm presuming you're looking for a three-dot domain and six numbers after it).
And always use '' for strings when you don't need variable extrapolation. it's much safer.
get-content test.csv |
foreach -object {
$_ -match ".+\s([a-z\.]+)\s[\s\d]+\S+\s\S+\s+crossdomain\s+\S+$" > $nul
$matches[1]
}
Related
I am attempting to rename a folder based on the first 10 characters inside a file using a powershell command.
I got as far as far as pulling the data I need to use to rename but I don't know how to pass it.
Get-Content 'C:\DATA\Company.dat' |
Select-Object -first 10 |
rename 'C:\DATA\FOLDER' 'C:\DATA\FOLDER (first 10)'
the part I'm stuck on is (first 10), I don't know what to pass to that section to complete my task?
Select-Object -first 10 will take the first 10 objects. In your case this will be the first 10 lines of the file, not 10 characters.
You can use something like this
Rename-Item -Path 'C:\DATA\FOLDER' -NewName "C:\DATA\$((Get-Content 'C:\DATA\Company.dat' | Select-Object -first 1).Substring(0,10))"
Using -first 1 to get the first line and .Substring(0,10) to get the first 10 characters.
Edit:
Or as #AdminOfThings mentioned, without the Select-Object
Rename-Item -Path 'C:\DATA\FOLDER' -NewName "C:\DATA\$((Get-Content 'C:\DATA\Company.dat' -raw).Substring(0,10))"
To complement Michael B.'s helpful answer with a 3rd approach:
If the characters of interest are known to be all on the 1st line (which is a safe assumption in your case), you can use Get-Content -First 1 ... (same as: Get-Content -TotalCount 1 ...) to retrieve that 1st line directly (and exclusively), which:
performs better than Get-Content ... | Select-Object -First 1
avoids having to read the entire file into memory with Get-Content -Raw ...
Rename-Item 'C:\DATA\FOLDER' `
"FOLDER $((Get-Content -First 1 C:\DATA\Company.dat).Substring(0, 10))"
Note:
It is sufficient to pass only the new name to Rename-Item's 2nd positional argument (the -NewName parameter); e.g., FOLDER 1234567890, not the whole path. While you can pass the whole path, it must refer to the same location as the input path.
The substring-extraction command is embedded inside an expandable string ("...") by way of $(...) the subexpression operator.
As for what you tried:
Select-Object -First 10 gets the first 10 input objects, which are the file's lines output by Get-Content; in other words: you'll send 10 lines rather than 10 characters through the pipeline, and even if they were characters, they'd be sent one by one.
While it is possible to solve this problem in the pipeline, it would be cumbersome and slow:
-join ( # -join, given an array of chars., returns a string
Get-Content -First 1 C:\DATA\Company.dat | # get 1st line
ForEach-Object ToCharArray | # convert to a char. array
Select-Object -First 10 # get first 10 chars.
) |
Rename-Item 'C:\DATA\FOLDER' { 'FOLDER ' + $_ }
That said, you could transform the above into something faster and more concise:
-join (Get-Content -First 1 C:\DATA\Company.dat)[0..9] |
Rename-Item 'C:\DATA\FOLDER' { 'FOLDER ' + $_ }
Note:
Get-Content -First 1 returns (at most) 1 line, in which case PowerShell returns that line as-is, not wrapped in an array.
Indexing into a string ([...]) with the range operator (..') - e.g., [0..9] - implicitly extracts the characters at the specified positions as an array; it is as if you had called .ToCharArray()[0..9]
Note how the new name is determined via a delay-bind script-block argument ({ ... }) in which $_ refers to the input object (the 10-character string, in this case); it is this technique that enables a renaming command to operate on multiple inputs, where each new name is derived from the specific input at hand.
We are migrating data from one filer to another and we have home directories setup for a bunch of users under their Active Directory profile. Some users have \\\x.x.x.x\sharename\etc, some have \\\hostname\sharename\etc and some have \\\hostname.domain.global\sharename\etc.
I need to replace the first portion of x.x.x.x, hostname and hostname.domain.global with the new DFS name of domain.global\sharename\etc, leaving everything else the same.
How can I go about doing this?
Thank you.
You should be able to use regular expressions. Examples:
PS C:\> '\\1.2.3.4\sharename1\etc1' |
Select-String '^\\\\[^\\]+\\(.+)' |
ForEach-Object { "\\domain.global\{0}" -f $_.Matches[0].Groups[1].Value }
\\domain.global\sharename1\etc1
PS C:\> '\\hostname\sharename2\etc2' |
Select-String '^\\\\[^\\]+\\(.+)' |
ForEach-Object { "\\domain.global\{0}" -f $_.Matches[0].Groups[1].Value }
\\domain.global\sharename2\etc2
PS C:\> '\\hostname.domain.global\sharename3\etc3' |
Select-String '^\\\\[^\\]+\\(.+)' |
ForEach-Object { "\\domain.global\{0}" -f $_.Matches[0].Groups[1].Value }
\\domain.global\sharename3\etc3
The regular expression pattern is:
^\\\\[^\\]+\\(.+)
This translates to: Start with (^) two \ characters, get 1 or more characters that are not \, followed by a single \, and then one or more characters (.+). The ( ) capture the path portion and make it available in the string replacement (i.e., $_.Matches[0].Groups[1].Value).
So I am a complete beginner at Powershell but need to write a script that will take a file, compare it against another file, and tell me what strings are different in the first compared to the second. I have had a go at this but I am struggling with the outputs as my script will currently only tell me on which line things are different, but it also seems to count lines that are empty too.
To give some context for what I am trying to achieve, I would like to have a static file of known good Windows processes ($Authorized) and I want my script to pull a list of current running processes, filter by the process name column so to just pull the process name strings, then match anything over 1 character, sort the file by unique values and then compare it against $Authorized, plus finally either outputting the different process strings found in $Processes (to the ISE Output Pane) or just to output the different process names to a file.
I have spent today attempting the following in Powershell ISE and also Googling around to try and find solutions. I heard 'fc' is a better choice instead of Compare-Object but I could not get that to work. I have thus far managed to get it to work but the final part where it compares the two files it seems to compare line by line, for which would always give me false positives as the line position of the process names in the file supplied would change, furthermore I only want to see the changed process names, and not the line numbers which it is reporting ("The process at line 34 is an outlier" is what currently gets outputted).
I hope this makes sense, and any help on this would be very much appreciated.
Get-Process | Format-Table -Wrap -Autosize -Property ProcessName | Outfile c:\users\me\Desktop\Processes.txt
$Processes = 'c:\Users\me\Desktop\Processes.txt'
$Output_file = 'c:\Users\me\Desktop\Extracted.txt'
$Sorted = 'c:\Users\me\Desktop\Sorted.txt'
$Authorized = 'c:\Users\me\Desktop\Authorized.txt'
$regex = '.{1,}'
select-string -Path $Processes -Pattern $regex |% { $_.Matches } |% { $_.Value } > $Output_file
Get-Content $Output_file | Sort-Object -Unique > $Sorted
$dif = Compare-Object -ReferenceObject $(Get-Content $Sorted) -DifferenceObject $(get-content $Authorized) -IncludeEqual
$lineNumber = 1
foreach ($difference in $dif)
{
if ($difference.SideIndicator -ne "==")
{
Write-Output "The Process at Line $linenumber is an Outlier"
}
$lineNumber ++
}
Remove-Item c:\Users\me\Desktop\Processes.txt
Remove-Item c:\Users\me\Desktop\Extracted.txt
Write-Output "The Results are Stored in $Sorted"
From the length and complexity of your script, I feel like I'm missing something, but your description seems clear
Running process names:
$ProcessNames = #(Get-Process | Select-Object -ExpandProperty Name)
.. which aren't blank: $ProcessNames = $ProcessNames | Where-Object {$_ -ne ''}
List of authorised names from a file:
$AuthorizedNames = Get-Content 'c:\Users\me\Desktop\Authorized.txt'
Compare:
$UnAuthorizedNames = $ProcessNames | Where-Object { $_ -notin $AuthorizedNames }
optional output to file:
$UnAuthorizedNames | Set-Content out.txt
or in the shell:
#(gps).Name -ne '' |? { $_ -notin (gc authorized.txt) } | sc out.txt
1 2 3 4 5 6 7 8
1. #() forces something to be an array, even if it only returns one thing
2. gps is a default alias of Get-Process
3. using .Property on an array takes that property value from every item in the array
4. using an operator on an array filters the array by whether the items pass the test
5. ? is an alias of Where-Object
6. -notin tests if one item is not in a collection
7. gc is an alias of Get-Content
8. sc is an alias of Set-Content
You should use Set-Content instead of Out-File and > because it handles character encoding nicely, and they don't. And because Get-Content/Set-Content sounds like a memorable matched pair, and Get-Content/Out-File doesn't.
I've tried to parse own custom log files rows and I except to get some spesific rows which header value is involved querystring values and included spesific key-value
I spesified some example log file rows as below :
2016-01-04 14:07:06 192.168.1.0 - example.com GET MainPage a=2&b=20&c=12-10&d=apple
2016-01-04 14:07:07 192.168.1.0 - example.com GET Search x=2&y=20&c=56-32-12&d=orange
2016-01-04 14:07:08 192.168.1.0 - example.com GET ProdView r=1&b=20&c=24&d=orange
2016-01-04 14:09:38 192.168.1.0 - example.com GET ProdView a=2&b=20&c=1-23e&d=orange
What I want for my own powershell result
2016-01-04 14:07:08 192.168.1.0 - example.com GET ProdView r=1&b=20&c=24&d=orange
Output must show which querystring parameters contains c and it points just numeric value(it mustn't contains any '-' character)
My own query as below :
Import-Csv -Encoding UTF8 .\logs\x.log -header A,B,C,D,E,F,G,H -delimiter ' ' | where {$_.H -match "c"} | select-string H
But I've blocked when I attempt to parse querystring parameters
Best regards
You could use a regular expression to match the log entries where the query-string contains the information you describe - parameter c is present, has only numerical digits as its value:
$_.H -match '(^|&)c=\d+(&|$)'
# (^|&) either start-of-string or the character &
# c= the literal string "c="
# \d+ one or more digits
# ($|&) either end-of-string or the character &
in your example, you have double and triple spaces, but this example only works if there is just a single space.
Import-Csv -Encoding UTF8 .\logs\x.log -header A,B,C,D,E,F,G,H -delimiter ' ' | ? h -match 'c[^-]+(&|$)' | select -exp h
or maybe something like this:
gc .\logs\x.log | % {$($_ -replace ' +', ' ').trim().split(' ')[-1]} | ? {$_ -match 'c[^-]+(&|$)'}
I have an XML command that returns a list of URLs, example
PS > $xml.rss.channel.item.link
http://example.com/20140704.exe
http://example.com/20140704.tar.xz
http://example.com/20140624.exe
http://example.com/20140624.tar.xz
http://example.com/20140507.tar.xz
From this list, I would like to return the first .tar.xz line. I have this
command
$xml.rss.channel.item.link | ? {$_ -match '.tar.xz'} | select -first 1
But I would prefer a command with only one pipe if possible.
You don't need a pipe at all:
(Select-Xml -Xml $xml -XPath "(//link[contains(.,'.tar.xz')])[1]").Node.InnerText
Note: XPath is case-sensitive. If that is an issue, you can use a trick with translate() function and force it to ignore the case.
A different way using two pipes
$xml.rss.channel.item.link | Select-String .tar.xz | select -first 1
One pipe
($xml.rss.channel.item.link | Select-String .tar.xz)[0]