I am trying to setup a log which would pull different information from another log file to log assets build by MDT using PowerShell. I can extract a line of log using simple get-content | select-string to get the lines i need so output looks like that
[LOG[Validate Domain Credentials [domain\user]]LOG]!
time="16:55:42.000+000" date="10-20-2017" component="Wizard"
context="" type="1" thread="" file="Wizard"
and I am curious if there is a way of capturing things like domain\user, time and date in a separate variables so it can be later passed with another data captured in a similar way in output file in a single line.
This is how you could do it:
$line = Get-Content "<your_log_path>" | Select-String "Validate Domain Credentials" | select -First 1
$regex = '\[(?<domain>[^\\[]+)\\(?<user>[^]]+)\].*time="(?<time>[^"]*)".*date="(?<date>[^"]*)".*component="(?<component>[^"]*)".*context="(?<context>[^"]*)".*type="(?<type>[^"]*)".*thread="(?<thread>[^"]*)".*file="(?<file>[^"]*)"'
if ($line -match $regex) {
$user = $Matches.user
$date = $Matches.date
$time = $Matches.time
# ... now do stuff with your variables ...
}
You might want to build in some error checking etc. (e.g. when no line is found or does not match etc.)
Also you could greatly simplify the regex if you only need those 3 values. I designed it so that all fields from the line are included.
Also, you could convert the values into more appropriate types, which (depending on what you want to do with them afterwards) might make handling them easier:
$type = [int]$Matches.type
$credential = New-Object System.Net.NetworkCredential($Matches.user, $null, $Matches.domain)
$datetime = [DateTime]::ParseExact(($Matches.date + $Matches.time), "MM-dd-yyyyHH:mm:ss.fff+000", [CultureInfo]::InvariantCulture)
Related
I'm trying to use the PowerShell command 'Get-GPOReport' to get GPO information in XML string format so I can search it for sub-Element values with unknown and different Element tag names (I don't think XML Object format will work for me, so I didn't perform a cast with "[xml]"), but I haven't been able to parse the XML output so that I can grab the line or two after a desired "Name" Element line that matches the text I'm searching for.
After, I have been trying to use 'Select-String' or 'Select-XML' with XPath (formatting is unclear and I don't know if I can use a format for various policy record locations) to match text and grab a value, but I haven't had any luck.
Also, if anyone know how to search for GPMC GUI names (i.e. "Enforce password history") instead of needing to first locate back-end equivalent names to search for (i.e. "PasswordHistorySize"), that would also be more helpful.
The following initial code is the part that works:
$String = "PasswordHistorySize" # This is an example string, as I will search for various strings eventually from a file, but I'm not sure if I could search for equivalent Group Policy GUI text "Enforce password history", if anyone knows how to do that.
$CurrentGPOReport = Get-GPOReport -Guid $GPO.Id -ReportType Xml -Domain $Domain -Server $NearestDC
If ($CurrentGPOReport -match $String)
{
Write-Host "Policy Found: ""$($String)""" -Foregroundcolor Green
#
#
# The following code is what I've tried to use to get value data, without any luck:
#
$ValueLine1 = $($CurrentGPOReport | Select-String -Pattern $String -Context 0,2)
$Value = $($Pattern = ">(.*?)</" ; [regex]::match($ValueLine1, $Pattern).Groups[1].Value)
}
I've been looking at this since yesterday and didn't understand why Select-String wasn't working, and I figured it out today... The report is stored as a multi-line string, rather than an array of strings. You could do a -match against it for the value, but Select-String doesn't like the multi-line formatting it seems. If you -split '[\r\n]+' on it you can get Select-String to find your string.
If you want to use RegEx to just snipe the setting value you can do it with a multi-line regex search like this:
$String = "PasswordHistorySize" # This is an example string, as I will search for various strings eventually from a file, but I'm not sure if I could search for equivalent Group Policy GUI text "Enforce password history", if anyone knows how to do that.
$CurrentGPOReport = Get-GPOReport -Guid $GPO.Id -ReportType Xml -Domain $Domain -Server $NearestDC
$RegEx = '(?s)' + [RegEx]::Escape($String) + '.+?Setting.*?>(.*?)<'
If($CurrentGPOReport -match $RegEx)
{
Write-Host "Policy Found: ""$String""" -Foregroundcolor Green
$Value = $Matches[1]
}
I'm not sure how to match the GPMC name, sorry about that, but this should get you closer to your goals.
Edit: To try and get every setting separated out into it's own chunk of text and not just work on that one policy I had to alter my RegEx a bit. This one's a little more messy with the output, but can be cleaned up simply enough I think. This will split a GPO into individual settings:
$Policies = $CurrentGPOReport -split '(\<(q\d+:.+?>).+?\<(?:\/\2))' | Where { $_ -match ':Name' }
That will get you a collection of things that look like this:
<q1:Account>
<q1:Name>PasswordHistorySize</q1:Name>
<q1:SettingNumber>21</q1:SettingNumber>
<q1:Type>Password</q1:Type>
</q1:Account>
From there you just have to filter for whatever setting you're looking for.
I have tried this with XPath, as you'll have more control navigating in the XML nodes:
[string]$SearchQuery = "user"
[xml]$Xml = Get-GPOReport -Name "Default Domain Policy" -ReportType xml
[array]$Nodes = Select-Xml -Xml $Xml -Namespace #{gpo="http://www.microsoft.com/GroupPolicy/Settings"} -XPath "//*"
$Nodes | Where-Object -FilterScript {$_.Node.'#text' -match $SearchQuery} | ForEach-Object -Process {
$_.Name #Name of the found node
$_.Node.'#text' #text in between the tags
$_.Node.ParentNode.ChildNodes.LocalName #other nodes on the same level
}
After testing we found that in the XML output of the Get-GPOReport cmdlet, the setting names does not always match that of the HTML output. For example: "Log on as a service" is found as "SeServiceLogonRight" in the XML output.
I'm trying to write a simple usage logger into my script that would store information about the time when user opened the script, finished using the script and the user name.
The first part of the logger where I gather the first two data works fine and adds two necessary columns with values to the CSV file. Yet when I run the second part of the logger it does not add a new column to my existing CSV file.
#Code I will add at the very beginning of my script
$FileNameDate = Get-Date -Format "MMM_yyyy"
$FilePath = "C:\Users\Username\Desktop\Script\Logs\${FileNameDate}_MonthlyLog.csv"
$TimeStamp = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
$UserName = [string]($env:UserName)
$LogArray = #()
$LogArrayDetails = #{
Username = $UserName
StartDate = $TimeStamp
}
$LogArray += New-Object PSObject -Property $LogArrayDetails | Export-Csv $FilePath -Notypeinformation -Append
#Code I will add at the very end of my script
$logArrayFinishDetails = #{FinishDate = $TimeStamp}
$LogCsv = Import-Csv $FilePath | Select Username, StartDate, #{$LogArrayFinishDetails} | Export-Csv $FilePath -NoTypeInformation -Append
CSV file should look like this when the script is closed:
Username StartDate FinishDate
anyplane 08/Apr/2018 23:47:55 08/Apr/2018 23:48:55
Yet it looks like this:
StartDate Username
08/Apr/2018 23:47:55 anyplane
The other weird thing is that it puts the StartDate first while I clearly stated in $LogArrayDetails that Username goes first.
Assuming that you only ever want to record the most recent run [see bottom if you want to record multiple runs] (PSv3+):
# Log start of execution.
[pscustomobject] #{ Username = $env:USERNAME; StartDate = $TimeStamp } |
Export-Csv -Notypeinformation $FilePath
# Perform script actions...
# Log end of execution.
(Import-Csv $FilePath) |
Select-Object *, #{ n='FinishDate'; e={ (Get-Date).toString("dd/MMM/yyyy HH:mm:ss") } } |
Export-Csv -Notypeinformation $FilePath
As noted in boxdog's helpful answer, using -Append with Export-Csv won't add additional columns.
However, since you're seemingly attempting to rewrite the entire file, there is no need to use
-Append at all.
So as to ensure that the old version of the file has been read in full before you attempt to replace it with Export-Csv, be sure to enclose your Import-Csv $FilePath call in (...), however.
This is not strictly necessary with a 1-line file such as in this case, but a good habit to form for such rewrites; do note that this approach is somewhat brittle in general, as something could go wrong while rewriting the file, resulting in potential data loss.
#{ n='FinishDate'; e={ (Get-Date).toString("dd/MMM/yyyy HH:mm:ss") } is an example of a calculated property/column that is appended to the preexisting columns (*)
The other weird thing is that it puts the StartDate first while I clearly stated in $LogArrayDetails that Username goes first.
You've used a hashtable (#{ ... }) to declare the columns for the output CSV, but the order in which a hashtable's entries are enumerated is not guaranteed.
In PSv3+, you can use an ordered hashtable instead ([ordered] #{ ... }) to achieve predictable enumeration, which you also get if you convert the hashtable to a custom object by casting to [pscustomobject], as shown above.
If you do want to append to the existing file, you can use the following, but note that:
this approach does not scale well, because the entire log file is read into memory every time (and converted to objects), though limiting the entries to a month's worth should be fine.
as stated, the approach is brittle, as things can go wrong while rewriting the file; consider simply writing 2 rows per execution instead, which allows you to append to the file line by line.
there's no concurrency management, so the assumption is that only ever one instance of the script is run at a time.
$FilePath = './t.csv'
$TimeStamp = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
$env:USERNAME = $env:USER
# Log start of execution. Note the empty 'FinishDate' property
# to ensure all rows ultimately have the same column structure.
[pscustomobject] #{ Username = $env:USERNAME; StartDate = $TimeStamp; FinishDate = '' } |
Export-Csv -Notypeinformation -Append $FilePath
# Perform script actions...
# Log end of execution:
# Read the entire existing file...
$logRows = Import-Csv $FilePath
# ... update the last row's .FinishDate property
$logRows[-1].FinishDate = (Get-Date).toString("dd/MMM/yyyy HH:mm:ss")
# ... and rewrite the entire file, keeping only the last 30 entries
$logRows[-30..-1] | Export-Csv -Notypeinformation $FilePath
Because your CSV already has a structure (i.e. defined headers), PowerShell honours this when appending and doesn't add additional columns. It is (sort of) explained in this excerpt from the Export-Csv help:
When you submit multiple objects to Export-CSV, Export-CSV organizes
the file based on the properties of the first object that you submit.
If the remaining objects do not have one of the specified properties,
the property value of that object is null, as represented by two
consecutive commas. If the remaining objects have additional
properties, those property values are not included in the file.
You could include the FinishDate property in the original file (even though it would be empty), but the best option might be to export your output to a different CSV at the end, perhaps deleting the original after import then recreating it with the additional data. In fact, just removing the -Append will likely give the result you want.
I have a script written that pulls a number of CSV files from an FTP server and downloads to a network location.
The content of this CSV file follows the example I have provided in this link
File Example
In short working with this file I need to:
Using the 12 characters (alpha-numeric) which follow Ords: on line two define a variable which will used later in a query. (A)
GB0000000001
Would become
$OrderVariable = "GB0000000001"
I have read about
.TrimStart([Characters_to_remove])
but am unsure how it would skip the first row and then how I would remove everything following the next 12 letters.
Using the entire line two information excluding the Ords: define this as a variable e.g.
GB0000000001 – Promotion Event
would become
$TitleEvent = "GB0000000001 – Promotion Event"
The CSV contains all the customers that an email needs to be sent to e.g.
D|300123123|BBA
D|300321312|DDS
D|A0123950|BBA
D|A0999950|ZZG
These items I would expect to be written into a hashtable which I thought would be simple enough except I cannot find any way to exclude everything which precedes it.
$mytable = Import-Csv -Path $filePath -Header D,Client,Suffix
$HashTable = #{}
foreach ($r in $mytable) {
$HashTable[$r.Client] = $r.Data
}
UPDATE
I have managed to get most of this element into a variable with the following
$target = "\\Messaging"
cd $target
$Clients = Import-Csv example.txt | where {$_ -like "*D|*"}
$Clients = $Clients[1..($Clients.count - 1)]
$Clients | Export-Csv "Test.csv" -NoTypeInformation
But I cannot get it to import with custom headers or without the first "H|" delimitation...
End of update 1
I believe this is roughly what is going to be required as the only element that I will need to define and use in a later query is the Client themselves.
The next would define all the text that remains as the message content
This is a Promotion Event and action needs to be taken by you. The
deadline for your instruction is 2pm on 12 September 2016.
The deadline for this event has been extended.
To notify us of your instruction you can send a secure message.
This can differ on each occasion massively so cannot simply be a removal of X numbers of lines the content will always follow the Ords: (line two) and End at the start for the D| delimitation.
Most of the other code I need to put together I am 'fairly confident' with (famous last words) and have a fully working script that is pulling the files I need, I am just not great at working with .csv's when I have them.
The data format is flexible without a global table/grid structure so let's use regexps (the breakdown), which is quite a universal method of parsing such texts.
$text = [IO.File]::ReadAllText('inputfile.txt', [Text.Encoding]::UTF8)
$data = ([regex]('ORDS: (?<order>.+?) [-–—] (?<title>.+)[\r\n]+' +
'(?<info>[\s\S]+?)[\r\n]+' +
'(?<clients>D\|[\s\S]+?)[\r\n]+' +
'T\|(?<T>\d+)')
).Matches($text) |
forEach {
$g = $_.groups
#{
order = $g['order'].value
info = $g['info'].value -join ' '
clients = $g['clients'].value -split '[\r\n]+' |
where { $_ -match 'D\|(.+?)\|(.+)' } |
forEach {
#{
id = $matches[1]
suffix = $matches[2]
}
}
T = $g['T']
}
}
$data is now a record (or an array of records if the file has multiple entries):
Name Value
---- -----
T 000004
info This is a Promotion Event and action needs to be take...
order GB0000000001
clients {System.Collections.Hashtable, System.Collections.Has...
$data.clients is an array of records:
Name Value
---- -----
id 300123123
suffix BBA
id 300321312
suffix DDS
id A0123950
suffix BBA
id A0999950
suffix ZZG
I have a text file to process. Text file has some configuration data and some networking commands. I want to run all those network commands and redirect output in some log file.
At starting of text file,there are some configuration information like File-name and file location. This can be used for naming log file and location of log file. These line starts with some special characters like '<#:'. just to know that rest of the line is config data about file not the command to execute.
Now, before i want start executing networking commands (starts with some special characters like '<:'), first i want to read all configuration information about file i.e. file name, location, overwrite flag etc. Then i can run all commands and dump output into log file.
I used get-content iterator to loop over entire text file.
Question: Is there any way to start looping over file from a specific line again?
So that i can process config information first (loop till i first encounter command to execute, remember this line number), create log file and then keep running commands and redirect output to log file (loop from last remembered line number).
Config File looks like:
<#Result_File_Name:dump1.txt
<#Result_File_Location:C:\powershell
<:ping www.google.com
<:ipconfig
<:traceroute www.google.com
<:netsh interface ip show config
My powerhsell script looks like:
$content = Get-Content C:\powershell\config.txt
foreach ($line in $content)
{
if($line.StartsWith("<#Result_File_Name:")) #every time i am doing this, even for command line
{
$result_file_arr = $line.split(":")
$result_file_name = $result_file_arr[1]
Write-Host $result_file_name
}
#if($line.StartsWith("<#Result_File_Location:"))#every time i am doing this, even for command line
#{
# $result_file_arr = $line.split(":")
# $result_file_name = $result_file_arr[1]
#}
if( $conf_read_over =1)
{
break;
}
if ($line.StartsWith("<:")) #In this if block, i need to run all commands
{
$items = $line.split("<:")
#$items[0]
#invoke-expression $items[2] > $result_file_name
invoke-expression $items[2] > $result_file_name
}
}
If all the config information starts with <# just process those out first separately. Once that is done you can assume the rest are commands?
# Collect config lines and process
$config = $content | Where-Object{$_.StartsWith('<#')} | ForEach-Object{
$_.Trim("<#") -replace "\\","\\" -replace "^(.*?):(.*)" , '$1 = $2'
} | ConvertFrom-StringData
# Process all the lines that are command lines.
$content | Where-Object{!$_.StartsWith('<#') -and ![string]::IsNullOrEmpty($_)} | ForEach-Object{
Invoke-Expression $_.trimstart("<:")
}
I went a little over board with the config section. What I did was convert it into a hashtable. Now you will have your config options, as they were in file, accessible as an object.
$config
Name Value
---- -----
Result_File_Name dump1.txt
Result_File_Location C:\powershell
Small reconfiguration of your code, with some parts missing, would look like the following. You will most likely need to tweak this to your own needs.
# Collect config lines and process
$config = ($content | Where-Object{$_.StartsWith('<#')} | ForEach-Object{
$_.Trim("<#") -replace "\\","\\" -replace "^(.*?):(.*)" , '$1 = $2'
} | Out-String) | ConvertFrom-StringData
# Process all the lines that are command lines.
$content | Where-Object{!$_.StartsWith('<#') -and ![string]::IsNullOrEmpty($_)} | ForEach-Object{
Invoke-Expression $_.trimstart("<:") | Add-Content -Path $config.Result_File_Name
}
As per your comment you are still curious about your restart loop logic which was part of your original question. I will add this as a separate answer to that. I would still prefer my other approach.
# Use a flag to determine if we have already restarted. Assume False
$restarted = $false
$restartIndexPoint = 4
$restartIndex = 2
for($contentIndex = 0; $contentIndex -lt $content.Length; $contentIndex++){
Write-Host ("Line#{0} : {1}" -f $contentIndex, $content[$contentIndex])
# Check to see if we are on the $restartIndexPoint for the first time
if(!$restarted -and $contentIndex -eq $restartIndexPoint){
# Set the flag so this does not get repeated.
$restarted = $true
# Reset the index to repeat some steps over again.
$contentIndex = $restartIndex
}
}
Remember that array indexing is 0 based when you are setting your numbers. Line 20 is element 19 in the string array for example.
Inside the loop we run a check. If it passes we change the current index to something earlier. The write-host will just print the lines so you can see the "restart" portion. We need a flag to be set so that we are not running a infinite loop.
I have a requirement like:
Have a text file containing the following in the following pattern
172.26.xxy.zxy:Administrator:Password
172.26.xxy.yyx:Administrator:Password
172.26.xxy.yyy:Administrator:Password
172.26.xxy.yxy:Administrator:Password
I need my powershell script to read each word and use that word whereever required. For example,
foreach(something)
{
I want the IP's(172.26.---.---) to read and store the value as a variable.
I want to store the two words after **:** in seperate variables.
}
How can this be done? I know to read an entire file or get some specific string. But I need the same to be done on each line.Any help would be really appreciated.
Something like this? You can just split on the : and then store your variables based on the index
$contents = Get-Content C:\your\file.txt
foreach($line in $contents) {
$s = $line -split ':'
$ip = $s[0]
$user = $s[1]
$pass = $s[2]
write-host $ip $user $pass
}
minor edit: "t" missing in content.
You can write a regular expression to replace to remove the parts you do not need
$ip_address= '172.26.xxy.zxy:Administrator:Password' -replace '^(.+):(.+):(.+)$','$1'
$user= '172.26.xxy.zxy:Administrator:Password' -replace '^(.+):(.+):(.+)$','$2'
$pwd= '172.26.xxy.zxy:Administrator:Password' -replace '^(.+):(.+):(.+)$','$3'
I think the more generic and pure Powershell way would be something like this:
Select-String "(.*):(.*):(.*)" c:\file.txt |
Select #{Name="IP"; Expression = {$_.Matches.Groups[1]}},
#{Name="User"; Expression = {$_.Matches.Groups[2]}},
#{Name="Password"; Expression = {$_.Matches.Groups[3]}}
The Output would be then an array of objects each having three properties IP, User and Password. So you can now use them for your purposes, or just add more commands at the end of the pipe.