I am using get-winevent to convert an evtx log to .json file. Then I've send it to ELK. Get-WinEvent -Path .\log.evtx | ConvertTo-Json|Format-List | Out-File log.json
The file looks like a normal string containing file on windows. But when I take it to linux, it contains binary data and cannot be parsed to ELK.
Even if I use out-string, nothing changes. $result = Get-WinEvent -Path .\user-creation-1log.evtx | ConvertTo-Json| Format-List
$result | Out-String | out-file log.jsonThis also appears like a binary file in linux. (Although I remember export-csv with get-winevent created complete text file, but this makes a really ugly formatted csv file). I really liked the way convertTo-json formatted and valued the json data and would prefer it. (if someone can provide a different way to convert the evtx data in its fullest form to json, happy to take).
I've tried evtx2csv python module, but that doesn't write output to a file.
First, don't use Format-List if you intend to export JSON. This is only for formatting objects as a nice visual representation in the console.
Also, I don't use Linux, but I guess it's safest to specify utf8 as encoding explicitly to make sure it's compatible:
Get-WinEvent -Path .\log.evtx | ConvertTo-Json | Out-File log.json -Encoding utf8
Related
In a powershell script, I am creating a txt file:
#Create a logfile
$logfile = "E:\scripts\Password expiration\PasswordExpiryLog.txt"
#Initialize the log file with the date
$date = Get-Date -Format "dd.MM.yyyy HH:mm:ss"
$message= "********************" + $date + "********************"
Add-Content $logfile $message
After that, I do some other things to get some info. When I try to send that info into the TXT file, formatting the info as a table, it writes the information in a way that the information is not in columns, missing the break of line. I mean, if I press enter where the next line should start, it is written in a column.
$Expired | FORMAT-TABLE #{Name='UserPrincipalName';Expression={"$($_.UserPrincipalName)"};align ='left'}, #{Name='rapasswordexpiring';Expression={"$($_.rapasswordexpiring)"};align ='center'}, #{Name='PasswordLastSet';Expression={"$($_.PasswordLastSet)"};align ='left'} -AutoSize | Out-File -Append 'E:\scripts\Password expiration\PasswordExpiryLog.txt'
I have also realized, while writing this message, that between the characters there is always a space
How it looks
How it should look
If I instead of out-file to the same logfile I use a new txt file, the information is written correctly.
$Expired | FORMAT-TABLE #{Name='UserPrincipalName';Expression={"$($_.UserPrincipalName)"};align ='left'}, #{Name='rapasswordexpiring';Expression={"$($_.rapasswordexpiring)"};align ='center'}, #{Name='PasswordLastSet';Expression={"$($_.PasswordLastSet)"};align ='left'} -AutoSize | Out-File -Append 'E:\scripts\Password expiration\table.txt'
Correct
However, I cannot get here the date and time.
Can you tell me what am I doing wrong or how to improve the output? I would be able to use cvs or html file if would be better.
Different default encoding:
Add-Content:
-Encoding Specifies the type of encoding for the target file. The
default value is Default.
Default Uses the encoding that corresponds to the system's active
code page (usually ANSI).
Out-File
-Encoding Specifies the type of encoding for the target file. The
default value is unicode.
unicode Uses UTF-16 with the little-endian byte order.
This will be easy for most of you answer, but I'm struggling with it.
For instance, if I input Get-Tpm, the return I get is,
Get-tpm written to console, as shown below.
If I enter
ECHO Get-tpm > C:\dssupport\output.txt
the output is a text file containing 'Get-tpm:'
Is there a way to do both, so that I can see it on my monitor,
and also saved as a text file?
I'm grateful for any help at all.
Thank you
Most PowerShell cmdlets like Get-Tpm output objects which usually display on the console screen formatted as table or as list.
You can easily write output like that to a file, usually a structured Csv file you can open in Excel using Export-Csv.
That last cmdlet however does not have something neat like a parameter called -PassThru, but what you can do is use a cmdlet that does let you do that.
In your example, you could do
# show on screen AND write to CSV file
Get-Tpm | ConvertTo-Csv -NoTypeInformation | Set-Content -Path 'X:\TPM.csv' -PassThru
or
# show on screen AND write to JSON file
Get-Tpm | ConvertTo-Json | Set-Content -Path 'X:\TPM.json' -PassThru
or even
# show on screen as list AND write to text file
Get-Tpm | Format-List | Out-String | Set-Content -Path 'X:\TPM.txt' -PassThru
Although a file written like that is not really meant to be used for anything other than display purposes.
Or you can do this in separate lines of code where you show on screen as list for instance and in the other line write the csv file.
For that, you need to capture the results from the cmdlet in a variable:
$result = Get-Tpm
# show on screen as list (or table)
$result | Format-List # or do Format-Table -AutoSize
# show in a separate grid view window
$result | Out-GridView -Title "My TPM results"
# output to CSV file
$result | Export-Csv -Path 'X:\TPM.csv' -NoTypeInformation
I have an array of objects in Powershell. It was working, but now when I do an Export-Csv on the array, it property and value names are transformed like:
Account_No -> +ACI-Account+AF8-No+ACI-
Does anyone know why it is doing this?
Thanks
I am using PS 5.1, and the command is:
$rowsWithErrs | Export-Csv -Path $rowErrCsvPath -NoTypeInformation -Encoding UTF7
It looks like there isn't anything wrong with what you are doing. Everything is getting sent out in the format that you are expecting.
The only problem is that the application that you are using to view your data is not using the same encoding that was used to write the data.
The extra characters are what you see when interpreting text as UTF8 or something similar or compatible with UTF8 (which is the standard for most systems) instead of UTF7 when the text was encoded as UTF7.
example
> "Account_No" | Out-File -FilePath test.txt -Encoding UTF7
> Get-Content test.txt -Encoding UTF8
Account+AF8-No
> Get-Content test.txt -Encoding UTF7
Account_No
if reading csv data in Powershell you can do the following
> $csv = Import-Csv -FilePath $filepath -Encoding UTF7
if reading csv data in Excel, on the data tab select From Text/CSV at the top of the import window select File Origin 65000: Unicode (UTF-7)
For other applications like VS Code or Notepad++ you may be out of luck if you want to view the data there because it looks like they do not support UTF-7 encoding.
I would like to convert the StartupInfo XML log file (Windows 10) to CSV.
C:\Windows\System32\WDI\LogFiles\StartupInfo\<SID>_StartupInfo<NUMBER>.xml
I tried:
[xml]$StartupInfo = Get-Content "C:\Windows\System32\WDI\LogFiles\<SID>_StartupInfo<NUMBER>.xml"
$StartupInfo.StartupData.Process | Export-Csv -Path C:\Users\<USER>\Desktop\StartupInfo.csv -NoTypeInformation
But I have still problems with the columns "CommandLine", "DiskUsage" and "CpuUsage".
Thx for assistance.
I am sure there are more distinguished methods to do this like XSLT transforms. But for this particular type of XML file I think this method works:
$StartupInfo.StartupData.Process | Select-Object Name,PID,StartedInTraceSec,
#{name='CommandLine';Expression={$PSitem.CommandLine.'#cdata-section'}},
#{name='DiskUsage';Expression={$PSItem.DiskUsage.'#text'}},
#{name='CPUUsage';Expression={$PSItem.CPUUsage.'#text'}} |
Export-Csv -Path C:\Users\<USER>\Desktop\StartupInfo.csv -NoTypeInformation
The problem is that XML files aren't flat like CSV files are. If you want an explanation of the technique I used have a look here: https://mcpmag.com/articles/2017/01/19/using-powershell-calculated-properties.aspx
I am sorting a large directory of files and I am trying to select individual lines from the output of an ls command and show those only, but I get weird results and I am not familiar enough with powershell to know what I'm doing wrong.
this approach works:
ls > data.txt
select-string 2012 data.txt
rm data.txt
but it seems wasteful to me to create a file just to read the data that I already have to fill into the file. I want to pipe the output directly to select-string.
I have tried this approach:
ls | select-string 2012
but that does not give me the appropriate output.
My guess is that I need to convert the output from ls into something select-string can work with, but I have no idea how to do that, or even whether that is actually the correct approach.
PowerShell is object-oriented, not pure text like cmd. If you want to get fileobjects(lines) that were modified in 2012, use:
Get-ChildItem | Where-Object { $_.LastWriteTime.Year -eq 2012 }
If you want to get fileobjects with "2012" in the filename, try:
Get-ChildItem *2012*
When you use
ls | select-string 2012
you're actually searching for lines with "2012" INSIDE every file that ls / get-childitem listed.
If you really need to use select-string on the output from get-childitem, try converting it to strings, then splitting up into lines and then search it. Like this:
(Get-ChildItem | Out-String) -split "`n" | Select-String 2012
I found another simple way to convert objects to strings:
Get-ChildItem | Out-String -stream | Select-String 2012
in this very interesting article:
http://blogs.msdn.com/b/powershell/archive/2006/04/25/how-does-select-string-work-with-pipelines-of-objects.aspx
If you wanted Select-String to work on the Monad formatted output, you'll need to get that as a string. Here is the thing to grok about
our outputing. When your command sequence emits a stream of strings,
we emit it without processing. If instead, your command sequence
emits a stream of objects, then we redirect those objects to the
command Out-Default. Out-Default looks at the type of the object and
the registered formating metadata to see if there is a default view
for that object type. A view defines a FORMATTER and the metadata for
that command. Most objects get vectored to either Format-Table or
Format-List (though they could go to Format-Wide or Format-Custom).
THESE FORMATTERS DO NOT EMIT STRINGS! You can see this for yourself
by the following: "These formating records are then vectored to an
OUT-xxx command to be rendered into the appropriate data for a
particular output device. By default, they go to Out-Host but you can
pipe this to Out-File, Out-Printer or Out-String. (NOTE: these
OUT-xxx commands are pretty clever, if you pipe formating objects to
them, they'll render them. If you pipe raw object to them, they'll
first call the appropriate formatter and then render them.)