I am new to azure data explorer and Kusto Queries. I am learning from below online sample
https://dataexplorer.azure.com/clusters/help/databases/Samples
Here is the query which i am getting results in Data Explorer but unable to display in power shell
StormEvents
| where DamageProperty >0
| limit 2
| project StormSummary.TotalDamages
Below is reference link for code which i am trying to run query in Powershell (Example2 in below link page)
https://learn.microsoft.com/en-us/azure/data-explorer/kusto/api/powershell/powershell
I had changed only "$Query" and modified last code line as like below
$dataView | Format-Table -AutoSize
i am getting output as
StormSummary_TotalDamages
-------------------------
{}
I tried modifying query without "TotalDamages" in "StormSummary.TotalDamages" but resulting dataview which i am unable to get again "TotalDamages" .
StormSummary
------------
{TotalDamages, StartTime, EndTime, Details}
Some one helped me to fix my issue. I am posting it to helps others.
Explanation:
Query result is importing to Json string, converting it from json and converting the columns&rows data-layout back into individual PSObjects really helped my issue.
Code:
As per the Example 2 mentioned in my question . we are calling
$reader = $queryProvider.ExecuteQuery($query, $crp)
After this I removed existing code and modified like below:
Modified Code to get Projected Field data(TotalDamages):
$json = [Kusto.Cloud.Platform.Data.ExtendedDataReader]::ToJsonString($reader)
$data = $json | ConvertFrom-Json
$columns = #{}
$count = 0
foreach ($column in $data.Tables[0].Columns) {
$columns[$column.ColumnName] = $count
$count++
}
$items = foreach ($row in $data.Tables[0].Rows) {
$hash = #{}
foreach ($property in $columns.Keys){
$hash[$property] = $row[$columns[$property]]
}
[PSCustomObject]$hash
}
foreach($item in $items)
{
Write-Host "TotalDamages: "$item.StormSummary.TotalDamages
}
Output:
TotalDamages: 6200000
TotalDamages: 2000
Related
Apologies if this is irrelevant but I'm new to powershell and I've been scratching my head on this for a few days on and off now. I'm trying to write a script that will output two columns of data to a html document. I've achieved most of it by learning through forums and testing different combinations.
The problem is although it gives me the result I need within powershell itself; it will not properly display the second column results for Net Log Level.
So the script looks at some folders and pulls the * value which is always three digits (this is the Site array). It then looks within each of these folders to the Output folder and grabs a Net Log Level node from a file inside there. The script is correctly listing the Sites but is only showing the last value for Net Log Level which is 2. You can see this in the screenshot above. I need this to take every value for each Site and display as appropriate. The image of the incorrect result is below. I need the result to be 1,4,2,2,2. Any help would be greatly appreciated!
function getSite {
Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | foreach {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
New-Object -TypeName System.Collections.ArrayList
$List1 += #([System.Collections.ArrayList]#($Site))
New-Object -TypeName System.Collections.ArrayList
$List2 += #([System.Collections.ArrayList]#($NetLogLevel))
}
$Results = #()
ForEach($Site in $List1){
$Results += [pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site','Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
Restructure your code as follows:
Get-ChildItem 'C:\Scripts\ServiceInstalls\*\Output\Config.exe.config' |
ForEach-Object {
$site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content -Raw $_.FullName
$netLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']").InnerText
# Construct *and output* a custom object for the file at hand.
[pscustomobject] #{
'Site ID' = $site
'Net Log Level' = $netLogLevel
}
} | # Pipe the stream of custom objects directly to ConvertTo-Html
ConvertTo-Html | # No need to specify -Property if you want to use all properties.
Set-Content Output.html
As for what you tried:
New-Object -TypeName System.Collections.ArrayList in effect does nothing: it creates an array-list instance but doesn't save it in a variable, causing it to be enumerated to the pipeline, and since there is nothing to enumerate, nothing happens.
There is no point in wrapping a [System.Collections.ArrayList] instance in #(...): its elements are enumerated and then collected in a regular [object[]] array - just use #(...) by itself.
Using += to "grow" an array is quite inefficient, because a new array must be allocated behind the scenes every time; often there is no need to explicitly create an array - e.g. if you can simply stream objects to another command via the pipeline, as shown above, or you can let PowerShell itself implicitly create an array for you by assigning the result of a pipeline or foreach loop as a whole to a variable - see this answer.
Also note that when you use +=, the result is invariably a regular [object[] array, even if the RHS is a different collection type such as ArrayList.
There are still cases where iteratively creating an array-like collection is necessary, but you then need to use the .Add() method of such a collection type in order to grow the collection efficiently - see this answer.
Instead of populating two separate lists, simply create the resulting objects in the first loop:
function getSite {
$Results = Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | ForEach-Object {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
[pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site', 'Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
I have been researching this for weeks now and can't seem to make much ground on the subject. I have a large PDF (900+ pages), that is the result of a mail merge. The result is 900+ copies of the same document which is one page, with the only difference being someone's name on the bottom. What I am trying to do, is have a powershell script read the document using itextsharp and save pages that contain a specific string (the person's name) into their respective folder.
This is what I have managed so far.
Add-Type -Path C:\scripts\itextsharp.dll
$reader = New-Object iTextSharp.text.pdf.pdfreader -ArgumentList
"$pwd\downloads\TMs.pdf"
for($page = 1; $page -le $reader.NumberOfPages; $page++) {
$pageText = [iTextSharp.text.pdf.parser.PdfTextExtractor]::GetTextFromPage($reader,$page).Split([char]0x000A)
if($PageText -match 'DAN KAGAN'){
Write-Host "DAN FOUND"
}
}
As you can see I am only using one name for now for testing. The script finds the name properly 10 times. What I cannot seem to find any information on, is how to extract pages that this string appears on.
I hope this was clear. If I can be of any help, please let me know.
Thanks!
I actually just finished writing a very similar script. With my script, I need to scan a PDF of report cards, find a student's name and ID number, and then extract that page and name it appropriately. However, each report card can span multiple pages.
It looks like you're using iTextSharp 5, which is good because so am I. iTextSharp 7's syntax is wildly different and I haven't learned it yet.
Here's the logic that does the page extraction, roughly:
$Document = [iTextSharp.text.Document]::new($PdfReader.GetPageSizeWithRotation($StartPage))
$TargetMemoryStream = [System.IO.MemoryStream]::new()
$PdfCopy = [iTextSharp.text.pdf.PdfSmartCopy]::new($Document, $TargetMemoryStream)
$Document.Open()
foreach ($Page in $StartPage..$EndPage) {
$PdfCopy.AddPage($PdfCopy.GetImportedPage($PdfReader, $Page));
}
$Document.Close()
$NewFileName = 'Elementary Student Record - {0}.pdf' -f $Current.Student_Id
$NewFileFullName = [System.IO.Path]::Combine($OutputFolder, $NewFileName)
[System.IO.File]::WriteAllBytes($NewFileFullName, $TargetMemoryStream.ToArray())
Here is the complete working script. I've removed as little as possible to provide you a near working example:
Import-Module -Name SqlServer -Cmdlet Invoke-Sqlcmd
Add-Type -Path 'C:\...\itextsharp.dll'
# Get table of valid student IDs
$ServerInstance = '...'
$Database = '...'
$Query = #'
select student_id, student_name from student
'#
$ValidStudents = #{}
Invoke-Sqlcmd -Query $Query -ServerInstance $ServerInstance -Database $Database -OutputAs DataRows | ForEach-Object {
[void]$ValidStudents.Add($_.student_id.trim(), $_.student_name)
}
$PdfFiles = Get-ChildItem "G:\....\*.pdf" -File |
Select-Object -ExpandProperty FullName
$OutputFolder = 'G:\...'
$StudentIDSearchPattern = '(?mn)^(?<Student_Id>\d{6,7}) - (?<Student_Name>.*)$'
foreach ($PdfFile in $PdfFiles) {
$PdfReader = [iTextSharp.text.pdf.PdfReader]::new($PdfFile)
$StudentStack = [System.Collections.Stack]::new()
# Map out the PDF file.
foreach ($Page in 1..($PdfReader.NumberOfPages)) {
[iTextSharp.text.pdf.parser.PdfTextExtractor]::GetTextFromPage($PdfReader, $Page) |
Where-Object { $_ -match $StudentIDSearchPattern } |
ForEach-Object {
$StudentStack.Push([PSCustomObject]#{
Student_Id = $Matches['Student_Id']
Student_Name = $Matches['Student_Name']
StartPage = $Page
IsValid = $ValidStudents.ContainsKey($Matches['Student_Id'])
})
}
}
# Extract the pages and save the files
$LastPage = $PdfReader.NumberOfPages
while ($StudentStack.Count -gt 0) {
$Current = $StudentStack.Pop()
$StartPage = $Current.StartPage
$EndPage = $LastPage
$Document = [iTextSharp.text.Document]::new($PdfReader.GetPageSizeWithRotation($StartPage))
$TargetMemoryStream = [System.IO.MemoryStream]::new()
$PdfCopy = [iTextSharp.text.pdf.PdfSmartCopy]::new($Document, $TargetMemoryStream)
$Document.Open()
foreach ($Page in $StartPage..$EndPage) {
$PdfCopy.AddPage($PdfCopy.GetImportedPage($PdfReader, $Page));
}
$Document.Close()
$NewFileName = 'Elementary Student Record - {0}.pdf' -f $Current.Student_Id
$NewFileFullName = [System.IO.Path]::Combine($OutputFolder, $NewFileName)
[System.IO.File]::WriteAllBytes($NewFileFullName, $TargetMemoryStream.ToArray())
$LastPage = $Current.StartPage - 1
}
}
In my test environment this processes about 500 students across 5 source PDFs in about 15 seconds.
I tend to use constructors instead of New-Object, but there's no real difference between them. I just find them easier to read.
I'm trying to get something that looks like UNIX ls output in PowerShell. This is getting there:
Get-ChildItem | Format-Wide -AutoSize -Property Name
but it's still outputting the items in row-major instead of column-major order:
PS C:\Users\Mark Reed> Get-ChildItem | Format-Wide -AutoSize -Property Name
Contacts Desktop Documents Downloads Favorites
Links Music Pictures Saved Games
Searches Videos
Desired output:
PS C:\Users\Mark Reed> My-List-Files
Contacts Downloads Music Searches
Desktop Favorites Pictures Videos
Documents Links Saved Games
The difference is in the sorting: 1 2 3 4 5/6 7 8 9 reading across the lines, vs 1/2/3 4/5/6 7/8/9 reading down the columns.
I already have a script that will take an array and print it out in column-major order using Write-Host, though I found a lot of PowerShellish idiomatic improvements to it by reading Keith's and Roman's takes. But my impression from reading around is that's the wrong way to go about this. Instead of calling Write-Host, a script should output objects, and let the formatters and outputters take care of getting the right stuff written to the user's console.
When a script uses Write-Host, its output is not capturable; if I assign the result to a variable, I get a null variable and the output is written to the screen anyway. It's like a command in the middle of a UNIX pipeline writing directly to /dev/tty instead of standard output or even standard error.
Admittedly, I may not be able to do much with the array of Microsoft.PowerShell.Commands.Internal.Format.* objects I get back from e.g. Format-Wide, but at least it contains the output, which doesn't show up on my screen in rogue fashion, and which I can recreate at any time by passing the array to another formatter or outputter.
This is a simple-ish function that formats column major. You can do this all in PowerShell Script:
function Format-WideColMajor {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)]
[AllowNull()]
[AllowEmptyString()]
[PSObject]
$InputObject,
[Parameter()]
$Property
)
begin {
$list = new-object System.Collections.Generic.List[PSObject]
}
process {
$list.Add($InputObject)
}
end {
if ($Property) {
$output = $list | Foreach {"$($_.$Property)"}
}
else {
$output = $list | Foreach {"$_"}
}
$conWidth = $Host.UI.RawUI.BufferSize.Width - 1
$maxLen = ($output | Measure-Object -Property Length -Maximum).Maximum
$colWidth = $maxLen + 1
$numCols = [Math]::Floor($conWidth / $colWidth)
$numRows = [Math]::Ceiling($output.Count / $numCols)
for ($i=0; $i -lt $numRows; $i++) {
$line = ""
for ($j = 0; $j -lt $numCols; $j++) {
$item = $output[$i + ($j * $numRows)]
$line += "$item$(' ' * ($colWidth - $item.Length))"
}
$line
}
}
}
Is this possible?
I'm brand new to powershell and am currently in the process of converting a vbscript script to Powershell. The following one-liner command seems to do exactly what the entire vbscript does:
Repadmin /istg
which outputs
Repadmin: running command /istg against full DC ST-DC7.somestuff.com
Gathering topology from site BR-CORP (ST-DC7.somestuff.com):
Site ISTG
================== =================
Portland ST-DC4
Venyu ST-DC5
BR-Office ST-DC3
BR-CORP ST-DC7
The problem is I need to return this info (namely the last 4 lines) as objects which contain a "Site" and "ISTG" field. I tried the following:
$returnValues = Repadmin /istg
$returnValues
But this didin't return anything (possibly because Repadmin writes out the lines instead of actually returning the data?)
Is there a way to get the Info from "Repadmin /istg" into an array?
Here's one possible way, using regular expressions:
$output = repadmin /istg
for ( $n = 10; $n -lt $output.Count; $n++ ) {
if ( $output[$n] -ne "" ) {
$output[$n] | select-string '\s*(\S*)\s*(\S*)$' | foreach-object {
$site = $_.Matches[0].Groups[1].Value
$istg = $_.Matches[0].Groups[2].Value
}
new-object PSObject -property #{
"Site" = $site
"ISTG" = $istg
} | select-object Site,ISTG
}
}
You have to start parsing the 10th item of output and ignore empty lines because repadmin.exe seems to insert superflous line breaks (or at least, PowerShell thinks so).
I need help with loop processing an array of arrays. I have finally figured out how to do it, and I am doing it as such...
$serverList = $1Servers,$2Servers,$3Servers,$4Servers,$5Servers
$serverList | % {
% {
Write-Host $_
}
}
I can't get it to process correctly. What I'd like to do is create a CSV from each array, and title the lists accordingly. So 1Servers.csv, 2Servers.csv, etc... The thing I can not figure out is how to get the original array name into the filename. Is there a variable that holds the list object name that can be accessed within the loop? Do I need to just do a separate single loop for each list?
You can try :
$1Servers = "Mach1","Mach2"
$2Servers = "Mach3","Mach4"
$serverList = $1Servers,$2Servers
$serverList | % {$i=0}{$i+=1;$_ | % {New-Object -Property #{"Name"=$_} -TypeName PsCustomObject} |Export-Csv "c:\temp\$($i)Servers.csv" -NoTypeInformation }
I take each list, and create new objects that I export in a CSV file. The way I create the file name is not so nice, I don't take the var name I just recreate it, so if your list is not sorted it will not work.
It would perhaps be more efficient if you store your servers in a hash table :
$1Servers = #{Name="1Servers"; Computers="Mach1","Mach2"}
$2Servers = #{Name="2Servers"; Computers="Mach3","Mach4"}
$serverList = $1Servers,$2Servers
$serverList | % {$name=$_.name;$_.computers | % {New-Object -Property #{"Name"=$_} -TypeName PsCustomObject} |Export-Csv "c:\temp\$($name).csv" -NoTypeInformation }
Much like JPBlanc's answer, I kinda have to kludge the filename... (FWIW, I can't see how you can get that out of the array itself).
I did this example w/ foreach instead of foreach-object (%). Since you have actual variable names you can address w/ foreach, it seems a little cleaner, if nothing else, and hopefully a little easier to read/maintain:
$1Servers = "apple.contoso.com","orange.contoso.com"
$2Servers = "peach.contoso.com","cherry.contoso.com"
$serverList = $1Servers,$2Servers
$counter = 1
foreach ( $list in $serverList ) {
$fileName = "{0}Servers.csv" -f $counter++
"FileName: $fileName"
foreach ( $server in $list ) {
"-- ServerName: $server"
}
}
I was able to resolve this issue myself. Because I wasn't able to get the object name through, I just changed the nature of the object. So now my server lists consist of two columns, one of which is the name of the list itself.
So...
$1Servers = += [pscustomobject] #{
Servername = $entry.Servername
Domain = $entry.Domain
}
Then...
$serverList = $usaServers,$devsubServers,$wtencServers,$wtenclvServers,$pcidevServers
Then I am able to use that second column to name the lists within my foreach loop.