Are earlier historical metrics published to CloudWatch Delayed? - powershell

I've got a custom CloudWatch metric based on a historical value, so I'm publishing them with explicit timestamps. I've seen an issue using both the C# and PowerShell APIs where I don't see all my metrics right away.
In the example below, I'm creating sample points for every hour for the past two weeks, but I only see the last ~48 hours in my graph in the CloudWatch console.
Initialize-AWSDefaults
$cwNamespace = 'Historical CW Sanity Check'
# generate some data
$now = [DateTime]::Now.ToUniversalTime()
$startTime = $now - [TimeSpan]::FromDays(14)
$t = $startTime
$x = 0
while ($t -le $now)
{
$t += [TimeSpan]::FromHours(1)
$datum = New-Object -TypeName 'Amazon.CloudWatch.Model.MetricDatum'
$datum.Unit = [Amazon.CloudWatch.StandardUnit]::Count
$datum.Value = 2 * $x
$datum.Timestamp = $t
$datum.MetricName = 'Test 2'
$x +=1.0
Write-Host "($t, $x)"
Write-CWMetricData -Namespace $cwNamespace -MetricData $datum
}
From the docs, I thought it supported up to two weeks' worth of historical data. I'm wondering why I see the new points before the old points (or if the old points are going to show up at all). It looks like I'm only getting two days' worth.

From the help text on Write-CWMetricData:
Data that is timestamped 24 hours or more in the past may
take in excess of 48 hours to become available from submission time using
GetMetricStatistics.
I'm betting the same limit holds true for the graphs in the Console.

Related

PowerShell, simple timed script delivers unreliable results

I´m quite new to PowerShell and just a bit more experienced with programming as such.
The goal of the current excercise our teacher gave us, is basically to code a simple crossroads traffic light.
I use Windows Forms to display labels, that change colors.
Now I coded a button, that is supposed to run through the traffic light "phases" once.
Thanks to prior experiences I went with the method to have a incrementing variable and a Function that checks the value of said variable.
But the colors turn the intended way once every 4 tries or so, without a recognizable logic to it.
Am I missing something basic in the way PS reads code and the timing thereof?
Button, supposed to reset the variable, then increment it, then call the function. Due to the Write-Host I can at lease say that the variable does count to 4 as intended
$Button1 = New-Object System.Windows.Forms.Button;
$Button1.Location = New-Object System.Drawing.Size(300,100);
$Button1.Text = "Next Phase";
$Window.Controls.Add($Button1);
$Button1.Add_Click({
#Set Phase 0, then increment & update Function Phase
$current_phase = 0;
Do {
$current_phase++;
Write-Host $current_phase;
Phase_update;
} Until($current_phase -gt 3)
Two of the functions minus some of the color changes, intended to change the colors and then wait a while
Function Phase_update
{
#Phase A
if ( $current_phase -eq 1 )
{
$LightTop.ForeColor = 'red';
}
#Phase B
elseif ( $current_phase -eq 2 )
{
$LightTop.ForeColor = 'green'
}
Start-Sleep -Seconds 2
}
# Show Form
[void]$Window.ShowDialog()
Thanks in advance for any input, problem specific and in general

Graph's API bitlocker/recoveryKeys calls return less than a handful of keys per call

I have been following a couple of solutions to retrieve a list of Azure AD registered BitLocker keys using the MS Graph API. The code I am using is:
$AzureTokenURI = "https://login.microsoftonline.com/$TenantID/oauth2/v2.0/token"
$BitlockerURI = "https://graph.microsoft.com/v1.0/informationProtection/bitlocker/recoveryKeys"
$TokenRequestBody = "grant_type=password&username=$($SACred.UserName)&client_id=$ClientId&client_secret=$ClientSecret&password=$($SACred.GetNetworkCredential().Password)&scope=https://graph.microsoft.com/.default"
$TSStart = [datetime]::Now
$BitlockerKeys = #()
while ($BitlockerURI) {
If (!$ResponseJSON -or ([datetime]::Now - $TSStart).TotalSeconds -gt ($ResponseJSON.expires_in - 300)) {
$TSStart = [datetime]::Now
$ResponseJSON = Invoke-WebRequest -Uri $AzureTokenURI -Method POST -Body $TokenRequestBody -UseBasicParsing |ConvertFrom-Json
$BitlockerHeader = #{Authorization = "$($ResponseJSON.token_type) $($ResponseJSON.access_token)"; 'ocp-client-name' = 'BL_List_Keys_IDs'; 'ocp-client-version' = '1.0'}
Write-Host "Token Renewed"
}
$BitlockerKeysResponse = Invoke-RestMethod -Uri $BitlockerURI -Headers $BitlockerHeader -Method Get
$BitlockerKeys += $BitlockerKeysResponse.Value
Write-Host $BitlockerKeys.Count.ToString().PadRight(6), (New-TimeSpan $TSStart ([datetime]::Now)).ToString('hh\:mm\:ss\.fff')
$BitlockerURI = $BitlockerKeysResponse.'#odata.nextLink'
}
Now everything works, however when the keys are coming (little less than 5,000 in total) I realize that each call brings back just a few keys and I cannot seem to find a method to force the server side paging to a reasonable value (say a couple thousand keys per call). See the output below for the first 30 keys:
Keys
Time Lapsed
1
00:00:13.343
5
00:00:24.452
6
00:00:31.984
10
00:00:40.765
14
00:00:49.171
18
00:00:56.577
22
00:01:03.937
27
00:01:12.468
30
00:01:20.218
This implies that just 5k keys take about 2 hours to download and this is nearly as bad as querying each device one by one which defeats the purpose of using the "/informationProtection/bitlocker/recoveryKeys" method in first instance. There is only one call in the list that returned 200 keys and that took 8 seconds which tells me that lapsed time does not depend on no of keys returned. Therefore the only way to make this method efficient is to force hundreds of keys to be returned at the same time. How can I force the API to return hundreds or thousands of keys per each call?
have you tried using $top={number} to get the specific number of calls that you want? Please refer to the paging - docs. Let us know if that worked for you and please feel free to reach out to us.

Counting date entries from file. - Powershell

I have a file which have several lines with different timestamps, however I want to count the timestamps from the last 15 minutes.
Counting timestamps would be easy enough, but I cannot figure out how to get the timestamps from the last 15 minutes.
Example of timestamps
2020-07-20T18:00:00Z
I have tried several things, but I couldn't get them to work. Perhaps it's just me...
I'm not sure what the formatting in your original file looks like but this code will count the entries within the last 15 minutes.
$lastFifteenMinutesCount = 0
$comparisonDate = [datetime]::UtcNow
Get-Content C:\tmp\overflow\63003625\stuff.txt | % {
if ($_ -match '(\d{4}-\d{2}-\d{2}T\d{1,2}:\d{1,2}:\d{1,2}Z)') {
$sDate = [datetime]::Parse($Matches[1]).ToUniversalTime().AddMinutes(15)
if ($sDate -ge $comparisonDate) {
$lastFifteenMinutesCount++
}
}
}
$lastFifteenMinutesCount

store data by day and hour - powershell

Id like to be able to store, change, and retrieve data by day and hour using powershell.
For instance, I have a list of data from which I can extract the dayOfMonth, hour, max, active, average.
I would like to be able to stick the 3 latter values in an array indexed by day and hour.
The raw data might look like this: (the avg val wouldnt be there yet):
12/13/19,08:13:04,13,13
12/13/19,08:43:04,10,9
12/13/19,09:13:04,11,9
12/13/19,09:43:04,12,7
12/13/19,08:13:04,10,12
12/14/19,08:13:04,8,2
12/14/19,08:43:04,10,4
12/14/19,09:13:04,8,7
12/14/19,09:43:04,11,1
12/14/19,10:13:04,9,7
I wish to be able to store the maxChannel,maxActive, and average, for each hour of each day, so the data in the array would look like this:
day,hr,max,actv,avg
13,8,13,13,11
13,9,12,9,8
13,10,12,2,2
14,8,10,4,3
14,9,11,7,4
14,10,9,7,7
I tried to store it in a previously created array
$arrOut2 = #() #summary data
like this(which needless to say didnt work):
initialize each needed row
ForEach ($line in $arrFile)
{
$arrLine = $line.split(",") #create an array with each word in line its own element
$Date=$arrLine[0]
$Time=$arrLine[1]
$ChanM=$arrLine[2]
$ChanA=$arrLine[3]
$DateA=$Date.Split("/")
$Day=$DateA[1]
$TimeA=$Time.Split(":")
$Hour=[int]$TimeA[0]
$arrOut2 += ,#[$Day][$Hour](0,0,0)
}
ForEach ($line in $arrFile)
{
$arrLine = $line.split(",") #create an array with each word in line its own element
$Date=$arrLine[0]
$Time=$arrLine[1]
$ChanM=$arrLine[2]
$ChanA=$arrLine[3]
$DateA=$Date.Split("/")
$Day=$DateA[1]
$TimeA=$Time.Split(":")
$Hour=[int]$TimeA[0]
IF (($Hour -ge 8) -And ($Hour -lt 17)) {
$AvCnt=$AvCnt+1 #count Intervals for averaging
$AvTot=$AvTot+$ChanA #sum active channels for averaging
$AvgActChan=$AvTot/$AvCnt
IF($ChanC -GT $arrOut2[$Hour][0]){$arrOut2[$Hour][0]=$ChanC} #grab max chanC frm current hr
IF($ChanA -GT $arrOut2[$Hour][1]){$arrOut2[$Hour][0]=$ChanC} #grab max chan Actv frm curr hr
$ArrOut2[$Day][$Hour][2]=$AvgActChan #grab curnt chan Avg Active from current hour
}
}
As you can see in the first forloop, I am attempting to use day and hour as indices. Powershell just doesnt like that - at least in that format.
Does anyone have any ideas of how I can accomplish this? I know I could keep track of the current day and hour, then process it in separate variables, then when I see the day or hour change, write those to a separate array, but the data is quite a bit messier than Ive represented in this example and that would make the whole thing quite messy, when processing it within an array would be so much cleaner.
Appreciate any suggestions
thanks
I would sort it this way:
$data = import-csv data.txt -header date,time,a,b
$data | sort date,time

Creating PowerShell if statement for True over time

I have CSV document. Each row of the CSV has a timestamp, along with various performance metrics as they were when recorded at that time.
What I want to be able to do is consume the csv and perform a logic test on it of this kind of nature:
If said metric is above a specified threshold for a specified amount of time = True
I can easily setup a logic statement comparing two numbers/metrics, but I wasn't sure how to add in the component as to whether that condition is true for a certain amount of time.
CSV might look something like this:
time,KB/sec,IOPS,Avg. Latency ms
8/6/2017 10:30,10616.79606,117.2243058,35.63250298
8/6/2017 10:20,11337.82872,149.4084982,28.55670254
8/6/2017 10:17,12276.47016,172.2729097,25.43457276
8/6/2017 10:14,10125.01863,106.2432985,37.38492431
8/6/2017 10:11,12185.1857,127.452136,32.40784617
So I might want to know if the latency is consistently over a certain amount for a couple of hours for instance
I could use a bit of direction and would greatly appreciate any help anyone can provide.
First you need to convert the values from your CSV from string to the proper type (DateTime and double respectively).
Import-Csv 'C:\path\to\your.csv' | ForEach-Object {
$_.time = [DateTime]$_.time
$_.'KB/sec' = [double]$_.'KB/sec'
$_.IOPS = [double]$_.IOPS
...
}
Check if the value in question is above the given threshold. You have 3 cases to cover:
The value exceeds the threshold for the first time: remember the current timestamp.
The value exceeds the threshold and also exceeded it in the previous iteration: calculate the difference between the current timestamp and the previously remembered timestamp. Return $true if the timespan is greater than the maximum duration you want to accept.
The value does not exceed the threshold: forget the previously remembered timestamp.
Well not knowing exactly what you're aiming for perhaps the below will help.
function Monitor-Something
{
Param
(
$Date = '8/6/2017',
$StartTime = '10:17',
$StopTime = '10:30',
$Threshold,
$CSVLogfile = 'C:\temp\test.csv'
)
Begin
{
}
Process
{
$csv = Import-Csv $CSVLogfile
[datetime]$monitorStart = $("$Date $StartTime")
[datetime]$monitorStop = $("$Date $StopTime")
foreach($item in $csv)
{
[datetime]$objDateTime = $item.time
if($objDateTime -gt $monitorStart -and $objDateTime -lt $monitorStop)
{
#do stuff here, compare against your threshold report things build an object for an output
$item.IOPS #This is here just to show you a successful result
}
}
}
End
{
#return some report or object that you can analyze
}
}