How do I generate a list of times by minute in KDB? - kdb

Let's say I have two datetimespans:
s: 2019.06.06D11:00:00.000000000
e: 2022.03.01D23:53:00.000000000
What would be the cleanest way to get a list of every inclusive minute between these two? (casting to z might be easier).

This approach would be faster that looping:
q)\ts a:s+0D00:01*til 1+"j"$(e-s)%0D00:01
17 33555136
q)\ts b:(e>)(+[00:01])\s
1640 48195648
q)a~b
1b

I solved my own issue with a while.
(e>)(+[00:01])\s
If there is a better way, please let me know!

Related

promql example with related fields but different labels

I'm using Prometheus and Grafana, and I'm trying to track a web server app.
I want to graph the average duration in ms of a particular query. I think I can get there from the data below, but I'm struggling.
My two sets of values:
rate(http_server_request_duration_seconds_sum[5m])
Element Value
{instance="dbserver:5000",job="control-tower",method="get",path="/api/control/v1/node/config.json"} 0.0010491088980113385
{instance="dbserver:5000",job="control-tower",method="get",path="/api/schedule/v1/programs/:id.json"} 0
{instance="dbserver:5000",job="control-tower",method="get",path="/api/schedule/v1/users.json"} 0
{instance="dbserver:5000",job="control-tower",method="get",path="/metrics"} 0.00009133616130826839
{instance="dbserver:5000",job="control-tower",method="post",path="/api/caption/v1/messages.json"} 0
{instance="dbserver:5000",job="control-tower",method="post",path="/api/caption/v1/sessions.json"} 0
{instance="dbserver:5000",job="control-tower",method="post",path="/api/schedule/v1/programs.json"} 0
{instance="dbserver:5000",job="control-tower",method="put",path="/api/caption/v1/sessions/captioners.json"} 0
{instance="dbserver:5000",job="control-tower",method="put",path="/api/control/v1/agents/:id.json"}
rate(http_server_requests_total[5m])
Element Value
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="get",path="/api/control/v1/node/config.json"} 0.03511075688258612
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="get",path="/api/schedule/v1/programs/:id.json"} 0
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="get",path="/api/schedule/v1/users.json"} 0
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="get",path="/metrics"} 0.06671043807691363
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="post",path="/api/caption/v1/sessions.json"} 0
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="post",path="/api/schedule/v1/programs.json"} 0
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="put",path="/api/caption/v1/sessions/captioners.json"} 0
{code="200",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="put",path="/api/control/v1/agents/:id.json"} 0
{code="422",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="post",path="/api/schedule/v1/programs.json"} 0
{code="502",host="dbserver:5000",instance="dbserver:5000",job="control-tower",method="post",path="/api/caption/v1/messages.json"}
They have different labels. For this, I only care where path="/api/caption/v1/messages.json".
I think I need to use a combination of rate, sum, and "on" or "ignore", but I haven't been able to get on or ignore to work at all.
I can get the numerator (in seconds) with:
rate( http_server_request_duration_seconds_sum { path="/api/caption/v1/messages.json" }[5m])
And that returns:
{instance="dbserver:5000", job="control-tower", method="post", path="/api/caption/v1/messages.json"}
But the denominator can have different return codes, so I have to sum those, and I need to do some ignore or on or something, but I haven't found an example that helps me out, and I'm really new at this.
Anyone?
Okay, I continued to play. Because I only have one path I worry about, i figured out I could sum the rates. I think this works:
sum( rate( http_server_request_duration_seconds_sum {path="/api/caption/v1/messages.json"}[2h])) / sum( rate( http_server_requests_total{ path="/api/caption/v1/messages.json"}[2h]))
I changed the sample rate as my sample data fell off my 5-minute window, and I had zeros.
I THINK what this is doing is summing the rates, which gets rid of all the labels. And I THINK what it's also doing is using 2 hours of data. I think the rate value is how quickly the value changed over that 2 hour period.
I would love comments.
This solution won't work if I want one chart to include other paths, and I'm still not sure what to do about that, so this solves my current problem but still doesn't help me figure out how to do something similar with ignore or on.

using the same function but on different rows

I want to estimate the co(variance) matrix of two assets classes (with different derivatives) of every month.. So let's assume that each month is 25 days. I will get the following code,
covar=cov([elec(1:25,:) ngas(1:25,:)])
However, I have like 5 years of data so rewrite everything seems like a waste of time, and I think there has to be an easier way to fix this problem.
ps. I do think the answer to my question is already answered somewhere, but I do not know the words the search on. Thanks for your reply
Sounds like you just need a for loop?
counter = 0;
cover{floor(size(elec,1)/25)} = []; %//Pre-allocation
for day = 1:25:size(elec,1)
counter = counter + 1;
covar{counter}=cov([elec(day:day+25-1,:) ngas(day:day+25-1,:)])
end

Finding a Range in CoffeeScript

I understand how to define an array range in CoffeeScript
lng[1..10]
However if I have
data = 10
What's the best way to find if 10 is within a range of 1 and 11?
if data is between(1..11)
return true
There is no "between" keyword, but you can utilize a normal array-range:
if data in [1..11]
alert 'yay'
But that's a bit of an overkill, so in simple cases I'd recommend a normal comparison:
if 1 <= data <= 11
alert 'yay'
If you don't mind polluting the native prototypes, you can add a between method to the Number objects:
Number::between = (min, max) ->
min <= this <= max
if 10.between(1, 11)
alert 'yay'
Although i personally wouldn't use it. if 1 <= something <= 11 is more direct and everyone will understand it. The between method, instead, has to be looked up if you want to know what it does (or you'd have to guess), and i think it doesn't add that much.

LINQPad taking long time to execute?

//LINQPad version 4.31 - for me this code takes 1 minute and 40 seconds to execute. Can anyone else //confirm / explain whats going on
var v = new {Name="jon",Age=33};
v.Age.Dump();
v.GetType().Dump();
It's instant for me with V4.38.03.
The output from v.GetType().Dump(); is quite complex due to the anonymous type. I expect something has been improved in the latest version.

How to round the minute of a timestamp to increments of 15?

I'm building an application to record when my cat has an asthma attack. I'm not interested in the exact time since glancing at the time in interval of 15 minutes is easier to review (e.g. rounding 9:38am should be recorded as 9:45am).
I looked for a UDF at cflib.org for this but couldn't find one. I tinkered with CF's round function but I'm not getting it to do what I want.
Any advice?
This could do with a bit more polish (like data type validation) but it will take a time value and return it rounded to the nearest 15-minute increment.
<cfscript>
function roundTo15(theTime) {
var roundedMinutes = round(minute(theTime) / 15 ) * 15;
var newHour=hour(theTime);
if (roundedMinutes EQ 60) {
newHour=newHour + 1;
roundedMinutes=0;
}
return timeFormat(createTime(newHour,roundedMinutes,0),"HH:mm");
}
</cfscript>
I'm not familiar with the format of the timestamp here, but generally when I want to round I do something like floor((val + increment / 2) / increment) * increment
I like Al Everett's answer, or alternatively store the actual time to preserve the most accurate time, then use query of query in the view and use between :00 and :15 to show the time in 15min period.
If you use Henry's suggestion to store the precise time in the database (principle: if there's no cost, prefer to preserve data), then you can simply use Al Everett's rounding function whenever you display the data to the user.