How to group dates within a certain time interval - matlab

I have an array of dates and i would like to discard any dates that don't have at least one another date in a specific time interval, for example 5 minutes. I need to find a smart way to do it, as loops take forever with a larger dataset.
input data:
2009 07 07 16:01:30
2009 07 07 16:04:06
2009 07 07 16:05:00
2009 07 07 16:12:00
2009 07 07 16:19:43
2009 07 07 16:24:00
results:
2009 07 07 16:01:30
2009 07 07 16:04:06
2009 07 07 16:05:00
2009 07 07 16:19:43
2009 07 07 16:24:00
The value 2009 07 07 16:12:00 was discarded because it was more than 5 minutes away from any other timestamp.
Thanks,
Cristi
Secondary issue:
Both Dan and nkjt suggested an implementation that worked, thanks! What if the dates are part of 2 groups: A or B and i want to find if there exist a date from group A that has a corresponding date in group B that is within a number of seconds/minutes apart? if not just remove the date from group A..

You can use diff. You'll need to use datenum to convert your data into a vector of values. In MATLAB datenums, "1" is a single day, so you can define a datenum step in terms of a time unit divided by the number of those in a day:
s = num_mins/(24*60);
Here's the trick with diff:
x = datenum(mydata);
s = num_mins/(24*60);
% for increasing times we shouldn't need the `abs` but to be safe
d = abs(diff(x));
q = [d (s+1)]>s&[(s+1) d]>s;
(You can use datestr to convert back, or apply q to the original data)
How it works:
The output of diff is one shorter than the original - it's just the difference between neighbouring values. We need it to be directional - to check each value against the one that comes before and after.
[d (s+1)]>s makes a vector the same length as the original, and checks if the difference values are larger than s. Because we set the last value to be s+1, the final value will always return true. This is a check to whether there's a gap between a value and the one following it (so for the final value this is always true).
[(s+1) d]>s does the same but on the other side. Again, we are setting one value, this time the first, to be larger than s so it's always true.
Combining these gives us the points where the difference is more than five minutes on either side (or for the end points, on one side).

Related

Identify cells which satisfy a specific condition and copy them Excel

I have a peculiar problem which I need some help with. I have a reading taken every day of the year for last 10 years. Column A contains dates from 01 Jan to 31 Dec, whereas column B contains values taken every day in 2012, Column C contains values taken everyday in 2013, and so on and so forth.
What I am looking for is to copy data from Col A and Col B and paste it on new worksheet only where reading was 0 on consecutive days. For example:
Col A: Date
Col B: 2012
01 Jan
0
02 Jan
1
03 Jan
0
04 Jan
0
05 Jan
3
06 Jan
0
07 Jan
0
08 Jan
0
09 Jan
2
10 Jan
0
From the above, I need to pick 03 Jan, 04 Jan, 06 Jan, 07 Jan, and 08 Jan only, since these are the dates where 0 occurs consecutively.
Many thanks in advance for help!
I understand that isolating dates where 0 occurs is very simple. Its the condition of consecutive which has proven tricky for me.

Subtract 2 text dimensions in Tableau

I wanted to know if it is possible to subtract 2 dimensions which are text fields. For eg.
Kpi Name Jun May Apr
$Deposits 234 567 321
Here I have Kpi name in rows and date in columns. Kpi_value is text.Is it possible to take difference between Jun value and May Value? If this difference is positive I want to put a colored shape next to it.
Thanks!

Where is my interpretation of ASN1 der wrong?

Here is what my structure looks like:
SET OF
SEQUENCE:
INTEGER: XX
INTEGER: YY
My encoding looks like this:
11 08 10 06 02 01 XX 02 01 YY
11 08 -- SET OF
10 06 -- SEQUENCE
However, when I decode with openssl, I don't see the expected output. It looks like
0:d=0 hl=2 l= 8 prim: SET
0000 - 10 06 02 01 XX 02 01 YY-
This is not what I expected to see. (Look at the structure I wanted it to look like)
I am not sure what I am missing. Any help would be much appreciated.
A SET and SEQUENCE are constructed types. That means that the bit that indicates a constructed type in the tag needs to be set. That would be bit 5 or 6 (depending if you start with bit 0 or 1). If the bit isn't set then the parser will view it as a primitive type, which means it has a single value instead of children. This is why you get prim in your output. The tag number is still 17 or 16 which denotes a SET OF or SEQUENCE, so the structure is still seen to be a SET.
So instead of 11 and 10 you should be using values 31 and 30. Then your code should parse correctly.

How to identify date numbers corresponding to non-business days and replace them with by the next available business day in MATLAB

I am working with time series data in MATLAB. I have got two vectors of date numbers, one of which relies on a somewhat subjective data source. If both vectors were perfectly accurate, all date numbers should correspond to trading days while one vector would be a 'proper subset' of the other. Unfortunately this is not the case because one of the vectors contains several date numbers which correspond to non-business days. I would like to find a way to replace all non-business days in this vector with the next available business day.
Example:
datenumbers = [736062;736063;736064;736065;736066;736067]
% corresponds to [wed, thu, fri, sat, sun, mon]
This contains [736065;736066] which corresponds to the upcoming weekend. Because these are not working days I would like to identify the date number corresponding to the subsequent monday and change both entries so that:
datenumbers = [736062;736063;736064;736067;736067;736067]
% corresponds to [wed, thu, fri, mon, mon, mon]
One way to do it is simply to loop through all your values and change them if they are Saturday or Sunday.
datenumbers = [736062;736063;736064;736065;736066;736067];
for i = 1:length(datenumbers)
weekDay = mod(datenumbers(i),7);
if weekDay == 1 || weekDay == 2
datenumbers(i) = (3-weekDay) + datenumbers(i);
end
end

How to convert a large number to base 36 using DC or other

I am trying to represent the maximum 64-bit unsigned value in different bases.
For base 2 (binary) it would be 64 1's:
1111111111111111111111111111111111111111111111111111111111111111
For base 16 (hex) it would be 16 F's
FFFFFFFFFFFFFFFF
For base 10 (decimal) it would be:
18446744073709551615
I'm trying to get the representation of this value in base 36 (it uses 0-9 and A-Z). There are many online base converters, but they all fail to produce the correct representation because they are limited by 64-bit math.
Does anyone know how to use DC (which is an extremely hard to use string math processors that can handle numbers of unlimited magnitude) and know how to do this conversion? Either that or can anyone tell me how I can perform this conversion with a calculator that won't fail due to integer roll-over?
I mad a quick test with ruby:
i = 'FFFFFFFFFFFFFFFF'.to_i(16)
puts i #18446744073709551615
puts i.to_s(36) #3w5e11264sgsf
You may also use larger numbers:
i = 'FFFFFFFFFFFFFFFF'.to_i(16) ** 16
puts i
puts i.to_s(36)
result:
179769313486231590617005494896502488139538923424507473845653439431848569886227202866765261632299351819569917639009010788373365912036255753178371299382143631760131695224907130882552454362167933328609537509415576609030163673758148226168953269623548572115351901405836315903312675793605327103910016259918212890625
1a1e4vngailcqaj6ud31s2kk9s94o3tyofvllrg4rx6mxa0pt2sc06ngjzleciz7lzgdt55aedc9x92w0w2gclhijdmj7le6osfi1w9gvybbfq04b6fm705brjo535po1axacun6f7013c4944wa7j0yyg93uzeknjphiegfat0ojki1g5pt5se1ylx93knpzbedn29
A short explanation what happens with big numbers:
Normal numbers are Fixnums. If you get larger numbers, the number becomes a Bignum:
small = 'FFFFFFF'.to_i(16)
big = 'FFFFFFFFFFFFFFFF'.to_i(16) ** 16
puts "%i is a %s" % [ small, small.class ]
puts "%i\n is a %s" % [ big, big.class ]
puts "%i^2 is a %s" % [ small, (small ** 2).class ]
Result:
268435455 is a Fixnum
179769313486231590617005494896502488139538923424507473845653439431848569886227202866765261632299351819569917639009010788373365912036255753178371299382143631760131695224907130882552454362167933328609537509415576609030163673758148226168953269623548572115351901405836315903312675793605327103910016259918212890625
is a Bignum
268435455^2 is a Bignum
From the documentation of Bignum:
Bignum objects hold integers outside the range of Fixnum. Bignum objects are created automatically when integer calculations would otherwise overflow a Fixnum. When a calculation involving Bignum objects returns a result that will fit in a Fixnum, the result is automatically converted.
It can be done with dc, but the output is not extremely useful.
$ dc
36
o
16
i
FFFFFFFFFFFFFFFF
p
03 32 05 14 01 01 02 06 04 28 16 28 15
Here's the explanation:
Entering a number by itself pushes that number
o pops the stack and sets the output radix.
i pops the stack and sets the input radix.
p prints the top number on the stack, in the current output radix. However, dc prints any output with a higher radix than 16 as binary (not ASCII).
In dc, the commands may be all put on the same line, like so:
$ dc
36o16iFFFFFFFFFFFFFFFFp
03 32 05 14 01 01 02 06 04 28 16 28 15
Get any language that can handle arbitrarily large integers. Ruby, Python, Haskell, you name it.
Implement the basic step: modulo 36 gives you the next digit, division by 36 gives you the number with the last digit cut out.
Map the digits to characters the way you like. For instance, '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ'[digit] is fine by me. Append digits to the result as you produce them.
???
Return the concatenated string of digits. Profit!