I have a vector of electrical power consumption data which consists of transient, steady and power off states. I would like to identify steady-state starting point by the following condition:
The 5 consecutive elements of the data have difference values between each adjacent element <= threshold value (for this case, let say =10 W)
The first element that meets the condition shows the starting point of steady-state.
Example:
data = [0 0 0 40 70 65 59 50 38 30 32 33 30 33 37 19 ...
0 0 0 41 73 58 43 34 25 39 33 38 34 31 35 38 19 0]
abs(diff(data)) = [0 0 40 30 15 7 9 12 8 3 2 1 3 4 18 19 ...
0 0 41 32 15 9 14 6 5 4 3 4 3 19 19 0]
The sequences of abs(diff(data)) that meet the condition are 8 3 2 1 3 and 6 5 4 3 4. Therefore, the output should show the 10th data element (=30) and 27th data element (=33) as starting point of steady-state (There are 2 times of steady-state detected).
How would I write MATLAB code for this scenario?
(PS: data = 0 shows power off state)
Here's one approach using nlfilter (if the function is not available, you can implement a sliding window yourself):
data = [0 0 0 40 70 65 59 50 38 30 32 33 30 33 37 19 0 0 0 41 73 58 43 34 25 39 33 38 34 31 35 38 19 0];
difs = abs(diff(data));
% Use sliding window to find windows of consecutive elements below threshold
steady = nlfilter(difs, [1, 5], #(x)all(x <= 10));
% Find where steady state starts (1) and ends (-1)
start = diff(steady);
% Return indices of starting steady state
ind = find(start == 1);
Related
numb is a list of numbers:
q))input
42 58 74 51 63 23 41 40 43 16 64 29 35 37 30 3 34 33 25 14 4 39 66 49 69 13..
31 41 39 27 9 21 7 25 34 52 60 13 43 71 10 42 19 30 46 50 17 33 44 28 3 62..
15 57 4 55 3 28 14 21 35 29 52 1 50 10 39 70 43 53 46 68 40 27 13 69 20 49..
3 34 11 53 6 5 48 51 39 75 44 32 43 23 30 15 19 62 64 69 38 29 22 70 28 40..
18 30 60 56 12 3 47 46 63 19 59 34 69 65 26 61 50 67 8 71 70 44 39 16 29 45..
I want to iterate through each row and calculate the sum of the first 2 and then 3 and then 4 numbers etc. If that sum is greater than 1000 I want to stop the iteration on that particualr row and jump on the next row and do the same thing. This is my code:
{[input]
tot::tot+{[x;y]
if[1000<sum x;:count x;x,y]
}/[input;input]
}each numb
My problem here is that after the count of x is added to tot the over keeps going on the same row. How can I exit over and jump on the next row?
UPDATE: (QUESTION STILL OPEN) I do appreciate all the answers so far but I am not looking for an efficient way to sum the first n numbers. My question is how do I break the over and jump on the next line. I would like to achieve the same thing as with those small scripts:
C++
for (int i = 0; i <= 100; i++) {
if (i = 50) { printf("for loop exited at: %i ", i); break; }
}
Python
for i in range(100):
if i == 50:
print(i);
break;
R
for(i in 1:100){
if(i == 50){
print(i)
break
}
}
I think this is what you are trying to accomplish.
sum {(x & sums y) ? x}[1000] each input
It takes a cumulative sum of each row and takes an element wise minimum between that sum and the input limit thereby capping the output at the limit like so:
q)(100 & sums 40 43 16 64 29)
40 83 99 100 100
It then uses the ? operator to find the first occurance of that limit (i.e the element where this limit was equaled or passed) adding one as it is 0 indexed. In the example the first 100 occurs after 3 elements. You might want add one to include the first element after the limit in the count.
q)40 83 99 100 100 ? 100
3
And then it sums this count over all rows of the input.
You could use coverage in this case to exit when you fail to satisfy a condition
https://code.kx.com/q/ref/adverbs/#converge-repeat
The first parameter would be a function that does your check based on the current value of x which will be the next value to be passed in the main function.
For your example ive made a projection using the main input line then increase the indexes of what i am summing each time:
q)numb
98 11 42 97 89 80 73 35 4 30
86 33 38 86 26 15 83 71 21 22
23 43 41 80 56 11 22 28 47 57
q){[input] {x+1}/[{100>sum (y+1)#x}[input;];0] }each numb
1 1 2
this returns the first index of each where running sum is over 100
However this isn't really an ideal use case of KDB
could instead be done with something like
(sums#/:numb) binr\: 100
maybe your real example makes more sense
You can use while loops in KDB although all KDB developers are generally too afraid of being openly mocked and laughed at for doing so
q){i:0;while[i<>50;i+:1];:"loop exited at ",string i}`
"loop exited at 50"
Kdb does have a "stop loop" mechanism but only in the case of a monadic function with single seed value
/keep squaring until number is no longer less than 1000, starting at 2
q){x*x}/[{x<1000};2]
65536
/keep dealing random numbers under 20 until you get an 18 (seed value 0 is irrelevant)
q){first 1?20}\[18<>;0]
0 19 17 12 15 10 18
However this doesn't really fit your use case and as other people have pointed out, this is not how you would/should solve this problem in kdb.
I am working on Matlab and I have a 16x16 matrix where the column values are equal to the row values matrix of nucleotide substitutions
I would like to reshape it so I have only 1 row containing only the unique values (in other words, I would like a row with:
7816 0 ....6432 0 ....8148 20.....
I tried B = reshape(matrix,1,[]); and it works but unfortunately it is giving me also the non-unique values (it is basically taking every row and pasting it just next to the previous one).
Is there any way to do this? Thanks!
Given a symmetric input matrix A:
>> A = randi(30, 5)
A =
9 13 19 23 8
13 4 5 14 19
16 25 13 11 27
12 3 20 25 11
1 12 9 20 27
>> A = A + A.'
A =
18 26 35 35 9
26 8 30 17 31
35 30 26 31 36
35 17 31 50 31
9 31 36 31 54
>> A(A < 10) = 0
A =
18 26 35 35 0
26 0 30 17 31
35 30 26 31 36
35 17 31 50 31
0 31 36 31 54
You can extract the lower triangular portion and turn it into a vector like so:
>> B = A(find(tril(ones(size(A))))).'
B =
18 26 35 35 0 0 30 17 31 26 31 36 50 31 54
Notice that this skips the 26 in the second column, the 35, 30 in the second column, and so on.
This takes the lower triangular portion of a matrix of 1's the same size as A and finds the indices of all of the 1 values. (That gets around the 0 values in the original matrix.) Then it uses the locations of the 1's returned by find to index into the original matrix A. Transpose to make it a row vector.
I have implemented matching pursuit algorithm but i m unable to get the required result.
Here is my code:
D=[1 6 11 16 21 26 31 36 41 46
2 7 12 17 22 27 32 37 42 47
3 8 13 18 23 28 33 38 43 48
4 9 14 19 24 29 34 39 44 49
5 10 15 20 25 30 35 40 45 50];
b=[6;7;8;9;10];
n=size(D);
A1=zeros(n);
R=b;
H=10;
if(H <= 0)
error('The number of iterations needs to be greater then 0')
end;
for k=1:1:H
[c,d] = max(abs(D'*R)); %//'
A1(:,d)=D(:,d);
D(:,d)=0;
y = A1\b;
R = b-A1*y;
end
Output
y=
0.8889
0
0
0
0
0
0
0
0
0.1111
I should get only non-zero value at (2,1) and other values should be zero but I'm getting 2 non-zero value. Can you please help me find out where the error is?
Thanks.
I checked with:
http://www.scholarpedia.org/article/Matching_pursuit
Your functions need to be normalized!
D = D./repmat(sum(D,1),5,1);
I get the following algorithm:
D=[1 6 11 16 21 26 31 36 41 46
2 7 12 17 22 27 32 37 42 47
3 8 13 18 23 28 33 38 43 48
4 9 14 19 24 29 34 39 44 49
5 10 15 20 25 30 35 40 45 50];
D = D./repmat(sum(D,1),5,1);
b=[6;7;8;9;10];
n=size(D);
A1=zeros(n);
R=b;
H=100;
if(H <= 0)
error('The number of iterations needs to be greater then 0')
end;
a = zeros(1,H);
G = zeros(size(D,1),H);
for k=1:1:H
ip = D'*R;
[~,d] = max(abs(ip)); %//'
G(:,k) = D(:,d);
a(k) = ip(d);
R = R-a(k)*G(:,k);
end
% recover signal:
Rrec = zeros(size(R));
for i=1:H
Rrec = Rrec + a(i)*G(:,i);
end
figure();
plot(b);
hold on;
plot(Rrec)
It approximates the signal quite well. But not with D(:,2) at first as expected. Maybe it is a starting point...
Here is the updated code. This is based on the algorithm provided at https://en.wikipedia.org/wiki/Matching_pursuit
clc;
clear all;
D=[1 6 11 16 21 26 31 36 41 46
2 7 12 17 22 27 32 37 42 47
3 8 13 18 23 28 33 38 43 48
4 9 14 19 24 29 34 39 44 49
5 10 15 20 25 30 35 40 45 50];
b=[6;7;8;9;10];
H=10;
for index=1:10
G(:,index)=D(:,index)./norm(D(:,index));
end
G1=G;
n=size(G);
R=b;
if(H <= 0)
error('The number of iterations needs to be greater then 0')
end;
if(H >size(D,2))
error('The number of iterations needs to be less than dictionary size')
end;
bIndex=1:size(G,2);
for k=H:-1:1
innerProduct=[];
for index=1:size(G,2)
innerProduct(index)=dot(R,G(:,index));
end
[c,d] = max(abs(innerProduct));
An(H-k+1)=innerProduct(d);
R = R-(An(H-k+1)*G(:,d));
G(:,d)=[];
strong(H-k+1)=bIndex(d);
bIndex(d)=[];
end
G_new=G1(:,strong);
%% reconstruction
bReconstructed=zeros(size(G_new,1),1);
for index=1:size(G_new,2)
bReconstructed(:,index) = (An(index)*G_new(:,index));
end
b_new=sum(bReconstructed,2)
Yes the atoms in the dictionary must be normalized so that the inner products of the current residual with different atoms can be compared fairly.
You may want to check my OMP implementation which also includes incremental Cholesky updates for the least square step in OMP at https://github.com/indigits/sparse-plex/blob/master/library/%2Bspx/%2Bpursuit/%2Bsingle/omp_chol.m
I have written detailed tutorial notes on OMP in my library documentation at https://sparse-plex.readthedocs.io/en/latest/book/pursuit/omp/index.html
My library sparse-plex contains a C implementation of OMP which is close to 4 times faster than fastest MATLAB implementations. See the discussion here https://sparse-plex.readthedocs.io/en/latest/book/pursuit/omp/fast_omp.html
I am currently doing a project that involves MATLAB and I just can't seem to figure out a way to solve this problem. I have a data set that looks like this:
262 23 34
262 23 34
262 23 35
262 23 38
262 23 38
262 23 39
262 23 40
262 23 41
262 23 42
262 23 43
262 23 45
262 23 46
262 23 47
262 23 48
262 23 50
262 23 50
262 23 51
262 23 52
262 23 55
262 23 57
262 23 58
263 0 0
263 0 2
263 0 4
263 0 7
263 0 10
263 0 15
263 0 25
263 0 29
263 0 32
263 0 39
263 1 1
272 23 28
272 23 30
272 23 56
273 0 1
273 0 2
273 0 3
273 0 3
273 0 4
273 0 4
273 0 5
273 0 5
273 0 6
273 0 8
273 0 10
273 0 32
273 0 37
From the left to the right represents Julian day, hour in UTC, and minute when a tip of a rain gauge was made.
I need to calculate 5 minute totals and its accumulation of each day, for example, on the day 262 the rain tips total from 13-15 minute (since the information before 23:34 is not provided), 13-20 accumulated, 13-25, 13-30,... etc. Like I said each time recorded is when one tip was made and the precipitation amount of one tip is 0.01 inch. So all I need to know is how many tips were made within one day in 5 minute interval.
Could anyone help me please?
Perhaps something like this:
%# all possible days represented in the data
%# (you could also do this for all 1:365 days)
days = unique(X(:,1));
%# cumulative counts in each 5 minutes intervals
%# (each column represents one of the days)
counts = zeros(24*60/5,numel(days));
for i=1:numel(days)
%# indices of the instances in that day
idx = (X(:,1) == days(i));
%# convert hour/minute into minute units
m = X(idx,2).*60 + X(idx,3);
%# count occurences in each 5 minute bin
c = accumarray(fix(m/5)+1, 1, [24*60/5 1]);
%# take the cumulative sum and store it
counts(:,i) = cumsum(c);
end
so for day=262 we have:
>> counts(end-6:end,1)
ans =
0 %# [00:00, 23:30)
2 %# [00:00, 23:35)
6 %# [00:00, 23:40)
10 %# [00:00, 23:45)
14 %# [00:00, 23:50)
18 %# [00:00, 23:55)
21 %# [00:00, 00:00 of next day)
you can convert day-hour-min into minutes. Suppose your data is stored in an n-by-3 matrix called data (how original). Then
>> minutes = data * [24*60; 60; 1]; % counting minutes from the begining
Now you have to define the bins' edges (the intervals for summation):
>> edges = min(minutes) : 5 : max(minutes); % you might want to round the lower and upper limits to align with a 5 minute interval.
Use histc to count how many drops in each bin
>> drops = histc(minutes, edges);
We have a MxN matrix and a constrain cstrn = 100;.
The constrain is the summarize limit of column's elements (per column):
sum(matrix(:,:))<=cstrn.
For a given example as the following:
Columns 1 to 5:
15 18 -5 22 19
50 98 -15 39 -8
70 -15 80 45 38
31 52 9 80 72
-2 63 52 71 6
7 99 32 58 41
I want to find the max number of element per column who fulfill this constrain.
How can i summarize every column element with the others elements in same column and find which sum combinations uses the max number of elements per column?
In the given example solution is:
4 3 5 2 5
where
column 1: 15 + 50 + 31 +7 +(-2)
column 2: 18 +(-15) + 52 or 63 etc.
Thank you in advance.
Since it is always easier to fit small elements into a sum, you can do a sort, followed by the cumulative sum:
m= [
15 18 -5 22 19
50 98 -15 39 -8
70 -15 80 45 38
31 52 9 80 72
-2 63 52 71 6
7 99 32 58 41];
cs = cumsum(sort(m))
cs =
-2 -15 -15 22 -8
5 3 -20 61 -2
20 55 -11 106 17
51 118 21 164 55
101 216 73 235 96
171 315 153 315 168
Now you easily identify at which element you cross the threshold cnstrn (thanks, #sevenless)!
out = sum(cs <= cnstrn)
out =
4 3 5 2 5
I'd add to Jonas's answer, that you can impose your constraint in a way that outputs a logical matrix then sum over the 1's and 0's of that matrix like so:
cstrn = 100
m= [
15 18 -5 22 19
50 98 -15 39 -8
70 -15 80 45 38
31 52 9 80 72
-2 63 52 71 6
7 99 32 58 41];
val_matrix = cumsum(sort(m))
logical_matrix = val_matrix<=cstrn
output = sum(logical_matrix)
Giving output:
cstrn =
100
val_matrix =
-2 -15 -15 22 -8
5 3 -20 61 -2
20 55 -11 106 17
51 118 21 164 55
101 216 73 235 96
171 315 153 315 168
logical_matrix =
1 1 1 1 1
1 1 1 1 1
1 1 1 0 1
1 0 1 0 1
0 0 1 0 1
0 0 0 0 0
output =
4 3 5 2 5
Here is a logic, on mobile so can't give a code.
Check this out. Go to a column, sort it ascending order, loop to sum, break when hits <=100. Get counter. Refer back to original column, get the indices of elements matching the elements you just summed up :-)