I have the code below in which I am trying to start the "number_panels" and "number_turbines" for loop from a non zero number.
More specifically I am trying for 3000 to 4000 "number_panels" with 500 panel interval
and from 5 to 8 "number_turbines" with one turbine interval
i.e
number_of_days = 2;
for number_panels = 3000:500:4000 % range of PV panel units examined
for number_turbines = 5:8 % range of wind turbine units examined
for h=1:24 %# hours
for d = 1:number_of_days %# which day
n = h + 24*(d-1);
% hourly_deficit_1(...,..., h, d)= Demand(n)-(PV_supply(n)... %
hourly_deficit(number_panels + 1, number_turbines + 1, h,d) = hourly_annual_demand(n) - (hourly_annual_PV(n)*number_panels) - (hourly_annual_WT(n)*number_turbines);% hourly power deficit (RES supply with demand)
if hourly_deficit(number_panels + 1, number_turbines + 1, h,d)< 0 % zero out negative hourly deficit values (this is power surplus from RES)
hourly_deficit(number_panels + 1, number_turbines + 1, h,d) = 0;
end
When I do this I get a size(hourly_deficit) = 4001,9,24,2 whereas I am expecting and trying to achieve a 3,4,24,2 size. Does anyone know where I am going wrong?
The value of the variable number_panels starts at 3000 instead of 0 or 1. Thus when you index a matrix with that variable as the index value Matlab thinks you are wanting the 3001st index and thus gives you a matrix that is 3000 zeros with the 3001st being set to what you ask.
If you follow the loop into it's next cycle the value of number_panels becomes 3500. You now are indexing at 3501, based on your code. This means that all of the places from 3002 to 3500 will be filled with zeros and 3501 will be set to whatever value you give it.
The same logic applies to number_turbines The only difference is that you'll be indexing by 1 instead of by 500 like you are with number_panels.
If you want to get back to the size matrix you are expecting you'll need to modify the way you call the index values. This could be done a number of ways. You could have a counter within the for-loop or you could use modulus math. Modulus math doesn't work well when you're using a step size that isn't 1. It also doesn't work when you get to the point where you have an index value that is a multiple of your starting index.
You'll have to work out what will work best for you in that arena. Especially since you want to use a step size that isn't 1. But for the number_turbines that goes from 5 to 8, you can simple index using number_turbines - 5 + 1 or more concise number_turbines-4.
For clarity, here is the code you provided with the necessary tweaks to show the use of what was mentioned in the comments. Please take note that you will need to modify the -4 for the number_turbines index value should you start at something other than 5. Also note that you need to index the number_panels vector now since it's not a looped value.
Hope this helps!
number_of_days = 2;
number_panels = 3000:500:4000;
for idx_number_panels = 1:length(number_panels) % range of PV panel units examined
for number_turbines = 5:8 % range of wind turbine units examined
for h=1:24 %# hours
for d = 1:number_of_days %# which day
n = h + 24*(d-1);
% hourly_deficit_1(...,..., h, d)= Demand(n)-(PV_supply(n)... %
hourly_deficit(idx_number_panels , number_turbines -4, h,d) = hourly_annual_demand(n) - (hourly_annual_PV(n)*number_panels(idx_number_panels)) - (hourly_annual_WT(n)*number_turbines);% hourly power deficit (RES supply with demand)
if hourly_deficit(idx_number_panels, number_turbines -4, h,d)< 0 % zero out negative hourly deficit values (this is power surplus from RES)
hourly_deficit(idx_number_panels, number_turbines -4, h,d) = 0;
end
end
end
end
end
Related
I have been trying to write a MATLAB function that does the following:
Takes inputs "upper" and "number" which are both positive integers. The program should take the interval [0,"upper"] and cut it into "number" subintervals which have integer endpoints and are the same length, except for the last interval (if the interval isn't cleanly divisible by number, then the last interval can contain less integer entries).
Then the program should make a vector x with entries that are the beginnings/ends of those subintervals.
For example, if I have upper = 11 and number = 3, then the program should take the vector
R = [0 1 2 3 4 5 6 7 8 9 10 11]
and generate
x = [0 4 8 11].
It should first create subinterval vectors
r1 = [0 1 2 3 4]
r2 = [4 5 6 7 8]
r3 = [8 9 10 11]
and then loop through x and assign 0 to the first entry and the endpoints of the rs to the other entries of x but I am at a loss as to how to tell MATLAB to do this.
My other problem is that I don't know how to tell MATLAB how to figure out if the interval [0, "upper"] is cleanly divisible by "number" and if not, how to then do to make the last subinterval shorter than the others.
I've sat at this for several days, but I'm not getting anywhere.
Any ideas or pointers in the right direction would be greatly appreciated,
thank you.
I've figured out how to handle cases for when mod(upper, number) = 0, i.e., when upper is cleanly divisible by number. For those cases, I simply make
delta = upper/number;
and then
for i=1:(number+1)
x(i) = (i-1)*delta;
end
Where the difficulties arise is when upper and number aren't cleanly divisible.
Given
upper = 12;
number = 3;
You can generate R using
R = 0:upper; % [0, 1, 2, ..., upper]
If number was a divisor of upper then the output interval would be upper/number. If not then this quotient should be rounded up, so there's a smaller "chunk" left in the final interval. So
interval = ceil(upper/number);
Now we just need to build the output array x. If number was a divisor of upper then the colon notation would give what you want
x = 0:interval:upper; % only useful if number is a divisor of upper
Otherwise, this will not actually include upper; we can handle the two possible cases in an if:
if mod(upper,number) == 0
x = 0:interval:upper; % When number divides upper, this includes upper
else
x = [0:interval:upper, upper]; % Otherwise we need to append upper
end
You could instead handle this by subtracting 0.5 (or any positive value <1) from upper during the creation of x which would ensure it isn't included, and then always append it. Essentially forcing the else case above.
Then the overall code becomes this, note we don't even need to generate R unless you want it as an output:
upper = 11; % Upper limit
number = 3; % Number of intervals
interval = ceil(upper/number); % Interval size
x = [0:interval:(upper-0.5), upper]; % Create output
TL;DR: I need to find all possible combinations of N row vectors (of size 1xB), whose row-wise sum produces the desired result vector (also of size 1xB).
I have a binary matrix (1 or 0 entries only) of size N x B where N denotes the number of units and B denotes the number of bins. Each unit, i.e., each row, of the matrix can be in one of 2^B states. That is, if B=2, the states possible are {0,0}, {0,1}, {1,0} or {1,1}. If B=3, then the possible states are {0,0,0}, {0,0,1}, {0,1,0}, {0,1,1}, {1,0,0}, {1,0,1}, {1,1,0} or {1,1,1}. Basically the binary representation of the numbers from 0 to 2^B-1.
For the matrix, I know the sum over the rows of the matrix, for example, {1,2}. This sum can be achieved through different binary matrices like [0,0;0,1;1,1] or [0,1;0,1;1,0]. The number of units in each state are {1,1,0,1} and {0,2,1,0}, respectively for each of the matrices, where the first number corresponds to the first state {0,0}, second to the second state {0,1} and so on in increasing order. My problem is to find all possible vectors of these numbers of states that satisfy a particular matrix sum.
Now to implement this in MATLAB, I used recursion and a global variable. This to me was the easiest approach, however, it takes a lot of time. The code I used is given below:
function output = getallstate()
global nState % stores all the possible vectors
global nStateRow % stores the current row of the vector
global statebin %stores the binary representation of all the possible states
nState = [];
nStateRow = 1;
nBin = 2; % number of columns or B
v = [1 2]; % should always be of the size 1 x nBin
N = 3; % number of units
statebin = de2bi(0:(2 ^ nBin - 1), nBin) == 1; % stored as logical because I use it to index later
getnstate(v, 2 ^ nBin - 1, nBin) % the main function
checkresult(v, nState, nBin) % will result in false if even one of the results is incorrect
% adjust for max number of units, because the total of each row cannot exceed this number.
output = nState(1:end-1, :); % last row is always repeated (needs to be fixed somehow)
output(:, 1) = N - sum(output(:, 2:end), 2); % the first column, that is the number of units in the all 0 state is always determined by the number of units in the other states
if any(output(:, 1) < 0)
output(output(:, 1) < 0, :) = [];
end
end
function getnstate(r, state, nBin)
global nState
global nStateRow
global statebin
if state == 0
if all(r == 0)
nStateRow = nStateRow + 1;
nState(nStateRow, :) = nState(nStateRow - 1, :);
end
else
for a = 0:min(r(statebin(state + 1, :)))
nState(nStateRow, state + 1) = a;
getnstate(r - a * statebin(state + 1, :), state - 1, nBin);
end
end
end
function allOk = checkresult(r, nState, nBin)
% just a function that checks whether the obtained vectors all result in the correct sum
allstate = de2bi(0:(2 ^ nBin - 1), nBin);
allOk = true;
for iRow = 1:size(nState, 1)
sumR = sum(bsxfun(#times, allstate, nState(iRow, :).'), 1);
allOk = allOk & isequal(sumR,r);
end
end
function b = de2bi(d, n)
d = d(:);
[~, e] = log2(max(d));
b = rem(floor(d * pow2(1-max(n, e):0)), 2);
end
The above code works fine and gives all possible states but, as is expected, it gets slower as you increase the number of columns (B) and the number of units (N). Also, it uses globals. The following are my questions:
Is there a way to generate these without using globals?
Is there a non-recursive way for this algorithm?
EDIT 1
In what way do the above and still have an optimised algorithm which is faster than the current version?
EDIT 2
Added the de2bi function to remove dependency on the Communications Toolbox.
I have a set of measured data that should theoretically store only the power peaks arriving at a receiver, and I know these peaks should come in intervals of 4 seconds (approximately at least, because in the real case scenario I should expect it to deviate a little).
The problem is that the system can also receive random data from sources other than the one I'm interested in studying or as an echo from the same source, like in the image example:
In this image, the blue data is the real data, and the red data is random data that should be ignored.
What's the best way using MATLAB (and possibly some statistics knowledge) to detect those that are most probably the wanted data?
(sometimes the "parasite" data can also be spaced of 4 seconds, if it's an echo)
The following code finds times tags that belongs to the longest series with gaps that are close to multiple of 4.
The algorithm assumes that valid gap might be missing from the series (not searching for continuity).
%T is the X coordinate of your graph (time tag).
%Notice: The amplitude is irrelevant here.
T = [1, 2, 5, 6, 7, 10, 12, 14];
%Create all possible combinations of indexes of T.
[Y, X] = meshgrid(1:length(T));
%G matrix is the combinations of all gaps:
%T(1) - T(1), T(2) - T(1), T(3) - T(1)...
%It is inefficient to compute all gaps (even in reverse and T(1) - T(1)),
%But it is a common way to solve problems using Matlab.
G = T(X) - T(Y);
%Ignore sign of gaps.
G = abs(G);
%Remove all gaps that are not multiple of 4 with 0.1 hysteresis.
%Remove gaps like 5, 11, and 12.7...
G((mod(G, 4) > 0.1) & (mod(G, 4) < 3.9)) = 0;
%C is a counter vector - counts all gaps that are not zeros.
%Now C holds the number of elements in the relevant series of each time sample.
C = sum(G > 0, 1);
%Only indexes belongs to the maximum series are valid.
ind = (C == max(C));
%Result: time tags belongs to the longest series.
resT = T(ind)
Note:
In case you are looking for longest series without gaps you can use the following code:
T = [1, 2, 5, 6, 7, 10, 12, 14];
len = length(T);
C = zeros(1, len);
for i = 1:len-1
j = i;
k = i+1;
while (k <= len)
gap = T(k) - T(j);
if (abs(gap - 4) < 0.1)
C(i) = C(i) + 1; %Increase series counter.
%Continue searching from j forward.
j = k;
k = j+1;
else
k = k+1;
end
if (gap > 4.1)
%Break series if gap is above 4.1
break;
end
end
end
%now find(C == max(C)) is the index of the beginning of the longest contentious series.
Problem : How do I use a continuous map - The Link1: Bernoulli Shift Map to model binary sequence?
Concept :
The Dyadic map also called as the Bernoulli Shift map is expressed as x(k+1) = 2x(k) mod 1. In Link2: Symbolic Dynamics, explains that the Bernoulli Map is a continuous map and is used as the Shift Map. This is explained further below.
A numeric trajectory can be symbolized by partitioning into appropriate regions and assigning it with a symbol. A symbolic orbit is obtained by writing down the sequence of symbols corresponding to the successive partition elements visited by the point in its orbit. One can learn much about the dynamics of the system by studying its symbolic orbits. This link also says that the Bernoulli Shift Map is used to represent symbolic dynamics.
Question :
How is the Bernoulli Shift Map used to generate the binary sequence? I tried like this, but this is not what the document in Link2 explains. So, I took the numeric output of the Map and converted to symbols by thresholding in the following way:
x = rand();
y = mod(2* x,1) % generate the next value after one iteration
y =
0.3295
if y >= 0.5 then s = 1
else s = 0
where 0.5 is the threshold value, called the critical value of the Bernoulli Map.
I need to represent the real number as fractions as explained here on Page 2 of Link2.
Can somebody please show how I can apply the Bernoulli Shift Map to generate symbolized trajectory (also called time series) ?
Please correct me if my understanding is wrong.
How do I convert a real valued numeric time series into symbolized i.e., how do I use the Bernoulli Map to model binary orbit /time series?
You can certainly compute this in real number space, but you risk hitting precision problems (depending on starting point). If you're interested in studying orbits, you may prefer to work in a rational fraction representation. There are more efficient ways to do this, but the following code illustrates one way to compute a series derived from that map. You'll see the period-n definition on page 2 of your Link 2. You should be able to see from this code how you could easily work in real number space as an alternative (in that case, the matlab function rat will recover a rational approximation from your real number).
[EDIT] Now with binary sequence made explicit!
% start at some point on period-n orbit
period = 6;
num = 3;
den = 2^period-1;
% compute for this many steps of the sequence
num_steps = 20;
% for each step
for n = 1:num_steps
% * 2
num = num * 2;
% mod 1
if num >= den
num = num - den;
end
% simplify rational fraction
g = gcd(num, den);
if g > 1
num = num / g;
den = den / g;
end
% recover 8-bit binary representation
bits = 8;
q = 2^bits;
x = num / den * q;
b = dec2bin(x, bits);
% display
fprintf('%4i / %4i == 0.%s\n', num, den, b);
end
Ach... for completeness, here's the real-valued version. Pure mathematicians should look away now.
% start at some point on period-n orbit
period = 6;
num = 3;
den = 2^period-1;
% use floating point approximation
x = num / den;
% compute for this many steps of the sequence
num_steps = 20;
% for each step
for n = 1:num_steps
% apply map
x = mod(x*2, 1);
% display
[num, den] = rat(x);
fprintf('%i / %i\n', num, den);
end
And, for extra credit, why is this implementation fast but daft? (HINT: try setting num_steps to 50)...
% matlab vectorised version
period = 6;
num = 3;
den = 2^period-1;
x = zeros(1, num_steps);
x(1) = num / den;
y = filter(1, [1 -2], x);
[a, b] = rat(mod(y, 1));
disp([a' b']);
OK, this is supposed to be an answer, not a question, so let's answer my own questions...
It's fast because it uses Matlab's built-in (and highly optimised) filter function to handle the iteration (that is, in practice, the iteration is done in C rather than in M-script). It's always worth remembering filter in Matlab, I'm constantly surprised by how it can be turned to good use for applications that don't look like filtering problems. filter cannot do conditional processing, however, and does not support modulo arithmetic, so how do we get away with it? Simply because this map has the property that whole periods at the input map to whole periods at the output (because the map operation is multiply by an integer).
It's daft because it very quickly hits the aforementioned precision problems. Set num_steps to 50 and watch it start to get wrong answers. What's happening is the number inside the filter operation is getting to be so large (order 10^14) that the bit we actually care about (the fractional part) is no longer representable in the same double-precision variable.
This last bit is something of a diversion, which has more to do with computation than maths - stick to the first implementation if your interest lies in symbol sequences.
If you only want to deal with rational type of output, you'll first have to convert the starting term of your series into a rational number if it is not. You can do that with:
[N,D] = rat(x0) ;
Once you have a numerator N and a denominator D, it is very easy to calculate the series x(k+1)=mod(2*x(k), 1) , and you don't even need a loop.
for the part 2*x(k), it means all the Numerator(k) will be multiplied by successive power of 2, which can be done by matrix multiplication (or bsxfun for the lover of the function):
so 2*x(k) => in Matlab N.*(2.^(0:n-1)) (N is a scalar, the numerator of x0, n is the number of terms you want to calculate).
The Mod1 operation is also easy to translate to rational number: mod(x,1)=mod(Nx,Dx)/Dx (Nx and Dx being the numerator and denominator of x.
If you do not need to simplify the denominator, you could get all the numerators of the series in one single line:
xn = mod( N.*(2.^(0:n-1).'),D) ;
but for visual comfort, it is sometimes better to simplify, so consider the following function:
function y = dyadic_rat(x0,n)
[N,D] = rat(x0) ; %// get Numerator and Denominator of first element
xn = mod( N.*(2.^(0:n-1).'),D) ; %'// calculate all Numerators
G = gcd( xn , D ) ; %// list all "Greatest common divisor"
y = [xn./G D./G].' ; %'// output simplified Numerators and Denominators
If I start with the example given in your wiki link (x0=11/24), I get:
>> y = dyadic_rat(11/24,8)
y =
11 11 5 2 1 2 1 2
24 12 6 3 3 3 3 3
If I start with the example given by Rattus Ex Machina (x0=3/(2^6-1)), I also get the same result:
>> y = dyadic_rat(3/63,8)
y =
1 2 4 8 16 11 1 2
21 21 21 21 21 21 21 21
I have a bunch of times-series each described by two components, a timestamp vector (in seconds), and a vector of values measured. The time vector is non-uniform (i.e. sampled at non-regular intervals)
I am trying to compute the mean/SD of each 1-minutes interval of values (take X minute interval, compute its mean, take the next interval, ...).
My current implementation uses loops. This is a sample of what I have so far:
t = (100:999)' + rand(900,1); %' non-uniform time
x = 5*rand(900,1) + 10; % x(i) is the value at time t(i)
interval = 1; % 1-min interval
tt = ( floor(t(1)):interval*60:ceil(t(end)) )'; %' stopping points of each interval
N = length(tt)-1;
mu = zeros(N,1);
sd = zeros(N,1);
for i=1:N
indices = ( tt(i) <= t & t < tt(i+1) ); % find t between tt(i) and tt(i+1)
mu(i) = mean( x(indices) );
sd(i) = std( x(indices) );
end
I am wondering if there a faster vectorized solution. This is important because I have a large number of time-series to process each much longer than the sample shown above..
Any help is welcome.
Thank you all for the feedback.
I corrected the way t is generated to be always monotonically increasing (sorted), this was not really an issue..
Also, I may not have stated this clearly but my intention was to have a solution for any interval length in minutes (1-min was just an example)
The only logical solution seems to be...
Ok. I find it funny that to me there is only one logical solution, but many others find other solutions. Regardless, the solution does seem simple. Given the vectors x and t, and a set of equally spaced break points tt,
t = sort((100:999)' + 3*rand(900,1)); % non-uniform time
x = 5*rand(900,1) + 10; % x(i) is the value at time t(i)
tt = ( floor(t(1)):1*60:ceil(t(end)) )';
(Note that I sorted t above.)
I would do this in three fully vectorized lines of code. First, if the breaks were arbitrary and potentially unequal in spacing, I would use histc to determine which intervals the data series falls in. Given they are uniform, just do this:
int = 1 + floor((t - t(1))/60);
Again, if the elements of t were not known to be sorted, I would have used min(t) instead of t(1). Having done that, use accumarray to reduce the results into a mean and standard deviation.
mu = accumarray(int,x,[],#mean);
sd = accumarray(int,x,[],#std);
You could try and create a cell array and apply mean and std via cellfun. It's ~10% slower than your solution for 900 entries, but ~10x faster for 90000 entries.
[t,sortIdx]=sort(t); %# we only need to sort in case t is not monotonously increasing
x = x(sortIdx);
tIdx = floor(t/60); %# convert seconds to minutes - can also convert to 5 mins by dividing by 300
tIdx = tIdx - min(tIdx) + 1; %# tIdx now is a vector of indices - i.e. it starts at 1, and should go like your iteration variable.
%# the next few commands are to count how many 1's 2's 3's etc are in tIdx
dt = [tIdx(2:end)-tIdx(1:end-1);1];
stepIdx = [0;find(dt>0)];
nIdx = stepIdx(2:end) - stepIdx(1:end-1); %# number of times each index appears
%# convert to cell array
xCell = mat2cell(x,nIdx,1);
%# use cellfun to calculate the mean and sd
mu(tIdx(stepIdx+1)) = cellfun(#mean,xCell); %# the indexing is like that since there may be missing steps
sd(tIdx(stepIdx+1)) = cellfun(#mean,xCell);
Note: my solution does not give the exact same results as yours, since you skip a few time values at the end (1:60:90 is [1,61]), and since the start of the interval is not exactly the same.
Here's a way that uses binary search. It is 6-10x faster for 9900 elements and about 64x times faster for 99900 elements. It was hard to get reliable times using only 900 elements so I'm not sure which is faster at that size. It uses almost no extra memory if you consider making tx directly from the generated data. Other than that it just has four extra float variables (prevind, first, mid, and last).
% Sort the data so that we can use binary search (takes O(N logN) time complexity).
tx = sortrows([t x]);
prevind = 1;
for i=1:N
% First do a binary search to find the end of this section
first = prevind;
last = length(tx);
while first ~= last
mid = floor((first+last)/2);
if tt(i+1) > tx(mid,1)
first = mid+1;
else
last = mid;
end;
end;
mu(i) = mean( tx(prevind:last-1,2) );
sd(i) = std( tx(prevind:last-1,2) );
prevind = last;
end;
It uses all of the variables that you had originally. I hope that it suits your needs. It is faster because it takes O(log N) to find the indices with binary search, but O(N) to find them the way you were doing it.
You can compute indices all at once using bsxfun:
indices = ( bsxfun(#ge, t, tt(1:end-1)') & bsxfun(#lt, t, tt(2:end)') );
This is faster than looping but requires storing them all at once (time vs space tradeoff)..
Disclaimer: I worked this out on paper, but haven't yet had the opportunity to check it "in silico"...
You may be able to avoid loops or using cell arrays by doing some tricky cumulative sums, indexing, and calculating the means and standard deviations yourself. Here's some code that I believe will work, although I am unsure how it stacks up speed-wise to the other solutions:
[t,sortIndex] = sort(t); %# Sort the time points
x = x(sortIndex); %# Sort the data values
interval = 60; %# Interval size, in seconds
intervalIndex = floor((t-t(1))./interval)+1; %# Collect t into intervals
nIntervals = max(intervalIndex); %# The number of intervals
mu = zeros(nIntervals,1); %# Preallocate mu
sd = zeros(nIntervals,1); %# Preallocate sd
sumIndex = [find(diff(intervalIndex)) ...
numel(intervalIndex)]; %# Find indices of the interval ends
n = diff([0 sumIndex]); %# Number of samples per interval
xSum = cumsum(x); %# Cumulative sum of x
xSum = diff([0 xSum(sumIndex)]); %# Sum per interval
xxSum = cumsum(x.^2); %# Cumulative sum of x^2
xxSum = diff([0 xxSum(sumIndex)]); %# Squared sum per interval
intervalIndex = intervalIndex(sumIndex); %# Find index into mu and sd
mu(intervalIndex) = xSum./n; %# Compute mean
sd(intervalIndex) = sqrt((xxSum-xSum.*xSum./n)./(n-1)); %# Compute std dev
The above computes the standard deviation using the simplification of the formula found on this Wikipedia page.
The same answer as above but with the parametric interval (window_size).
Issue with the vector lengths solved as well.
window_size = 60; % but it can be any value 60 5 0.1, which wasn't described above
t = sort((100:999)' + 3*rand(900,1)); % non-uniform time
x = 5*rand(900,1) + 10; % x(i) is the value at time t(i)
int = 1 + floor((t - t(1))/window_size);
tt = ( floor(t(1)):window_size:ceil(t(end)) )';
% mean val and std dev of the accelerations at speed
mu = accumarray(int,x,[],#mean);
sd = accumarray(int,x,[],#std);
%resolving some issue with sizes (for i.e. window_size = 1 in stead of 60)
while ( sum(size(tt) > size(mu)) > 0 )
tt(end)=[];
end
errorbar(tt,mu,sd);