Good evening,
May I please get help with a script I'm writing? I have a parfor loop nested within a for loop. The goal is to iterate over a set of data that consists of 10 data subsets generated from an earlier parsim simulink analysis (it's labeled as 1x10 SimulationOutput). Each data subset is 24 rows deep, and a variable length of columns (usually about 200,000 to 300,000 columns of data). Part of the process is to find the maximum or minimum values in each data set. Once that is done, it is to be put into a table, appending data to that table. Ideally, I should have a 6x10 table by the end of it. See below for the code:
% Run Time
tic
% Preallocate memory to increase speed
b=zeros(24,1); %Make space for this array.
c=zeros(500000,1);
d=zeros(500000,1);
e=zeros(500000,1);
f=zeros(500000,1);
g=zeros(500000,1);
h=zeros(500000,1);
%table=[];
for j = 1:length(out(1,:)) %iterate over each run
parfor i = 1:length(out(1,j).PN.time) % Set length of vector
b=out(1,j).PN.signals.values(:,i); % Find the values to work on
c(i)=b(19,:); % Distance to target (m)
d(i)=b(20,:); % Lat. Accelerations, integrated twice (m)
e(i)=b(21,:); % Long. Acceleration, integrated twice (m)
f(i)=b(22,:); % Lat. Guidance Error
g(i)=b(23,:); % Long. Guidance Error
h(i)=b(24,:); % time to target (sec)
end
%For c_min, there's extranous zeros popping up, exclude them
tc = c;
tc(tc <= 0) = nan;
[c_min, I_1] = min(tc);
% [c_min,I_1]=min(c(c>0)); % Collect the closest missile/target approach (most
critical value)
[d_max,I_2]=max(d); % We need to find the max value per run, but wish for the min value
%over all runs.
[e_max,I_3]=max(e); % We need to find the max value per run, but wish for the min value
%over all runs.
[f_min,I_4]=min(f); % We just want the minimum value here.
[g_min,I_5]=min(g); % We just want the minimum value here.
[h_max,I_6]=max(h); % The minimum time is 2nd most critical value, after distance to
%target.
table(:,j)=[ c_min d_max e_max f_min g_min h_max]; %d_max e_max f_min g_min h_max
end
toc
The issue that I am having is that, while I can input the correct data sets in the correct locations in the table if I set a constant j value (example: if j = 7, then the 7th column in the table gets the correct data) I can't seem to get all the values inputted correctly. What I mean is that, the outputted table (6x10) will have repeated values across columns, values from one column in another column, and so on). It is as if the script cannot differentiate between columns anymore, so values just go wherever.
If anyone has any advice, I'd greatly appreciate it. Thank you,
Related
I have some EEG data that I would like to break down into 30-second windows and run a fast Fourier transform on each window of data. I've tried to implement a for-loop and increment the index value by the number of samples in the time window. When I run this, I can see that (1) this works for the first window of data, but not the rest of them because (I think) the "number of samples minus one" leads to fewer elements than necessary for data_fft and thus doesn't have the same dimensions as f, which are both being plotted in a figure. (2) I tried to update the index value by adding the number of samples in a window, but after i = 1, it goes to i = 2 in my workspace and not to i = 7681 as I'd hoped. I've spent an embarrassingly long time on trying to figure out how to change this so it works correctly, so any advice is appreciated! Code is below. Let me know if I can clarify anything.
data_ch6 = data(:,6); % looking at just 1 electrode
tmax = 2*60; % total time in sec I want to analyze; just keeping it to 2 minutes for this exercise
tmax_window = 30; %30 sec window
times = tmax/tmax_window; % number of times fft should be run
Nsamps = tmax*hdr.SPR; % total # samples in tmax; sample rate is 256 hz
Nsamps_window = tmax_window*hdr.SPR; % # samples in time window
f = hdr.SPR*(0:((Nsamps_window-1)/2))/Nsamps_window; % frequency for plotting
for i = 1:Nsamps; % need to loop through data in 30 second windows in tmax
data_fft = abs(fft(data_ch6(i:i+Nsamps_window-1))); %run fft on data window
data_fft = data_fft(i:((i+Nsamps_window-1)/2)); %discard half the points
figure
plot(f, data_fft)
i = i+Nsamps_window;
end
Well there are a few things that are wrong in your code. First, let me start be saying that i is a very poor choice for a variable name since in matlab it usually stand for sqrt{-1}.
As for your code, I assume that you intend to perform windowed FFT without overlapping.
1) Your loop goes from 1 to Nsamps with an increment of 1. That means the each time you advance 1 sample. In other words you have Nsamps_window-1 overlap. You can use perhaps i=1:Nsamps_window:Nsamps-Nsamps_window-1 if you are not interested in overlapping.
2) The length of data_fft is Nsamps_window, so I think what you wanted to do is data_fft = data_fft(1:round(Nsamps_window/2));
3) When plotting FFT results, I suggest using dB: plot(20*log10(abs(data_fft)));
4) The line i = i+Nsamps_window; is meaningless since i is your loop variable (it has not effect).
I am trying to create a set of 320 matrices, each having dimensions 1152 x 241. Each matrix represents a different time step. I am trying to populate each cell, using a random value from another file. This other file is also dimensioned 1152 x 241, but there are ~2520 time steps from which to choose.
So what is supposed to happen is pick a cell, populate with a value from a random time step from the big file, and move onto the adjacent cell and do the same thing. Repeat until 320 matrices have been created.
Problem is I run the code and I only create one matrix. What do I need to do to fix my code so that 320 matrices are created? Thanks!
clear all;
clc;
% Load datafile
load 1979_1999_tropics_subset_3mmhr.mat
% Create empty maps
rain_fake_timeseries = zeros(1152,241,320);
for i = 1:1152; % set longitude
%disp(i)
for j = 1:241; % set latitude
%disp(j)
%for k = 1:320; % create map
%disp(k)
rain_fake_timeseries = datasample(rain_sample_1979_1999,1,3);
%disp(rain_fake_timeseries)
%save random_clus_fake_timeseries.mat rain_fake_timeseries -v7.3;
%end
end
end
save random_clus_fake_timeseries.mat rain_fake_timeseries -v7.3;
This is because you are not properly indexing into your time series array to store the data. What you are doing is that you are only saving the last randomly chosen slice in your time series array. If you look at your loop closely, you are simply overwriting the output array at each iteration of the for loop.
You are also not creating your for loop correctly. If I understand you correctly, each location in a slice represents a unique (x,y) coordinate. For each matrix that you have, you want to sample from this exact same location but temporally search through your ~2500 time instances. As such, you need to use all of your loop variables i, j and k to index into your 3D matrix. You also need to access all time slices at position (i,j) and randomly sample from all of the slices. If I can suggest a small optimization change, we can do this with only two for loops rather than three, randomly choose 320 points at this position for all of the time slices, and store it into the 3D matrix.
In other words:
clear all;
clc;
% Load datafile
load 1979_1999_tropics_subset_3mmhr.mat
% Create empty maps
rain_fake_timeseries = zeros(1152,241,320);
for i = 1 : size(rain_fake_timeseries,1)
for j = 1 : size(rain_fake_timeseries,2)
rain_fake_timeseries(i,j,:) = datasample(rain_sample_1979_1999(i,j,:), ...
size(rain_fake_timeseries,3), 3);
end
end
save random_clus_fake_timeseries.mat rain_fake_timeseries -v7.3;
Note that I have replaced the dimensions in the for loop with calls to size so that you can easily change the size of the matrices and it'll still work without you having to change any constants.
Suppose you have 5 vectors: v_1, v_2, v_3, v_4 and v_5. These vectors each contain a range of values from a minimum to a maximum. So for example:
v_1 = minimum_value:step:maximum_value;
Each of these vectors uses the same step size but has a different minimum and maximum value. Thus they are each of a different length.
A function F(v_1, v_2, v_3, v_4, v_5) is dependant on these vectors and can use any combination of the elements within them. (Apologies for the poor explanation). I am trying to find the maximum value of F and record the values which resulted in it. My current approach has been to use multiple embedded for loops as shown to work out the function for every combination of the vectors elements:
% Set the temp value to a small value
temp = 0;
% For every combination of the five vectors use the equation. If the result
% is greater than the one calculated previously, store it along with the values
% (postitions) of elements within the vectors
for a=1:length(v_1)
for b=1:length(v_2)
for c=1:length(v_3)
for d=1:length(v_4)
for e=1:length(v_5)
% The function is a combination of trigonometrics, summations,
% multiplications etc..
Result = F(v_1(a), v_2(b), v_3(c), v_4(d), v_5(e))
% If the value of Result is greater that the previous value,
% store it and record the values of 'a','b','c','d' and 'e'
if Result > temp;
temp = Result;
f = a;
g = b;
h = c;
i = d;
j = e;
end
end
end
end
end
end
This gets incredibly slow, for small step sizes. If there are around 100 elements in each vector the number of combinations is around 100*100*100*100*100. This is a problem as I need small step values to get a suitably converged answer.
I was wondering if it was possible to speed this up using Vectorization, or any other method. I was also looking at generating the combinations prior to the calculation but this seemed even slower than my current method. I haven't used Matlab for a long time but just looking at the number of embedded for loops makes me think that this can definitely be sped up. Thank you for the suggestions.
No matter how you generate your parameter combination, you will end up calling your function F 100^5 times. The easiest solution would be to use parfor instead in order to exploit multi-core calculation. If you do that, you should store the calculation results and find the maximum after the loop, because your current approach would not be thread-safe.
Having said that and not knowing anything about your actual problem, I would advise you to implement a more structured approach, like first finding a coarse solution with a bigger step size and narrowing it down successivley by reducing the min/max values of your parameter intervals. What you have currently is the absolute brute-force method which will never be very effective.
Here's my goal:
I'm trying to find a way to search through a data signal and find (index) locations where a known, repeating binary data sequence is located. Then, because the spreading code and demodulation is known, pull out the corresponding chip of data and read it. Currently, I believe xcorr will do the trick.
Here's my problem:
I can't seem to interpret my result from xcorr or xcorr2 to give me what I'm looking for. I'm either having a problem cross-referencing from the vector location of my xcorr function to my time vector, or a problem properly identifying my data sequence with xcorr, or both. Other possibilities may exist.
Where I am at/What I have:
I have created a random BPSK signal that consists of the data sequence of interest and garbage data over a repeating period. I have tried processing it using xcorr, which is where I am stuck.
Here's my code:
%% Clear Variables
clc;
clear all, close all;
%% Create random data
nbits = 2^10;
ngarbage = 3*nbits;
data = randi([0,1],1,nbits);
garbage = randi([0,1],1,ngarbage);
stream = horzcat(data,garbage);
%% Convert from Unipolar to Bipolar Encoding
stream_b = 2*stream - 1;
%% Define Parameters
%%% Variable Parameters
nsamples = 20*nbits;
nseq = 5 %# Iterate stream nseq times
T = 10; %# Number of periods
Ts = 1; %# Symbol Duration
Es = Ts/2; %# Energy per Symbol
fc = 1e9; %# Carrier frequency
%%% Dependent Parameters
A = sqrt(2*Es/Ts); %# Amplitude of Carrier
omega = 2*pi*fc %# Frequency in radians
t = linspace(0,T,nsamples) %# Discrete time from 0 to T periods with nsamples samples
nspb = nsamples/length(stream) %# Number of samples per bit
%% Creating the BPSK Modulation
%# First we have to stretch the stream to fit the time vector. We can quickly do this using _
%# simple matrix manipulation.
% Replicate each bit nspb/nseq times
repStream_b = repmat(stream_b',1,nspb/nseq);
% Tranpose and replicate nseq times to be able to fill to t
modSig_proto = repmat(repStream_b',1,nseq);
% Tranpose column by column, then rearrange into a row vector
modSig = modSig_proto(:)';
%% The Carrier Wave
carrier = A*cos(omega*t);
%% Modulated Signal
sig = modSig.*carrier;
Using XCORR
I use xcorr2() to eliminate the zero padding effect of xcorr on unequal vectors. See comments below for clarification.
corr = abs(xcorr2(data,sig); %# pull the absolute correlation between data and sig
[val,ind] = sort(corr(:),'descend') %# sort the correlation data and assign values and indices
ind_max = ind(1:nseq); %# pull the nseq highest valued indices and send to ind_max
Now, I think this should pull the five highest correlations between data and sig. These should correspond to the end bit of data in the stream for every iteration of stream, because I would think that is where the data would most strongly cross-correlate with sig, but they do not. Sometimes the maxes are not even one stream length apart. So I'm confused here.
Question
In a three part question:
Am I missing a certain step? How do I use xcorr in this case to find where data and sig are most strongly correlated?
Is my entire method wrong? Should I not be looking for the max correlations?
Or should I be attacking this problem from another angle, id est, not use xcorr and maybe use filter or another function?
Your overall method is great and makes a lot of sense. The problem you're having is that you're getting some actual correlation with your garbage data. I noticed that you shifted all of your sream to be zero-centered, but didn't do the same to your data. If you zero-center the data, your correlation peaks will be better defined (at least that worked when I tried it).
data = 2*data -1;
Also, I don't recommend using a simple sort to find your peaks. If you have a wide peak, which is especially possible with a noisy signal, you could have two high points right next to each other. Find a single maximum, and then zero that point and a few neighbors. Then just repeat however many times you like. Alternatively, if you know how long your epoch is, only do a correlation with one epoch's worth of data, and iterate through the signal as it arrives.
With #David K 's and #Patrick Mineault's help I manage to track down where I went wrong. First #Patrick Mineault suggested I flip the signals. The best way to see what you would expect from the result is to slide the small vector along the larger, searched vector. So
corr = xcorr2(sig,data);
Then I like to chop off the end there because it's just extra. I did this with a trim function I made that simply takes the signal you're sliding and trims it's irrelevant pieces off the end of the xcorr result.
trim = #(x,s2) x(1:end - (length(s2) - 1));
trim(corr,data);
Then, as #David K suggests, you need to have the data stream you're looking for encoded the same as your searched signal. So in this case
data = 2*data-1;
Second, if you just have your data at it's original bit length, and not at it's stretched, iterated length, it can be found in the signal but it will be VERY noisy. To reduce the noise, simply stretch the data to match it's stretched length in the iterated signal. So
rdata = repmat(data',1,nspb/nseq);
rdata = repmat(rdata',1,nseq);
data = rdata(:)';
Now finally, we should have crystal clear correlations for this case. And to pull out the maxes that should correspond to those correlations I wrote
[sortedValues sortIndex] = sort(corr(:),'descend');
c = 0 ;
for r = 1 : length(sortedValues)
if sortedValues(r,:) == max(corr)
c = c + 1;
maxIndex(1,c) = sortIndex(r,:);
else break % If you don't do this, you get loop lock
end
end
Now c should end up being nseq for this case and you should have 5 index times where the corrs should be! You can easily pull out the bits with another loop and c or length(maxIndex). I've also made this into a more "real world" toy script, where there is a data stream, doppler, fading, and it's over a time vector in seconds instead of samples.
Thanks for the help!
Try flipping the signal, i.e.:
corr = abs(xcorr2(data,sig(end:-1:1));
Is that any better?
I am a total beginner in MATLAB and I hope to find some help here. I have some model prediction results for 80 individuals alltogether in one large matrix. I need to extract the data for each individual from the big matrix, assign them in a new variable/matrix, do some extra calculations and then plot certain information as needed.
To do so, I am trying to write a script with a loop function but in a complicated, or maybe more accurately: in a primitive way!
Simplified Example:
My matrix is called: All_Indi_Data .... its dimension is: 600 rows x 21 columns
%Column 1: grouping variable (e.g., code or ID with values 1,2,3,4,5, etc.);
%Column 2: independent var.;
%Column 3: t;
%Column 4: OBS;
%Column 5: PRED;
i= length (All_Indi_Data);
%% First Indi.
q=1; % indicating the ID of the indi for which I want to extract the data
j=1; % variable added to insure writing start from the first row
for r=1:i
if All_Indi_Data (r,1)==q
Indi_1 (j,1:21) = All_Indi_Data (r,1:21)
j=j+1
end
end
%% Second Indi.
q=q+1
j=1
for r=1:i
if All_Indi_Data (r,1)==q
Indi_2 (j,1:21) = All_Indi_Data (r,1:21)
j=j+1
end
end
.
.
.
1) My first question is: can I allocate these data in new variables (Indi_1, Indi_2, ect.) in a more simple way with or without the loop function?!!! I would appreciate your help a lot.
2) Is there any code or any way to plot these selected parts (according to the grouping variable, e.g. data for Indi_1) from the previously mentioned big matrix without wasting a lot of time and space (wto recopying the core part of the code again and again) for the script, and using the loop function?! in other words, I would like to detect - with loop function & the grouping variable- which values are of interest and then to plot them (e.g. data in colum 3 with data from column 4 for each individual, starting from the first to the last)?!
I hope that I described my problem clearly and hope to hear something from the expert guys :) ...
Thanks a lot in advance ..
Try the following code:
for idx=1:80
pos=find(All_Indi_Data(:,1)==idx);
eval(['Indi_' num2str(idx) '=All_Indi_Data(pos,:);']);
end
What I do is: in each iteration, I search for a value of the ID, indicated in the variable idx. Note that I do not use ´i´ as the name of a variable, because Matlab uses it and ´j´ and the imaginary unit for complex numbers and that could cause problems.
Then, using find I search for the position (or positions) of All_Indi_Data in which I can find the information of that individual. Now I have in the variable ´pos´ the indexes of the rows in which there is information for the individual of interest.
Finally, using eval I extract the data for each individual into a variable. Note that eval combined with a loop makes it easy to create lots of variables. I indicate the rows I want to extract with ´pos´ and, as I want all the columns, I use just ´:´ (you could use ´1:21´ too).
With another similar loop you can plot the information you want. For example:
for idx=1:80
eval(['x=Indi_' num2str(idx) ';']);
% Now I have in X the information for this individual
%Plot the columns of x I want
plot(x(:, 3), x(:,4));
pause; %stay here until a press a key
end