I am looking for a tip to feed extra columns of px matrix with data, as my code ends up with Error: Subscripted assignment dimension mismatch from z=2. ps: While breaking the code it only works fine for z=1. I've also tried
px(:,z:ind)=getsqldata(data{z,3}) as a way around but without success.
ind=20;
T=250; % time period
z=1; %initialization
for i=z:ind
while (data{z,6}~='C' ||data{i,6}~='OT')
px(:,z)=getsqldata(data{z,3});
if px(1,1)=='No Data'
error('test');
end
ret(:,z) = -price2ret(px(:,z));
ret=ret(1:T,1);
z=z+1;
end
end
Best,
ps: getsqldata() is my user-defined function, retrieving data from database. I can confirm that there is nothing wrong with it, as I've used it for a while.
Related
I wanted to plot the load voltage across the resistor in series with a diode using matlab. This is a rather simple example involving piecewise functions, however I ran into an unexpected error.
t=[0:0.001:10]
vs=4*sin(pi * t)
for i =1:length(vs)
if(vs(i)<=0.7)
v(i)=0;
else
v(i)=vs(i)-0.7;
end
end
plot(t,v)
t is the time, vs is the source voltage, and v is the load voltage. Now, running this gave me an error saying "error, t and v are of different sizes..". Using length() I found out that while t and vs are of lengths 10001, v is somehow of length 1000001.
This is truly baffling to me. How can v and vs possibly differ in size? every element of vs was mapped to an element of v, and yet the size of v comes out to be about 100 times the size of vs.
Being new to matlab, I still am not very comfortable with not explicitly declaring the array v before using it in the for loop. However, I went through with it, because the example I worked on prior to this, used the same thing and it worked without any problems. We simply had a plot a piecewise function:
x=[-2 : 0.00001 : 20];
for i=1: length(x)
if(x(i)>=-2 && x(i)<0)
y(i)=sqrt(x(i)^2+1);
else if(x(i)>=0 && x(i)<10)
y(i)=3*x(i)+1;
else
y(i)=9*sin(5*x(i)-50);
end
end
end
plot(x,y)
This code worked flawlessly, and in my opinion the initial code is fundamentally doing the same thing, so again, I'm clueless as to why it failed.
The original code works if you initialise v to be of the same size as t (and therefore, vs), but still, I want to know why the code involving x,y worked (where y wasn't initialised) and the code involving (t,v) failed.
Also, my friend copy pasted the entire code into the command window of matlab 2016, and it worked. (and I'm using the 2021 version).
Its good practice to initialize variables before entering a loop. It will help avoid undefinied behaviour when you run the script multiple times. If you run the script with different lengths for t, it would fail the second run. One solution would be:
t=0:0.001:10;
vs=4*sin(pi * t);
v=nan(size(t));
for i =1:length(vs)
if(vs(i)<=0.7)
v(i)=0;
else
v(i)=vs(i)-0.7;
end
end
figure;
plot(t,v);
You could also avoid the for loop and use matrix operations instead:
t=0:0.001:10;
vs=4*sin(pi * t);
v=vs-0.7;
v(vs<=0.7)=0;
figure;
plot(t,v);
I am currently processing a bunch of files that I have imported into a structure, but have hit a bump in the road while trying to loop over the data.
First of all, here is my structure:
Ice
1.1 az160, az240, az300...
1.1.2 zen15, zen30,zen45...
1.1.2.1 Data
1.1.2.2 Textdata
I am trying to extract a value from each "textdata" cell array and use it to divide a column in data of the same structure. To do so, I am looping through the structure in the following way:
az_names = fieldnames(ice)
for m = 1:numel(az_names)
snames = fieldnames(ice.(az_names{m}))
for k = 1:numel(snames)
inttime = strrep(ice.(az_names{m}).(snames{k}).textdata(9,1), 'Integration > Time (usec): ','');
inttime = strrep(inttime, ' (USB2+E00040)','');
integration = cellfun(#str2num,inttime)
line 17 ice.(az_names{m}).(snames{k}).data(:,4) = ice.(az_names{m}).(snames{k}).data(:,3)/integration
end
end
I get the following error:
Index exceeds matrix dimensions
Edit: Matlab gives me the error at line 17. If I run the code up to "integration" and also write:
ice.(az_names{m}).(snames{k}).data(:,4)
I don't get a problem, Matlab prints to screen the right number and the data column.
I thought this would loop through each field in the structure and do the operation (dividing a column of values by a number), but I seem to be missing a point here. Can anybody see my mistake?
Regards,
If the error occurs when you try to execute this fragment:
ice.(az_names{m}).(snames{k}).data(:,4)
Then the cause is quite simple.
The variables m and k seem to be handled properly (due to the numel in the loop they should never be too big), meaning that the 4 is simply too big.
Check
ice.(az_names{m}).(snames{k}).data(:,4)
Or even more directly
size(ice.(az_names{m}).(snames{k}).data)
And you should find that the second element of the size is less than 4.
On second thought, this fragment works:
a.b=1;
a.b(:,4)=1
So I suspect that the error occurs when trying to read in this part:
ice.(az_names{m}). (snames{k}).data(:,3)
Meaning that the second element of the size should even be less than 3.
Also I would recommend removing the space.
I have written code to plot data from very large .txt files (20Gb to 60Gb). The .txt files contain two columns of data, that represent the outputs of two sensors from an experiment that I did. The reason the data files are so large is that the data was recorded at 4M samples/s.
The code works well for plotting relatively small .txt files (10Gb), however when I try to plot my larger data files (60Gb) I get the following error message:
Attempted to access TIME(0); index must be a
positive integer or logical.
Error in textscan_loop (line 17)
TIME =
((TIME(end)+sample_rate):sample_rate:(sample_rate*(size(d,1)))+(TIME(end)));%shift
Time along
The basic idea behind my code is to conserve RAM by reading Nlines of data from .txt on disk to Matlab variable C in RAM, plotting C then clearing C. This process occurs in loop so the data is plotted in chunks until the end of the .txt file is reached. The code can be found below:
Nlines = 1e6; % set numbe of lines to sample per cycle
sample_rate = (1); %sample rate
DECE= 1000;% decimation factor
TIME = (0:sample_rate:sample_rate*((Nlines)-1));%first inctance of time vector
format = '%f\t%f';
fid = fopen('H:\PhD backup\Data/ONK_PP260_G_text.txt');
while(~feof(fid))
C = textscan(fid, format, Nlines, 'CollectOutput', true);
d = C{1}; % immediately clear C at this point you need the memory!
clearvars C ;
TIME = ((TIME(end)+sample_rate):sample_rate:(sample_rate*(size(d,1)))+(TIME(end)));%shift Time along
plot((TIME(1:DECE:end)),(d(1:DECE:end,:)))%plot and decimate
hold on;
clearvars d;
end
fclose(fid);
I think the while loop does around 110 cycles before the code stops executing and the error message is displayed, I know this because the graph shows around 110e7 data points and the loop processes 1e6 data points at a time.
If anyone knows why this error might be occurring please let me know.
Cheers,
Jim
The error that you encounter is in fact not in the plotting, but in the line of reference.
Though I have been unable to reproduce the exact error, I suspect it to be related to this:
Time = 1:0
Time(end)
In any case, the way forward is clear. You need to run this code with dbstop if error and observe all relevant variables in the line that throws the error.
From here you will likely figure out what is causing the problem, hopefully just something simple like your code being unable to deal with data size that is an exact multiple of 1000 or so.
Trying to use plot for big data is problematic as matlab is trying to plot every single data point.
Obviously the screen will not display all of these points (many will overlap), and therefore it is recommended to plot only the relevant points. One could subsample and do this manually as you seem to have tried, but fortunately we have a ready to use solution for this:
The Plot (Big) File Exchange Submission
Here is the introduction:
This simple tool intercepts data going into a plot and reduces it to
the smallest possible set that looks identical given the number of
pixels available on the screen. It then updates the data as a user
zooms or pans. This is useful when a user must plot a very large
amount of data and explore it visually.
This works with MATLAB's built-in line plot functions, allowing the
functionality of those to be preserved.
Instead of:
plot(t, x);
One could use:
reduce_plot(t, x);
Most plot options, such as multiple series and line properties, can be
passed in too, such that 'reduce_plot' is largely a drop-in
replacement for 'plot'.
h = reduce_plot(t, x(1, :), 'b:', t, x(2, :), t, x(3, :), 'r--*');
This function works on plots where the "x" data is always increasing,
which is the most common, such as for time series.
psz=length(pic)
p=0; %masking counter
for i=1:outs:(psz) % dividing in blocks
for j=1:outs:(psz)
p=p+1
blocks(:,:,p)=pic(i:i+outs-1,j:j+outs-1);
ins(:,p)=reshape(blocks(:,:,p)',1,ins')';
end
end
so to begin with i am trying to reproduce sanger rule for pca using neural networks so if somone wants to duscuss about it or give him my code he can message me:)
i get the following error
Error using reshape
To RESHAPE the number of elements must not change.
Error in train (line 30)
ins(:,p)=reshape(blocks(:,:,p)',1,ins')';
Converting blocks(:,:,p) to a column vector should clear the error as long as there are the same number of elements in blocks(:,:,p) as the row length of ins
col_vec = blocks(:,:,p);
ins(:,p) = col_vec(:);
The size of blocks(:,:,p) is outs-by-outs so to make a column vector, it has to be (outs*outs)-by-1. To do this, the command would be:
ins(:,p)=reshape(blocks(:,:,p)',outs*outs,1); % no need for '
However, be sure that size(ins,1) is outs*outs or it won't work. What is the size of ins (and blocks, out of curiosity)? Also, be sure you really want the ' on blocks because the command will work with or without it.
I'm trying to modify someone else's code that uses global variables and keep getting an error that I can't figure out. This is my first time using global variables, so I suspect it has to do with that. I'm getting this error:
Attempt to reference field of non-structure array.
Error in likelihood (line 35)
X=ModelInfo.X;
This is the code I run:
clearvars -global
clearvars
global ModelInfo
%Pull in data from MySQL database
ModelInfo.X=..... %76x2 double type
ModelInfo.y=..... %76x1 double type
addpath('C:/.../Sampling Plans');
addpath('C:/.../Constructing a Surrogate');
%Number of variables
k=size(ModelInfo.X,2);
%Number of sample points
n=size(ModelInfo.X,1);
%Set upper and lower bounds for search of log theta
UpperTheta=ones(1,k).*2;
LowerTheta=ones(1,k).*-3;
%Run GA search of likelihood
[ModelInfo.Theta,MinNegLnLikelihood]=ga(#likelihood,k,[],[],[],[],LowerTheta,UpperTheta);
...which references an outside function likelihood (written by someone else):
function [NegLnLike,Psi,U]=likelihood(x)
global ModelInfo
X=ModelInfo.X;
y=ModelInfo.y;
theta=10.^x;
p=2;
n=size(X,1);
one=ones(n,1);
% Pre-allocate memory
Psi=zeros(n,n);
% Build upper half of correlation matrix
for i=1:n
for j=i+1:n
Psi(i,j)=exp(-sum(theta.*abs(X(i,:)-X(j,:)).^p));
end
end
% Add upper and lower halves and diagonal of ones plus
% small number to reduce ill-conditioning
Psi=Psi+Psi'+eye(n)+eye(n).*eps;
% Cholesky factorisation
[U,p]=chol(Psi);
% Use penalty if ill-conditioned
if p>0
NegLnLike=1e4;
else
% Sum lns of diagonal to find ln(abs(det(Psi)))
LnDetPsi=2*sum(log(abs(diag(U))));
% Use back-substitution of Cholesky instead of inverse
mu=(one'*(U\(U'\y)))/(one'*(U\(U'\one)));
SigmaSqr=((y-one*mu)'*(U\(U'\(y-one*mu))))/n;
NegLnLike=-1*(-(n/2)*log(SigmaSqr) - 0.5*LnDetPsi);
end
UPDATE:
Figured out the problem. I declared ModelInfo as a global variable at the beginning, and then within the %Pull in data from MySQL database section I created a database connection and then cleared some variables we wouldn't need anymore using the clearvars -except command, but I didn't include ModelInfo in that list. After that I put data into ModelInfo. Since it's global, I guess that deleted either the local copy or the global copy (not quite sure), but it still appeared in the list of variables in the workspace and isstruct(ModelInfo) still returned 1...so visibly there was nothing to indicate an issue other than the error.
If that is indeed the code you run:
%Pull in data from MySQL database
ModelInfo.X=..... %76x2 double type
ModelInfo.y=..... %76x1 double type
I doubt it has anything to do with global variables. You should have some data of some sort instead of the 5 dots ..... in order to make any code work (as the comment that is placed above indicate "pull data from..."). I'd also try to make this data in the form that the comments suggest if it isn't of the same class, i.e. make it double and of the size mentioned.
For example, try to see if replacing these lines with this makes your code work
ModelInfo.X=rand(76,2) %76x2 double type
ModelInfo.y=rand(76,1) %76x1 double type