I am using Matlab 2016a. I have four matrices of size 2044x1572x84 and am trying to regress each column of each matrix to produce a new 2044x1572 matrix of regression coefficients. I need to use parfor; a for loop would take way too long.
When I use the below code using test data (e.g. using rand to make four matrices of 50x50x40) the code executes with no errors. However, when I try using the same code in a cluster with the full 2044x1572x84 matrices I get a transparency violation error with regards to the table: Error using table (line 247) Transparency violation error. I've tried modifying the table code to fix this but only get a suite of other errors.
I'm unsure how to fix the error in this case, particularly given that the success of the code seems to be dependent on the size of the input data. I'm not particularly familiar with parfor, and any feedback on what I may be doing wrong would be greatly appreciated.
COEFF_LST=ones(2044,1572);
parfor i=1:2044
for j=1:1572
ZZ=squeeze(ARRAY_DETREND_L2_LST(i,j,:));
XX=squeeze(ARRAY_DETREND_L2_ONDVI(i,j,:));
YY=squeeze(ARRAY_DETREND_WB_85(i,j,:));
LL=squeeze(ARRAY_DETREND_L2_CNDVI(i,j,:));
T=table(ZZ,XX,YY,LL,'VariableNames',{'LST','ONDVI','DROUGHT','NDVI'});
lm=fitlm(T);
array=table2array(lm.Coefficients);
COEFF_LST(i,j)=array(3,1);
end
end
The table constructor uses inputname under certain circumstances - that can cause transparency violations inside parfor. I realise it's inconvenient, but perhaps you could try "hiding" the table call inside a separate function. I.e.
parfor ...
T = myTableBuilder(ZZ,XX,...);
end
function t = myTableBuilder(varargin)
t = table(varargin{:});
end
In this case I'm getting a transparency error with table, so a simple solution that works is to not use table.
In this case the code would be:
Predictor_Matrix=horzcat(ZZ,XX,YY);
lm = fitlm(Predictor_Matrix,WW);
This works on a cluster without throwing any errors.
Related
I am trying to use a table within a parfor loop in MATLAB. This gives me "Transparency violation error. See Parallel Computing Toolbox about transparency" I'm trying to build this table so I can make a prediction using a trained classifier from the MATLAB classification learner app (trainedClassifier.prefictFcn(T))...so either I need to build a table within the parfor loop or need some alternative to a table that I can still feed into the classifier.
parfor i=1:100
acheck=1;
bcheck=2;
ccheck=3;
T=table(acheck,bcheck,ccheck);
end
This solution works for your particular problem:
parfor i=1:100
acheck=1;
bcheck=2;
ccheck=3;
T(i,:)=table([acheck,bcheck,ccheck]);
end
Note that in your original program you just overwrite existing values and end up with a one row table. I assumed that that was not intended. Actually, that would be the outcome of a for.
Also, since this is a parfor and T is created inside the loop (as well as acheck, etc.) using just T creates nothing at all. The variable is a temporary one, visible to each process locally and destroyed in global scope (more can be found here).
To fix both overwriting and accessibility the program assigns the each set of variables to each row of T. If square brackets are omitted, the program throws a transparency error. Unfortunately, I do not know why is that but it may be that the operations done by the table data-structure cause that. Maybe someone else will know the answer, for now this seem to solve your problem though.
I want to find possible differences between different conditions. I have n subjects for which I have a mean value for every condition for every subject respectively. The values between subjects vary a lot, that's why I wanted to perform a repeated measures anova to control for that.
My within subject factor would be the condition then and I don't have any between subjects factor.
So far I have the following code:
%% create simulated numbers
meanPerf = randn(20,3);
%% create a table array with the mean performance for every condition
tableData = table(meanPerf(:,1),meanPerf(:,2),meanPerf(:,3),'VariableNames',{'meanPerf1','meanPerf2','meanPerf3'})
tableInfo = table([1,2,3]','VariableNames',{'Conditions'})
%% fit repeated measures model to the table data
repMeasModel = fitrm(tableData,'meanPerf1meanPerf3~1','WithinDesign',tableInfo);
%% perform repeated measures anova to check for differences
ranovaTable = ranova(repMeasModel)
My first question is: Am I doing this correctly?
The second question is: How can I perform a post hoc analysis to find out which of the condition are significantly different from each other?
I tried using:
multcompare(ranovaTable,'Conditions');
but that produced the following error:
Error using internal.stats.parseArgs (line 42)
Wrong number of arguments.
I am using Matlab 2015b.
Would be great if you could help me out. I think I'm loosing my mind over this.
Best,
Phill
I was trying the same thing using Matlab R2016a, and I get the following multcompare error message: "STATS must be a stats output structure from ANOVA1, ANOVA2, ANOVAN, AOCTOOL, KRUSKALWALLIS, or FRIEDMAN.".
However, this discussion was helpful for me:
https://www.mathworks.com/matlabcentral/answers/140799-3-way-repeated-measures-anova-pairwise-comparisons-using-multcompare
You might try something like:
multcompare(repMeasModel,'Factor1','By','Factor2)
I believe you'll need to create factors in the within structure of your model too.
I am a newbie to Matlab and I am currently trying to optimize a nested for loop as below. The loop is currently running forever for my input.
for i = 1:size(mat,1)
for j = 1:size(mat,2)
mat(i,j) = some_mapping(mat(i,j)+1);
end
end
However I can't find a way to vectorize it. I have tried bsxfun and arrayfun but it does not seem to work (or even run more slowly than the loop).
Maybe I was doing it in a wrong way. Any help is appreciated!
As suggested by Andras Deak, if some_mapping is simply a look-up-table operation, then
mat = some_mapping( mat+1 );
Notes:
- In order of the mapping to work, the values of mat must be integers in the range [0..numel(some_mapping)-1].
- The size of some_mapping does not affect the size of the result, it will be identical in size to mat.
I keep getting this error. I received this error after I corrected for logical integer error. The only way I could think of programming this was defining the initial value and then start euler's method at the next t value since it uses the previous solution to find the next one. But by doing this I am getting this error? I'm unsure how to fix it. I tried to end one step from my final value but that didn't work either. Thanks for the help. For the problem we had to create the function and call it. I was initially using n=8.
function [exeuler] = pb3(n)
%using explicit euler to solve ODE with input n and outputting exeuler as
%the answer
%n=steps t,y are initial conditions
h=3/n;
t=logical((1+h):h:4);
back=logical(t-h);
exeuler(1)=2; %defines the initial value
exeuler(t)=exeuler(back)+h*(t.^2-(2*exeuler(back)/t));
end
Well, I am sorry, but you haven't showed much of effort. For the future, please provide the complete code (i.e. with example function and so on). This is very unclear.
I think your problem is in these lines:
t=logical((1+h):h:4);
exeuler(t)=exeuler(back)+h*(t.^2-(2*exeuler(back)/t));
cause t is a vector. You are calling vector with non-integer values as index. Then, I guess, you get the wrong amount of exeuler(t)'s which leads to the error.
And for-loop is missing, isn't it? Because you don't use the Euler method stepwise andd therefore f(t+1) doesn't really depend on f(t).
So, my advice in general would be not to correct this error, but to rethink your algorithm.
I am working on a project that needs to use hidden markov models. I downloaded Kevin Murphy's toolbox. I have some problems about the usage. In the toolbox webpage, he says that first input of dhmm_em and dhmm_logprob are symbol sequence data. On their examples, they give row vectors as data. So, when I give my symbol sequence as row vector, I get error;
??? Error using ==> assert at 9
assertion violated:
Error in ==> fwdback at 105
assert(approxeq(sum(alpha(:,t)),1))
Error in ==> dhmm_logprob at 17
[alpha, beta, gamma, ll] = fwdback(prior,
transmat, obslik, 'fwd_only', 1);
Error in ==> mainCourseProject at 110
loglik(train_act) =
dhmm_logprob(orderedSymbols,
hmm{train_act}.prior,
hmm{train_act}.trans,
hmm{act}.emiss);
However, before giving this error, code works for some symbol vectors. When I give my data as column vector, functions work fine, no errors. So why exactly am I getting this error?
You might say that I should be giving not single vectors, but vector sets, I also tried to collect my feature vectors in a struct and give row vectors as such, but nothing changed, I still get assertion error.
By the way, my symbol sequence does not have any zeros, I am doing everything almost the same as they showed in their examples, so I would be greatful if anyone could help me please.
Im not sure, but from the function call stack shown above, shouldn't the last line be hmm{train_act}.emiss instead of hmm{act}.emiss.
In other words when you computing the log-probability of a sequence, you should pass components that belong to the same HMM model (transition matrix, emission matrix, and prior probabilities).
By the way, the ASSERT in the code is a sanity check that a vector of probabilities should sum to 1. Oftentimes, when working with very small values (log-probabilities), numerical stability issues can creep in... You could edit the APPROXEQ function to relax the comparison a bit, by giving it a bigger margin of error
This error message and the code it refers to are human-readable. An assertion is a guard put in by the programmer, to ensure that certain conditions are met. In this case, what is the condition? approxeq(sum(alpha(:,t)),1) I'd venture to say that approxeq wants the values to be approximately equal, so this boils down to: sum(alpha(:,t)) ~= 1
Without knowing anything about the code, I'd also guess that these refer to probabilities. The probabilities of a node's edges must sum to one. Hopefully this starts you down a productive debugging path. If you can't figure out what's wrong with your input that produces this condition, start wading into the code a bit to see where this alpha vector comes from, and how it ended up invalid.