I would like to calculate standard errors of contrasts in a linear mixed-effect model (fitlme) in MATLAB.
y = randn(100,1);
area = randi([1 3],100,1);
mea = randi([1 3],100,1);
sub = randi([1 5],100,1);
data = array2table([area mea sub y],'VariableNames',{'area','mea','sub','y'});
data.area = nominal(data.area,{'A','B','C'});
data.mea = nominal(data.mea,{'Baseline','+1h','+8h'});
data.sub = nominal(data.sub);
lme = fitlme(data,'y~area*mea+(1|sub)')
% Plot Area A on three measurements
coefv = table2array(dataset2table(lme.Coefficients(:,2)));
bar([coefv(1),sum(coefv([1 4])),sum(coefv([1 5]))])
Calculating the contrast means, e.g. area1-measurement1 vs area1-measurement2 vs area1-measurement3 can be done by summing the related coefficient parameters. However, does anyone know how to calculate the related standard errors?
I know a hypothesis test can be done by coefTest(lme,H), but only p values can be extracted.
An example for Area A is shown below:
I have resolved this issue!
Matlab uses the 'predict' function to estimate contrasts. To find confidence intervals for area A, at measurement +8h in this particular example use:
dsnew = dataset();
dsnew.area = nominal('A');
dsnew.mea = nominal('+8h');
dsnew.sub = nominal(1);
[yh yCI] = predict(lme,dsnew,'Conditional',false)
A result is shown below:
Related
After reading the docs about "stateEstimatorPF" I get a little confused about how to create the StateTransitionFcn for my case. In my case I have 10 sensors measurments that decay exponentially and I want to find the best parameters for my function model.
The function model is x = exp(B*deltaT)*x_1, where x are the hypotheses, deltaT is the constant time delta in my measurments and x_1 is the true previous state. I would like to use the particle filter to estimate the parameter B. If I guess right, B should be the particles and the weighted mean of this particles should be what I'm looking for.
How can I write the StateTransitionFcn and use the "stateEstimatorPF" to solve this problem?
The code below is what I get so far (and it does not work):
pf = robotics.ParticleFilter
pf.StateTransitionFcn = #stateTransitionFcn
pf.StateEstimationMethod = 'mean';
pf.ResamplingMethod = 'systematic';
initialize(pf,5000,[0.9],1);
measu = [1.0, 0.9351, 0.8512, 0.9028, 0.7754, 0.7114, 0.6830, 0.6147, 0.5628, 0.7090]
states = []
for i=1:10
[statePredicted,stateCov] = predict(pf);
[stateCorrected,stateCov] = correct(pf,measu(i));
states(i) = getStateEstimate(pf)
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function predictParticles = stateTransitionFcn(pf, prevParticles,x_1)
predictParticles = exp(prevParticles)*x_1 %how to properly use x_1?%;
end
I want to ask your help in EEG data classification.
I am a graduate student trying to analyze EEG data.
Now I am struggling with classifying ERP speller (P300) with SWLDA using Matlab
Maybe there is something wrong in my code.
I have read several articles, but they did not cover much details.
My data size is described as below.
size(target) = [300 1856]
size(nontarget) = [998 1856]
row indicates the number of trials, column indicates spanned feature
(I stretched data [64 29] (for visual representation I did not select ROI)
I used stepwisefit function in Matlab to classify target vs non-target
Code is attached below.
ingredients = [targets; nontargets];
heat = [class_targets; class_nontargets]; % target: 1, non-target: -1
randomized_set = shuffle([ingredients heat]);
for k=1:10 % 10-fold cross validation
parition_factor = ceil(size(randomized_set,1) / 10);
cv_test_idx = (k-1)*parition_factor + 1:min(k * parition_factor, size(randomized_set,1));
total_idx = 1:size(randomized_set,1);
cv_train_idx = total_idx(~ismember(total_idx, cv_test_idx));
ingredients = randomized_set(cv_train_idx, 1:end-1);
heat = randomized_set(cv_train_idx, end);
[W,SE,PVAL,INMODEL,STATS,NEXTSTEP,HISTORY]= stepwisefit(ingredients, heat, 'penter', .1);
valid_id = find(INMODEL==1);
v_weights = W(valid_id)';
t_ingredients = randomized_set(cv_test_idx, 1:end-1);
t_heat = randomized_set(cv_test_idx, end); % true labels for test set
v_features = t_ingredients(:, valid_id);
v_weights = repmat(v_weights, size(v_features, 1), 1);
predictor = sum(v_weights .* v_features, 2);
m_result = predictor > 0; % class A: +1, B: 0
t_heat(t_heat==-1) = 0;
acc(k) = sum(m_result==t_heat) / length(m_result);
end
p.s. my code is currently very inefficient and might be bad..
In my assumption, stepwisefit calculates significant coefficients every steps, and valid column would be remained.
Even though it's not LDA, but for binary classification, LDA and linear regression are not different.
However, results were almost random chance.. (for other binary data on the internet, it worked..)
I think I made something wrong, and your help can correct me.
I will appreciate any suggestion and tips to implement classifier for ERP speller.
Or any idea for implementing SWLDA in Matlab code?
The name SWLDA is only used in the context of Brain Computer Interfaces, but I bet it has another name in a more general context.
If you track the recipe of SWLDA you will end up in Krusienski 2006 papers ("A comparison..." and "Toward enhanced P300..") and from there the book where stepwise logarithmic regression is explained: "Draper Smith, Applied Regression Analysis, 1981". However, as far as I am aware of, no paper gives actually the complete recipe on how to implement it (and their details and secrets).
My approach was using stepwiseglm:
H=predictors;
TH=variables;
lbs=labels % (1,2)
if (stepwiseflag)
mdl = stepwiseglm(H', lbs'-1,'constant','upper','linear','distr','binomial');
if (mdl.NumEstimatedCoefficients>1)
inmodel = [];
for i=2:mdl.NumEstimatedCoefficients
inmodel = [inmodel str2num(mdl.CoefficientNames{i}(2:end))];
end
H = H(inmodel,:);
TH = TH(inmodel,:);
end
end
lbls = classify(TH',H',lbs','linear');
You can also use a k-fold cross validaton approach using matlab cvpartition.
c = cvpartition(lbs,'k',10);
opts = statset('display','iter');
fun = #(XT,yT,Xt,yt)...
(sum(~strcmp(yt,classify(Xt,XT,yT,'linear'))));
I'm trying to estimate the (unknown) original datapoints that went into calculating a (known) moving average. However, I do know some of the original datapoints, and I'm not sure how to use that information.
I am using the method given in the answers here: https://stats.stackexchange.com/questions/67907/extract-data-points-from-moving-average, but in MATLAB (my code below). This method works quite well for large numbers of data points (>1000), but less well with fewer data points, as you'd expect.
window = 3;
datapoints = 150;
data = 3*rand(1,datapoints)+50;
moving_averages = [];
for i = window:size(data,2)
moving_averages(i) = mean(data(i+1-window:i));
end
length = size(moving_averages,2)+(window-1);
a = (tril(ones(length,length),window-1) - tril(ones(length,length),-1))/window;
a = a(1:length-(window-1),:);
ai = pinv(a);
daily = mtimes(ai,moving_averages');
x = 1:size(data,2);
figure(1)
hold on
plot(x,data,'Color','b');
plot(x(window:end),moving_averages(window:end),'Linewidth',2,'Color','r');
plot(x,daily(window:end),'Color','g');
hold off
axis([0 size(x,2) min(daily(window:end))-1 max(daily(window:end))+1])
legend('original data','moving average','back-calculated')
Now, say I know a smattering of the original data points. I'm having trouble figuring how might I use that information to more accurately calculate the rest. Thank you for any assistance.
You should be able to calculate the original data exactly if you at any time can exactly determine one window's worth of data, i.e. in this case n-1 samples in a window of length n. (In your case) if you know A,B and (A+B+C)/3, you can solve now and know C. Now when you have (B+C+D)/3 (your moving average) you can exactly solve for D. Rinse and repeat. This logic works going backwards too.
Here is an example with the same idea:
% the actual vector of values
a = cumsum(rand(150,1) - 0.5);
% compute moving average
win = 3; % sliding window length
idx = hankel(1:win, win:numel(a));
m = mean(a(idx));
% coefficient matrix: m(i) = sum(a(i:i+win-1))/win
A = repmat([ones(1,win) zeros(1,numel(a)-win)], numel(a)-win+1, 1);
for i=2:size(A,1)
A(i,:) = circshift(A(i-1,:), [0 1]);
end
A = A / win;
% solve linear system
%x = A \ m(:);
x = pinv(A) * m(:);
% plot and compare
subplot(211), plot(1:numel(a),a, 1:numel(m),m)
legend({'original','moving average'})
title(sprintf('length = %d, window = %d',numel(a),win))
subplot(212), plot(1:numel(a),a, 1:numel(a),x)
legend({'original','reconstructed'})
title(sprintf('error = %f',norm(x(:)-a(:))))
You can see the reconstruction error is very small, even using the data sizes in your example (150 samples with a 3-samples moving average).
I am optimizing portfolio of N stocks over M levels of expected return. So after doing this I get the time series of weights (i.e. a N x M matrix where where each row is a combination of stock weights for a particular level of expected return). Weights add up to 1.
Now I want to plot something called portfolio composition map (right plot on the picture), which is a plot of these stock weights over all levels of expected return, each with a distinct color and length (at every level of return) is proportional to it's weight.
My questions is how to do this in Julia (or MATLAB)?
I came across this and the accepted solution seemed so complex. Here's how I would do it:
using Plots
#userplot PortfolioComposition
#recipe function f(pc::PortfolioComposition)
weights, returns = pc.args
weights = cumsum(weights,dims=2)
seriestype := :shape
for c=1:size(weights,2)
sx = vcat(weights[:,c], c==1 ? zeros(length(returns)) : reverse(weights[:,c-1]))
sy = vcat(returns, reverse(returns))
#series Shape(sx, sy)
end
end
# fake data
tickers = ["IBM", "Google", "Apple", "Intel"]
N = 10
D = length(tickers)
weights = rand(N,D)
weights ./= sum(weights, dims=2)
returns = sort!((1:N) + D*randn(N))
# plot it
portfoliocomposition(weights, returns, labels = tickers)
matplotlib has a pretty powerful polygon plotting capability, e.g. this link on plotting filled polygons:
ploting filled polygons in python
You can use this from Julia via the excellent PyPlot.jl package.
Note that the syntax for certain things changes; see the PyPlot.jl README and e.g. this set of examples.
You "just" need to calculate the coordinates from your matrix and build up a set of polygons to plot the portfolio composition graph. It would be nice to see the code if you get this working!
So I was able to draw it, and here's my code:
using PyPlot
using PyCall
#pyimport matplotlib.patches as patch
N = 10
D = 4
weights = Array(Float64, N,D)
for i in 1:N
w = rand(D)
w = w/sum(w)
weights[i,:] = w
end
weights = [zeros(Float64, N) weights]
weights = cumsum(weights,2)
returns = sort!([linspace(1,N, N);] + D*randn(N))
##########
# Plot #
##########
polygons = Array(PyObject, 4)
colors = ["red","blue","green","cyan"]
labels = ["IBM", "Google", "Apple", "Intel"]
fig, ax = subplots()
fig[:set_size_inches](5, 7)
title("Problem 2.5 part 2")
xlabel("Weights")
ylabel("Return (%)")
ax[:set_autoscale_on](false)
ax[:axis]([0,1,minimum(returns),maximum(returns)])
for i in 1:(size(weights,2)-1)
xy=[weights[:,i] returns;
reverse(weights[:,(i+1)]) reverse(returns)]
polygons[i] = matplotlib[:patches][:Polygon](xy, true, color=colors[i], label = labels[i])
ax[:add_artist](polygons[i])
end
legend(polygons, labels, bbox_to_anchor=(1.02, 1), loc=2, borderaxespad=0)
show()
# savefig("CompositionMap.png",bbox_inches="tight")
Can't say that this is the best way, to do this, but at least it is working.
I have a portfolio with assets and I like to calculate the variance of the portfolio. I calculated the weights of the assets and the covariance matrix. Both are time-varying. To get the variance I need the formula below:
Wt'*SIGMA_1*Wt = variance portfolio
In Matlab I used the following code to calculate the weight of each asset:
for t=1:n
At_1(:,t) = inv(SIGMA_1(:,:,t))*FIT_1';
Wt_1(t,1) = At_1(1,t)/sum(At_1(:,t));
Wt_1(t,2) = At_1(2,t)/sum(At_1(:,t));
Wt_1(t,3) = At_1(3,t)/sum(At_1(:,t));
Wt_1(t,4) = At_1(4,t)/sum(At_1(:,t));
Wt_1(t,5) = At_1(5,t)/sum(At_1(:,t));
Wt_1(t,6) = At_1(6,t)/sum(At_1(:,t));
Wt_1(t,7) = At_1(7,t)/sum(At_1(:,t));
Wt_1(t,8) = At_1(8,t)/sum(At_1(:,t));
Wt_1(t,9) = At_1(9,t)/sum(At_1(:,t));
Wt_1(t,10) = At_1(10,t)/sum(At_1(:,t));
end
where SIGMA_1 is the covariance matrix.
Now I need a loop to calculate the time-varying variance of the portfolio. I have 10 time-varying weights, and the time-varying covariance SIGMA_1.I am stuck with writing a loop for that. Can somebody help me with this?
Have you tried this?
for t=1:n
At_1(:,t) = inv(SIGMA_1(:,:,t))*FIT_1';
for k=1:10
Wt_1(t,k) = At_1(k,t)/sum(At_1(:,t));
end
end