I want to calculate the mean value if the first column numbers are same. e.g:
M = [ 2 0.99 0.15 0.60 0.12 0.76 0.16 0.81 0.02 0.75 0.32
2 0.17 0.38 0.34 0.02 0.74 0.67 0.75 0.92 0.23 0.81
2 0.26 0.16 0.30 0.29 0.74 0.89 0.12 0.65 0.06 0.79
3 0.40 0.76 0.45 0.32 0.11 0.52 0.53 0.93 0.77 0.85
3 0.07 0.87 0.42 0.65 0.68 0.70 0.33 0.16 0.67 0.51
3 0.68 0.35 0.36 0.96 0.46 0.15 0.55 0.92 0.72 0.64
3 0.40 0.69 0.56 0.94 0.21 0.95 0.40 0.79 0.64 0.95
4 0.98 0.29 0.74 0.46 0.10 0.54 0.42 0.58 0.42 0.44
4 0.40 0.53 0.42 0.24 0.82 0.68 0.18 0.44 0.39 0.06
4 0.62 0.83 0.43 0.76 0.18 0.04 0.26 0.26 0.82 0.87 ]
Out=[ 2 0.47 0.23 0.41 0.15 0.75 0.57 0.56 0.53 0.35 0.64
3 0.39 0.67 0.45 0.72 0.37 0.58 0.45 0.70 0.70 0.74
4 0.67 0.55 0.53 0.49 0.37 0.42 0.28 0.43 0.54 0.46 ]
G = findgroups(M(:,1 ));
Out = [unique(M(:,1)) splitapply(#mean, M(:,2:end), G)]
G = findgroups(A) returns G, a vector of group numbers created from the grouping variable A. Here G = findgroups(M(:,1 )) means pick up the first column out the matrics.
Y = splitapply(func,X,G) splits X into groups specified by G and applies the function func to each group.
Related
The following script fits a curve bowing-like via curve_fit (from scipy.optimize), see below:
ydata = numpy.array[ 1.6504 1.63928044 1.62855028 1.6181874 1.60817119 1.59848249 1.58910347 1.58001759 1.57120948 1.56266487 1.55437054 1.54631424 1.5384846 1.53087109 1.52346397 1.5162542 1.5092334 1.50239383 1.4957283 1.48923013 1.48289315 1.47671162 1.4706802 1.46479393 1.45904821 1.45343874 1.44796151 1.44261281 1.43738913 1.43228723 1.42730406 1.42243677 1.4176827 1.41303936 1.40850439 1.40407561 1.39975096 1.39552851 1.39140647 1.38738314 1.38345695 1.37962642 1.37589018 1.37224696 1.36869555 1.36523487 1.36186389 1.35858169 1.35538741 1.35228028 1.34925958 1.34632469 1.34347504 1.34071015 1.33802957 1.33543295 1.33291998 1.33049042 1.32814407 1.32588081 1.32370057 1.32160331 1.31958908 1.31765795 1.31581005 1.31404556 1.31236472 1.3107678 1.30925513 1.30782709 1.30648411 1.30522666 1.3040553 1.30297062 1.30197327 1.30106398 1.30024355 1.29951286 1.29887287 1.29832464 1.29786933 1.29750821 1.29724268 1.29707426 1.29700463 1.29703564 1.29716927 1.29740773 1.2977534 1.29820885 1.29877688 1.29946049 1.3002629 1.30118751 1.30223793 1.30341792 1.30473139 1.30618232 1.30777475 1.30951267 1.3114 ]
xdata = numpy.array[ 0. 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.2 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.3 0.31 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.5 0.51 0.52 0.53 0.54 0.55 0.56 0.57 0.58 0.59 0.6 0.61 0.62 0.63 0.64 0.65 0.66 0.67 0.68 0.69 0.7 0.71 0.72 0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.8 0.81 0.82 0.83 0.84 0.85 0.86 0.87 0.88 0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1. ]
sigma = np.ones(len(xdata))
sigma[[0, -1]] = 0.01
def function_cte(x, b):
return 1.31*x + 1.57*(1-x) - b*x*(1-x)
def function_linear(x, c1, c2):
return 1.31*x + 1.57*(1-x) - (c1+c2*x)*x*(1-x)
popt_cte, pcov_cte = curve_fit(function_cte, xdata, ydata, sigma=sigma)
popt_lin, pcov_lin = curve_fit(function_linear, xdata, ydata, sigma=sigma)
But I'm getting the plot in figure ,
i.e., the initial points from both functions disagree with the data to fit (xdata, ydata).
I would like a fit constrained on the endpoints (0.0, 1.57) and (1.0, 1.31) at the same point that minimize the error. Any idea based on this code or it is better taking another way?
thanks!
I have a 256*256 matrix, some values are 0 (close the each other); and I find the coordinates' of 0 values.
% finding missing rows and cols: xi, yi
[row,col]=find(~X);
MIS=[row,col];
MISWO=[MIS zeros(size(MIS,1),1) ];
MISWO
...
168 224 0
169 224 0
170 224 0
171 224 0
172 224 0
173 224 0
174 224 0
Part of the X matrix:
0.57 0.58 0.00 0.55 0.54
0.55 0.54 0.00 0.55 0.52
0.56 0.55 0.00 0.55 0.53
0.56 0.55 0.00 0.53 0.52
0.56 0.00 0.00 0.53 0.54
0.55 0.00 0.00 0.53 0.52
0.55 0.00 0.00 0.55 0.51
0.55 0.00 0.00 0.53 0.51
0.56 0.00 0.00 0.51 0.53
0.55 0.00 0.00 0.51 0.51
0.55 0.00 0.00 0.51 0.49
0.55 0.00 0.00 0.52 0.49
0.56 0.00 0.53 0.51 0.48
My goal is finding the zero values 5-10 neighbors with coordinates and values.
Can anybody help me?
All the best
In order to find all nearest neighbors in a 5x5 box around each zero pixel we can use 2d convolution:
X1=conv2(double(~X),ones(5),'same')>0;
This yields a binary matrix with 1 in the places of ALL the nearest neighbors positions around zero pixels. finding the rows and cols for all the nearest neighbors without the zeros is just:
[row2 col2]=find(X1.*X);
Then the matrix that you want is:
MIS2=[row2 col2 X(row2, col2)];
I have data in excel like this:
id date value
a 9/17/2012 0.25
a 9/18/2012 0.48
a 9/19/2012 0.29
a 9/20/2012 0.46
a 9/21/2012 0.17
a 9/24/2012 0.89
a 9/25/2012 0.20
a 9/26/2012 0.65
a 9/27/2012 0.26
b 9/17/2012 0.83
b 9/18/2012 0.87
b 9/19/2012 0.40
b 9/20/2012 0.33
b 9/21/2012 0.71
b 9/24/2012 0.13
b 9/25/2012 0.91
b 9/26/2012 0.73
b 9/27/2012 0.87
c 9/17/2012 0.47
c 9/18/2012 0.15
c 9/19/2012 0.73
c 9/20/2012 0.47
c 9/21/2012 0.03
c 9/24/2012 0.23
c 9/25/2012 0.21
c 9/26/2012 0.39
c 9/27/2012 0.77
and I would like to use Matlab to re-arrange to:
date a b c
9/17/2012 0.25 0.83 0.47
9/18/2012 0.48 0.87 0.15
9/19/2012 0.29 0.40 0.73
9/20/2012 0.46 0.33 0.47
9/21/2012 0.17 0.71 0.03
9/24/2012 0.89 0.13 0.23
9/25/2012 0.20 0.91 0.21
9/26/2012 0.65 0.73 0.39
9/27/2012 0.26 0.87 0.77
What's the easiest way to do this?
Use:
importdata or xlsread
join (statistics toolbox) or see MATLAB Combine matrices of different dimensions, filling values of corresponding indices.
This is a follow up question related to how to find out the scaling factors to match two curves in matlab?
I use the following code to figure out the scaling factors to match two curves
function err = sqrError(coeffs, x1, y1, x2, y2)
y2sampledInx1 = interp1(coeffs(1)*x2,y2,x1);
err = sum((coeffs(2)*y2sampledInx1-y1).^2);
end
and I used fmincon to optimize the result.
options = optimset('Algorithm','active-set','MaxFunEvals',10000,'TolCon',1e-7)
A0(1)=1; A0(2)=1; LBA1=0.1; UBA1=5; LBA2=0.1; UBA2=5;
LB=[LBA1 LBA2]; UB=[UBA1 UBA2];
coeffs = fmincon(#(c) sqrError(c,x1, y1, x2, y2),A0,[],[],[],[],LB,UB,[],options);
when I test with my data with the function,
x1=[-0.3
-0.24
-0.18
-0.12
-0.06 0
0.06
0.12
0.18
0.24
0.3
0.36
0.42
0.48
0.54
0.6
0.66
0.72
0.78
0.84
0.9
0.96
1.02
1.08
1.14
1.2
1.26
1.32
1.38
1.44
1.5
1.56
1.62
1.68
1.74
1.8
1.86
1.92
1.98
2.04 ] y1=[0.00
0.00
0.00
0.01
0.03
0.09
0.13
0.14
0.14
0.16
0.20
0.22
0.26
0.34
0.41
0.52
0.62
0.72
0.81
0.91
0.95
0.99
0.98
0.96
0.90
0.82
0.74
0.66
0.58
0.52
0.47
0.40
0.36
0.32
0.27
0.22
0.19
0.15
0.12
0.10 ];
x2=[-0.3
-0.24
-0.18
-0.12
-0.06 0
0.06
0.12
0.18
0.24
0.3
0.36
0.42
0.48
0.54
0.6
0.66
0.72
0.78
0.84
0.9
0.96
1.02
1.08
1.14
1.2
1.26
1.32
1.38
1.44
1.5
1.56
1.62
1.68
1.74
1.8
1.86
1.92
1.98
2.04 ]; y2=[0.00
0.00
0.00
0.00
0.05
0.15
0.15
0.13
0.11
0.11
0.13
0.18
0.24
0.33
0.43
0.54
0.66
0.76
0.84
0.90
0.93
0.94
0.94
0.91
0.87
0.81
0.75
0.69
0.63
0.55
0.49
0.43
0.37
0.32
0.27
0.23
0.19
0.16
0.13
0.10 ];
The error message shows up as follows:
??? Error using ==> interp1 at 172 NaN is not an appropriate value for
X.
Error in ==> sqrError at 2 y2sampledInx1 =
interp1(coeffs(1)*x2,y2,x1);
Error in ==> #(c)sqrError(c,x1,y1,x2,y2)
Error in ==> nlconst at 805
f =
feval(funfcn{3},x,varargin{:});
Error in ==> fmincon at 758
[X,FVAL,LAMBDA,EXITFLAG,OUTPUT,GRAD,HESSIAN]=...
Error in ==>coeffs = fmincon(#(c) sqrError(c,x1, y1, x2,
y2),A0,[],[],[],[],LB,UB,[],options);
What is wrong in the code and how should I get around with it.
Thanks for the help.
Your scaling is likely pushing the interpolated axis out of range of the x-axis of the data. i.e.
x1 < min(x2*coeffs(1)) or x1 > max(x2*coeffs(1)) for at least one x1 and the value of coeffs(1) chosen by the fitting algorithm
You can fix this by giving an extrapolation value for data outside the range. Alternately, you can use extrapolation to guess at these values. So try one of these
y2sampledInx1 = interp1(coeffs(1)*x2,y2,x1,'Linear', 'Extrap');
y2sampledInx1 = interp1(coeffs(1)*x2,y2,x1,'Linear', Inf);
y2sampledInx1 = interp1(coeffs(1)*x2,y2,x1,'Linear', 1E18); %if Inf messes with the algorithm
I need to sort out few small matrices from 1 huge raw matrix ...according to sorting 1st column (1st column contain either 1, 2, or 3)...
if 1st column is 1, then randomly 75% of the 1 save in file A1, 25% of the 1 save in file A2.
if 1st column is 2, then randomly 75% of the 2 save in file B1, 25% of the 2 save in file B2.
if 1st column is 3, then randomly 75% of the 3 save in file C1, 25% of the 3 save in file C2.
how am i going to write the code?
Example:
a raw matrix has 15 rows x 6 columns:
7 rows are 1 in 1st column, 5 rows are 2 in 1st column, and 3 rows are 3 in 1st column.
1 -0.05 -0.01 0.03 0.07 0.11
1 -0.4 -0.36 -0.32 -0.28 -0.24
1 0.3 0.34 0.38 0.42 0.46
1 0.75 0.79 0.83 0.87 0.91
1 0.45 0.49 0.53 0.57 0.61
1 0.8 0.84 0.88 0.92 0.96
1 0.05 0.09 0.13 0.17 0.21
2 0.5 0.54 0.58 0.62 0.66
2 0.4 0.44 0.48 0.52 0.56
2 0.9 0.94 0.98 1.02 1.06
2 0.85 0.89 0.93 0.97 1.01
2 0.75 0.79 0.83 0.87 0.91
3 0.36 0.4 0.44 0.48 0.52
3 0.6 0.64 0.68 0.72 0.76
3 0.4 0.44 0.48 0.52 0.56
7 rows got 1 in 1st column, randomly take out 75% of 7 rows (which is 7*0.75=5.25) to be new matrix (5rows x 6 columns), the rest of 25% become another new matrix
5 rows got 2 in 1st column, randomly take out 75% of 5 rows (which is 5*0.75=3.75) to be new matrix (4rows x 6 columns), the rest of 25% become another new matrix
3 rows got 3 in 1st column, randomly take out 75% of 3 rows (which is 3*0.75=2.25) to be new matrix (2rows x 6 columns), the rest of 25% become another new matrix
Result:
A1=
1 -0.4 -0.36 -0.32 -0.28 -0.24
1 0.3 0.34 0.38 0.42 0.46
1 0.75 0.79 0.83 0.87 0.91
1 0.8 0.84 0.88 0.92 0.96
1 -0.05 -0.01 0.03 0.07 0.11
B1=
2 0.9 0.94 0.98 1.02 1.06
2 0.85 0.89 0.93 0.97 1.01
2 0.5 0.54 0.58 0.62 0.66
2 0.75 0.79 0.83 0.87 0.91
C1=
3 0.36 0.4 0.44 0.48 0.52
3 0.4 0.44 0.48 0.52 0.56
here is one possible solution to your problem using the function randperm:
% Create matrices
firstcol=ones(15,1);
firstcol(8:12)=2;
firstcol(13:15)=3;
mat=[firstcol rand(15,5)];
% Sort according to first column
A=mat(mat(:,1)==1,:);
B=mat(mat(:,1)==2,:);
C=mat(mat(:,1)==3,:);
% Randomly rearrange lines
A=A(randperm(size(A,1)),:);
B=B(randperm(size(B,1)),:);
C=C(randperm(size(C,1)),:);
% Select first 75% lines (rounding)
A1=A(1:round(0.75*size(A,1)),:);
A2=A(round(0.75*size(A,1))+1:end,:);
B1=B(1:round(0.75*size(B,1)),:);
B1=B(round(0.75*size(B,1))+1:end,:);
C1=C(1:round(0.75*size(C,1)),:);
C1=C(round(0.75*size(C,1))+1:end,:);
Hope it helps.