I have experimental data on how diameter (D/D0) of a fluid filament thins over time (t). D/D0 (ydata) and t (xdata) are np arrays.
I would like to fit the data to the following equation:
In the equation, my known variables are D/D1 = D/D0, D1=D0, and t.
D/D0 = np.array([0.129 , 0.1613, 0.1935, 0.2258, 0.2581, 0.2903, 0.3226,0.3548, 0.3871, 0.4194, 0.4516, 0.4839, 0.5161, 0.5484, 0.5806, 0.6129, 0.6452, 0.6774, 0.7097, 0.7419, 0.7742, 0.8065, 0.8387, 0.871 , 0.9032, 0.9355, 1.])
t = np.array([0.7705, 0.7495, 0.5768, 0.5699, 0.503 , 0.493 , 0.4102, 0.3693, 0.3234, 0.2774, 0.2116, 0.2076, 0.1856, 0.1627, 0.1397, 0.1198, 0.0908, 0.0848, 0.0609, 0.0499, 0.0399, 0.0349, 0.0289, 0.02 , 0.012 , 0.007 , 0.002 ])
alpha, a, eta_0 and lambda are unknown to me and I would like to find these values and plot the equation with my experimental data.
How best can I achieve this in Python?
It is important to replace the three dependent parameters by only one independent parameter into the model equation. Then one can use any software for non-linear regression.
Related
I have a relatively complex non-linear system to solve in MATLAB, which can be described as a function of unknown coefficients and basis functions, that I am trying to fit to some initial trajectory data to get the coefficients.
I currently have a working solution using MATLAB's fittype function, but it is limited as only scalar values can be input as coefficients, whereas I need to run this fit multiple times using a different number of unknown coefficients Xi to quantify my wider algorithm's performance with a varying number of basis functions. An example of the fit is below:
s1_fit_1 = fittype( #(xi1, xi2, xi3, xi4, xi5, xi6, xi7, xi8, xi9, xi10, r1_1, v1_1, t) ...
( (( chebyshevT((mBasis-1), (time_to_z(t0, t1, t)))' ...
- (omega1(getT1(t), dt1)' .* h0) ...
- (omega2(getT1(t), dt1)' .* hf) ...
- (omega3(getT1(t), dt1)' .* h0_dot) ...
- (omega4(getT1(t), dt1)' .* hf_dot))' * [xi1; xi2; xi3; xi4; xi5; xi6; xi7; xi8; xi9; xi10]) ...
+ (omega1(getT1(t), dt1) * r0_1) ...
+ (omega2(getT1(t), dt1) * r1_1) ...
+ (omega3(getT1(t), dt1) * v0_1) ...
+ (omega4(getT1(t), dt1) * v1_1) ), ...
'independent', 't', 'dependent', 's1_r_1', ...
'coefficients', {'xi1', 'xi2', 'xi3', 'xi4', 'xi5', 'xi6', 'xi7', 'xi8', 'xi9', 'xi10', 'r1_1', 'v1_1'});
For reference, functions omega[x] and time_to_z() are not affected by this and can be neglected.
As you can see I currently have 10 Xi[x] coefficients; I would like to vary this number over multiple script runs, up to approximately 50 as a maximum. It can be done manually, but that would be very time-consuming. The simplest way for me to do this would be to define Xi as a vector or cell where I dynamically update the size, but fittype does not seem to support this.
Is there a way for me to achieve this, or perhaps another non-linear solver I can use that would achieve the same result?
Many thanks.
vector-valued coefficients are better supported by function lsqcurvefit from Optimization Toolbox:
https://www.mathworks.com/matlabcentral/answers/470283-using-a-vector-of-coefficients-in-fittype
I have the system of second order ODEs:
A*u(t) + B*u''(t) = q(t) + b_A + b_B.
Here, A and B are known matrices, b_A is a known vector, b_B is a known vector, and q(t) is a time dependent vector which I can compute for a given t-value.
The goal of my problem is to numerically approximate the functions u_1 , ... , u_n, which are entries in u(t). Also, u''(t) denotes the second time derivative of u(t). I also have the initial condition vector:
u0 = zeros(n,1).
How would I solve this problem using MATLABs built in ode solvers (ode45)?
All of the examples I have seen thus far involve converting the system of second order ODEs into a system of first order ODEs, but they have all been very small examples. Thanks for the help.
To convert this into a system of first order ODEs, I would do
y_1 = u
y_2 = u', so :
y_1' = y_2, and
y_2' = A^(-1)*(q(t) + b_A + b_B - A*y_1).
How should I implement this in MATLAB?
I was training to do some PCA reconstroctions of MNIST on python and compare them to my (old) reconstruction in maltab and I happened to discover that my reconstruction don't agree. After some debugging I decided to print a unique characteristic of the principal components of each one to reveal if they were the same and I discovered to my surprised that they were not the same. I printing the sum of all components and I got different numbers. I did the following in matlab:
[coeff, ~, ~, ~, ~, mu] = pca(X_train);
U = coeff(:,1:K)
U_fingerprint = sum(U(:))
%print 31.0244
and in python/scipy:
pca = pca.fit(X_train)
U = pca.components_
print 'U_fingerprint', np.sum(U)
# prints 12.814
why are the twi PCA's not computing the same value?
All my attempts and solving this issue:
The way I discovered this was because when I was reconstructing my MNIST images, the python reconstructions where much much closer to their original images by a lot. I got error of 0.0221556788645 in python while in MATLAB I got errors of size 29.07578. To figure out where the difference was coming from I decided to finger print the data sets (maybe they were normalized differently). So I got two independent copies the MNIST data set (that were normalized by dividing my 255) and got the finger prints (summing all numbers in data set):
print np.sum(x_train) # from keras
print np.sum(X_train)+np.sum(X_cv) # from TensorFlow
6.14628e+06
6146269.1585420668
which are (essentially) same (one copy from tensorflow MNIST and the other from Keras MNIST, note MNIST train data set has about 1000 less training set so you need to append the missing ones). To my surprise, my MATLAB data had the same finger print:
data_fingerprint = sum(X_train(:))
% prints data_fingerprint = 6.1463e+06
meaning the data sets are exactly the same. Good, so the normalization data is not the issue.
In my MATLAB script I am actually computing the reconstruction manually as follow:
U = coeff(:,1:K)
X_tilde_train = (U * U' * X_train);
train_error_PCA = (1/N_train)*norm( X_tilde_train - X_train ,'fro')^2
%train_error_PCA = 29.0759
so I thought that might be the problem because I was using the interface python gave for computing the reconstructions as in:
pca = PCA(n_components=k)
pca = pca.fit(X_train)
X_pca = pca.transform(X_train) # M_train x K
#print 'X_pca' , X_pca.shape
X_reconstruct = pca.inverse_transform(X_pca)
print 'tensorflow error: ',(1.0/X_train.shape[0])*LA.norm(X_reconstruct_tf - X_train)
print 'keras error: ',(1.0/x_train.shape[0])*LA.norm(X_reconstruct_keras - x_train)
#tensorflow error: 0.0221556788645
#keras error: 0.0212030354818
which results in different error values 0.022 vs 29.07, shocking difference!
Thus, I decided to code that exact reconstruction formula in my python script:
pca = PCA(n_components=k)
pca = pca.fit(X_train)
U = pca.components_
print 'U_fingerprint', np.sum(U)
X_my_reconstruct = np.dot( U.T , np.dot(U, X_train.T) )
print 'U error: ',(1.0/X_train.shape[0])*LA.norm(X_reconstruct_tf - X_train)
# U error: 0.0221556788645
to my surprise, it has the same error as my MNIST error computing by using the interface. Thus, concluding that I don't have the misconception of PCA that I thought I had.
All that lead to me to check what the principal components actually where and to my surprise scipy and MATLAB have different fingerprint for their PCA values.
Does anyone know why or whats going on?
As warren suggested, the pca components (eigenvectors) might have different sign. After doing a finger print by adding all components in magnitude only I discovered they have the same finger print:
[coeff, ~, ~, ~, ~, mu] = pca(X_train);
K=12;
U = coeff(:,1:K)
U_fingerprint = sumabs(U(:))
% U_fingerprint = 190.8430
and for python:
k=12
pca = PCA(n_components=k)
pca = pca.fit(X_train)
print 'U_fingerprint', np.sum(np.absolute(U))
# U_fingerprint 190.843
which means the difference must be because of the different sign of the (pca) U vector. Which I find very surprising, I thought that should make a big difference, I didn't even consider it making a big difference. I guess I was wrong?
I don't know if this is the problem, but it certainly could be. Principal component vectors are like eigenvectors: if you multiply the vector by -1, it is still a valid PCA vector. Some of the vectors computed by matlab might have a different sign than those computed in python. That will result in very different sums.
For example, the matlab documentation has this example:
coeff = pca(ingredients)
coeff =
-0.0678 -0.6460 0.5673 0.5062
-0.6785 -0.0200 -0.5440 0.4933
0.0290 0.7553 0.4036 0.5156
0.7309 -0.1085 -0.4684 0.4844
I have my own python PCA code, and with the same input as in matlab, it produces this coefficient array:
[[ 0.0678 0.646 -0.5673 0.5062]
[ 0.6785 0.02 0.544 0.4933]
[-0.029 -0.7553 -0.4036 0.5156]
[-0.7309 0.1085 0.4684 0.4844]]
So, instead of simply summing the coefficient array, try summing the absolute values of the coefficients. Alternatively, ensure that all the vectors have the same sign convention before summing. You could do that by, say, multiplying each column by the sign of the first element in that column (assuming none of them are zero).
This is my code in IBM SPSS:
FACTOR
/VARIABLES VAR00001 VAR00002 VAR00003 VAR00004 VAR00005 VAR00006
/MISSING LISTWISE
/ANALYSIS VAR00001 VAR00002 VAR00003 VAR00004 VAR00005 VAR00006
/PRINT UNIVARIATE INITIAL CORRELATION SIG DET KMO INV REPR AIC EXTRACTION ROTATION
/PLOT EIGEN ROTATION
/CRITERIA MINEIGEN(1) ITERATE(25)
/EXTRACTION PC
/CRITERIA ITERATE(25)
/ROTATION VARIMAX
/METHOD=CORRELATION.
and this is code of MATLAB R2015b to do the same:
[lambda,psi,T,stats,F]=factoran(DATA,2,'rotate','varimax');
SPSS output for roteted component matrix:
Rotated Component Matrix
Component
1 2
VAR00001 .973 -.062
VAR00002 .911 -.134
VAR00003 .833 -.035
VAR00004 .972 -.102
VAR00005 -.236 .823
VAR00006 .062 .878
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
a Rotation converged in 3 iterations.
MATLAB lambda output:
0.993085200854508 -0.0537771548307969
0.875990644597448 -0.147112975689921
0.748570753047806 -0.0343768914779775
0.987459815125692 -0.0988807726538385
-0.203059229288894 0.976610007465447
0.00719025397609984 0.475514010080256
Why these outputs are different? I want same results in MATLAB. As you know SPSS ignores eigenvalues smaller than 1. I want same structure in MATLAB. How can I do this?
PS.
MATLAB T output:
0.622170579007477 -0.782881709211232
0.782881709211232 0.622170579007477
MATLAB psi output:
0.0108898014620571
0.210998162961140
0.438460057014266
0.0151457063113246
0.00500000000002244
0.773834726466399
Other SPSS outputs:
Component Matrix
Component
1 2
VAR00001 .964 .144
VAR00002 .919 .061
VAR00003 .821 .141
VAR00004 .971 .105
VAR00005 -.404 .755
VAR00006 -.124 .871
Extraction Method: Principal Component Analysis.
a 2 components extracted.
Component Transformation Matrix
Component 1 2
1 .977 -.211
2 .211 .977
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
Matlab extracts factors using the maximum likelihood method. I don't think you can change this. SPSS extracts methods using principle components as its default, and this is the method that you have chosen for your SPSS analysis. That's yet another difference...
I'm using the PRTools MATLAB library to train some classifiers, generating test data and testing the classifiers.
I have the following details:
N: Total # of test examples
k: # of
mis-classification for each
classifier and class
I want to do:
Calculate and plot Bayesian posterior distributions of the unknown probabilities of mis-classification (denoted q), that is, as probability density functions over q itself (so, P(q) will be plotted over q, from 0 to 1).
I have that (math formulae, not matlab code!):
Posterior = Likelihood * Prior / Normalization constant =
P(q|k,N) = P(k|q,N) * P(q|N) / P(k|N)
The prior is set to 1, so I only need to calculate the likelihood and normalization constant.
I know that the likelihood can be expressed as (where B(N,k) is the binomial coefficient):
P(k|q,N) = B(N,k) * q^k * (1-q)^(N-k)
... so the Normalization constant is simply an integral of the posterior above, from 0 to 1:
P(k|N) = B(N,k) * integralFromZeroToOne( q^k * (1-q)^(N-k) )
(The Binomial coefficient ( B(N,k) ) can be omitted though as it appears in both the likelihood and normalization constant)
Now, I've heard that the integral for the normalization constant should be able to be calculated as a series ... something like:
k!(N-k)! / (N+1)!
Is that correct? (I have some lecture notes with this series, but can't figure out if it is for the normalization constant integral, or for the overall distribution of mis-classification (q))
Also, hints are welcome as how to practically calculate this? (factorials are easily creating truncation errors right?) ... AND, how to practically calculate the final plot (the posterior distribution over q, from 0 to 1).
I really haven't done much with Bayesian posterior distributions ( and not for a while), but I'll try to help with what you've given. First,
k!(N-k)! / (N+1)! = 1 / (B(N,k) * (N + 1))
and you can calculate the binomial coefficients in Matlab with nchoosek() though it does say in the docs that there can be accuracy problems for large coefficients. How big are N and k?
Second, according to Mathematica,
integralFromZeroToOne( q^k * (1-q)^(N-k) ) = pi * csc((k-N)*pi) * Gamma(1+k)/(Gamma(k-N) * Gamma(2+N))
where csc() is the cosecant function and Gamma() is the gamma function. However, Gamma(x) = (x-1)! which we'll use in a moment. The problem is that we have a function Gamma(k-N) on the bottom and k-N will be negative. However, the reflection formula will help us with that so that we end up with:
= (N-k)! * k! / (N+1)!
Apparently, your notes were correct.
Let q be the probability of mis-classification. Then the probability that you would observe k mis-classifications in N runs is given by:
P(k|N,q) = B(N,k) q^k (1-q)^(N-k)
You need to then assume a suitable prior for q which is bounded between 0 and 1. A conjugate prior for the above is the beta distribution. If q ~ Beta(a,b) then the posterior is also a Beta distribution. For your info the posterior is:
f(q|-) ~ Beta(a+k,b+N-k)
Hope that helps.