I'm having some serious issues fitting an exponential function (Beer-Lambert law) to my data. The optimization toolset function that I'm using produces terrible fits:
function [ Coefficients ] = fitting_new( Modified_Spectrum_Data,trajectory )
x_axis = trajectory;
fun = #(x,x_axis) (x(1)*exp((-x(2))*x_axis));
start = [Modified_Spectrum_Data(1) 0.05];
nlm = nlinfit(x_axis,Modified_Spectrum_Data,fun,start,opts);
Coefficients = nlm;
end
Data:
Modified_Spectrum_Data = [1.11111111111111, 1.08784976353957, 1.06352170731165, 1.04099672033640, 1.02649723285838, 1.00423806910703, 0.994116452961827, 0.975928861361604, 0.963081773802984, 0.953191520906905, 0.940636278551651, 0.930360007604054, 0.922259178548511, 0.916659345499171, 0.909149956799775, 0.901241601559703, 0.895375741449218, 0.893308346234150, 0.887985459843162, 0.884657500398024, 0.883852990694089, 0.877158499678129, 0.874817832833850, 0.875428444059047, 0.873170360623947, 0.871461252768665, 0.867913776631497, 0.866459074988087, 0.863819528471106, 0.863228815347816 ,0.864369045426273 ,0.860602502500599, 0.862653463581049, 0.861169231463016, 0.858658616425390, 0.864588421841755, 0.858668693409622, 0.857993365648639]
trajectory = [0.0043, 0.9996, 2.0007, 2.9994, 3.9996, 4.9994, 5.9981, 6.9978, 7.9997, 8.9992, 10.0007, 10.9993, 11.9994, 12.9992, 14.0001, 14.9968, 15.9972, 16.9996, 17.9996, 18.999, 19.9992, 20.9996, 21.9994, 23.0003, 23.9992, 24.999, 25.9987, 26.9986, 27.999, 28.9991, 29.999, 30.9987, 31.9976, 32.9979, 33.9983, 34.9988, 35.999, 36.9991]
I've tried using multiple different fitting functions and messing around with the options, but they don't seem to make too big of a difference. Additionally, I've tried changing the initial guess, but again that doesn't really make a difference.
Excel seems to be able to fit the data perfectly fine, but I have 900 rows of data I want to fit so doing it in Excel is not possible.
Any help would be greatly appreciated, thank you.
You'll want to use the cftool. Your data looks to follow a power law. Then choose 'Modified Spectrum Data' as your x axis and 'Trajectory' as your y. Select 'Power' from the drop down menu towards the top of the GUI.
Modified_Spectrum_Data = [1.11111111111111, 1.08784976353957, 1.06352170731165, 1.04099672033640, 1.02649723285838, 1.00423806910703, 0.994116452961827, 0.975928861361604, 0.963081773802984, 0.953191520906905, 0.940636278551651, 0.930360007604054, 0.922259178548511, 0.916659345499171, 0.909149956799775, 0.901241601559703, 0.895375741449218, 0.893308346234150, 0.887985459843162, 0.884657500398024, 0.883852990694089, 0.877158499678129, 0.874817832833850, 0.875428444059047, 0.873170360623947, 0.871461252768665, 0.867913776631497, 0.866459074988087, 0.863819528471106, 0.863228815347816 ,0.864369045426273 ,0.860602502500599, 0.862653463581049, 0.861169231463016, 0.858658616425390, 0.864588421841755, 0.858668693409622, 0.857993365648639]
trajectory = [0.0043, 0.9996, 2.0007, 2.9994, 3.9996, 4.9994, 5.9981, 6.9978, 7.9997, 8.9992, 10.0007, 10.9993, 11.9994, 12.9992, 14.0001, 14.9968, 15.9972, 16.9996, 17.9996, 18.999, 19.9992, 20.9996, 21.9994, 23.0003, 23.9992, 24.999, 25.9987, 26.9986, 27.999, 28.9991, 29.999, 30.9987, 31.9976, 32.9979, 33.9983, 34.9988, 35.999, 36.9991]
cftool
Screenshot:
For more information on the curve fitting (cftool), see: https://www.mathworks.com/help/curvefit/curvefitting-app.html
Related
I am new to Julia and trying to use the Julia package DifferentialEquations to simultaneously solve for several conditions of the same set of coupled ODEs. My system is a model of an experiment and in one of the conditions, I increase the amount of one of the dependent variables at mid-way through the process.
I would like to be able to adjust the condition of this single trajectory, however so far I am only able to adjust all the trajectories at once. Is it possible to access a single one using callbacks? If not, is there a better way to do this?
Here is a simplified example using the lorentz equations for what I want to be doing:
#Differential Equations setup
function lorentz!(du,u,p,t)
a,r,b=p
du[1]= a*(u[2]-u[1])
du[2]=u[1]*(r-u[3])-u[2]
du[3]=u[1]*u[2]-b*u[3];
end
#function to cycle through inital conditions
function prob_func(prob,i,repeat)
remake(prob; u0 = u0_arr[i]);
end
#inputs
t_span=[(0.0,100.0),(0.0,100.0)];
u01=[0.0;1.0;0.0];
u02=[0.0;1.0;0.0];
u0_arr = [u01,u02];
p=[10.,28.,8/3];
#initialising the Ensemble Problem
prob = ODEProblem(lorentz!,u0_arr[1],t_span[1],p);
CombinedProblem = EnsembleProblem(prob,
prob_func = prob_func, #-> (prob),#repeat is a count for how many times the trajectories had been repeated
safetycopy = true # determines whether a safetly deepcopy is called on the prob before the prob_func (sounds best to leave as true for user-given prob_func)
);
#introducing callback
function condition(u,t,repeat)
return 50 .-t
end
function affect!(repeat)
repeat.u[1]=repeat.u[1] +50
end
callback = DifferentialEquations.ContinuousCallback(condition, affect!)
#solving
sim=solve(CombinedProblem,Rosenbrock23(),EnsembleSerial(),trajectories=2,callback=callback);
# Plotting for ease of understanding example
plot(sim[1].t,sim[1][1,:])
plot!(sim[2].t,sim[2][1,:])
I want to produce something like this:
Example_desired_outcome
But this code produces:
Example_current_outcome
Thank you for your help!
You can make that callback dependent on a parameter and make the parameter different between problems. For example:
function f(du,u,p,t)
if p == 0
du[1] = 2u[1]
else
du[1] = -2u[1]
end
du[2] = -u[2]
end
condition(t,u,integrator) = u[2] - 0.5
affect!(integrator) = integrator.prob.p = 1
For more information, check out the FAQ on this topic: https://diffeq.sciml.ai/stable/basics/faq/#Switching-ODE-functions-in-the-middle-of-integration
I'm testing the y = SinC(x) function with single hidden layer feedforward neural networks (SLFNs) with 20 neurons.
With a SLFN, in the output layer, the output weight(OW) can be described by
OW = pinv(H)*T
after adding regularized parameter gamma, which
OW = pinv(I/gamma+H'*H)*H'*T
with
gamma -> Inf, pinv(H'*H)*H'*T == pinv(H)*T, also pinv(H'*H)*H' == pinv(H).
But when I try to calculate pinv(H'*H)*H' and pinv(H), I find a huge difference between these two when neurons number is over 5 (under 5, they are equal or almost the same).
For example, when H is 10*10 matrix, cond(H) = 21137561386980.3, rank(H) = 10,
H = [0.736251410036783 0.499731137079796 0.450233920602169 0.296610970576716 0.369359425954153 0.505556211442208 0.502934880027889 0.364904559142718 0.253349959726753 0.298697900877265;
0.724064281864009 0.521667364351399 0.435944895257239 0.337878535128756 0.364906002569385 0.496504064726699 0.492798607017131 0.390656915261343 0.289981152837390 0.307212326718916;
0.711534656474153 0.543520341487420 0.421761457948049 0.381771374416867 0.360475582262355 0.487454209236671 0.482668250979627 0.417033287703137 0.329570921359082 0.315860145366824;
0.698672860220896 0.565207057974387 0.407705930918082 0.427683127210120 0.356068794706095 0.478412571446765 0.472552121296395 0.443893207685379 0.371735862991355 0.324637323886021;
0.685491077062637 0.586647027111176 0.393799811411985 0.474875155650945 0.351686254239637 0.469385056318048 0.462458480695760 0.471085139463084 0.415948455902421 0.333539494486324;
0.672003357663056 0.607763454504209 0.380063647372632 0.522520267708374 0.347328559602877 0.460377531907542 0.452395518357816 0.498449772544129 0.461556360076788 0.342561958147251;
0.658225608290477 0.628484290731116 0.366516925684188 0.569759064961507 0.342996293691614 0.451395814182317 0.442371323528726 0.525823695636816 0.507817005881821 0.351699689941632;
0.644175558300583 0.648743139215935 0.353177974096445 0.615761051907079 0.338690023332811 0.442445652121229 0.432393859824045 0.553043275759248 0.553944175102542 0.360947346089454;
0.629872705346690 0.668479997764613 0.340063877672496 0.659781468051379 0.334410299080102 0.433532713184646 0.422470940392161 0.579948548513999 0.599160649563718 0.370299272759337;
0.615338237874436 0.687641820315375 0.327190410302607 0.701205860709835 0.330157655029498 0.424662569229062 0.412610204098877 0.606386924575225 0.642749594844498 0.379749516620049];
T=[-0.806458764562879 -0.251682808380338 -0.834815868451399 -0.750626822371170 0.877733363571576 1 -0.626938984683970 -0.767558933097629 -0.921811074815239 -1]';
There is a huge difference between pinv(H'*H)*H*T and pinv(H)*T, where
pinv(H'*H)*H*T = [-4803.39093243484 3567.08623820149 668.037919243849 5975.10699147077
1709.31211566970 -1328.53407325092 -1844.57938928594 -22511.9388736373
-2377.63048959478 31688.5125271114]';
pinv(H)*T = [-19780274164.6438 -3619388884.32672 -76363206688.3469 16455234.9229156
-135982025652.153 -93890161354.8417 283696409214.039 193801203.735488
-18829106.6110445 19064848675.0189]'.
I also find that if I round H , round(H,2), pinv(H'*H)*H*T and pinv(H)*T return the same answer. So I guess one of the reason might be the float calculation issue inside the matlab.
But since cond(H) is large, any small change of H may result in large difference in the inverse of H. I think the round function may not be a good option to test. As Cris Luengo mentioned, with large cond,the numerical imprecision will affect the accuracy of inverse.
In my test, I use 1000 training samples Input:[-10,10], with noise between [-0.2,0.2], and test samples are noise free. 20 neurons are selected. The OW = pinv(H)*Tcan give reasonable results for SinC training, while the performance for OW = pinv(H'*H)*T is worse. Then I try to increase the precision of H'*H by pinv(vpa(H'*H)), there's no significant improvement.
Does anyone know how to solve this?
After some research, the answer is that ELM is very sentive to scaling and activation function.
Please refer to this paper for details: https://dl.acm.org/citation.cfm?id=2797143.2797161
And paper: https://ieeexplore.ieee.org/document/8533625 demonstrated a noval algorithm to improve the perforamance of ELM for scaling.
My program results in text (notepad) file, I want to plot FAR Vs FRR and EER Vs Threshold. Following is the result.
FAR, FRR, Accuracy, EER, Threshold
21.3248 46.6667 78.0417 21.9583 0.5467
23.5897 41.6667 75.9583 24.0417 0.5007
25.7265 40.8333 73.8958 26.1042 0.5168
28.8889 50.8333 70.5625 29.4375 0.5591
26.9658 43.3333 72.6250 27.3750 0.3973
17.0085 50.8333 82.1458 17.8542 0.4310
22.9274 43.3333 76.5625 23.4375 0.3339
16.0470 46.6667 83.1875 16.8125 0.4013
16.4530 43.3333 82.8750 17.1250 0.5091
18.8462 41.6667 80.5833 19.4167 0.5055
First, you need to load your data using:
load data.txt
result = data;
After that, you plot your ROC by referring to the following code:
plotroc(targets,outputs)
plotroc(targets1,outputs2,'name1',...)
you may see here: https://www.mathworks.com/help/deeplearning/ref/plotroc.html
I have the data of electricity consumption of a region during the year of 2017. So I have to matrix 1x1, one with the month and other with the consumption. I want to use the command forecast to forecast the consumption of the first month of 2018, but I don't know how to do this even after reading the examples on MATLAB's help page.
Example:
data = {1166974.25000000, 1132479.36000000, 1137173.86000000, 1145853.58000000, 1118875.72000000, 1071456.85000000 ,1047171.87000000, 1071179.65000000 ,1077986.32000000 ,1112111.10000000, 1149668.47000000 ,1161649.19000000, 1175576.25000000 ,1126753.31000000 ,1204843.11000000 ,1183946.03000000, 1153080.36000000, 1120182.07000000, 1104726.03000000 ,1108110.02000000 ,1137729.28000000 ,1189699.45000000, 1252975.55000000, 1218118.20000000 ,1259580 ,1208193 ,1194430, 1244458, 1218867, 1205705 ,1177362, 1185584, 1164758, 1226991 ,1286044 ,1305312, 1360681.70000000 ,1332020 ,1306497.90000000 ,1299819.10000000 ,1316167.70000000 ,1246959.40000000 ,1256700.20000000 ,1266490.60000000, 1275642.90000000, 1358839.80000000, 1361440.10000000, 1398059.40000000};
data = [data{:}];
sys = ar(data,4)
K = 49;
p = forecast(sys,data,K);
plot(data,'b',p,'r'), legend('measured','forecasted')
Why does this not work?
I hope you found a solution to your problem. If you have not, maybe I can be of assistance.
MathWork's documentation of the function notes that the "PastData" entry (labeled "data" in your code) can either be an iddata object or an N x N_y matrix of doubles. Your implementation uses a matrix, so I decided to try out the code with an iddata object.
rawdat = [1166974.25000000, 1132479.36000000, 1137173.86000000, 1145853.58000000, 1118875.72000000, 1071456.85000000 ,1047171.87000000, 1071179.65000000 ,1077986.32000000 ,1112111.10000000, 1149668.47000000 ,1161649.19000000, 1175576.25000000 ,1126753.31000000 ,1204843.11000000 ,1183946.03000000, 1153080.36000000, 1120182.07000000, 1104726.03000000 ,1108110.02000000 ,1137729.28000000 ,1189699.45000000, 1252975.55000000, 1218118.20000000 ,1259580 ,1208193 ,1194430, 1244458, 1218867, 1205705 ,1177362, 1185584, 1164758, 1226991 ,1286044 ,1305312, 1360681.70000000 ,1332020 ,1306497.90000000 ,1299819.10000000 ,1316167.70000000 ,1246959.40000000 ,1256700.20000000 ,1266490.60000000, 1275642.90000000, 1358839.80000000, 1361440.10000000, 1398059.40000000];
data = iddata(rawdat',[]);
sys = ar(data,4);
K = 49;
p = forecast(sys,data,K);
plot(data,'b',p,'r'), legend('measured','forecasted')
Notice that I also changed the initial data's variable name and type.
The above code leads to the following figure.
Please update us. Thanks.
This is an extension of my previous question: https://dsp.stackexchange.com/questions/28095/choosing-low-pass-filter-parameters
I am recording people from an overheard camera. I have tracks of each's head using some software. I want to periodicity from tracks due to head wobbling.
I apply low-pass butterworth filter. I want the starting point and ending point of the filtered to be same as unfiltered tracks.
Data:
K>> [xcor_i,ycor_i ]
ans =
-101.7000 -77.4040
-102.4200 -77.4040
-103.6600 -77.4040
-103.9300 -76.6720
-103.9900 -76.5130
-104.0000 -76.4780
-105.0800 -76.4710
-106.0400 -77.5660
-106.2500 -77.8050
-106.2900 -77.8570
-106.3000 -77.8680
-106.3000 -77.8710
-107.7500 -78.9680
-108.0600 -79.2070
-108.1200 -79.2590
-109.9500 -80.3680
-111.4200 -80.6090
-112.8200 -81.7590
-113.8500 -82.3750
-115.1500 -83.2410
-116.1500 -83.4290
-116.3700 -83.8360
-117.5000 -84.2910
-117.7400 -84.3890
-118.8800 -84.7770
-119.8400 -85.2270
-121.1400 -85.3250
-123.2200 -84.9800
-125.4700 -85.2710
-127.0400 -85.7000
-128.8200 -85.7930
-130.6500 -85.8130
-132.4900 -85.8180
-134.3300 -86.5500
-136.1700 -87.0760
-137.6500 -86.0920
-138.6900 -86.9760
-140.3600 -87.9000
-142.1600 -88.4660
-144.7200 -89.3210
Code(answer by #SleuthEye):
dataOut_x = xcor_i(1)+filter(b,a,xcor_i-xcor_i(1));
dataOut_y = ycor_i(1)+filter(b,a,ycor_i-ycor_i(1));
Output:
In the above example, the endpoint(to the left) is different for filtered and unfiltered tracks. How can I ensure it is same?
Your question is pretty ambiguous, and doesn't really have a specific question. I'm assuming you want to have your filtered data start at the same points as the measured data, but are unsure why this is not happening already, and how to do so.
A low pass filter is a filter which lowers the effect of rapid changes. One way of doing this, and the method which appears to be used here, is by using a rolling average. A rolling average is simply an average (mean) of the previous data points. It looks like you are using a rolling average of 5 data points. Therefore you need five points of raw data before your filter will give you a single data point.
-101.7000 -77.4040 }
-102.4200 -77.4040 } }
-103.6600 -77.4040 } }
-103.9300 -76.6720 } }
-103.9900 -76.5130 } Filter point 1. }
-104.0000 -76.4780 } Filter point 2.
-105.0800 -76.4710
-106.0400 -77.5660
-106.2500 -77.8050
-106.2900 -77.8570
-106.3000 -77.8680
-106.3000 -77.8710
In order to solve this problem, you could just append the first data point to the data set four times, as this means that the filter will produce the same number of points. This is a pretty rough solution, however, as you are creating new data. This could be achieved quite simply, for example if your dataset is called myArray:
firstEntry = myArray(1,:);
myNewArray = [firstEntry; firstEntry; firstEntry; firstEntry; myArray];
This will create four data points equal to your first data point, which should then allow you to apply the low pass filter to your data, and have it start at the same point.
Hope this helps, although it's worth bearing in mind that filtering ALWAYS results in a loss of data.
Because you don't want to implement it but want someone else to:
The theory as above is correct, but instead you need to add 2 values at the end of your vectors:
x_last = xcor_i(end);
y_last = ycor_i(end);
xcor_i = [xcor_i;x_last;x_last];
ycor_i = [ycor_i;y_last;y_last];
This gives the following:
As you can see the ends are pretty close to being the same now.