Unclear documentation for ANN parameters - matlab

I'm dealing with some problem about reproducing results of ANN. Namely in the parameters of ANN there are 2 that I cannot address:
net. Inputs{i}.range
net.output.range
The documentation is very lapidary and cannot help me to understand how it works. Both seem to have massive impact on the output. Let's consider this MWE:
net=feedforwardnet(10);
net.inputs{1}.size=3;
%net.inputs{1}.range=[0 100;0 100;0 100];
%net.output.range=[1 200;];
net.layers{2}.size=1;
L1=[-1.1014, -2.1138, -2.6975;
-2.3545, 0.7693, 1.7621;
-1.1258, -1.4171, -3.1113;
-0.7845, -3.7105, 0.1605;
0.3993, 0.7042, 3.5076;
0.283, -3.914, -1.3428;
-2.0566, -3.4762, 1.3239;
-1.0626, 0.3662, 2.9169;
0.1367, 2.5801, 2.5867;
0.7155, 2.6237, 2.5376;];
B1=[3.5997, 3.1386, 2.7002, 1.8243, -1.9267, -1.6754, 0.8252, 1.0865, -0.0005, 0.6126];
L2=[0.5005, -1.0932, 0.34, -1.5099, 0.5896, 0.5881, 0.4769, 0.6728, -0.9407, -1.0296];
B2=0.1567;
net.IW{1}=L1;
net.Lw{2,1}=L2;
net.b{1}=B1';
net.b{2}=B2;
input=[40; 30; 20];
output=net(input)
If you uncomment line 3 and 4 the result rises from 0.1464 to 119.1379. I'm trying to reproduce this aspect of Matlab ANN in another environment, but the documentation is too short and does not explain anything.
What do these two parameters exactly do? I mean, what exact function is applied on the input and output data?

Related

pinv(H) is not equal to pinv(H'*H)*H'

I'm testing the y = SinC(x) function with single hidden layer feedforward neural networks (SLFNs) with 20 neurons.
With a SLFN, in the output layer, the output weight(OW) can be described by
OW = pinv(H)*T
after adding regularized parameter gamma, which
OW = pinv(I/gamma+H'*H)*H'*T
with
gamma -> Inf, pinv(H'*H)*H'*T == pinv(H)*T, also pinv(H'*H)*H' == pinv(H).
But when I try to calculate pinv(H'*H)*H' and pinv(H), I find a huge difference between these two when neurons number is over 5 (under 5, they are equal or almost the same).
For example, when H is 10*10 matrix, cond(H) = 21137561386980.3, rank(H) = 10,
H = [0.736251410036783 0.499731137079796 0.450233920602169 0.296610970576716 0.369359425954153 0.505556211442208 0.502934880027889 0.364904559142718 0.253349959726753 0.298697900877265;
0.724064281864009 0.521667364351399 0.435944895257239 0.337878535128756 0.364906002569385 0.496504064726699 0.492798607017131 0.390656915261343 0.289981152837390 0.307212326718916;
0.711534656474153 0.543520341487420 0.421761457948049 0.381771374416867 0.360475582262355 0.487454209236671 0.482668250979627 0.417033287703137 0.329570921359082 0.315860145366824;
0.698672860220896 0.565207057974387 0.407705930918082 0.427683127210120 0.356068794706095 0.478412571446765 0.472552121296395 0.443893207685379 0.371735862991355 0.324637323886021;
0.685491077062637 0.586647027111176 0.393799811411985 0.474875155650945 0.351686254239637 0.469385056318048 0.462458480695760 0.471085139463084 0.415948455902421 0.333539494486324;
0.672003357663056 0.607763454504209 0.380063647372632 0.522520267708374 0.347328559602877 0.460377531907542 0.452395518357816 0.498449772544129 0.461556360076788 0.342561958147251;
0.658225608290477 0.628484290731116 0.366516925684188 0.569759064961507 0.342996293691614 0.451395814182317 0.442371323528726 0.525823695636816 0.507817005881821 0.351699689941632;
0.644175558300583 0.648743139215935 0.353177974096445 0.615761051907079 0.338690023332811 0.442445652121229 0.432393859824045 0.553043275759248 0.553944175102542 0.360947346089454;
0.629872705346690 0.668479997764613 0.340063877672496 0.659781468051379 0.334410299080102 0.433532713184646 0.422470940392161 0.579948548513999 0.599160649563718 0.370299272759337;
0.615338237874436 0.687641820315375 0.327190410302607 0.701205860709835 0.330157655029498 0.424662569229062 0.412610204098877 0.606386924575225 0.642749594844498 0.379749516620049];
T=[-0.806458764562879 -0.251682808380338 -0.834815868451399 -0.750626822371170 0.877733363571576 1 -0.626938984683970 -0.767558933097629 -0.921811074815239 -1]';
There is a huge difference between pinv(H'*H)*H*T and pinv(H)*T, where
pinv(H'*H)*H*T = [-4803.39093243484 3567.08623820149 668.037919243849 5975.10699147077
1709.31211566970 -1328.53407325092 -1844.57938928594 -22511.9388736373
-2377.63048959478 31688.5125271114]';
pinv(H)*T = [-19780274164.6438 -3619388884.32672 -76363206688.3469 16455234.9229156
-135982025652.153 -93890161354.8417 283696409214.039 193801203.735488
-18829106.6110445 19064848675.0189]'.
I also find that if I round H , round(H,2), pinv(H'*H)*H*T and pinv(H)*T return the same answer. So I guess one of the reason might be the float calculation issue inside the matlab.
But since cond(H) is large, any small change of H may result in large difference in the inverse of H. I think the round function may not be a good option to test. As Cris Luengo mentioned, with large cond,the numerical imprecision will affect the accuracy of inverse.
In my test, I use 1000 training samples Input:[-10,10], with noise between [-0.2,0.2], and test samples are noise free. 20 neurons are selected. The OW = pinv(H)*Tcan give reasonable results for SinC training, while the performance for OW = pinv(H'*H)*T is worse. Then I try to increase the precision of H'*H by pinv(vpa(H'*H)), there's no significant improvement.
Does anyone know how to solve this?
After some research, the answer is that ELM is very sentive to scaling and activation function.
Please refer to this paper for details: https://dl.acm.org/citation.cfm?id=2797143.2797161
And paper: https://ieeexplore.ieee.org/document/8533625 demonstrated a noval algorithm to improve the perforamance of ELM for scaling.

Issues fitting an exponential function

I'm having some serious issues fitting an exponential function (Beer-Lambert law) to my data. The optimization toolset function that I'm using produces terrible fits:
function [ Coefficients ] = fitting_new( Modified_Spectrum_Data,trajectory )
x_axis = trajectory;
fun = #(x,x_axis) (x(1)*exp((-x(2))*x_axis));
start = [Modified_Spectrum_Data(1) 0.05];
nlm = nlinfit(x_axis,Modified_Spectrum_Data,fun,start,opts);
Coefficients = nlm;
end
Data:
Modified_Spectrum_Data = [1.11111111111111, 1.08784976353957, 1.06352170731165, 1.04099672033640, 1.02649723285838, 1.00423806910703, 0.994116452961827, 0.975928861361604, 0.963081773802984, 0.953191520906905, 0.940636278551651, 0.930360007604054, 0.922259178548511, 0.916659345499171, 0.909149956799775, 0.901241601559703, 0.895375741449218, 0.893308346234150, 0.887985459843162, 0.884657500398024, 0.883852990694089, 0.877158499678129, 0.874817832833850, 0.875428444059047, 0.873170360623947, 0.871461252768665, 0.867913776631497, 0.866459074988087, 0.863819528471106, 0.863228815347816 ,0.864369045426273 ,0.860602502500599, 0.862653463581049, 0.861169231463016, 0.858658616425390, 0.864588421841755, 0.858668693409622, 0.857993365648639]
trajectory = [0.0043, 0.9996, 2.0007, 2.9994, 3.9996, 4.9994, 5.9981, 6.9978, 7.9997, 8.9992, 10.0007, 10.9993, 11.9994, 12.9992, 14.0001, 14.9968, 15.9972, 16.9996, 17.9996, 18.999, 19.9992, 20.9996, 21.9994, 23.0003, 23.9992, 24.999, 25.9987, 26.9986, 27.999, 28.9991, 29.999, 30.9987, 31.9976, 32.9979, 33.9983, 34.9988, 35.999, 36.9991]
I've tried using multiple different fitting functions and messing around with the options, but they don't seem to make too big of a difference. Additionally, I've tried changing the initial guess, but again that doesn't really make a difference.
Excel seems to be able to fit the data perfectly fine, but I have 900 rows of data I want to fit so doing it in Excel is not possible.
Any help would be greatly appreciated, thank you.
You'll want to use the cftool. Your data looks to follow a power law. Then choose 'Modified Spectrum Data' as your x axis and 'Trajectory' as your y. Select 'Power' from the drop down menu towards the top of the GUI.
Modified_Spectrum_Data = [1.11111111111111, 1.08784976353957, 1.06352170731165, 1.04099672033640, 1.02649723285838, 1.00423806910703, 0.994116452961827, 0.975928861361604, 0.963081773802984, 0.953191520906905, 0.940636278551651, 0.930360007604054, 0.922259178548511, 0.916659345499171, 0.909149956799775, 0.901241601559703, 0.895375741449218, 0.893308346234150, 0.887985459843162, 0.884657500398024, 0.883852990694089, 0.877158499678129, 0.874817832833850, 0.875428444059047, 0.873170360623947, 0.871461252768665, 0.867913776631497, 0.866459074988087, 0.863819528471106, 0.863228815347816 ,0.864369045426273 ,0.860602502500599, 0.862653463581049, 0.861169231463016, 0.858658616425390, 0.864588421841755, 0.858668693409622, 0.857993365648639]
trajectory = [0.0043, 0.9996, 2.0007, 2.9994, 3.9996, 4.9994, 5.9981, 6.9978, 7.9997, 8.9992, 10.0007, 10.9993, 11.9994, 12.9992, 14.0001, 14.9968, 15.9972, 16.9996, 17.9996, 18.999, 19.9992, 20.9996, 21.9994, 23.0003, 23.9992, 24.999, 25.9987, 26.9986, 27.999, 28.9991, 29.999, 30.9987, 31.9976, 32.9979, 33.9983, 34.9988, 35.999, 36.9991]
cftool
Screenshot:
For more information on the curve fitting (cftool), see: https://www.mathworks.com/help/curvefit/curvefitting-app.html

Matlab: efficienting portion of code, random start

I have the following code that generates a matrix of 15 blocks that will then be used in a Montecarlo approach as multiple starting points. How can I get the same exact result in a smarter way?
assume that J=15*100 are the total simulation and paramNum the number of parameters
[10^-10*ones(paramNum,round(J/15)) 10^-9*ones(paramNum,round(J/15)) 10^-8*ones(paramNum,round(J/15)) 10^-7*ones(paramNum,round(J/15)) 10^-6*ones(paramNum,round(J/15)) 10^-5*ones(paramNum,round(J/15)) rand*10^-5*ones(paramNum,round(J/15)) 10^-4*ones(paramNum,round(J/15)) rand*10^-4*ones(paramNum,round(J/15)) 10^-3*ones(paramNum,round(J/15)) 10^-2*ones(paramNum,round(J/15)) 10^-1*ones(paramNum,round(J/15)) 10^-abs(randn/2)*ones(paramNum,round(J/15))];
you could do
v = 10.^[-10:-5 rand*10^-5 -4:-1 10^-abs(randn/2)];
repmat(repelem(v, 1, round(J/15)), paramNum) .* ...
repmat(ones(paramNum,round(J/15)), numel(v))
Or mimic the repmat/repelem functionality with a for loop. The first is shorter, the later is more understandable.
By the way... it's less than 15 blocks...

DST transform in Matlab Vs FFTW3

I am porting a Matlab/Octave code to C. One of the implementations I need to port is the DST transform.
In Matlab/Octave, this is done using
A =[0.00000 4.24264 5.65685 7.07107 8.48528 0.00000]
res = dst(A)
ans =
22.3599 -4.7499 -4.4287 2.9465 -6.8864 1.9383
In C, I have done what the FFTW3 documentation says:
n = 6;
fftw_plan plan = fftw_plan_r2r_1d( n, in, out, FFTW_RODFT10, FFTW_ESTIMATE);
fftw_execute ( plan );
where in is the input array. The result of this is, out:
42.588457
-9.899495
0.000000
2.449490
-11.411543
5.656854
Why are those results so different? Any ideas of what I am doing wrong here, if any?
If you are looking for exactly the same tranform, you may try the RODFT00 flag instead.
According to docs of fftw, matlab, octave, it seems to fit, except for a 2 factor. Output of fftw may be twice the one of matlab or octave.
http://www.mathworks.fr/fr/help/pde/ug/dst.html
http://octave.sourceforge.net/signal/function/dst.html
http://www.fftw.org/doc/1d-Real_002dodd-DFTs-_0028DSTs_0029.html
Bye,
tried this x=[ 0
-0.041389004581424
-0.049979175343607
-0.057007496876302
-0.062473969179509
-0.066378592253228
-0.068721366097459
-0.069502290712203
-0.068721366097459
-0.066378592253228
-0.062473969179509
-0.057007496876302
-0.049979175343607
-0.041389004581424
0.500000000000000
-0.500000000000000]
matlab's dst gives :
dst(x)=[-0.524649937006448
-0.208549082037091
0.175438242940290
-0.201365706563574
0.122632326416342
0.023410695189363
-0.170916792532103
0.404635063593086
-0.557336293807673
0.766827077578399
-0.834391996423464
0.916008576298954
-0.836787527646708
0.743037852780101
-0.522660424688284
0.285317021943821]
using FFTW with "FFTW_RODFT00" flag
gives
[-1.0493
-0.417098
0.350876
-0.402731
0.245265
0.0468214
-0.341834
0.80927
-1.11467
1.53365
-1.66878
1.83202
-1.67358
1.48608
-1.04532
0.570634]
Basically a factor of two is the difference, even if one uses same flag

Performing Kernel Density Estimations in MATLAB

I have been using MATLAB to perform Kernel Density Estimations (KDE) on UTM data (X and Y coordinates). I ran into a problem that I do not seem to be understanding.
I perform the KDEs with a sample of 45 points. Everything works fine and I produce the graphs with contours.
[bandwidth,density,X,Y]=kde2d(data)
The function kde2d is code by Zdravko Botev. I obtained it from his file exchange on MathWorks. The variable 'data' is a 45x2 array of my data. The first column holds the X coordinates and the second the Y.
The problem comes when I try to do the same line of code on a subset of those 45 points. I get a recurring error:
Error using fzero (line 274)
The function values at the interval endpoints must differ in sign.
Error in kde2d (line 101)
t_star=fzero(#(t)(t-evolve(t)),[0,0.1]);
I get the same error for a bunch of those subsets on a bunch of different sets of 45 points.
The complete set has these 45 values:
1594436.281 572258.1272
1594418.48 572357.5859
1594471.362 572385.5186
1594516.726 572266.8206
1594415.313 572369.2754
1594519.701 572272.7153
1594415.377 572363.4139
1594468.365 572381.5779
1594518.139 572276.6059
1594425.496 572271.6874
1594524.259 572272.7651
1594502.555 572172.8749
1594516.747 572264.867
1594485.314 572360.2689
1594476.027 572375.7997
1594556.087 572419.6609
1594522.718 572274.7021
1594472.775 572395.3039
1594554.568 572419.6443
1594527.255 572276.7054
1594474.315 572393.3669
1594522.697 572276.6557
1594471.319 572389.4262
1594460.854 572373.6799
1594546.022 572228.0609
1594460.79 572379.5414
1594468.323 572385.4855
1594466.953 572371.7926
1594519.722 572270.7614
1594396.76 572398.3826
1594468.131 572403.0693
1594418.288 572375.1697
1594396.377 572433.5499
1594448.287 572271.9361
1594510.541 572276.523
1594424.466 572226.7345
1594413.773 572371.2124
1594511.848 572296.0774
1594513.367 572296.094
1594424.488 572224.7805
1594468.152 572401.1153
1594421.37 572371.2953
1594446.768 572271.9195
1594468.152 572401.1153
1594448.799 572225.0457
One of the subsets I am trying to use is this:
1594436.281 572258.1272
1594418.48 572357.5859
1594471.362 572385.5186
1594516.726 572266.8206
1594415.313 572369.2754
1594519.701 572272.7153
1594415.377 572363.4139
1594468.365 572381.5779
1594518.139 572276.6059
1594425.496 572271.6874
I am not sure if I should include any of Botev's code. I am hoping that the error message can be explained on its own. If not I can provide more. Thank you very much.