Can I export my (Matlab-based) neural network to PHP? - matlab

I have trained a neural network in Matlab (Using the neural network toolbox). Now I would like to export the calculated weights and biases to another platform (PHP) in order to make calculations with them. Is there a way to create a function or equation to do this?
I found this related question: Equation that compute a Neural Network in Matlab.
Is there a way to do what I want and port the results of my NN (29 inputs, 10 hidden layers, 1 output) to PHP?

Yes, the net properties also referenced in the other question are simple matrices:
W1=net.IW{1,1};
W2=net.LW{2,1};
b1=net.b{1,1};
b2=net.b{2,1};
So you can write them to a file, say, as comma-separated-values.
csvwrite('W1.csv',W1)
Then, in PHP read this data and convert or use it as you like.
<?php
if (($handle = fopen("test.csv", "r")) !== FALSE) {
$data = fgetcsv($handle, 1000, ",");
}
?>
Than, to process the weights, you can use the formula from the other question by replacing the tansig function, which is calculated according to:
n = 2/(1+exp(-2*n))-1
This is mathematically equivalent to tanh(N)
Which exists in php as well.
source: http://dali.feld.cvut.cz/ucebna/matlab/toolbox/nnet/tansig.html

Transferring all of these is pretty trivial. You will need:
Write the code for matrix multiplication, which are a pretty simple couple of for loops.
Second, observe that according to the Matlab documentation tansig(n) = 2/(1+exp(-2*n))-1. I'm pretty sure that PHP has exp (and if not, it is has a pretty simple polynomial expansion which you can write yourself)
Read, understand and apply Jasper van den Bosch's excellent answer to your question.

Hence the solution becomes (after correcting all wrong parts)
Here I am giving a solution in Matlab, but if you have tanh() function, you may easily convert it to any programming language. For PHP, tanh() function exists: php tanh(). It is for just showing the fields from network object and the operations you need.
Assume you have a trained ann (network object) that you want to export
Assume that the name of the trained ann is trained_ann
Here is the script for exporting and testing.
Testing script compares original network result with my_ann_evaluation() result
% Export IT
exported_ann_structure = my_ann_exporter(trained_ann);
% Run and Compare
% Works only for single INPUT vector
% Please extend it to MATRIX version by yourself
input = [12 3 5 100];
res1 = trained_ann(input')';
res2 = my_ann_evaluation(exported_ann_structure, input')';
where you need the following two functions
First my_ann_exporter:
function [ my_ann_structure ] = my_ann_exporter(trained_netw)
% Just for extracting as Structure object
my_ann_structure.input_ymax = trained_netw.inputs{1}.processSettings{1}.ymax;
my_ann_structure.input_ymin = trained_netw.inputs{1}.processSettings{1}.ymin;
my_ann_structure.input_xmax = trained_netw.inputs{1}.processSettings{1}.xmax;
my_ann_structure.input_xmin = trained_netw.inputs{1}.processSettings{1}.xmin;
my_ann_structure.IW = trained_netw.IW{1};
my_ann_structure.b1 = trained_netw.b{1};
my_ann_structure.LW = trained_netw.LW{2};
my_ann_structure.b2 = trained_netw.b{2};
my_ann_structure.output_ymax = trained_netw.outputs{2}.processSettings{1}.ymax;
my_ann_structure.output_ymin = trained_netw.outputs{2}.processSettings{1}.ymin;
my_ann_structure.output_xmax = trained_netw.outputs{2}.processSettings{1}.xmax;
my_ann_structure.output_xmin = trained_netw.outputs{2}.processSettings{1}.xmin;
end
Second my_ann_evaluation:
function [ res ] = my_ann_evaluation(my_ann_structure, input)
% Works with only single INPUT vector
% Matrix version can be implemented
ymax = my_ann_structure.input_ymax;
ymin = my_ann_structure.input_ymin;
xmax = my_ann_structure.input_xmax;
xmin = my_ann_structure.input_xmin;
input_preprocessed = (ymax-ymin) * (input-xmin) ./ (xmax-xmin) + ymin;
% Pass it through the ANN matrix multiplication
y1 = tanh(my_ann_structure.IW * input_preprocessed + my_ann_structure.b1);
y2 = my_ann_structure.LW * y1 + my_ann_structure.b2;
ymax = my_ann_structure.output_ymax;
ymin = my_ann_structure.output_ymin;
xmax = my_ann_structure.output_xmax;
xmin = my_ann_structure.output_xmin;
res = (y2-ymin) .* (xmax-xmin) /(ymax-ymin) + xmin;
end

Related

How to interprete the regression plot obtained at the end of neural network regression for multiple outputs?

I have trained my Neural network model using MATLAB NN Toolbox. My network has multiple inputs and multiple outputs, 6 and 7 respectively, to be precise. I would like to clarify few questions based on it:-
The final regression plot showed at the end of the training shows a very good accuracy, R~0.99. However, since I have multiple outputs, I am confused as to which scatter plot does it represent? Shouldn't we have 7 target vs predicted plots for each of the output variable?
According to my knowledge, R^2 is a better method of commenting upon the accuracy of the model, whereas MATLAB reports R in its plot. Do I treat that R as R^2 or should I square the reported R value to obtain R^2.
I have generated the Matlab Script containing weight, bias and activation functions, as a final Result of the training. So shouldn't I be able to simply give my raw data as input and obtain the corresponding predicted output. I gave the exact same training set using the indices Matlab chose for training (to cross check), and plotted the predicted output vs actual output, but the result is not at all good. Definitely, not along the lines of R~0.99. Am I doing anything wrong?
code:
function [y1] = myNeuralNetworkFunction_2(x1)
%MYNEURALNETWORKFUNCTION neural network simulation function.
% X = [torque T_exh lambda t_Spark N EGR];
% Y = [O2R CO2R HC NOX CO lambda_out T_exh2];
% Generated by Neural Network Toolbox function genFunction, 17-Dec-2018 07:13:04.
%
% [y1] = myNeuralNetworkFunction(x1) takes these arguments:
% x = Qx6 matrix, input #1
% and returns:
% y = Qx7 matrix, output #1
% where Q is the number of samples.
%#ok<*RPMT0>
% ===== NEURAL NETWORK CONSTANTS =====
% Input 1
x1_step1_xoffset = [-24;235.248;0.75;-20.678;550;0.799];
x1_step1_gain = [0.00353982300884956;0.00284355877067267;6.26959247648903;0.0275865874012055;0.000366568914956012;0.0533831576137729];
x1_step1_ymin = -1;
% Layer 1
b1 = [1.3808996210168685;-2.0990163849711894;0.9651733083552595;0.27000953282929346;-1.6781835509820286;-1.5110463684800366;-3.6257438832309905;2.1569498669085361;1.9204156230460485;-0.17704342477904209];
IW1_1 = [-0.032892214008082517 -0.55848270745152429 -0.0063993424771670616 -0.56161004933654057 2.7161844536020197 0.46415317073346513;-0.21395624254052176 -3.1570133640176681 0.71972178875396853 -1.9132557838515238 1.3365248285282931 -3.022721627052706;-1.1026780445896862 0.2324603066452392 0.14552308208231421 0.79194435276493658 -0.66254679969168417 0.070353201192052434;-0.017994515838487352 -0.097682677816992206 0.68844109281256027 -0.001684535122025588 0.013605622123872989 0.05810686279306107;0.5853667840629273 -2.9560683084876329 0.56713425120259764 -2.1854386350040116 1.2930115031659106 -2.7133159265497957;0.64316656469750333 -0.63667017646313084 0.50060179040086761 -0.86827897068177973 2.695456517458648 0.16822164719859456;-0.44666821007466739 4.0993786464616679 -0.89370838440321498 3.0445073606237933 -3.3015566360833453 -4.492874075961689;1.8337574137485424 2.6946232855369989 1.1140472073136622 1.6167763205944321 1.8573696127039145 -0.81922672766933646;-0.12561950922781362 3.0711045035224349 -0.6535751823440773 2.0590707752473199 -1.3267693770634292 2.8782780742777794;-0.013438026967107483 -0.025741311825949621 0.45460734966889638 0.045052447491038108 -0.21794568374100454 0.10667240367191703];
% Layer 2
b2 = [-0.96846557414356171;-0.2454718918618051;-0.7331628718025488;-1.0225195290982099;0.50307202195645395;-0.49497234988401961;-0.21817117469133171];
LW2_1 = [-0.97716474643411022 -0.23883775971686808 0.99238069915206006 0.4147649511973347 0.48504023209224734 -0.071372217431684551 0.054177719330469304 -0.25963474838320832 0.27368380212104881 0.063159321947246799;-0.15570858147605909 -0.18816739764334323 -0.3793600124951475 2.3851961990944681 0.38355142531334563 -0.75308427071748985 -0.1280128732536128 -1.361052031781103 0.6021878865831336 -0.24725687748503239;0.076251356114485525 -0.10178293627600112 0.10151304376762409 -0.46453434441403058 0.12114876632815359 0.062856969143306296 -0.0019628163322658364 -0.067809039768745916 0.071731544062023825 0.65700427778446913;0.17887084584125315 0.29122649575978238 0.37255802759192702 1.3684190468992126 0.60936238465090853 0.21955911453674043 0.28477957899364675 -0.051456306721251184 0.6519451272106177 -0.64479205028051967;0.25743349663436799 2.0668075180209979 0.59610776847961111 -3.2609682919282603 1.8824214917530881 0.33542869933904396 0.03604272669356564 -0.013842766338427388 3.8534510207741826 2.2266745660915586;-0.16136175574939746 0.10407287099228898 -0.13902245286490234 0.87616472446622717 -0.027079111747601223 0.024812287505204988 -0.030101536834009103 0.043168268669541855 0.12172932035587079 -0.27074383434206573;0.18714562505165402 0.35267726325386606 -0.029241400610813449 0.53053853235049087 0.58880054832728757 0.047959541165126809 0.16152268183097709 0.23419456403348898 0.83166785128608967 -0.66765237856750781];
% Output 1
y1_step1_ymin = -1;
y1_step1_gain = [0.114200879346771;0.145581598485951;0.000139011547272197;0.000456244862967996;2.05816254143146e-05;5.27704485488127;0.00284355877067267];
y1_step1_xoffset = [-0.045;1.122;2.706;17.108;493.726;0.75;235.248];
% ===== SIMULATION ========
% Dimensions
Q = size(x1,1); % samples
% Input 1
x1 = x1';
xp1 = mapminmax_apply(x1,x1_step1_gain,x1_step1_xoffset,x1_step1_ymin);
% Layer 1
a1 = tansig_apply(repmat(b1,1,Q) + IW1_1*xp1);
% Layer 2
a2 = repmat(b2,1,Q) + LW2_1*a1;
% Output 1
y1 = mapminmax_reverse(a2,y1_step1_gain,y1_step1_xoffset,y1_step1_ymin);
y1 = y1';
end
% ===== MODULE FUNCTIONS ========
% Map Minimum and Maximum Input Processing Function
function y = mapminmax_apply(x,settings_gain,settings_xoffset,settings_ymin)
y = bsxfun(#minus,x,settings_xoffset);
y = bsxfun(#times,y,settings_gain);
y = bsxfun(#plus,y,settings_ymin);
end
% Sigmoid Symmetric Transfer Function
function a = tansig_apply(n)
a = 2 ./ (1 + exp(-2*n)) - 1;
end
% Map Minimum and Maximum Output Reverse-Processing Function
function x = mapminmax_reverse(y,settings_gain,settings_xoffset,settings_ymin)
x = bsxfun(#minus,y,settings_ymin);
x = bsxfun(#rdivide,x,settings_gain);
x = bsxfun(#plus,x,settings_xoffset);
end
The above one is the automatically generated code. The plot which I generated to cross-check the first variable is below:-
% X and Y are input and output - same as above
X_train = X(results.info1.train.indices,:);
y_train = Y(results.info1.train.indices,:);
out_train = myNeuralNetworkFunction_2(X_train);
scatter(y_train(:,1),out_train(:,1))
To answer your question about R: Yes, you should square R to get the R^2 value. In this case, they will be very close since R is very close to 1.
The graphs give the correlation between the estimated and real (target) values. So R is the strenght of the correlation. You can square it to find the R-square.
The graph you draw and matlab gave are not the graph of the same variables. The ranges or scales of the axes are very different.
First of all, is the problem you are trying to solve a regression problem? Or is it a classification problem with 7 classes converted to numeric? I assume this is a classification problem, as you are trying to get the success rate for each class.
As for your first question: According to the literature it is recommended to use the value "All: R". If you want to get the success rate of each of your classes, Precision, Recall, F-measure, FP rate, TP Rate, etc., which are valid in classification problems. values ​​you need to reach. There are many matlab documents for this (help ROC) and you can look at the details. All the values ​​I mentioned and which I think you actually want are obtained from the confusion matrix.
There is a good example of this.
[x,t] = simpleclass_dataset;
net = patternnet(10);
net = train(net,x,t);
y = net(x);
[c,cm,ind,per] = confusion(t,y)
I hope you will see what you want from the "nntraintool" window that appears when you run the code.
Your other questions have already been answered. Alternatively, you can consider using a machine learning algorithm with open source software such as Weka.

EEG data classification with SWLDA using matlab

I want to ask your help in EEG data classification.
I am a graduate student trying to analyze EEG data.
Now I am struggling with classifying ERP speller (P300) with SWLDA using Matlab
Maybe there is something wrong in my code.
I have read several articles, but they did not cover much details.
My data size is described as below.
size(target) = [300 1856]
size(nontarget) = [998 1856]
row indicates the number of trials, column indicates spanned feature
(I stretched data [64 29] (for visual representation I did not select ROI)
I used stepwisefit function in Matlab to classify target vs non-target
Code is attached below.
ingredients = [targets; nontargets];
heat = [class_targets; class_nontargets]; % target: 1, non-target: -1
randomized_set = shuffle([ingredients heat]);
for k=1:10 % 10-fold cross validation
parition_factor = ceil(size(randomized_set,1) / 10);
cv_test_idx = (k-1)*parition_factor + 1:min(k * parition_factor, size(randomized_set,1));
total_idx = 1:size(randomized_set,1);
cv_train_idx = total_idx(~ismember(total_idx, cv_test_idx));
ingredients = randomized_set(cv_train_idx, 1:end-1);
heat = randomized_set(cv_train_idx, end);
[W,SE,PVAL,INMODEL,STATS,NEXTSTEP,HISTORY]= stepwisefit(ingredients, heat, 'penter', .1);
valid_id = find(INMODEL==1);
v_weights = W(valid_id)';
t_ingredients = randomized_set(cv_test_idx, 1:end-1);
t_heat = randomized_set(cv_test_idx, end); % true labels for test set
v_features = t_ingredients(:, valid_id);
v_weights = repmat(v_weights, size(v_features, 1), 1);
predictor = sum(v_weights .* v_features, 2);
m_result = predictor > 0; % class A: +1, B: 0
t_heat(t_heat==-1) = 0;
acc(k) = sum(m_result==t_heat) / length(m_result);
end
p.s. my code is currently very inefficient and might be bad..
In my assumption, stepwisefit calculates significant coefficients every steps, and valid column would be remained.
Even though it's not LDA, but for binary classification, LDA and linear regression are not different.
However, results were almost random chance.. (for other binary data on the internet, it worked..)
I think I made something wrong, and your help can correct me.
I will appreciate any suggestion and tips to implement classifier for ERP speller.
Or any idea for implementing SWLDA in Matlab code?
The name SWLDA is only used in the context of Brain Computer Interfaces, but I bet it has another name in a more general context.
If you track the recipe of SWLDA you will end up in Krusienski 2006 papers ("A comparison..." and "Toward enhanced P300..") and from there the book where stepwise logarithmic regression is explained: "Draper Smith, Applied Regression Analysis, 1981". However, as far as I am aware of, no paper gives actually the complete recipe on how to implement it (and their details and secrets).
My approach was using stepwiseglm:
H=predictors;
TH=variables;
lbs=labels % (1,2)
if (stepwiseflag)
mdl = stepwiseglm(H', lbs'-1,'constant','upper','linear','distr','binomial');
if (mdl.NumEstimatedCoefficients>1)
inmodel = [];
for i=2:mdl.NumEstimatedCoefficients
inmodel = [inmodel str2num(mdl.CoefficientNames{i}(2:end))];
end
H = H(inmodel,:);
TH = TH(inmodel,:);
end
end
lbls = classify(TH',H',lbs','linear');
You can also use a k-fold cross validaton approach using matlab cvpartition.
c = cvpartition(lbs,'k',10);
opts = statset('display','iter');
fun = #(XT,yT,Xt,yt)...
(sum(~strcmp(yt,classify(Xt,XT,yT,'linear'))));

(Matlab) Solving equation with different input

I have a task of creating a matlab program, and I was searching for "switch" and "if" functions for the task.
And the user will define any two of the variables, then matlab will solve for the other one. Normally, I would use "if" to account for different scenarios, but the increasing number of variables in the equation would increase the number of lines as well.
Updated with a demonstration:
% Mach Number after Shockwaves
M2=sqrt((((gamma-1).*M1.^2)+2)./(2*gamma.*M1.^2-(gamma-1)));
% Temperature Ratio
TR=((2*gamma.*M1.^2-(gamma-1)).*(((gamma-1).*M1.^2)+2))./(((gamma+1)^2).*M1.^2);
% Pressure Ratio
PR=(2*gamma.*M1.^2-(gamma-1))./(gamma+1);
% Density Ratio
rhoR=((gamma+1).*M1.^2)./(((gamma-1).*M1.^2)+2);
%Stagnant Pressure Ratio Before Shockwaves
P0R=(((1+0.2.*M2.^2)./(1+0.2.*M1.^2)).^(1.4/0.1)).*((2*gamma.*M1.^2-(gamma-1))./(gamma+1));
%Stagnant Pressure Ratio After Shockwaves
P1R=((1+0.2.*M2.^2).^(1.4/0.4)).*((2*gamma.*M1.^2-(gamma-1))./(gamma+1));
Is there any alternative? Also, the matlab running in my campus has no symbolic toolbox, so it would best to avoid it. I am at my wit's end now, for I am sure there is a simple solution to account for such scenario.
Further Updated with the exact equation:
The program will be able to do like, let's say the user enter "TR" and "gamma", and then Matlab will find the "M1". This would further be carried into subsequent equations, where I would get "M2","PR","rhoR","P0R" and "P1R". Also, I realised that since Matlab will read from top to bottom, is there any way to account for this?
As we already wrote in the comments, what you try to do is very complicated and rather infeasible without the symbolic toolbox.
I have written a hacky solution which only accounts for the case:
2 out of the 3 variables {TR, gamma, M1} are given, the third one is then automatically calculated. These 3 variables can then be used to solve the remaining equations.
This solution assumes that you have at least once access to the symbolic toolbox, but you won't need it when you use generated code.
We first generate MATLAB functions based on the symbolic expression for each of the following cases:
{TR, gamma} given, M1 missing
{TR, M1} given, gamma missing
{gamma, M1} given, TR missing
This results in 6 m-files to be written, sol_TR.m, cond_TR.m, etc.
syms gamma M1 TR;
assume(gamma, 'real');
assume(gamma > 0);
assume(M1, 'real');
assume(M1 > 0);
assume(TR, 'real');
assume(TR > 0);
eq = TR ==(((gamma - 1)*M1^2 + 2)*(2*gamma*M1^2 - gamma + 1))/(M1^2*(gamma + 1)^2);
vars = {gamma, M1, TR};
num_vars = size(vars,2);
for i=1:num_vars
current_var = vars{i};
[sol, ~, cond] = solve(eq,current_var, 'ReturnConditions', true);
matlabFunction(sol, 'File', sprintf('sol_%s',char(current_var)),'Vars', vars, 'Optimize', false);
matlabFunction(cond, 'File', sprintf('cond_%s',char(current_var)),'Vars', vars, 'Optimize', false);
end
These functions can then be used to calculate the missing variable:
function [input_vector] = calc_third(varname_1, var_value_1, varname_2, var_value_2)
varnames = {'gamma', 'M1', 'TR'};
num_vars = size(varnames,2);
var_index = 1:num_vars;
var_name_map = containers.Map(varnames,var_index);
input_vector = zeros(1,num_vars);
input_vector(var_index == var_name_map(varname_1)) = var_value_1;
input_vector(var_index == var_name_map(varname_2)) = var_value_2;
var_index(var_index == var_name_map(varname_1)) = [];
var_index(var_index == var_name_map(varname_2)) = [];
sol_func = sprintf('sol_%s(input_vector(1),input_vector(2),input_vector(3))', varnames{var_index});
cond_func = sprintf('cond_%s(input_vector(1),input_vector(2),input_vector(3))', varnames{var_index});
result = dot(eval(sol_func), eval(cond_func));
input_vector(var_index)= result;
end
Example run:
>> calc_third('gamma', 0.5, 'TR', 100)
ans =
0.5000 0.0669 100.0000
You could of course build upon that solution and create a symbolic system of equations which incorporates all your 8 variables. You would then have to generate 28 functions and select the approriate ones based on the given input variables.
However, I would not recommend that route. Try to get the symbolic toolbox where you need it, this should help you avoid a lot of headache.
You could use it then like this:
function [] = calc_third(varname_1, var_value_1, varname_2, var_value_2)
gamma = sym('gamma');
M1 = sym('M1');
TR = sym('TR');
eq = TR ==(((gamma - 1)*M1^2 + 2)*(2*gamma*M1^2 - gamma + 1))/(M1^2*(gamma + 1)^2);
subs_eq = (subs(eq,[sym(varname_1), sym(varname_2)],[var_value_1,var_value_2]));
missing_var = symvar(subs_eq)
solve(subs_eq,missing_var)
end
Sample run:
>> calc_third('gamma', 0.5, 'TR', 100)
missing_var =
M1
ans =
(2*2^(1/2))/(3*88609^(1/2) + 893)^(1/2)

Export a neural network trained with MATLAB in other programming languages

I trained a neural network using the MATLAB Neural Network Toolbox, and in particular using the command nprtool, which provides a simple GUI to use the toolbox features, and to export a net object containing the informations about the NN generated.
In this way, I created a working neural network, that I can use as classifier, and a diagram representing it is the following:
There are 200 inputs, 20 neurons in the first hidden layer, and 2 neurons in the last layer that provide a bidimensional output.
What I want to do is to use the network in some other programming language (C#, Java, ...).
In order to solve this problem, I try to use the following code in MATLAB:
y1 = tansig(net.IW{1} * input + net.b{1});
Results = tansig(net.LW{2} * y1 + net.b{2});
Assuming that input is a monodimensional array of 200 elements, the previous code would work if net.IW{1} is a 20x200 matrix (20 neurons, 200 weights).
The problem is that I noticed that size(net.IW{1}) returns unexpected values:
>> size(net.IW{1})
ans =
20 199
I got the same problem with a network with 10000 input. In this case, the result wasn't 20x10000, but something like 20x9384 (I don't remember the exact value).
So, the question is: how can I obtain the weights of each neuron? And after that, can someone explain me how can I use them to produce the same output of MATLAB?
I solved the problems described above, and I think it is useful to share what I've learned.
Premises
First of all, we need some definitions. Let's consider the following image, taken from [1]:
In the above figure, IW stands for initial weights: they represent the weights of neurons on the Layer 1, each of which is connected with each input, as the following image shows [1]:
All the other weights, are called layer weights (LW in the first figure), that are also connected with each output of the previous layer. In our case of study, we use a network with only two layers, so we will use only one LW array to solve our problems.
Solution of the problem
After the above introduction, we can proceed by dividing the issue in two steps:
Force the number of initial weights to match with the input array length
Use the weights to implement and use the neural network just trained in other programming languages
A - Force the number of initial weights to match with the input array length
Using the nprtool, we can train our network, and at the end of the process, we can also export in the workspace some information about the entire training process. In particular, we need to export:
a MATLAB network object that represents the neural network created
the input array used to train the network
the target array used to train the network
Also, we need to generate a M-file that contains the code used by MATLAB to create the neural network, because we need to modify it and change some training options.
The following image shows how to perform these operations:
The M-code generated will be similar to the following one:
function net = create_pr_net(inputs,targets)
%CREATE_PR_NET Creates and trains a pattern recognition neural network.
%
% NET = CREATE_PR_NET(INPUTS,TARGETS) takes these arguments:
% INPUTS - RxQ matrix of Q R-element input samples
% TARGETS - SxQ matrix of Q S-element associated target samples, where
% each column contains a single 1, with all other elements set to 0.
% and returns these results:
% NET - The trained neural network
%
% For example, to solve the Iris dataset problem with this function:
%
% load iris_dataset
% net = create_pr_net(irisInputs,irisTargets);
% irisOutputs = sim(net,irisInputs);
%
% To reproduce the results you obtained in NPRTOOL:
%
% net = create_pr_net(trainingSetInput,trainingSetOutput);
% Create Network
numHiddenNeurons = 20; % Adjust as desired
net = newpr(inputs,targets,numHiddenNeurons);
net.divideParam.trainRatio = 75/100; % Adjust as desired
net.divideParam.valRatio = 15/100; % Adjust as desired
net.divideParam.testRatio = 10/100; % Adjust as desired
% Train and Apply Network
[net,tr] = train(net,inputs,targets);
outputs = sim(net,inputs);
% Plot
plotperf(tr)
plotconfusion(targets,outputs)
Before start the training process, we need to remove all preprocessing and postprocessing functions that MATLAB executes on inputs and outputs. This can be done adding the following lines just before the % Train and Apply Network lines:
net.inputs{1}.processFcns = {};
net.outputs{2}.processFcns = {};
After these changes to the create_pr_net() function, simply we can use it to create our final neural network:
net = create_pr_net(input, target);
where input and target are the values we exported through nprtool.
In this way, we are sure that the number of weights is equal to the length of input array. Also, this process is useful in order to simplify the porting to other programming languages.
B - Implement and use the neural network just trained in other programming languages
With these changes, we can define a function like this:
function [ Results ] = classify( net, input )
y1 = tansig(net.IW{1} * input + net.b{1});
Results = tansig(net.LW{2} * y1 + net.b{2});
end
In this code, we use the IW and LW arrays mentioned above, but also the biases b, used in the network schema by the nprtool. In this context, we don't care about the role of biases; simply, we need to use them because nprtool does it.
Now, we can use the classify() function defined above, or the sim() function equally, obtaining the same results, as shown in the following example:
>> sim(net, input(:, 1))
ans =
0.9759
-0.1867
-0.1891
>> classify(net, input(:, 1))
ans =
0.9759
-0.1867
-0.1891
Obviously, the classify() function can be interpreted as a pseudocode, and then implemented in every programming languages in which is possibile to define the MATLAB tansig() function [2] and the basic operations between arrays.
References
[1] Howard Demuth, Mark Beale, Martin Hagan: Neural Network Toolbox 6 - User Guide, MATLAB
[2] Mathworks, tansig - Hyperbolic tangent sigmoid transfer function, MATLAB Documentation center
Additional notes
Take a look to the robott's answer and the Sangeun Chi's answer for more details.
Thanks to VitoShadow and robott answers, I can export Matlab neural network values to other applications.
I really appreciate them, but I found some trivial errors in their codes and want to correct them.
1) In the VitoShadow codes,
Results = tansig(net.LW{2} * y1 + net.b{2});
-> Results = net.LW{2} * y1 + net.b{2};
2) In the robott preprocessing codes,
It would be easier extracting xmax and xmin from the net variable than calculating them.
xmax = net.inputs{1}.processSettings{1}.xmax
xmin = net.inputs{1}.processSettings{1}.xmin
3) In the robott postprocessing codes,
xmax = net.outputs{2}.processSettings{1}.xmax
xmin = net.outputs{2}.processSettings{1}.xmin
Results = (ymax-ymin)*(Results-xmin)/(xmax-xmin) + ymin;
-> Results = (Results-ymin)*(xmax-xmin)/(ymax-ymin) + xmin;
You can manually check and confirm the values as follows:
p2 = mapminmax('apply', net(:, 1), net.inputs{1}.processSettings{1})
-> preprocessed data
y1 = purelin ( net.LW{2} * tansig(net.iw{1}* p2 + net.b{1}) + net.b{2})
-> Neural Network processed data
y2 = mapminmax( 'reverse' , y1, net.outputs{2}.processSettings{1})
-> postprocessed data
Reference:
http://www.mathworks.com/matlabcentral/answers/14517-processing-of-i-p-data
This is a small improvement to the great Vito Gentile's answer.
If you want to use the preprocessing and postprocessing 'mapminmax' functions, you have to pay attention because 'mapminmax' in Matlab normalizes by ROW and not by column!
This is what you need to add to the upper "classify" function, to keep a coherent pre/post-processing:
[m n] = size(input);
ymax = 1;
ymin = -1;
for i=1:m
xmax = max(input(i,:));
xmin = min(input(i,:));
for j=1:n
input(i,j) = (ymax-ymin)*(input(i,j)-xmin)/(xmax-xmin) + ymin;
end
end
And this at the end of the function:
ymax = 1;
ymin = 0;
xmax = 1;
xmin = -1;
Results = (ymax-ymin)*(Results-xmin)/(xmax-xmin) + ymin;
This is Matlab code, but it can be easily read as pseudocode.
Hope this will be helpful!
I tried to implement a simply 2-layer NN in C++ using OpenCV and then exported the weights to Android which worked quiet well. I wrote a small script which generates a header file with the learned weights and this is used in the following code snipped.
// Map Minimum and Maximum Input Processing Function
Mat mapminmax_apply(Mat x, Mat settings_gain, Mat settings_xoffset, double settings_ymin){
Mat y;
subtract(x, settings_xoffset, y);
multiply(y, settings_gain, y);
add(y, settings_ymin, y);
return y;
/* MATLAB CODE
y = x - settings_xoffset;
y = y .* settings_gain;
y = y + settings_ymin;
*/
}
// Sigmoid Symmetric Transfer Function
Mat transig_apply(Mat n){
Mat tempexp;
exp(-2*n, tempexp);
Mat transig_apply_result = 2 /(1 + tempexp) - 1;
return transig_apply_result;
}
// Map Minimum and Maximum Output Reverse-Processing Function
Mat mapminmax_reverse(Mat y, Mat settings_gain, Mat settings_xoffset, double settings_ymin){
Mat x;
subtract(y, settings_ymin, x);
divide(x, settings_gain, x);
add(x, settings_xoffset, x);
return x;
/* MATLAB CODE
function x = mapminmax_reverse(y,settings_gain,settings_xoffset,settings_ymin)
x = y - settings_ymin;
x = x ./ settings_gain;
x = x + settings_xoffset;
end
*/
}
Mat getNNParameter (Mat x1)
{
// convert double array to MAT
// input 1
Mat x1_step1_xoffsetM = Mat(1, 48, CV_64FC1, x1_step1_xoffset).t();
Mat x1_step1_gainM = Mat(1, 48, CV_64FC1, x1_step1_gain).t();
double x1_step1_ymin = -1;
// Layer 1
Mat b1M = Mat(1, 25, CV_64FC1, b1).t();
Mat IW1_1M = Mat(48, 25, CV_64FC1, IW1_1).t();
// Layer 2
Mat b2M = Mat(1, 48, CV_64FC1, b2).t();
Mat LW2_1M = Mat(25, 48, CV_64FC1, LW2_1).t();
// input 1
Mat y1_step1_gainM = Mat(1, 48, CV_64FC1, y1_step1_gain).t();
Mat y1_step1_xoffsetM = Mat(1, 48, CV_64FC1, y1_step1_xoffset).t();
double y1_step1_ymin = -1;
// ===== SIMULATION ========
// Input 1
Mat xp1 = mapminmax_apply(x1, x1_step1_gainM, x1_step1_xoffsetM, x1_step1_ymin);
Mat temp = b1M + IW1_1M*xp1;
// Layer 1
Mat a1M = transig_apply(temp);
// Layer 2
Mat a2M = b2M + LW2_1M*a1M;
// Output 1
Mat y1M = mapminmax_reverse(a2M, y1_step1_gainM, y1_step1_xoffsetM, y1_step1_ymin);
return y1M;
}
example for a bias in the header could be this:
static double b2[1][48] = {
{-0.19879, 0.78254, -0.87674, -0.5827, -0.017464, 0.13143, -0.74361, 0.4645, 0.25262, 0.54249, -0.22292, -0.35605, -0.42747, 0.044744, -0.14827, -0.27354, 0.77793, -0.4511, 0.059346, 0.29589, -0.65137, -0.51788, 0.38366, -0.030243, -0.57632, 0.76785, -0.36374, 0.19446, 0.10383, -0.57989, -0.82931, 0.15301, -0.89212, -0.17296, -0.16356, 0.18946, -1.0032, 0.48846, -0.78148, 0.66608, 0.14946, 0.1972, -0.93501, 0.42523, -0.37773, -0.068266, -0.27003, 0.1196}};
Now, that Google published Tensorflow, this became obsolete.
Hence the solution becomes (after correcting all parts)
Here I am giving a solution in Matlab, but if you have tanh() function, you may easily convert it to any programming language. It is for just showing the fields from network object and the operations you need.
Assume you have a trained ann (network object) that you want to export
Assume that the name of the trained ann is trained_ann
Here is the script for exporting and testing.
Testing script compares original network result with my_ann_evaluation() result
% Export IT
exported_ann_structure = my_ann_exporter(trained_ann);
% Run and Compare
% Works only for single INPUT vector
% Please extend it to MATRIX version by yourself
input = [12 3 5 100];
res1 = trained_ann(input')';
res2 = my_ann_evaluation(exported_ann_structure, input')';
where you need the following two functions
First my_ann_exporter:
function [ my_ann_structure ] = my_ann_exporter(trained_netw)
% Just for extracting as Structure object
my_ann_structure.input_ymax = trained_netw.inputs{1}.processSettings{1}.ymax;
my_ann_structure.input_ymin = trained_netw.inputs{1}.processSettings{1}.ymin;
my_ann_structure.input_xmax = trained_netw.inputs{1}.processSettings{1}.xmax;
my_ann_structure.input_xmin = trained_netw.inputs{1}.processSettings{1}.xmin;
my_ann_structure.IW = trained_netw.IW{1};
my_ann_structure.b1 = trained_netw.b{1};
my_ann_structure.LW = trained_netw.LW{2};
my_ann_structure.b2 = trained_netw.b{2};
my_ann_structure.output_ymax = trained_netw.outputs{2}.processSettings{1}.ymax;
my_ann_structure.output_ymin = trained_netw.outputs{2}.processSettings{1}.ymin;
my_ann_structure.output_xmax = trained_netw.outputs{2}.processSettings{1}.xmax;
my_ann_structure.output_xmin = trained_netw.outputs{2}.processSettings{1}.xmin;
end
Second my_ann_evaluation:
function [ res ] = my_ann_evaluation(my_ann_structure, input)
% Works with only single INPUT vector
% Matrix version can be implemented
ymax = my_ann_structure.input_ymax;
ymin = my_ann_structure.input_ymin;
xmax = my_ann_structure.input_xmax;
xmin = my_ann_structure.input_xmin;
input_preprocessed = (ymax-ymin) * (input-xmin) ./ (xmax-xmin) + ymin;
% Pass it through the ANN matrix multiplication
y1 = tanh(my_ann_structure.IW * input_preprocessed + my_ann_structure.b1);
y2 = my_ann_structure.LW * y1 + my_ann_structure.b2;
ymax = my_ann_structure.output_ymax;
ymin = my_ann_structure.output_ymin;
xmax = my_ann_structure.output_xmax;
xmin = my_ann_structure.output_xmin;
res = (y2-ymin) .* (xmax-xmin) /(ymax-ymin) + xmin;
end

fit using lsqcurvefit

I want to fit some data to a lorentz function but I figure problems with fitting when I use parameters which are of different orders of magnitude.
This my lorentz function:
function [ value ] = lorentz( x,x0,gamma,amp )
value = amp * gamma^2 ./ ((x-x0).^2 + gamma^2);
end
Now the script to generate sample data:
x = linspace(2e14,6e14,200);
x0 = 4.525e14;
gamma = 0.5e14;
amp = 2e-14;
y = lorentz(x,x0,gamma,amp);
And the script for fitting lorentz to the sample data:
params = [4.475e14;0.4e14;1.8e-14];
opts = optimset('TolFun',1e-60,'TolX',1e-50,'Display','Iter');
fitfunc = #(params,x) lorentz(x,params(1),params(2),params(3));
fitparams = lsqcurvefit(fitfunc,params,x,y,[],[],opts)
figure(1);hold on;
plot(x,y,'.');
plot(x,lorentz(x,params(1),params(2),params(3)),'--');
plot(x,lorentz(x,fitparams(1),fitparams(2),fitparams(3)));
hold off;
This only varies the last parameter (the smallest, which is the amplitude). If I leave all exponentials out it works as expected. I assume there is some finetuning of opts to be done, but I don't know how. Any ideas how to do this?
As you suggested, you're going to run into numerical issues whenever you have parameters that vary over 28 (!) orders of magnitude. LSQCURVEFIT, for example, will try and estimate proper gradient steps, and those calculations may be sensitive to numerical stability (depending on the actual implementation - see http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm for a summary of how all this is done).
In my experience, you need to find a way to normalize the input parameters that make them more comparable. For example, you can take the log of all the values, and then exp() them inside you're objective function.
params = log([4.475e14;0.4e14;1.8e-14])
and
function [ value ] = lorentz( x,x0,gamma,amp )
gamma = exp(gamma);
amp = exp(amp);
x0 = exp(x0);
value = amp * gamma^2 ./ ((x-x0).^2 + gamma^2);
end
That may introduce other instabilities, but it should get you started.