I am working on an artificial neural network. I want to implement it in Matlab, but I am unable to find a proper activation function. I need a step function because my output is either 0 or 1. is there any function in Matlab that can be used for this kind of output. Also, I want the reverse function of the same activation function. logsig and tansig are not working for me.
Both tansig and logsig are part of the Neural Network Toolbox as the online documentation makes clear. So, if which tansig returns nothing, then you don't have that toolbox (or at least don't have a version current enough to contain that function). However, both of these functions are extremely simple, and the documentation even gives you the formulae under the "Algorithms" section: tansig, logsig. Both can be implemented as a one line anonymous function if you wanted.
If your question is actually about how to produce a Heaviside step function, Matlab has heaviside (it's part of the Symbolic Math toolbox but a pure numeric version is included – type edit heaviside to see the simple code). However, note that using such a non-differentiable function is problematic for some types of neural networks as this StackOverflow question and answer addresses.
Heaviside did not work for me.. i finally normalized my data between 1 and -1 and then applied tansig.
Thanks
Related
I am trying to implement bayesian optimization using gauss process regression, and I want to try the multiple output GP firstly.
There are many softwares that implemented GP, like the fitrgp function in MATLAB and the ooDACE toolbox.
But I didn't find any available softwares that implementd the so called multiple output GP, that is, the Gauss Process Model that predict vector valued functions.
So, Are there any softwares that implemented the multiple output gauss process that I can use directly?
I am not sure my answer will help you as you seem to search matlab libraries.
However, you can do co-kriging in R with gstat. See http://www.css.cornell.edu/faculty/dgr2/teach/R/R_ck.pdf or https://github.com/cran/gstat/blob/master/demo/cokriging.R for more details about usage.
The lack of tools to do cokriging is partly due to the relative difficulty to use it. You need more assumptions than for simple kriging: in particular, modelling the dependence between in of the cokriged outputs via a cross-covariance function (https://stsda.kaust.edu.sa/Documents/2012.AGS.JASA.pdf). The covariance matrix is much bigger and you still need to make sure that it is positive definite, which can become quite hard depending on your covariance functions...
There is a file called normxcorr2_general on MathWorks here that the author claims always gives correct answers while Matlab's built-in normxcorr2 gives incorrect answers when the two input matrices are close in size. After doing some testing, it is clear that the two functions do give significantly different outputs when the inputs are the same size.
Is normxcorr2_general actually more accurate? I don't have much experience in Matlab and I'm having trouble figuring that out from reading through the function script.
Edit: To clarify, if I understand it correctly then these functions are both implementing equation number (2) in this paper about computing normalized cross-correlations.
I'm trying to perform logistic regression to do classification using MATLAB. There seem to be two different methods in MATLAB's statistics toolbox to build a generalized linear model 'glmfit' and 'fitglm'. I can't figure out what the difference is between the two. Is one preferable over the other?
Here are the links for the function descriptions.
http://uk.mathworks.com/help/stats/glmfit.html
http://uk.mathworks.com/help/stats/fitglm.html
The difference is what the functions output. glmfit just outputs a vector of the regression coefficients (and some other stuff if you ask for it). fitglm outputs a regression object that packs all sorts of information and functionality inside (See the docs on GeneralizedLinearModel class). I would assume the fitglm is intended to replace glmfit.
In addition to Dan's answer, I would like to add the following.
The function fitglm, like newer functions from the statistics toolbox, accepts more flexible inputs than glmfit. For example, you can use a table as the data source, specifyy a formula of the form Y ~ X1 + X2 + ..., and use categorical variables.
As a side note, the function lassoglm uses (depends on) glmfit.
I have a function fun(x,y,z), such that say, x=1:10, y=50:60, z=100:105. Which optimization method (and how) I can use to get the minimum of this function, for example, where (x,y,z)=(3,52,101). I am working in Matlab.
Thank you for any help
Algorithms
There are many many algorithms out there that you can use for direct search optimization such as Nelder-Mead, Particle Swarm, Genetic Algorithm, etc.
I believe Nelder-Mead is a simplex optimization method which is used by fminsearch function in MATLAB.
Also, there is Genetic Algorithm which comes with MATLAB Global Optimization toolbox. You may want to give that a try as well.
Particle Swarm Optimization (PSO) is another direct search method that you can use. However, there is no official toolbox for Particle Swarm method built by Mathworks. The good news is there is quite a few PSO toolbox developed by other people. I personally have used this one and am quite happy with the performance. Its syntax is similar to Genetic Algorithm functions that come with Global Optimization Toolbox.
Discrete Optimization
Regarding your question that you are looking for a set of integer values namely x,y, and z corresponding to the minimum objective function value, I would add a part at the beginning of the objective function that rounds the variables to the closest integers and then feeds them to your main function fun(x,y,z). This way you would discretize your function space.
I hope my answer helps.
How to minimize multivariate function in Matlab by using derivatives?
So far, for minimizing single variable functions I used fminunc,
but now I need to work with multivariate functions.
Thank you
Use fminunc. If you want to use the gradient, just return it as the 2nd output of your objective function. You'll also need to indicate in an options object that you are passing the gradient.
options = optimoptions('fminunc','GradObj','on');
I believe the documentation has info on passing the Hessian (if one exists).