I would like to know how the smooth Dice coefficient is differentiable but not a smooth IoU. Exactly which part of smooth IoU makes is non-differentiable, since the equation for two seems quite similar. Calculus or differentiation isn't my strong suit, so I would appreciate if someone could explain or point me to a helpful resource.
Related
I am currently trying to teach me something about neural networks. So I bought myself this book called Applied Artificial Intelligence written by Wolfgang Beer and I am now stuck at understanding a part of his code. Actually I understand the code I just do not understand one mathematical step behind it...
The part looks like this:
for i in range(iterations):
guessed = sig(inputs*weights)
error = output - guessed
adjustment = error*sig_d(outpus)
#Why is there no learningrate?
#Why is the adjustment relative to the error
#muliplied by the derivative of your main function?
weights += adjustment
I tried to look up how the gradient descent method works, but I never got the part with ajusting the weights. How does the math behind it work and why do you use the derivative for it?
Alo when I started to look in the internet for other solutions I always saw them using a learning rate. I understand the consept of it but why is this method not used in this book? It would realy help me if someone could awnser me these questions...
And thanks for all these rapid responses in the past.
To train a regression model we start with arbitrary weights and adjust weights so that the error will be minimum. If we plot the error as a function of weights we will get a plot like above figure where error J(θ0,θ1) is a function of weights θ0,θ1. We will be succeeded when our error will be very bottom of the graph when its value is the minimum. The red arrows show the minimum points in the graph. To reach to the minimum point we take derivative (the tangential line to a function) of our error function. The slope of the tangent is the derivative at that point and it will give us a direction to move towards. We make steps down the cost function in the direction with the steepest descent. The size of each step is determined by the parameter α, which is called the learning rate.
The gradient descent algorithm is:
repeat until convergence:
θj:=θj −[ Derivative of J(θ0,θ1) in respect of θj]
where
j=0,1 represents the weights' index number.
In the above figure we plot error J(θ1) is a function of weight θ1. We start with an arbitrary value of θ1 and take derivative(slope of the tangent) of error J(θ1) to adjust weight θ1 so we can reach the bottom where error is minimum. If slope is positive we have to go left or decrease weight θ1. And if slope is negative we have to go right or increase θ1. We have to repeat this procedure until convergence or reaching minimum point.
If learning rate α is too small gradient descent converges too slow. And if α is too large gradient descent overshoots and fails to converge.
All the figures have been taken from Andrew Ng's machine learning course on coursera.org
https://www.coursera.org/learn/machine-learning/home/welcome
Why is there no learningrate?
there are lots of different flavors of neural networks, some will use learning rates and others probably just keep this constant
Why is the adjustment relative to the error
what else should it be relative to? If there is a lot of error then chances are you need to do a lot of adjustments, if there was only a little error then you would only want to adjust your weights a small amount.
muliplied by the derivative of your main function?
dont really have a good answer for this one.
I noticed that there are only not-a-knot and clamped spline in Matlab, which is assembled in the function spline. Can we change the original code in matlab a little bit so it can perform (or only perform) natural spline? I felt like it is a easy task since we need only to change the extra two degrees of freedom of parameters, such that it is satisfied to have zero second derivatives at the end points. But staring at the code of spline I don't know how to proceed. Can anyone help me out?
The documentation I am reading for curve fitting doesnt seem to be that complicated, but clearly I am doing something wrong.
Given x,y data, trying to fit a polynomial function of degree 3, the data points are seemingly ignored. Can someone explain both how to fix this, and what this curve is actually calculating, as opposed to what I think it should be calculating?
Data: http://pastebin.com/4EXu0FSv
You most likely are using the wrong regression model or interval (or points). Curve fitting is very very complex topic and can not be simply solved. Have a read of the Mathworks site about the curve fitting toolbox here .
However I would not be fitting a 3 order polynomial to this data. I would be more inclinded to fit positive reciprocal function - see if that gives you a better fit.
I am building receiver operating characteristic (ROC) curves to evaluate classifiers using the area under the curve (AUC) (more details on that at end of post). Unfortunately, points on the curve often go below the diagonal. For example, I end up with graphs that look like the one here (ROC curve in blue, identity line in grey) :
The the third point (0.3, 0.2) goes below the diagonal. To calculate AUC I want to fix such recalcitrant points.
The standard way to do this, for point (fp, tp) on the curve, is to replace it with a point (1-fp, 1-tp), which is equivalent to swapping the predictions of the classifier. For instance, in our example, our troublesome point A (0.3, 0.2) becomes point B (0.7, 0.8), which I have indicated in red in the image linked to above.
This is about as far as my references go in treating this issue. The problem is that if you add the new point into a new ROC (and remove the bad point), you end up with a nonmonotonic ROC curve as shown (red is the new ROC curve, and dotted blue line is the old one):
And here I am stuck. How can I fix this ROC curve?
Do I need to re-run my classifier with the data or classes somehow transformed to take into account this weird behavior? I have looked over a relevant paper, but if I am not mistaken, it seems to be addressing a slightly different problem than this.
In terms of some details: I still have all the original threshold values, fp values, and tp values (and the output of the original classifier for each data point, an output which is just a scalar from 0 to 1 that is a probability estimate of class membership). I am doing this in Matlab starting with the perfcurve function.
Note based on some very helpful emails about this from the people that wrote the articles cited above, and the discussion above, the right answer seems to be: do not try to "fix" individual points in an ROC curve unless you build an entirely new classifier, and then be sure to leave out some test data to see if that was a reasonable thing to do.
Getting points below the identity line is something that simply happens. It's like getting an individual classifier that scores 45% correct even though the optimal theoretical minimum is 50%. That's just part of the variability with real data sets, and unless it is significantly less than expected based on chance, it isn't something you should worry too much about. E.g., if your classifier gets 20% correct, then clearly something is amiss and you might look into the specific reasons and fix your classifier.
Yes, swapping a point for (1-fp, 1-tp) is theoretically effective, but increasing sample size is a safe bet too.
It does seem that your system has a non-monotonic response characteristic so be careful not to bend the rules of the ROC too much or you will impact the robustness of the AUC.
That said, you could try to use a Pareto Frontier Curve (Pareto Front). If that fits the requirements of "Repairing Concavities" then you'll basically sort the points so that the ROC curve becomes monotonic.
I used median filtering in Matlab to reduce noise in arrays. I was calculating the velocity and acceleration of an object. The velocity part worked fine, and the result I got was exactly what I expected.
I stumbled in to problems regarding the acceleration. I noticed the acceleration was not behaving well; I noticed rectangular shapes instead of the expected smooth shapes.
The plot for acceleration should be a sinusoidal wave for cam and follower in an engine but the problem is that the edges of the wave are not smooth but in form of straight lines.
For velocity I used vavg=medfilt2(v, [1 5])
For accel I used aavg=medfilt2(a, [1 5 ])
Is there any other filter that I should use to get accurate edges?
Are there other filter functions that could be useful?
Taking the derivative of something amplifies the noise, so it's not surprising that you're having problems when you're looking at the acceleration of a signal. You can use a noise-resistant method for taking the derivative. See this page for details:
http://www.holoborodko.com/pavel/numerical-methods/numerical-derivative/smooth-low-noise-differentiators/
It sounds like people use Savitz-Golay filters for this frequently - try the sgolay and sgolayfilt functions in Matlab.
In general, though, you should think about the characteristics of your signal and your noise. If you know that the signal is going to be something really smooth, then why not approximate it first enforcing its smoothness? It seems like splines could be appropriate for you (spline in matlab).