MATLAB fminunc stopped becaise it cannot decrease the objective formula? - matlab

Question

I am trying to using fminunc to obtain the optimal theta in logistic regression, however I keep getting that:
fminunc stopped because it cannot decrease the objective function
along the current search direction.
Searching online, I found that this is usually the result of a gradient error which I am implementing in logistic_costFunction.m. I re-checked my work but I cannot spot the root cause.
I am not sure how to solve this issue, any help would be appreciated.
Here is my code:
clear all; close all; clc;
%% Plotting data
x1 = linspace(0,3,50);
mqtrue = 5;
cqtrue = 30;
dat1 = mqtrue*x1+5*randn(1,50);
x2 = linspace(7,10,50);
dat2 = mqtrue*x2 + (cqtrue + 5*randn(1,50));
x = [x1 x2]'; % X
subplot(2,2,1);
dat = [dat1 dat2]'; % Y
scatter(x1, dat1); hold on;
scatter(x2, dat2, '*'); hold on;
classdata = (dat>40);
%% Compute Cost and Gradient
% Setup the data matrix appropriately, and add ones for the intercept term
[m, n] = size(x);
% Add intercept term to x and X_test
x = [ones(m, 1) x];
% Initialize fitting parameters
initial_theta = zeros(n + 1, 1);
% Compute and display initial cost and gradient
[cost, grad] = logistic_costFunction(initial_theta, x, dat);
fprintf('Cost at initial theta (zeros): %f\n', cost);
fprintf('Gradient at initial theta (zeros): \n');
fprintf(' %f \n', grad);
%% ============= Part 3: Optimizing using fminunc =============
% In this exercise, you will use a built-in function (fminunc) to find the
% optimal parameters theta.
% Set options for fminunc
options = optimset('GradObj', 'on', 'MaxIter', 400);
% Run fminunc to obtain the optimal theta
% This function will return theta and the cost
[theta, cost] = ...
fminunc(#(t)(logistic_costFunction(t, x, dat)), initial_theta, options);
logistic_costFunction.m
function [J, grad] = logistic_costFunction(theta, X, y)
% Initialize some useful values
m = length(y); % number of training examples
grad = zeros(size(theta));
H = sigmoid(X*theta);
T = y.*log(H) + (1 - y).*log(1 - H);
J = -1/m*sum(T);
% ====================Compute grad==================
for i = 1 : m
grad = grad + (H(i) - y(i)) * X(i,:)';
end
grad = 1/m*grad;
end
sigmoid.m
function g = sigmoid(z)
% Computes thes sigmoid of z
g = zeros(size(z));
g = 1 ./ (1 + (1 ./ exp(z)));
end