Linear regression cost function and gradient descent - linear-regression

Question

I have been studying data science and ML topics for a while and I always get sucked at one point that makes a great confusion for me.
In courses like Andrew Ng's, it is defined that the error between the predicted value and the true value from e.g. Linear regression is expressed by:
error = predicted_value - y
In some other tutorials/courses, the error is presented as:
error = y - predicted_value
Also, for instance, on Udacity's data science Nanodegree, the gradient descent weights update is given by:
error = y - predicted_value
W_new = W + learn_rate * np.matmul(error, X)
At the same time, in several other books/courses, the same procedure is given by :
error = predicted_value - y
W_new = W - learn_rate * np.matmul(error, X)
Could someone help me out with those different notations?
Thank you!
EDIT
Following #bottaio answer, I got the following:
First case :
# compute errors
y_pred = np.matmul(X, W) + b
error = y_pred - y
# compute steps
W_new = W - learn_rate * np.matmul(error, X)
b_new = b - learn_rate * error.sum()
return W_new, b_new
Second case :
# compute errors
y_pred = np.matmul(X, W) + b
error = y - y_pred
# compute steps
W_new = W + learn_rate * np.matmul(error, X)
b_new = b + learn_rate * error.sum()
return W_new, b_new
Running the first and second cases, I get :
Third case :
# compute errors
y_pred = np.matmul(X, W) + b
error = y_pred - y
# compute steps
W_new = W + learn_rate * np.matmul(error, X)
b_new = b + learn_rate * error.sum()
return W_new, b_new
Running the third case, I get :
That's exactly the intuition I'm trying to achieve.
Whats the relation between using the error = y - y_pred and having to use the step computation as positive W_new = W + learn_rate * np.matmul(error, X) instead of W_new = W - learn_rate * np.matmul(error, X) ?
Thank you for all the support!!!!!

Answer 1

error = predicted_value - y
error' = y - predicted_value = -error
W = W + lr * matmul(error, X) = W + lr * matmul(-error', X) = W - lr * matmul(-error', X)
These two expressions are two ways of looking at the same thing. You propagate error backwards.
To be honest, the second states more clearly what is going on under the hood - error is just a difference between what model predicted relative to ground truth (explains predicted - y). And gradient descent step is about changing weights in opposite direction to gradient (explains minus).