Is it there any way I can limit the number of bits of the weights obtained while solving the linearSVC?
http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC
Related
I'm working on a neural network with Keras using TensorFlow as the backend right now, and my model takes 5 inputs, all normalized to 0 to 1. The inputs' units vary from m/s to meters to m/s/s. So, for example, one input could vary from 0 m/s to 30 m/s, while another input could vary from 5 m to 200 m in the training dataset.
Is it better to individually and independently normalize all inputs so that I have different scales for each unit/input? Or would normalizing all inputs to one scale (mapping 0-200 to 0-1 for the example above) be better for accuracy?
Normalize individualy each input. Because if you normalize everything by dividing 200 some inputs will affect your network less than others. If one input vary between 0-30, after dividing by 200 you get 0-0.15 scale and scale for input which vary 0-200 will be 0-1 after division. So 0-30 input will have less numbers and you tell your network that input is not so relevant as one whith 0-200.
Say I have a training set with 50 vectors. I split this set into 5 sets each with 10 vectors and then I scale the vectors in each subset and normalise the subsets. Then I train my ANN with each vector from each subset.
After training is complete, I group my test set into subsets of 10 vectors each, scale the features of the vectors in each subset and normalise each subset and then feed it to the neural network to attempt to classify it.
Is this the right approach? Is it right to scale and normalise each subset, each with its own minimum, maximum, mean and standard deviation?
So I have something like this,
y=l3*[sin(theta1)*cos(theta2)*cos(theta3)+cos(theta1)*sin(theta2)*cos(theta3)-sin(theta1)*sin(theta2)*sin(theta3)+cos(theta1)*cos(theta2)sin(theta3)]+l2[sin(theta1)*cos(theta2)+cos(theta1)*sin(theta2)]+l1*sin(theta1)+l0;
and something similar for x. Where thetai is angles from specified interval and li some coeficients. Task is approximate inversion of equation, so you set x and y and result will be appropriate theta. So I random generate thetas from specified intervals, compute x and y. Then I norm x and y between <-1,1> and thetas between <0,1>. This data I used as training set in such way, inputs of network are normalized x and y, outputs are normalized thetas.
I train the network, tried different configuration and absolute error of network was still around 24.9% after whole night of training. It's so much, so I don't know what to do.
Bigger training set?
Bigger network?
Experiment with learning rate?
Longer training?
Technical info
As training algorithm was used error back propagation. Neurons have sigmoid activation function, units are biased. I tried topology: [2 50 3], [2 100 50 3], training set has length 1000 and training duration was 1000 cycle(in one cycle I go through all dataset). Learning rate has value 0.2.
Error of approximation was computed as
sum of abs(desired_output - reached_output)/dataset_lenght.
Used optimizer is stochastic gradient descent.
Loss function,
1/2 (desired-reached)^2
Network was realized in my Matlab template for NN. I know that is weak point, but I'm sure my template is right because(successful solution of XOR problem, approximation of differential equations, approximation of state regulator). But I show this template, because this information may be useful.
Neuron class
Network class
EDIT:
I used 2500 unique data within theta ranges.
theta1<0, 180>, theta2<-130, 130>, theta3<-150, 150>
I also experiment with larger dataset, but accuracy doesn't improve.
I have read in the documentation of crossval is that mcr = crossval('mcr',X,y,'Predfun',predfun) function in matlab calculate the misclassification rate, But if it's apply with 10-fold cross-validation, then we will have 10 different values for misclassification, since we done 10 testing, and each testing produce a result, but the value mcr is single or scalar , So does it take the average misclassification rates or it's take the minimum..etc ?
The average misclassification rate (across all folds and all monte-carlo repartitions) is used. The following line of crossval demonstrates the calculation of the average loss -
loss = sum(loss)/ (mcreps * sum(cvp.TestSize));
where loss is initially a vector of losses for each cross-validation fold and each repartition, mcreps is the number of repartitions and sum(cvp.TestSize) is the total size of the cross-validation test sets.
This is used for both the MSE (mean-squared error) and MCR loss functions.
I would like to integrate importance sampling in monte carlo to speed up the calculation.
Here I have created a simple example in matlab:
a=randn(10,10000);
a_sum=sum(a,1);
quantile(a_sum, 0.01)
The value at risk will amount to -7.3159 and around 100 scenarios are above the value at risk. So using a shift of -1. I can have more scenarios above value at risk:
b=a-1;
b_sum=sum(b,1);
A common way to calculate the value at risk is to use the likelihood ratios. Since I have shifted each random number by -1, I can calculate for each shift the likelihood ratio. But can I combine these likelihood ratios? And how can I calculate the value at risk?