I'm investigating RNNs and after read this paper, got the how Backrpop-through-time works on RNNs:
https://arxiv.org/pdf/1610.02583.pdf
But I have some confusion with the following implementation (from cs231):
for t in reversed(xrange(T)):
dh_current = dh[t] + dh_prev
dx_t, dh_prev, dWx_t, dWh_t, db_t = rnn_step_backward(dh_current, cache[t])
dx[t] += dx_t
dh0 = dh_prev
dWx += dWx_t
dWh += dWh_t
db += db_t
Why summing up dh[t] and dh_prev gradients, dh_current = dh[t] + dh_prev ?
Full source code: https://github.com/williamchan/cs231-assignment3/blob/master/cs231n/rnn_layers.py
Related
I am using a Logistic Regression model on PySpark through databricks but i am not able to get my precision and recall. Everything works fine and I am able to get my ROC but there is not attribute or lib for Precision and Recall
lrModel = LogisticRegression()
predictions = bestModel.transform(testData)
# Instantiate metrics object
results = predictions.select(['probability', 'label'])
results_collect = results.collect()
results_list = [(float(i[0][0]), 1.0-float(i[1])) for i in results_collect]
scoreAndLabels = sc.parallelize(results_list)
metrics = MulticlassMetrics(scoreAndLabels)
# Overall statistics
precision = metrics.precision()
recall = metrics.recall()
f1Score = metrics.fMeasure()
print("Summary Stats")
print("Precision = %s" % precision)
print("Recall = %s" % recall)
print("F1 Score = %s" % f1Score)
>>>Summary Stats
>>>Precision = 0.0
>>>Recall = 0.0
>>>F1 Score = 0.0
I was able to create my own function to do so. It returns everything and more. I am using the "MulticlassMetrics()" from mllib package. Since its a multiclass it calculates metrics for each label so, you have to specify which label you want to retrieve.
### Model Evaluator User Defined Functions
def udfModelEvaluator(dfPredictions, labelColumn='label'):
colSelect = dfPredictions.select(
[F.col('prediction').cast(DoubleType())
,F.col(labelColumn).cast(DoubleType()).alias('label')])
metrics = MulticlassMetrics(colSelect.rdd)
mAccuracy = metrics.accuracy
mPrecision = metrics.precision(1)
mRecall = metrics.recall(1)
mF1 = metrics.fMeasure(1.0, 1.0)
mMatrix = metrics.confusionMatrix().toArray().astype(int)
mTP = metrics.confusionMatrix().toArray()[1][1]
mTN = metrics.confusionMatrix().toArray()[0][0]
mFP = metrics.confusionMatrix().toArray()[0][1]
mFN = metrics.confusionMatrix().toArray()[1][0]
mResults = [mAccuracy, mPrecision, mRecall, mF1, mMatrix, mTP, mTN, mFP, mFN, "Return [[0]=Accuracy, [1]=Precision, [2]=Recall, [3]=F1, [4]=ConfusionMatrix, [5]=TP, [6]=TN, [7]=FP, [8]=FN]"]
return mResults
To call the function:
metricsList = udfModelEvaluator(predictionsData, "label")
metricsList
I´m using Spark 2.2.1 with Scala 2.11.12 version as a language to generate a recursive algorithm. First, I tried an implementation using RDD but the time when I used a lot of data was too much. I have made a new version using DataFrames but with very little data it takes too much, taking less data in each iteration than in the previous iteration.
I have tried to cache variables in different ways (types of persistence included), to use checkpoints in different moments, using the repartition method with different values and in different functions, and nothing works.
The code starts looking for the minimum distance between the points that make up the matrix (matrix is a DataFrame):
println("Finding minimum:")
val minDistRes = matrix.select(min("dist")).first().getFloat(0)
val clusterRes = matrix.where($"dist" === minDistRes)
println(s"New minimum:")
clusterRes.show(1)
Then, save the coordenates to the points for later calculations:
val point1 = clusterRes.first().getInt(0)
val point2 = clusterRes.first().getInt(1)
After, made several filters to use them in the new points generated in the next iteration (the creation of a broadcast variable is necessary to be able to access this data in a later map):
matrix = matrix.where("!(idW1 == " + point1 +" and idW2 ==" + point2 + " )").cache()
val dfPoints1 = matrix.where("idW1 == " + point1 + " or idW2 == " + point1).cache()
val dfPoints2 = matrix.where("idW1 == " + point2 + " or idW2 == " + point2).cache()
val dfPoints2Broadcast = spark.sparkContext.broadcast(dfPoints2)
val dfUnionPoints = dfPoints1.union(dfPoints2).cache()
val matrixSub = matrix.except(dfUnionPoints).cache()
Continued with the calculation of the new points and I return the matrix that will be used recursively by the algorithm:
val newPoints = dfPoints1.map{
r => val distAux = dfPoints2Broadcast.value.where("idW1 == " + r.getInt(0) +
" or idW1 == " + r.getInt(1) + " or idW2 == " + r.getInt(0) + " or idW2 == " +
r.getInt(1)).first().getFloat(2)
(newIndex.toInt, filterDF(r.getInt(0),r.getInt(1), point1, point2), math.min(r.getFloat(2), distAux))
}.asInstanceOf[Dataset[Row]]
matrix = matrixSub.union(newPoints)
Finalize each iteration caching the matrix variable and realized a checkpoint every so often:
matrix.cache()
if (a % 5 == 0)
matrix.checkpoint()
I have a Multilayer Perceptron code using Theano/Lasagne that when I run on a small dataset uses multiple cores correctly.
But, when I run it over a much larger dataset (with the same code) and I watch CPU utilisation in htop, I don't see it parallelize. It creates the number of processes as defined in .theanorc file which looks like:
[global]
OMP_NUM_THREADS=15
openmp=True
floatX = float32
[blas]
ldflags=-L/usr/lib/ -lblas -lgfortran
but most of the time (~ 90%) there is only one of the created processes is working (though some short amount of times utilisation goes up).
I imagine there is an operation that is not utilising multi-core while other operations are using it because when I run the code on a small dataset I see that most of the time all cores are utilised. All the operations are matrix multiplication (both sparse and dense) so I don't
know why some operations don't use multi-core.
This is the section of the code that is being run:
for n in xrange(self.n_epochs):
x_batch, y_batch = self.X, Y_train
l_train, acc_train = self.f_train(x_batch, y_batch, self.train_indices)
l_val, acc_val = self.f_val(self.X, Y_dev, self.dev_indices)
val_pred = self.f_predict(self.X, self.dev_indices)
if acc_val > best_val_acc:
best_val_loss = l_val
best_val_acc = acc_val
best_params = lasagne.layers.get_all_param_values(self.l_out)
n_validation_down = 0
else:
#early stopping
n_validation_down += 1
logging.info('epoch ' + str(n) + ' ,train_loss ' + str(l_train) + ' ,acc ' + str(acc_train) + ' ,val_loss ' + str(l_val) + ' ,acc ' + str(acc_val) + ',best_val_acc ' + str(best_val_acc))
if n_validation_down > self.early_stopping_max_down:
logging.info('validation results went down. early stopping ...')
break
My numpy/blas information:
lapack_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
blas_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
openblas_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
blis_info:
NOT AVAILABLE
openblas_lapack_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
lapack_mkl_info:
NOT AVAILABLE
blas_mkl_info:
NOT AVAILABLE
Note that the input matrices are sparse and I'm doing some extra sparse multiplications within custom layers using S.dot.
I was trying to use cross validation in following code:
Program:
TextDirectoryToArff d = new TextDirectoryToArff();
try {
Instances dataset = d.createDataset("C:\\mytest");
dataset.setClassIndex(dataset.numAttributes() - 1 );
double precision = 0, recall=0,fmeasure=0,error=0;
int size1 = dataset1.numInstances() / 10;
int begin = 0;
int end = size1 - 1 ;
for (int i=1 ; i<=10;i++)
{
System.out.println("iteration :" + 1);
Instances training = new Instances(dataset);
Instances testing = new Instances(dataset, begin , (end - begin));
for (int j=0;j < (end - begin); j++)
training.delete(begin);
Classifier tree = new NaiveBayes();
Instances filteredInstaces = training;
StringToNominal nominal ;
for(int a=0;a<training.numAttributes()-1;a++)
{
if(training.attribute(a).isString())
{
nominal = new StringToNominal();
nominal.setInputFormat(filteredInstaces);
training = Filter.useFilter(training, nominal);
}
}
tree.buildClassifier(training);
Evaluation eval = new Evaluation(testing);
eval.evaluateModel(tree, testing);
System.out.println("Precision:" + eval.precision(1));
System.out.println("Recall:" + eval.recall(1));
System.out.println("Fmeasure:" + eval.fMeasure(1));
System.out.println("Error:" + eval.errorRate());
I've some code for cross validation but not able to integrate with above code. Please suggest how can I integrate following code in above code to find cross validation?
Code:
Evaluation eval = new Evaluation(dataset);
eval.evaluateModel(cls, dataset2);
eval.crossValidateModel(cls,dataset1,10, dataset2.getRandomNumberGenerator(1));
System.out.println(eval.toSummaryString("\nResults\n======\n", false));
I think you are confused a bit about weka Evaluation.crossValidateModel method. It already calculate the 10 different train and test fold, and train 10 models on trains and evaluate the models on test, so not necessary to calculate it as you tried in your code.
so in your code:
TextDirectoryToArff d = new TextDirectoryToArff();
try {
Instances dataset = d.createDataset("C:\\mytest");
dataset.setClassIndex(dataset.numAttributes() - 1 );
// you already have a dataset
Classifier naiveBayes = new NaiveBayes();
//you need a classifier
Evaluation eval = new Evaluation(dataset);
//only call the crossValidateModel with your classifier, on your dataset, with 10 fold, and random
eval.crossValidateModel(naiveBayes, dataset, 10, new Random(1));
//print the results of 10 fold
System.out.println(classifier);
System.out.println(eval.toSummaryString());
System.out.println(eval.toMatrixString());
System.out.println(eval.toClassDetailsString());
you can find more at https://weka.wikispaces.com/Generating+cross-validation+folds+(Java+approach)
I have built a tricopter from scratch based on a .NET Micro Framework board from TinyCLR.com. I used the FEZ Mini which runs at 72 MHz. Read more about my project at: http://bit.ly/TriRot.
So after a pre-flight check where I initialise and test each component, like calibrating the IMU and spinning each motor, checking that I get receiver data, etc., it enters a permanent loop which then calls the flight controller method on each loop.
I'm trying to tune my PID controller now using the Ziegler-Nichols method, but I am always getting a progressively larger overshoot. I was eventually able to get a [mostly] stable oscillation using proportional control only (setting Ki and Kd = 0); timing the period K with a stopwatch averaged out to 3.198 seconds.
I came across the answer (by Rex Logan) on a similar question by chris12892.
I was initially using the "Duration" variable in milliseconds which made my copter highly aggressive, obviously because I was multiplying the running integrator error by thousands on each loop. I then divided it by another thousand to bring it to seconds, but I'm still battling...
What I don't understand from Rex's answer is:
Why does he ignore the time variable in the integral and differential parts of the equations? Is that right or is it a typo?
What he means by the remark
In a normal sampled system the delta term would be one...
One what? Should this be one second under normal circumstances? What
if this value fluctuates?
My flight controller method is below:
private static Single[] FlightController(Single[] imuData, Single[] ReceiverData)
{
Int64 TicksPerMillisecond = TimeSpan.TicksPerMillisecond;
Int64 CurrentTicks = DateTime.Now.Ticks;
Int64 TickCount = CurrentTicks - PreviousTicks;
PreviousTicks = CurrentTicks;
Single Duration = (TickCount / TicksPerMillisecond) / 1000F;
const Single Kp = 0.117F; //Proportional Gain (Instantaneou offset)
const Single Ki = 0.073170732F; //Integral Gain (Permanent offset)
const Single Kd = 0.001070122F; //Differential Gain (Change in offset)
Single RollE = 0;
Single RollPout = 0;
Single RollIout = 0;
Single RollDout = 0;
Single RollOut = 0;
Single PitchE = 0;
Single PitchPout = 0;
Single PitchIout = 0;
Single PitchDout = 0;
Single PitchOut = 0;
Single rxThrottle = ReceiverData[(int)Channel.Throttle];
Single rxRoll = ReceiverData[(int)Channel.Roll];
Single rxPitch = ReceiverData[(int)Channel.Pitch];
Single rxYaw = ReceiverData[(int)Channel.Yaw];
Single[] TargetMotorSpeed = new Single[] { rxThrottle, rxThrottle, rxThrottle };
Single ServoAngle = 0;
if (!FirstRun)
{
Single imuRoll = imuData[1] + 7;
Single imuPitch = imuData[0];
//Roll ----- Start
RollE = rxRoll - imuRoll;
//Proportional
RollPout = Kp * RollE;
//Integral
Single InstanceRollIntegrator = RollE * Duration;
RollIntegrator += InstanceRollIntegrator;
RollIout = RollIntegrator * Ki;
//Differential
RollDout = ((RollE - PreviousRollE) / Duration) * Kd;
//Sum
RollOut = RollPout + RollIout + RollDout;
//Roll ----- End
//Pitch ---- Start
PitchE = rxPitch - imuPitch;
//Proportional
PitchPout = Kp * PitchE;
//Integral
Single InstancePitchIntegrator = PitchE * Duration;
PitchIntegrator += InstancePitchIntegrator;
PitchIout = PitchIntegrator * Ki;
//Differential
PitchDout = ((PitchE - PreviousPitchE) / Duration) * Kd;
//Sum
PitchOut = PitchPout + PitchIout + PitchDout;
//Pitch ---- End
TargetMotorSpeed[(int)Motors.Motor.Left] += RollOut;
TargetMotorSpeed[(int)Motors.Motor.Right] -= RollOut;
TargetMotorSpeed[(int)Motors.Motor.Left] += PitchOut;// / 2;
TargetMotorSpeed[(int)Motors.Motor.Right] += PitchOut;// / 2;
TargetMotorSpeed[(int)Motors.Motor.Rear] -= PitchOut;
ServoAngle = rxYaw + 15;
PreviousRollE = imuRoll;
PreviousPitchE = imuPitch;
}
FirstRun = false;
return new Single[] {
(Single)TargetMotorSpeed[(int)TriRot.LeftMotor],
(Single)TargetMotorSpeed[(int)TriRot.RightMotor],
(Single)TargetMotorSpeed[(int)TriRot.RearMotor],
(Single)ServoAngle
};
}
Edit: I found that I had two bugs in my code above (fixed now). I was integrating and differentiating with the last IMU values as opposed to the last error values. That got rid of the runaway sitation completely. The only problem now is that it seems to be a bit slow. When I perturb the system, it responds very quickly and stop it from continuing, but it takes a long time to get back to the setpoint (0), about 10 seconds or more. Is this now just down to tuning the PID? I'll give the suggestions below a go, and let you know if any of them make a difference.
One question I have is:
being a .NET board, I don't want to bank on any kind of accurate timing, so instead of trying to work out at what frequency I am executing that method, surely if I calculate the actual time and factor that into the equations, it should be better, or am I misunderstanding something?