I tried programming a neural network in processing IDE.
I managed to do it quite well, until I tried using the MNIST handwritten digits data set. I tried the iris data set and few others from UCI machine learning repository, but when I used the MNIST data set it didn't worked. for some reason all of the outputs approached zero with time, and that caused the total error to be always equal to 1. I am almost sure that my problem is the activation function; so I tried using softmax for classification, but it wasn't very successful. I got the same results. I think maybe I should have use a different loss function, so I tried the negative log probability according to this video. the results now are the same cost value for each output neuron, and the sum of the outputs is not 1 as it should be.
Here are the functions for each part of the code that I have changed (I prefer not to share the full code because it's long and messy, and not really helpful):
softmax:
float[] softmax(float[] inputVector){
float[] result = new float[inputVector.length];
float sigma = 0;
for(int i = 0; i < inputVector.length; i++){
sigma += exp(inputVector[i]);
}
for(int i = 0; i < result.length; i++){
result[i] = exp(inputVector[i]) / sigma;
}
return result;
}
derivative of softmax:
float[] derivativeSoftmax(float[] inputVector){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * (1 - softmax(inputVector)[i]);
}
return result;
}
loss function:
for(int i = 0; i < outputNeuronsNumber; i++){
float tempSigma = 0;
for(int j = 0; j < outputNeuronsNumber; j++){
tempSigma += target[diffCounter2] * log(outputLayer[j]);
}
cost[i] = -tempSigma;
}
I can't see what is the problem with my code.
float[] derivativeSoftmax(float[] inputVector){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * (1 - softmax(inputVector)[i]);
}
return result;
}
I believe this is wrong, given the derivative of the softmax as defined on wikipedia.
float[] derivativeSoftmax(float[] inputVector, int k){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * ((i==k ? 1 : 0) - softmax(inputVector)[k]);
}
return result;
}
You should be taking the derivative at an index with respect to some other index. The equation as you have it, which is x*(1-x) doesn't make a lot of sense. But I may be wrong.
I am trying to convert a code in MatLab to OpenCV but I am stuck about the following lines as I don't know much programming
MatLab code:
[indx_row, indx_col] = find(mask ==1);
Indx_Row = indx_row;
Indx_Col = indx_col;
for ib = 1:nB;
istart = (ib-1)*n + 1;
iend = min(ib*n, N);
indx_row = Indx_Row(istart:iend);
indx_col = Indx_Col(istart:iend);
openCV code:
vector <Point> index_rowCol;
for(int i=0; i<mask.rows; i++)
{
for(int j=0; j<mask.cols; j++)
{
if( mask.at<float>(i,j) == 1 )
{
Point pixel;
pixel.x = j;
pixel.y = i;
index_rowCol.push_back(pixel);
}
}
}
//Code about the "for loop" in MatLab code
for(int ib=0 ; ib<nB; ib++)
{
int istart = (ib-1)*n;
int iend = std::min( ib*n, N );
index_rowCol.clear();// Clearing the "index_rowCol" so that we can fill it again from "istart" to "iend"4
for(int j = istart; j<iend; j++)
{
index_rowCol.push_back( Index_RowCol[j] );
}
}
I am unable to understand if it is ok or not?
I think that there is mistake in usage of min function.
Here
for ib = 1:nB;
istart = (ib-1)*n + 1;
iend = min(ib*n, N);
ib - is array [1,2,3..nB] and you compare each value with N. As the result you also get array.
So as result:
ib - is array, istart - is array and iend also an array.
In C++ implementation
for(int ib=0 ; ib<nB; ib++)
{
int istart = (ib-1)*n;
int iend = std::min( ib*n, N );
you work with scalars (here ib,istars and iend are scalars).
For better understand how the code above works use step-by-step debugging. (Set breakpoint and run the code then press (F10 key-for matlab) )
I am working on a game that requires a tile map saved in multidimensional array. In my game I have all of these maps made that are NSStrings with all the saved values needed, I'm looking to save all the 256 values of the NSString into an int 16x16 multidimensional array.
Here is my current code however it doesn't work
-(void)LoadMap:(NSString*)mapString
{
for(int h = 0; h < kMapSize; h++)
{
for(int w = 0; w < kMapSize; w++)
{
map[w][h] = [[mapString substringWithRange:NSMakeRange((h)+(w*kMapSize), 1)] intValue];
}
}
}
Any help would be great thankyou :)
There are two potential errors:
kMapSize is possibly not equal to 16. This variable has a misleading name since the casual reader would think it should be 256. Perhaps rename it kMapWidth.
mapString is possibly not 256 characters long. You might want to check [mapString length] at the beginning of LoadMap.
Below code works without problem. Maybe the second row is the reason of error.
// initial data
NSInteger kMapSize = 3;
char map[3][3]; <-- this row ?
NSString *mapString = #"000111222";
// put initial data into 'map' array
for(int h = 0; h < kMapSize; h++) {
for(int w = 0; w < kMapSize; w++) {
map[w][h] = [[mapString substringWithRange:NSMakeRange((h)+(w*kMapSize), 1)] intValue];
}
}
// confirm whether the data is stored successfully
for (int h = 0; h < kMapSize; h++) {
for (int w = 0; w < kMapSize; w++) {
NSLog(#"%d", map[w][h]);
}
}
I have an image of connected components(circles filled).If i want to segment them i can use watershed algorithm.I prefer writing my own function for watershed instead of using the inbuilt function in OPENCV.I have successfu How do i find the regionalmax of objects using opencv?
I wrote a function myself. My results were quite similar to MATLAB, although not exact. This function is implemented for CV_32F but it can easily be modified for other types.
I mark all the points that are not part of a minimum region by checking all the neighbors. The remaining regions are either minima, maxima or areas of inflection.
I use connected components to label each region.
I check each region for any point belonging to a maxima, if yes then I push that label into a vector.
Finally I sort the bad labels, erase all duplicates and then mark all the points in the output as not minima.
All that remains are the regions of minima.
Here is the code:
// output is a binary image
// 1: not a min region
// 0: part of a min region
// 2: not sure if min or not
// 3: uninitialized
void imregionalmin(cv::Mat& img, cv::Mat& out_img)
{
// pad the border of img with 1 and copy to img_pad
cv::Mat img_pad;
cv::copyMakeBorder(img, img_pad, 1, 1, 1, 1, IPL_BORDER_CONSTANT, 1);
// initialize binary output to 2, unknown if min
out_img = cv::Mat::ones(img.rows, img.cols, CV_8U)+2;
// initialize pointers to matrices
float* in = (float *)(img_pad.data);
uchar* out = (uchar *)(out_img.data);
// size of matrix
int in_size = img_pad.cols*img_pad.rows;
int out_size = img.cols*img.rows;
int x, y;
for (int i = 0; i < out_size; i++) {
// find x, y indexes
y = i % img.cols;
x = i / img.cols;
neighborCheck(in, out, i, x, y, img_pad.cols); // all regions are either min or max
}
cv::Mat label;
cv::connectedComponents(out_img, label);
int* lab = (int *)(label.data);
in = (float *)(img.data);
in_size = img.cols*img.rows;
std::vector<int> bad_labels;
for (int i = 0; i < out_size; i++) {
// find x, y indexes
y = i % img.cols;
x = i / img.cols;
if (lab[i] != 0) {
if (neighborCleanup(in, out, i, x, y, img.rows, img.cols) == 1) {
bad_labels.push_back(lab[i]);
}
}
}
std::sort(bad_labels.begin(), bad_labels.end());
bad_labels.erase(std::unique(bad_labels.begin(), bad_labels.end()), bad_labels.end());
for (int i = 0; i < out_size; ++i) {
if (lab[i] != 0) {
if (std::find(bad_labels.begin(), bad_labels.end(), lab[i]) != bad_labels.end()) {
out[i] = 0;
}
}
}
}
int inline neighborCleanup(float* in, uchar* out, int i, int x, int y, int x_lim, int y_lim)
{
int index;
for (int xx = x - 1; xx < x + 2; ++xx) {
for (int yy = y - 1; yy < y + 2; ++yy) {
if (((xx == x) && (yy==y)) || xx < 0 || yy < 0 || xx >= x_lim || yy >= y_lim)
continue;
index = xx*y_lim + yy;
if ((in[i] == in[index]) && (out[index] == 0))
return 1;
}
}
return 0;
}
void inline neighborCheck(float* in, uchar* out, int i, int x, int y, int x_lim)
{
int indexes[8], cur_index;
indexes[0] = x*x_lim + y;
indexes[1] = x*x_lim + y+1;
indexes[2] = x*x_lim + y+2;
indexes[3] = (x+1)*x_lim + y+2;
indexes[4] = (x + 2)*x_lim + y+2;
indexes[5] = (x + 2)*x_lim + y + 1;
indexes[6] = (x + 2)*x_lim + y;
indexes[7] = (x + 1)*x_lim + y;
cur_index = (x + 1)*x_lim + y+1;
for (int t = 0; t < 8; t++) {
if (in[indexes[t]] < in[cur_index]) {
out[i] = 0;
break;
}
}
if (out[i] == 3)
out[i] = 1;
}
The following listing is a function similar to Matlab's "imregionalmax". It looks for at most nLocMax local maxima above threshold, where the found local maxima are at least minDistBtwLocMax pixels apart. It returns the actual number of local maxima found. Notice that it uses OpenCV's minMaxLoc to find global maxima. It is "opencv-self-contained" except for the (easy to implement) function vdist, which computes the (euclidian) distance between points (r,c) and (row,col).
input is one-channel CV_32F matrix, and locations is nLocMax (rows) by 2 (columns) CV_32S matrix.
int imregionalmax(Mat input, int nLocMax, float threshold, float minDistBtwLocMax, Mat locations)
{
Mat scratch = input.clone();
int nFoundLocMax = 0;
for (int i = 0; i < nLocMax; i++) {
Point location;
double maxVal;
minMaxLoc(scratch, NULL, &maxVal, NULL, &location);
if (maxVal > threshold) {
nFoundLocMax += 1;
int row = location.y;
int col = location.x;
locations.at<int>(i,0) = row;
locations.at<int>(i,1) = col;
int r0 = (row-minDistBtwLocMax > -1 ? row-minDistBtwLocMax : 0);
int r1 = (row+minDistBtwLocMax < scratch.rows ? row+minDistBtwLocMax : scratch.rows-1);
int c0 = (col-minDistBtwLocMax > -1 ? col-minDistBtwLocMax : 0);
int c1 = (col+minDistBtwLocMax < scratch.cols ? col+minDistBtwLocMax : scratch.cols-1);
for (int r = r0; r <= r1; r++) {
for (int c = c0; c <= c1; c++) {
if (vdist(Point2DMake(r, c),Point2DMake(row, col)) <= minDistBtwLocMax) {
scratch.at<float>(r,c) = 0.0;
}
}
}
} else {
break;
}
}
return nFoundLocMax;
}
I do not know if it is what you want, but in my answer to this post, I gave some code to find local maxima (peaks) in a grayscale image (resulting from distance transform).
The approach relies on subtracting the original image from the dilated image and finding the zero pixels).
I hope it helps,
Good luck
I had the same problem some time ago, and the solution was to reimplement the imregionalmax algorithm in OpenCV/Cpp. It is not that complicated, because you can find the C++ source code of the function in the Matlab distribution. (somewhere in toolbox). All you have to do is to read carefully and understand the algorithm described there. Then rewrite it or remove the matlab-specific checks and you'll have it.
I want to generate the random numbers using this loop. When i runs the apps at everytime, i want to generate the random numbers without duplicates.
Eg:
for(int i = 0; i < 5; i++)
{
// int d = random() % i;
NSLog(#"The Value %d",i);
NSLog(#"The random Number %d",i);
}
Actual Number Random Number
1 4
2 5
3 2
4 1
5 3
It's Random Permutation Generation problem. Read this: http://www.techuser.net/randpermgen.html
The main idea is (in pseudo code):
for (i=1 to n) ar[i] = i;
for (i=1 to n) swap(ar[i], ar[Random(i,n)]);
In your case:
int ar[5],i,d,tmp;
for(i = 0; i < 5; i++) ar[i] = i+1;
for(i = 0; i < 5; i++) {
d = i + (random()%(5-i));
tmp = ar[i];
ar[i] = ar[d];
ar[d] = tmp;
NSLog(#"%d",ar[i]);
}
Can be something like this,
int rand[5] = {0};
int max = 5;
for(int i = 0; i < max; i++){
int r = random() % max + 1;
while([self foundNumber:r inArray:rand limit:i){
r = random() % max + 1;
}
rand[i] = r;
}
- (BOOL) foundNumber:r inArray:rand limit:l {
for(int i = 0; i < l; i++){
if(rand[i] == r) return YES;
}
return NO;
}