Converting and an arma::mat adjacency matrix into an igraph graph in C (Rcpp) - type-conversion

I use Armadillo objects in some (Rcpp) code where I work with matrices.
The matrices are adjacency matrices and I need to quickly compute the components of the underlying network and though I could do this via igraph.
But I fail already at converting the adjacency matrix into something that can be used with igraph.
#include <RcppArmadillo.h>
#include <iostream>
#include <igraph-0.7.1\include\igraph.h>
using namespace arma;
// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::export]]
vec component_membership(const mat& adjacencymatrix) {
igraph_t g;
igraph_adjacency(&g,&adjacencymatrix,IGRAPH_ADJ_DIRECTED);
// here is more code that is immaterial to my problem
}
On compilation it complains
cannot convert 'const mat* {aka const arma::Mat<double>*}' to
'igraph_matrix_t*' for argument '2' to
'int igraph_adjacency(igraph_t*, igraph_matrix_t*, igraph_adjacency_t)'
I understand why that is the case: I believe igraph_matrix_t and arma::matrix must be fundamentally different data types. How can I convert, i.e., but how do i fix this easily?

As you suspected, igraph_matrix_t and arma::matrix are completely different types. The igraph documentation lists no methods that would make use of a C array for constructing an igraph_matrix_t, so I think one has to do it by hand. Something like this might work (totally untested!):
igraph_matrix_t *m;
int rc = igraph_matrix_init(m, mat.n_rows, mat.n_cols);
for (unsigned long j = 0; j < mat.n_cols; ++j)
for (unsigned long i = 0; i < mat.n_rows; ++i)
igraph_matrix_set(m, i, j, mat(i, j));

Following #Ralf_Stubner's suggestion I ended up using the following code. Not sure it is smart, I thought I'd share it anyways
void armamat_to_igraph_matrix(const mat &x_in, igraph_matrix_t *x_out) {
igraph_matrix_init(x_out, x_in.n_rows, x_in.n_cols);
for (unsigned long j = 0; j < x_in.n_cols; ++j)
for (unsigned long i = 0; i < x_in.n_rows; ++i)
igraph_matrix_set(x_out, i, j, x_in(i, j));
return;
}
void igraph_vector_to_armauvec(const igraph_vector_t *x_in, uvec &x_out) {
x_out = uvec(igraph_vector_size(x_in));
for (unsigned long j = 0; j < igraph_vector_size(x_in); ++j)
x_out(j) = igraph_vector_e(x_in,j);
return;
}
void igraph_vector_to_armavec(const igraph_vector_t *x_in, vec &x_out) {
x_out = vec(igraph_vector_size(x_in));
for (unsigned long j = 0; j < igraph_vector_size(x_in); ++j)
x_out(j) = igraph_vector_e(x_in,j);
return;
}

Related

How to access the results of a Magma Routine

I am trying to access the results of the eigenvalue decomposition of a general real matrix, using the magma_sgeev routine. My code is as follows -
#include <cstdlib>
#include <iostream>
#include <map>
#include <typeindex>
#include <typeinfo>
#include <magma_v2.h>
#include <random>
#define N 10
#define LDA N
#define LDVL N
#define LDVR N
/* Main program */
int main()
{
magma_init();
/* Locals */
int n = N, lda = LDA, ldvl = LDVL, ldvr = LDVR, info, lwork;
float wkopt;
float *work;
/* Local arrays */
float wr[N], wi[N], vl[LDVL * N], vr[LDVR * N];
float *a = (float *)malloc(LDA * N * sizeof(float));
for (int i = 0; i < LDA * N; i++)
a[i] = rand();
printf("Done populating matrix\n");
lwork = -1;
magma_sgeev_m(MagmaNoVec, MagmaNoVec, n, a, lda, wr, wi, vl, ldvl, vr, ldvr,
work, lwork, &info);
lwork = (int)work[0];
work = (float *)malloc(lwork * sizeof(float));
printf("%d\n", lwork);
magma_sgeev_m(MagmaNoVec, MagmaNoVec, n, a, lda, wr, wi, vl, ldvl, vr, ldvr,
work, lwork, &info);
for (int i = 0; i < N; i++)
printf("%f ", wr[i]);
std::cout << std::endl;
for (int i = 0; i < N; i++)
printf("%f ", wi[i]);
printf("\n");
if (info > 0)
{
printf("The algorithm failed to compute eigenvalues.\n");
exit(1);
}
exit(0);
magma_finalize();
}
As the documentation says, the arrays wr and wi hold the results of the computation. However, there are two issues with this code,
When I try to access the wr and wi, I get a segmentation fault. I am not aware of the inner workings of the magma library, what am I missing, and how do I fix it ?
When I increase the size of the matrix, for example from 5x5 to 10x10. I get a segmentation fault in the first call of magma_sgeev itself. What am I missing
TIA

how to implement Softmax function for neural networks in processing 3 environment?

I tried programming a neural network in processing IDE.
I managed to do it quite well, until I tried using the MNIST handwritten digits data set. I tried the iris data set and few others from UCI machine learning repository, but when I used the MNIST data set it didn't worked. for some reason all of the outputs approached zero with time, and that caused the total error to be always equal to 1. I am almost sure that my problem is the activation function; so I tried using softmax for classification, but it wasn't very successful. I got the same results. I think maybe I should have use a different loss function, so I tried the negative log probability according to this video. the results now are the same cost value for each output neuron, and the sum of the outputs is not 1 as it should be.
Here are the functions for each part of the code that I have changed (I prefer not to share the full code because it's long and messy, and not really helpful):
softmax:
float[] softmax(float[] inputVector){
float[] result = new float[inputVector.length];
float sigma = 0;
for(int i = 0; i < inputVector.length; i++){
sigma += exp(inputVector[i]);
}
for(int i = 0; i < result.length; i++){
result[i] = exp(inputVector[i]) / sigma;
}
return result;
}
derivative of softmax:
float[] derivativeSoftmax(float[] inputVector){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * (1 - softmax(inputVector)[i]);
}
return result;
}
loss function:
for(int i = 0; i < outputNeuronsNumber; i++){
float tempSigma = 0;
for(int j = 0; j < outputNeuronsNumber; j++){
tempSigma += target[diffCounter2] * log(outputLayer[j]);
}
cost[i] = -tempSigma;
}
I can't see what is the problem with my code.
float[] derivativeSoftmax(float[] inputVector){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * (1 - softmax(inputVector)[i]);
}
return result;
}
I believe this is wrong, given the derivative of the softmax as defined on wikipedia.
float[] derivativeSoftmax(float[] inputVector, int k){
float[] result = new float[inputVector.length];
for(int i = 0; i < result.length; i++){
result[i] = softmax(inputVector)[i] * ((i==k ? 1 : 0) - softmax(inputVector)[k]);
}
return result;
}
You should be taking the derivative at an index with respect to some other index. The equation as you have it, which is x*(1-x) doesn't make a lot of sense. But I may be wrong.

where is cula "culaSgesv" answer for X?

I just downloaded Cula and I want to use it's implemented functions for solving system of linear equation I looked into Examples Directory and I saw below code but it's very confusing when they want to obtain X solution of A*X=B they just copy B in X and since A is identity diagonal matrix so the answer IS, "B" and in this line of code nothing happens
status = culaSgesv(N, NRHS, A, N, IPIV, X, N);
(changing X to B didn't help!)
would you please tell me whats going on? Please tell me how can I get the answer "X" from this?
if anyone need any further information please just tell me.
#ifdef CULA_PREMIUM
void culaDoubleExample()
{
#ifdef NDEBUG
int N = 4096;
#else
int N = 780;
#endif
int NRHS = 1;
int i;
culaStatus status;
culaDouble* A = NULL;
culaDouble* B = NULL;
culaDouble* X = NULL;
culaInt* IPIV = NULL;
culaDouble one = 1.0;
culaDouble thresh = 1e-6;
culaDouble diff;
printf("-------------------\n");
printf(" DGESV\n");
printf("-------------------\n");
printf("Allocating Matrices\n");
A = (culaDouble*)malloc(N*N*sizeof(culaDouble));
B = (culaDouble*)malloc(N*sizeof(culaDouble));
X = (culaDouble*)malloc(N*sizeof(culaDouble));
IPIV = (culaInt*)malloc(N*sizeof(culaInt));
if(!A || !B || !IPIV)
exit(EXIT_FAILURE);
printf("Initializing CULA\n");
status = culaInitialize();
checkStatus(status);
// Set A to the identity matrix
memset(A, 0, N*N*sizeof(culaDouble));
for(i = 0; i < N; ++i)
A[i*N+i] = one;
// Set B to a random matrix (see note at top)
for(i = 0; i < N; ++i)
B[i] = (culaDouble)rand();
memcpy(X, B, N*sizeof(culaDouble));
memset(IPIV, 0, N*sizeof(culaInt));
printf("Calling culaDgesv\n");
DWORD dw1 = GetTickCount();
status = culaDgesv(N, NRHS, A, N, IPIV, X, N);
DWORD dw2 = GetTickCount();
cout<<"Time difference is "<<(dw2-dw1)<<" milliSeconds"<<endl;
if(status == culaInsufficientComputeCapability)
{
printf("No Double precision support available, skipping example\n");
free(A);
free(B);
free(IPIV);
culaShutdown();
return;
}
checkStatus(status);
printf("Verifying Result\n");
for(i = 0; i < N; ++i)
{
diff = X[i] - B[i];
if(diff < 0.0)
diff = -diff;
if(diff > thresh)
printf("Result check failed: i=%d X[i]=%f B[i]=%f", i, X[i], B[i]);
}
printf("Shutting down CULA\n\n");
culaShutdown();
free(A);
free(B);
free(IPIV);
}
You mention Sgesv but the sample code you have shown is for Dgesv. Nevertheless, the answer is the same.
According to the Netlib LAPACK reference, the B matrix of RHS vectors is passed to the function as the 6th parameter:
[in,out] B
B is DOUBLE PRECISION array, dimension (LDB,NRHS)
On entry, the N-by-NRHS matrix of right hand side matrix B.
On exit, if INFO = 0, the N-by-NRHS solution matrix X.
And the X matrix is returned in the same parameter location. So B when passed to the function contains the NxNRHS right-hand-side vectors, and the same parameter returns the X result.
In the code you have shown, they are actually passing a variable called X and after the result is returned (in the same variable X) they are comparing it against a variable named B which is perhaps confusing, but the concept is the same.
Since the A matrix in the example is the identity matrix, the correct solution of Ax = b is x=b
For the general case, you should pass your B matrix of RHS vectors in the 6th parameter location. Upon completion of the function, the result (X) will be returned in the same parameter.

loop unrolling for matrix multiplication

I need to make a good implementation for matrix multiplication better than the naive method
here is the methods i used :
1- removed false dependencies which made the performance a lot better
2- used a recursive approach
and then there is something i need to try loop unrolling. The thing is each time i used it , it makes the performance worst i can't find an explanation for it
i need help here is the code
for (i = 0; i < M; i++)
for (j = 0; j < N; j++) {
double sum = 0;
#pragma unroll(5)
for (k = 0; k < K; k++)
{
sum += A[i + k*LDA] * B[k + j*LDB];
}
C[i + j*LDC] = sum ;
}

Resilient backpropagation neural network - question about gradient

First I want to say that I'm really new to neural networks and I don't understand it very good ;)
I've made my first C# implementation of the backpropagation neural network. I've tested it using XOR and it looks it work.
Now I would like change my implementation to use resilient backpropagation (Rprop - http://en.wikipedia.org/wiki/Rprop).
The definition says: "Rprop takes into account only the sign of the partial derivative over all patterns (not the magnitude), and acts independently on each "weight".
Could somebody tell me what partial derivative over all patterns is? And how should I compute this partial derivative for a neuron in hidden layer.
Thanks a lot
UPDATE:
My implementation base on this Java code: www_.dia.fi.upm.es/~jamartin/downloads/bpnn.java
My backPropagate method looks like this:
public double backPropagate(double[] targets)
{
double error, change;
// calculate error terms for output
double[] output_deltas = new double[outputsNumber];
for (int k = 0; k < outputsNumber; k++)
{
error = targets[k] - activationsOutputs[k];
output_deltas[k] = Dsigmoid(activationsOutputs[k]) * error;
}
// calculate error terms for hidden
double[] hidden_deltas = new double[hiddenNumber];
for (int j = 0; j < hiddenNumber; j++)
{
error = 0.0;
for (int k = 0; k < outputsNumber; k++)
{
error = error + output_deltas[k] * weightsOutputs[j, k];
}
hidden_deltas[j] = Dsigmoid(activationsHidden[j]) * error;
}
//update output weights
for (int j = 0; j < hiddenNumber; j++)
{
for (int k = 0; k < outputsNumber; k++)
{
change = output_deltas[k] * activationsHidden[j];
weightsOutputs[j, k] = weightsOutputs[j, k] + learningRate * change + momentumFactor * lastChangeWeightsForMomentumOutpus[j, k];
lastChangeWeightsForMomentumOutpus[j, k] = change;
}
}
// update input weights
for (int i = 0; i < inputsNumber; i++)
{
for (int j = 0; j < hiddenNumber; j++)
{
change = hidden_deltas[j] * activationsInputs[i];
weightsInputs[i, j] = weightsInputs[i, j] + learningRate * change + momentumFactor * lastChangeWeightsForMomentumInputs[i, j];
lastChangeWeightsForMomentumInputs[i, j] = change;
}
}
// calculate error
error = 0.0;
for (int k = 0; k < outputsNumber; k++)
{
error = error + 0.5 * (targets[k] - activationsOutputs[k]) * (targets[k] - activationsOutputs[k]);
}
return error;
}
So can I use change = hidden_deltas[j] * activationsInputs[i] variable as a gradient (partial derivative) for checking the sing?
I think the "over all patterns" simply means "in every iteration"... take a look at the RPROP paper
For the paritial derivative: you've already implemented the normal back-propagation algorithm. This is a method for efficiently calculate the gradient... there you calculate the δ values for the single neurons, which are in fact the negative ∂E/∂w values, i.e. the parital derivative of the global error as function of the weights.
so instead of multiplying the weights with these values, you take one of two constants (η+ or η-), depending on whether the sign has changed
The following is an example of a part of an implementation of the RPROP training technique in the Encog Artificial Intelligence Library. It should give you an idea of how to proceed. I would recommend downloading the entire library, because it will be easier to go through the source code in an IDE rather than through the online svn interface.
http://code.google.com/p/encog-cs/source/browse/#svn/trunk/encog-core/encog-core-cs/Neural/Networks/Training/Propagation/Resilient
http://code.google.com/p/encog-cs/source/browse/#svn/trunk
Note the code is in C#, but shouldn't be difficult to translate into another language.