I am using scala spark. I have a dataframe that 2 column each containing a Vector with the same cardinality/size. I want to find the distance between each element of the 2 Vectors and put the results in a vector in another column of the dataframe.
Example: [1, 3, 5, -2] - [-2, 5, 0, 1] = [3, 2, 5, 3]
I found sqdist method that can get me the sum of the square distances between 2 Vectors but how do I get the individual distances of each elements in the vector.
Related
Maybe the question is a little bit confused, I'll make an example below.
Let's say I have a 3 matrices a, b, c with same size.
a = [2, 5; 6, 9];
b = [3, 3; 8, 1];
c = [5, 5; 2, 7];
How can I get the new matrix max with each max element in all three matrices?
max = [5, 5; 8, 9]
I know I could create logical matrix like a>b and then do the math, calc it out, is there any other more efficient way to do it?
You can concatenate the matrices into one 2x2x3 matrix using
d=cat(3,a,b,c)
and then use max-function to get your desired output:
maxValues=max(d,[],3)
The 3rd input to max defines along which dimension of the first input you want to find the maximum value.
lets say that we have the next series of arrays:
A = [1, 2, -2, -24];
B = [1, 4, -7, -2];
C = [3, 1, -7, -14];
D = [11, 4, -7, -1];
E = [1, 2, -3, -4];
F = [5, 14, -17, -12];
I would like to create two arrays,
the first will be the maximum of each column for all arrays,
i.e.
Maxi = [11,14,-2 -1];
the second will be the minimum of each column for all arrays
i.e.
Mini= [1,1,-17 -24];
I am trying all day, using loops, with max, and abs but I cant make it work
in my problem have a matrix (100,200), so with the above example i am trying to easily approach the problem. The ultimate goal is to get a kinda fitting of the 100 y_lines of 200 x_points. The idea is to calculate two lines (i.e. max,min), that will be the "visual" boarders of all lines (maximum and minimum values for each x). The next step will be to calculate an array of the average of these two arrays, so in the end will be a line between all lines.
any help is more than welcome!
How about this?
Suppose you stack all the row vectors , namely A,B...,F as
arr=[A;B;C;D;E;F];% stack the vectors
And then use the max(), min() and mean() functions provided by Matlab. That is,
Maxi = max(arr); % Maxi is a row vector carrying the max of each column of arr
Mini = min(arr);
Meani = mean(arr);
You just have to stack them as shown above. But if you have 100s of row vectors, use a loop to stack them into array arr as shown above.
So say I have the below matrix
[1, 2, 3,
4, 5, 6,
7, 8, 9]
And I have a vector [1,3]
I want to access the 1st and 3rd row which would return
[1,2,3
7,8,9]
I need to be able to scale this up to about 1000 rows being grabbed based on values in the vector.
if A is your matrix and v your vector of index, you just have to do A(v,:)
Lets say I have the following vector in Matlab:
V = [4, 5, 5, 7];
How can I list (in a n-by-2 matrix for example) all the index pairs corresponding to unequal elements in the vector. For example for this particular vector the index pairs would be:
index pairs (1, 2) and (1, 3) corresponding to element pair (4,5)
index pair (1, 4) corresponding to element pair (4,7)
index pairs (2, 4) and (3, 4) corresponding to element pair (5,7)
The reason I need this is because I have a cost-function which takes a vector such as V as input and produces a cost-value.
I want to see how does the random swapping of two differing elements in the vector affect the cost value (using this for steepest descent hill climbing).
The order of the index pairs doesn't also matter. For my purposes (1,2) is the same as (2,1).
For example if my cost-function was evalCost(), then I could have V = [4, 5, 5, 7] and
evalCost(V) = 14
whereas for W = [4, 7, 5, 5] the cost could be:
evalCost(W) = 10
How to get the list of "swapping" pair indexes in Matlab. Hope my question is clear =)
I don't understand the cost function part, but the first part is simple:
[a,b]=unique(V)
C = combnk(b,2)
C contains the indices, and V(C) the values:
C = combnk(b,2)
C =
1 2
1 4
2 4
V(C)
ans =
4 5
4 7
5 7
Use bsxfun and then the two-ouput version of find to get the pairs. triu is applied to the output of bsxfun to consider only one of the two possible orders.
[ii jj] = find(triu(bsxfun(#ne, V, V.')));
pairs = [ii jj];
How can I create an index-matrix that specifies which elements of a matrix to address?
So for example I have a matrix A which is 80 by 50. I know that A(1:5,:) addresses only the first 5 elements, but what if I want to multiply A with another matrix which also changes the elements to be addressed? So I want to multiply B(1,:) with A(1:5,:), and B(2,:) with A(10:15,:) and so on. Is there a smart way to specify this index-matrix where the information (1:5; 10:15, etc.) is stored?
Yes you can certainly define indices into a matrix using another matrix. Here is a simple example using a cell array to store the index list:
X =[1,2,3,4,5,6]
Idx = { [1, 2, 3], [4, 5, 6] }
Y = X( Idx{1} ) .* X( {Idx{2} )
Y = [ 4, 10, 18]