I to do some indexing, something like what follows:
for c=1:size(params,1)
for d=1:size(data,2)
if params(c,8)==1
value1(c,d)=data(params(c,11),d);
elseif params(c,8)==2
value2(c,d)=data(params(c,11),d);
elseif params(c,8)==3
value3(c,d)=data(params(c,11),d);
end
end
end
The problems with this is that if we have params(:,8)=1,3,1,3,2,3,1... then value1 will contain all zeros in rows 2, 4, 5, 6, etc. These are the rows that do not have 1 in column 8 in params. Similarly, value2 will contains all zeros in rows 1, 2, 3, 4, 6, 7... and value3 will contain all zeros in row 1, 3, 5, 7, .... Could anyone tell me how to index so I don't have 'gaps' of zeros in between rows? Thanks!
Edit; below is a sample dataset:
data (1080x15 double)
168 432 45 86
170 437 54 82
163 423 52 83
178 434 50 84
177 444 42 87
177 444 58 85
175 447 48 77
184 451 59 86
168 455 52 104
174 437 62 88
175 443 55 85
179 456 51 92
168 450 73 82
175 454 60 68
params (72x12 double - we are interested in only column 8 and 11 ) so I'm showing only column 8-11 for the sake of space:
1 10 15 1
3 12 16 16
2 10 15 32
3 12 16 47
1 8 14 63
2 10 15 77
2 8 14 92
3 10 15 106
1 12 16 121
3 8 14 137
2 10 15 151
The expected output for value1, value2, and value3 should be 24x15. This is because there are 15 columns in data and value 1, 2, 3 occur 24 times each in column 8 in params.
You can use bsxfun to avoid for-loop (note that it is actually not vertorizing):
value1 = bsxfun(#times,data(params(:,11),:),(params(:,8)==1));
value2 = bsxfun(#times,data(params(:,11),:),(params(:,8)==2));
value3 = bsxfun(#times,data(params(:,11),:),(params(:,8)==3));
But it still gives you the results with zero rows. So you can remove zero-rows by:
value1(all(value1==0,2),:)=[];
value2(all(value2==0,2),:)=[];
value3(all(value3==0,2),:)=[];
You can also use above commands to remove zero-rows in your results without using bsxfun. It is not always good to loose the transparency.
Related
I want to change the base of a multiplication table to another base.
If I use
disp(dec2base((1:10).*(1:10)',7))
the numbers come flowing out individually. However I want them to stay in the exact position in the given matrix.
The numerical base is a display issue, numbers are always stored and manipulated in base 2 internally. So all you need to do is write a loop that displays the numbers in they way you want to. For example:
for ii=1:10
for jj=1:10
fprintf('%6s',dec2base(ii*jj,7));
end
fprintf('\n');
end
Output:
1 2 3 4 5 6 10 11 12 13
2 4 6 11 13 15 20 22 24 26
3 6 12 15 21 24 30 33 36 42
4 11 15 22 26 33 40 44 51 55
5 13 21 26 34 42 50 55 63 101
6 15 24 33 42 51 60 66 105 114
10 20 30 40 50 60 100 110 120 130
11 22 33 44 55 66 110 121 132 143
12 24 36 51 63 105 120 132 144 156
13 26 42 55 101 114 130 143 156 202
Storing base-7 representation of numbers as string array:
M = (1:10).*(1:10)';
out = strings(size(M));
for jj = 1:size(M,2)
for ii = 1:size(M,1)
out(ii,jj) = dec2base(M(ii,jj) ,7);
end
end
Sorry if this is a newbie question again.
I am trying to replicate the functionality of interfaces as seen in c++, rust etc. in kdb as is shown in a simple demonstration below:
q).iface.a.fun:{x*y+z}
q).iface.b.fun:{x*x+y+z}
q)ifaces:`a`b; // for demonstration purposes
q)tab:([]time:`datetime$();kind:`ifaces$();x:`long$();y:`long$();z:`long$());
q)n:10;
q)tab,:flip(n#.z.z;n?ifaces;n?10;n?10;n?10)
Now you would assume that the kind would be able to reference the `a`b fun methods of the iface interface as follows:
q)?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;`fun;`x;`y;`z))]
evaluation error:
fun
[0] ?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;`fun;`x;`y;`z))]
^
Obviously the functional nature of the select inhibits referencing the fun method on account of the symbol type field declarations.
You can avert this error by using enlist as follows:
q)?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;enlist`fun;`x;`y;`z))]
max ifaceval ..
-----------------------------------------------------------------------------..
9 77 154 95 65 0 128 153 126 60 49 77 154 95 65 0 128 153 126 60 49 77 154 ..
However this duplicates the result of fun for each row.
How might one effectively go about this without getting the above malformed responses?
Thanks again.
Selecting ifaceval first will ensure each row is returned. max x is a scalar, which forces all the ifaceval entries into one row. The scalar will be expanded across all rows if a vector column precedes it.
q)?[`tab;();0b;`ifaceval`max!((`.iface;`kind;enlist`fun;`x;`y;`z);(max;`x))]
ifaceval max
-------------------------------------
160 11 126 28 32 60 76 10 112 168 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
160 11 126 28 32 60 76 10 112 168 8
96 10 77 24 16 35 60 6 63 104 8
160 11 126 28 32 60 76 10 112 168 8
160 11 126 28 32 60 76 10 112 168 8
160 11 126 28 32 60 76 10 112 168 8
I'm not sure if this is exactly what you're looking for though. If you want to calculate ifaceval for each row in the table, this should work.
q)?[tab;();0b;`ifaceval`max!(((';(`.iface;::;enlist`fun));`kind;`x;`y;`z);(max;`x))]
ifaceval max
------------
160 8
10 8
77 8
24 8
16 8
60 8
60 8
10 8
112 8
168 8
One point to make is that it's probably best to avoid using kdb keywords for column names. Although it works in functional queries, it does not for qSQL ones.
q)select max:max x from tab
'assign
[0] select max:max x from tab
^
I have a matrix A of size 2500 x 500. I want to sum each 10 columns and get the result as a matrix B of size 2500 x 50. That is, the first column of B is the sum of the first 10 columns of A, the second column of B is the sum of second 10 columns of A, and so on.
How can I do that without a for loop? Since I have to do that hundreds of times and it is highly time consuming to do that using for loop.
First, we "block reshape" A, such that we have the desired number of columns. Therefore, we shamelessly steal the code from the great Divakar, and put in some minimal effort to generalize it. Then, we just need to sum along the second axis, and reshape to the original form.
Here's an example with five columns to be summed:
% Sample input data
A = reshape(1:100, 10, 10).'
[r, c] = size(A);
% Number of columns to be summed
n_cols = 5;
% Block reshape to n_cols, see https://stackoverflow.com/a/40508999/11089932
B = reshape(permute(reshape(A, r, n_cols, []), [1, 3, 2]), [], n_cols);
% Sum along second axis
B = sum(B, 2);
% Reshape to original form
B = reshape(B, r, c / n_cols)
That's the output:
A =
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99 100
B =
15 40
65 90
115 140
165 190
215 240
265 290
315 340
365 390
415 440
465 490
Hope that helps!
This can be done with splitapply. An advantage of this approach is that it works even if the group size does not divide the number of columns (the last group is smaller):
A = reshape(1:120, 12, 10).'; % example 10×12 data (borrowed from HansHirse)
n_cols = 5; % number of columns to sum over
result = splitapply(#(x)sum(x,2), A, ceil((1:size(A,2))/n_cols));
In this example,
A =
1 2 3 4 5 6 7 8 9 10 11 12
13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36
37 38 39 40 41 42 43 44 45 46 47 48
49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72
73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96
97 98 99 100 101 102 103 104 105 106 107 108
109 110 111 112 113 114 115 116 117 118 119 120
result =
15 40 23
75 100 47
135 160 71
195 220 95
255 280 119
315 340 143
375 400 167
435 460 191
495 520 215
555 580 239
I have a matrix A
A=[f magic(10)]
A=
931142103 92 99 1 8 15 67 74 51 58 40
931142103 98 80 7 14 16 73 55 57 64 41
931142103 4 81 88 20 22 54 56 63 70 47
459200101 85 87 19 21 3 60 62 69 71 28
459200101 86 93 25 2 9 61 68 75 52 34
459200101 17 24 76 83 90 42 49 26 33 65
459200101 23 5 82 89 91 48 30 32 39 66
37833100 79 6 13 95 97 29 31 38 45 72
37833100 10 12 94 96 78 35 37 44 46 53
37833100 11 18 100 77 84 36 43 50 27 59
The first column are firm codes. The rest columns are firms' data, with each row referring to the firm in Column 1 in a given year. Notice that years may not be balance for every firms.
I would like to subtract sub-matrices according to the first column. For instance, for A(1:3,2:11) for 931142103:
A(1:3,2:11)
ans =
92 99 1 8 15 67 74 51 58 40
98 80 7 14 16 73 55 57 64 41
4 81 88 20 22 54 56 63 70 47
Same as 459200101 (which would be A(4:7,2:11)) and A(8:10,2:11) for 37833100.
I get a sense that the code should like this:
indices=find(A(:,1));
obs=size(A(:,1));
for i=1:obs,
if i==indices(i ??)
A{i}=A(??,2:11);
end
end
I have difficulties in indexing these complicated codes: 459200101 and 37833100 in order to gather them together. And how can I write the rows of my submatrix A{i}?
Thanks so much!
One approach with arrayfun -
%// Get unique entries from first column of A and keep the order
%// with 'stable' option i.e. don't sort
unqA1 = unique(A(:,1),'stable')
%// Use arrayfun to select each such submatrix and store as a cell
%// in a cell array, which is the final output
outA = arrayfun(#(n) A(A(:,1)==unqA1(n),:),1:numel(unqA1),'Uni',0)
Or this -
[~,~,row_idx] = unique(A(:,1),'stable')
outA = arrayfun(#(n) A(row_idx==n,:),1:max(row_idx),'Uni',0)
Finally, you can verify results with a call to celldisp(outA)
If values in column 1 always appear grouped (as in your example), you can use mat2cell as follows:
result = mat2cell(A, diff([0; find(diff(A(:,1))); size(A,1)]));
If they don't, just sort the rows of A according to column 1 before applying the above:
A = sortrows(A,1);
result = mat2cell(A, diff([0; find(diff(A(:,1))); size(A,1)]));
If you don't mind the results internally not being ordered, you can use accumarray for this:
[~,~,I] = unique(A(:,1),'stable');
partitions = accumarray(I, 1:size(A,1), [], #(I){A(I,2:end)});
for a matrix A (10x100000) containing numbers between 1 and 100, how to interchange some elements of A by other values of A in both directions?
example:
replace numbers [5 7 9 18 55 4] by [47 78 41 1 99 98] and [47 78 41 1 99 98] by [5 7 9 18 55 4]
Use the two outputs of ismember:
n1 = [1 2 3]; %// first set of numbers
n2 = [4 5 6]; %// second set of numbers
[v1, i1] = ismember(A,n1);
[v2, i2] = ismember(A,n2);
A(v1) = n2(i1(v1));
A(v2) = n1(i2(v2));
Example:
>> A = randi(8,4,5)
A =
2 2 8 4 6
2 5 3 8 2
5 4 3 2 5
4 3 2 3 4
is transformed into
A =
5 5 8 1 3
5 2 6 8 5
2 1 6 5 2
1 6 5 6 1
bsxfun based approach -
%// Input matrix
A = randi(100,10,10)
vec1 = [5 7 9 18 55 4 , 47 78 41 1 99 98]; %// Numbers to be replaced
vec2 = [47 78 4 1 99 98, 5 7 9 18 55 4]; %// Numbers to be used as replacements
[v1,v2] = max(bsxfun(#eq,A(:),vec1),[],2);
A(find(v1)) = vec2(v2(v1))
Sample run -
Input A
A =
27 37 27 59 37 13 55 45 29 16
84 41 58 46 75 39 75 51 49 16
100 37 88 87 71 82 85 54 69 16
65 47 7 67 71 99 17 86 21 9
71 51 45 36 1 87 91 68 61 46
94 92 9 35 38 9 11 81 33 67
69 21 57 26 91 34 75 54 89 84
57 34 54 96 32 24 73 96 14 80
39 58 77 30 60 32 72 7 11 72
64 49 24 16 30 99 14 55 96 48
Output A
A =
27 37 27 59 37 13 99 45 29 16
84 9 58 46 75 39 75 51 49 16
100 37 88 87 71 82 85 54 69 16
65 5 78 67 71 55 17 86 21 4
71 51 45 36 18 87 91 68 61 46
94 92 4 35 38 4 11 81 33 67
69 21 57 26 91 34 75 54 89 84
57 34 54 96 32 24 73 96 14 80
39 58 77 30 60 32 72 78 11 72
64 49 24 16 30 55 14 99 96 48
As can be seen, the 7s from (4,3) and (9,8) in the original A are replaced by 78s and 47 in (4,2) by 5.
Matlab is a strange and mysterious place. Searching through the documentation I found a function called changem in the Mapping toolbox. I've never used it, but apparently if you have your original matrix A and two substitution vectors v1 and v2:
v1 = [ 5 7 9 18 55 4];
v2 = [47 78 41 1 99 98];
All you have to do is:
B = changem(A, [v1 v2], [v2 v1]);