Hi I am not sure how to explain what I need but I'll try. I need a query (if there is one) for counting if a tally was present in a point (column). So all species with more than one tally in a point will count only as one.
This is how the data looks:
Sp Site Pnt1 Pnt2 Pnt3 Total
A 1 1 1 1 3
A 2 1 1 2
A 3 1 1
B 1 1 1 1 3
B 2 1 1
C 1 1 1 2
C 2 0
I want to count if the sites have tally or not and if they are repeated by points (for the same species) I want to count them as one. I would like the resulting table to look like this.
Sp Pnt1 Pnt2 Pnt3 Total
A 1 1 1 3
B 1 1 1 3
C 0 1 1 2
Thanks for all the help you can provide.
Related
I'm trying to convolve a small vector (kernel) across a longer series.
The simplest possible kernel is (-1 1), which is the equivalent of the diff operator.
My failed attempt is:
{sum (x;y)*(-1;1)} scan til 10
0 1 1 2 2 3 3 4 4 5
This doesn't work, as it supplies the previous evaluation as the right component of the binary fn. What I should be doing is evaluating the function on each pair and storing the result. The result I'm looking for is:
1 1 1 1 1 1 1 1 1
What I can't figure out is an elegant KDB way of doing this calculation.
Is there an elegant way to do this in KDB?
You could use the each prior iterator to do this - https://code.kx.com/q/ref/maps/#each-prior
q)-':[til 10]
0 1 1 1 1 1 1 1 1 1
Maybe something like:
q)1_sum(1;-1)*1 prev\til 10
1 1 1 1 1 1 1 1 1
Can you provide more examples?
I'm dealing with matrices of this format:
M =
1 1 3
1 1 1
1 2 2
1 2 1
1 2 2
2 1 5
2 1 1
2 2 3
2 2 4
2 2 2
...
What I want to do is extract sub matrices where the values in the first and second column can be grouped such that:
M1 =
1 1 3
1 1 1
M2 =
1 2 2
1 2 1
1 2 2
M3 =
2 1 5
2 1 1
...
I have been trying to think hard about how to index the matrix for this and I have a matrix available:
I =
1 1
1 2
2 1
2 2
...
that I could use for indexing. I was wondering if I could use it but I'm not 100% sure how. I don't want to use a for loop since the matrixes can be rather large and the order of complexity can become very large.
Thank you for reading!
This is easily done with unique and accumarray:
M = [ 1 1 3
1 1 1
1 2 2
1 2 1
1 2 2
2 1 5
2 1 1
2 2 3
2 2 4
2 2 2 ]; %// data
[~, ~, u] = unique(M(:,1:2), 'rows'); %// unique labels of rows based on columns 1 and 2
M_split = accumarray(u(:), (1:size(M,1)).', [], #(x){M(sort(x),:)}); %'// group rows
% // based on labels
This gives a cell array containing the partial matrices. In your example,
M_split{1} =
1 1 3
1 1 1
M_split{2} =
1 2 2
1 2 1
1 2 2
M_split{3} =
2 1 5
2 1 1
M_split{4} =
2 2 3
2 2 4
2 2 2
Consider the following table:
myTable:
a b
-------
1
2
3 10
4 50
5 30
How do I replace the empty cells of b with a zero? So the result would be:
a b
-------
1 0
2 0
3 10
4 50
5 30
Right now I'm doing:
myTable: update b:{$[x~0Ni;0;x]}'b from myTable
But I am wondering whether there is a better/easier solution for doing this.
Using the fill operator (^)
Example Table:
q) tbl:flip`a`b!(2;0N)#10?0N 0N 0N,til 3
a b
---
0 2
1 1
1 1
1
1
Fill nulls in all columns with 0:
q)0^tbl
a b
---
0 2
1 1
1 1
0 1
1 0
Fill nulls only in selective columns with 0:
q)update 0^b from tbl
a b
---
0 2
1 1
1 1
1
1 0
In some cases when you want to fill the null values from the previous non-null value, you can use fills functions.
q)tbl:flip`a`b!(2;0N)#10?0N 0N 0N,til 3
a b
---
2 1
1
1 2
0
update fills a, fills b from tbl
q)a b
---
2 1
2 1
2 1
1 2
0 2
using fills with aggregation
q)tbl:update s:5?`g`a from flip`a`b!(2;0N)#10?0N 0N 0N,til 3
a b s
-----
1 a
2 a
0 0 g
2 2 g
0 a
q)update fills a, fills b by s from tbl
a b s
-----
1 a
1 2 a
0 0 g
2 2 g
0 2 a
Let's say I create an array of structs in matlab using:
mystruct = repmat(struct('field1',1, 'field2', ones(10,1)), 10, 1 );
For my purposes (simple example aside), I would find it very useful to get a vector output from using:
myvector = mystruct(:).field2(1)
However this gives me the error:
'Scalar index required for this type of multi-level indexing.'
EDIT: What I expect to get is the first element of the ones vector, from each struct in the array, hence a 10x1 vector of '1'.
I could easily manually using a for loop go through each value in my struct and assign to myvector but that seems incredibly cumbersome and also slow. Any thoughts?
I'm assuming you're trying to collect all of the field2 vectors into myvector:
myvector = [mystruct(:).field2];
Returns:
myvector =
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
EDIT: Per your comment, you can use the above and throw out the data you don't want (myvector(2:end,:) = []; in this case). This is a pretty memory intensive way to do it though. There may be a way to pull what you want using structfun or similar but I'd need to think about how to do it.
EDIT2: Try arrayfun(#(x) x.field2(1), mystruct) and see if this returns what you're looking for.
In two step you can:
get your struct filed2 as a matrix:
foo = [mystruct.field2];
get the first row (that contains the first indices of the field2)
myvector = foo(1, :);
Here's a small example of what I want. Given the following array:
1 1 2
2 2 1
1 1 1
1 1 6
Sorted (row sum is shown in parenthesis):
1 1 6 (8)
2 2 1 (5)
1 1 2 (4)
1 1 1 (3)
Is there a quick way to achieve this in Matlab?
Since sort returns the indexes in order as well as the sorted matrix, you can use these indices to access the original data -- try this:
% some data
A = [
1 1 2;
2 2 1;
1 1 1;
1 1 6;
];
% compute the row totals
row_totals = sum(A,2);
% sort the row totals (descending order)
[sorted, row_ids] = sort(row_totals, 'descend');
% and display the original data in that order (concatenated with the sums)
disp([A(row_ids,:), row_totals(row_ids)])
>>>
1 1 6 8
2 2 1 5
1 1 2 4
1 1 1 3
The ugliest one-liner I could come up with:
>> subsref( sortrows( [sum(A,2) A], -1 ), struct('type','()','subs',{{':',1+(1:size(A,2))}}) )
ans =
1 1 6
2 2 1
1 1 2
1 1 1
Disclaimer: I don't think anyone should write this kind of code, but it's a nice practice to keep your Matlab's skills sharp.
Just do something very simple like follows
temp = [1 1 2
2 2 1
1 1 1
1 1 6];
rowSums = sum(temp,2);
[~,idx] = sort(rowSums,'descend');
output = [temp(idx,:),rowSums(idx)];
EDIT
Changed the above code to make sure the sum is appended to the last column. I did not notice that this was a requirement when I read the question initially.
I leave it for you to judge if this is uglier than #Shai's:
fliplr(diff([sortrows(fliplr(-cumsum(A,2))) zeros(size(A,1),1) ],1,2))
Let's do some matrix multiplication
>> sortrows([sum(A,2) A], -1)*[zeros(1,size(A,2)); eye(size(A,2))]
returns
ans =
1 1 6
2 2 1
1 1 2
1 1 1