Let's say I have the following,
q)add2:{[x]:x+2};
q)Bidcols:`Bid1px`Bid2px`Bid3px;
q)table:([]time:9 11;Bid1px:4 5;Bid2px:7 3;Bid3px:6 8);
time Bid1px Bid2px Bid3px
-------------------------
9 4 7 6
11 5 3 8
and I want to apply this add2 function to each cols of the table like the below
q)table:update Bid1px:add2'[Bid1px],Bid2px:add2'[Bid2px],Bid3px:add2'[Bid3px] from table;
time Bid1px Bid2px Bid3px
-------------------------
9 6 9 8
11 7 5 10
My questions are:
Is there a way to do this using Bidcols?
What are the other efficient ways to achieve this?
Thanks in advance.
You can do this using a function select:
q)?[table; (); 0b; `time`Bid1px ! (`time; (each; add2; `Bid1px))]
time Bid1px
-----------
9 6
11 7
For (1), if you want to do it using Bidcols:
q)?[table; (); 0b; ] (cols table) ! {$[x in Bidcols; (each; add2; x); x]} each cols table
time Bid1px Bid2px Bid3px
-------------------------
9 6 9 8
11 7 5 10
I'm not sure what you mean for (2)? Are you asking for the most efficient way to do this?
Functional select/update are most general and flexible approaches. However, in this particular case reassignment works well:
table[Bidcols]: add2 table[Bidcols];
because add2 function already supports vectors.
If add didn't support vectors straightaway, e.g.
add: {[x]: $[x>10;x+2;x+3]}
Following reassignment would work:
table[Bidcols]: (add'') table[Bidcols];
Another solution:
q)#[table;Bidcols;add2]
time Bid1px Bid2px Bid3px
-------------------------
9 6 9 8
11 7 5 10
or
q)#[`table;Bidcols;add2]
for an in-place update.
Related
Let's say I've got a function that defines a matrix in terms of it's i and j coordinates:
f: {y+2*x}
I'm trying to create a square matrix that evaluates this function at all locations.
I know it needs to be something like f ' (til 5) /:\: til 5, but I'm struggling with rest.
Rephrasing your question a bit, you want to create a matrix A = [aij] where aij = f(i, j), i, j = 0..N-1.
In other words you want to evaluate f for all possible combinations of i and j. So:
q)N:5;
q)i:til[N] cross til N; / all combinations of i and j
q)a:f .' i; / evaluate f for all pairs (i;j)
q)A:(N;N)#a; / create a matrix using #: https://code.kx.com/q/ref/take/
0 1 2 3 4
2 3 4 5 6
4 5 6 7 8
6 7 8 9 10
8 9 10 11 12
P.S. No, (til 5) /:\: til 5 is not exactly what you'd need but close. You are generating a list of all pairs i.e. you are pairing or joining the first element of til 5 with every element of (another) til 5 one by one, then the second , etc. So you need the join operator (https://code.kx.com/q/ref/join/):
(til 5),/:\: til 5
You were close. But there is no need to generate all the coordinate pairs and then iterate over them. Each Right Each Left /:\: manages all that for you and returns the matrix you want.
q)(til 5)f/:\:til 5
0 1 2 3 4
2 3 4 5 6
4 5 6 7 8
6 7 8 9 10
8 9 10 11 12
Disclaimer: I am very new to the Q language so please excuse my silly question.
I have a function that currently is taking on 2 parameters (date;sym).It runs fine for 1 sym and 1 day. however, I need to perform this on multiple syms and dates which will take forever.
How do I create a loop that run the function on every sym, and on every date?
In python, it is straighforward as :
for date in datelist:
for sym in symlist:
func(date,sym)
How can I do something similar to this in Q? and how can I dynamically change the output table names and append them to 1 single table?
Currently, I am using the following:
output: raze .[function] peach paralist
where paralist is a list of parameter pairs: ((2020.06.01;ABC);(2020.06.01;XYZ)) but imho this is nowhere near efficient.
What would be the best way to achieve this in Q?
I'll generalize everything, if you have a given function foo which will operate on an atom dt with a vector s
q)foo:{[dt;s] dt +\: s}
q)dt:10?10
q)s:100?10
q)dt
8 1 9 5 4 6 6 1 8 5
q)s
4 9 2 7 0 1 9 2 1 8 8 1 7 2 4 5 4 2 7 8 5 6 4 1 3 3 7 8 2 1 4 2 8 0 5 8 5 2 8..
q)foo[;s] each dt
12 17 10 15 8 9 17 10 9 16 16 9 15 10 12 13 12 10 15 16 13 14 12 9 11 11 ..
5 10 3 8 1 2 10 3 2 9 9 2 8 3 5 6 5 3 8 9 6 7 5 2 4 4 ..
13 18 11 16 9 10 18 11 10 17 17 10 16 11 13 14 13 11 16 17 14 15 13 10 12 12 ..
9 14 7 12 5 6 14 7 6 13 13 6 12 7 9 10 9 7 12 13 10 11 9 6 8 8 ..
The solution is to project the symList over the function in question, then use each (or peach) for the date variable.
If your function requires an atomic date and sym, then you can just create a new function to implement this
q)bar:{[x;y] foo[x;] each y};
datelist:`date$10?10
symlist:10?`IBM`MSFT`GOOG
function:{0N!(x;y)}
{.[function;x]} peach datelist cross symlist
cross will return all combinations of sym and date
Is this what you need?
Try to use two "double" '
raze function'[datelist]'[symlist]
peach or each won't work here. They are not operators, but anonymous functions with two parameters: each is k){x'y}. That is why function each list1 each list2 statement is invalid, but function'[list1]'[list2] works.
From reading your response to another answer you are looking to save the results with unique names yes? Take a look at this solution using set to save and get to retrieve.
q)t:flip enlist each `colA`colB!(100;`name)
q)t
colA colB
---------
100 name
q)f:{[date;sym]tblName:`$string[date],string sym;tblName set update date:date,sym:sym from t}
q)newTbls:f'[.z.d+til 3;`AAA`BBB`CCC]
q)newTbls
`2020.09.02AAA`2020.09.03BBB`2020.09.04CCC
q)get each newTbls
+`colA`colB`date`sym!(,100;,`name;,2020.09.02;,`AAA)
+`colA`colB`date`sym!(,100;,`name;,2020.09.03;,`BBB)
+`colA`colB`date`sym!(,100;,`name;,2020.09.04;,`CCC)
q)get first newTbls
colA colB date sym
------------------------
100 name 2020.09.02 AAA
Does this meet your needs?
This could be a stab in the dark, but why not create a hdb rather than all these variables output20191005ABC, output20191006ABC .. etc and given you want to append them to 1 table.
Below I have outlined how to create a date partitioned hdb called outputHDB which has one table outputTbl. I created the hdb by running a function by date and sym and then upserting those rows to disk.
C:\Users\Matthew Moore>mkdir outputHDB
C:\Users\Matthew Moore>cd outputHDB
// can change the outputHDB as desired
// start q
h:hopen `::6789; // as a demo I connected to another hdb process and extracted some data per sym / date over IPC
hdbLoc:hsym `$"C:/Users/Matthew Moore/outputHDB";
{[d;sl]
{[d;s]
//output:yourFunc[date;sym];
// my func as a demo, I'm grabbing rows where price = max price by date and by sym from another hdb using the handle h
output:{[d;s]
h({[d;s] select from trades where date = d, sym = s, price = max price};d;s)
}[d;s];
// HDB Part
path:` sv (hdbLoc;`$string d;`outputTbl;`);
// change `outputTbl to desired table name
// dynamically creates the save location and then upserts one syms data directly to disk
// e.g. `:C:/Users/Matthew Moore/outputHDB/2014.04.21/outputTbl/
// extra / at the end saves the table as splayed i.e. each column is it's own file within the outputTbl directory
path upsert .Q.en[`:.;output];
// .Q.en enumerates syms in a table which is required when saving a table splayed
}[d;] each sl;
// applies the parted attribute to the sym column on disk, this speeds up querying for on disk data
#[` sv (hdbLoc;`$string d;`outputTbl;`);`sym;`p#];
}[;`AAPL`CSCO`DELL`GOOG`IBM`MSFT`NOK`ORCL`YHOO] each dateList:2014.04.21 2014.04.22 2014.04.23 2014.04.24 2014.04.25;
Now that the hdb has been created, you can load it from disk and query with qSQL
q)\l .
q)select from outputTbl where date = 2014.04.24, sym = `GOOG
date sym time src price size
------------------------------------------------------------
2014.04.24 GOOG 2014.04.24D13:53:59.182000000 O 46.43 2453
I've a problem to find common elements between all columns of a matrix in MATLAB, I've tried to solve it my self, the basic problem is intersect function set intersect just between two matrices, so I wrote a code like this
A=randi(n,m);
B=struct();
for k=1:size(A,2)-1
B.(['b' num2str(k)])=intersect(A(:,k),A(:,k+1));
end
unfortunately the problem isn't solve cause the dimension of A is unknown so we have same problem with B!
thanks all.
One vectorized approach using bsxfun -
unqA = unique(A)
out = unqA(all(any(bsxfun(#eq,A,permute(unqA,[2,3,1])),1),2))
Sample run -
A =
8 5 6 4 8
4 6 7 5 9
9 4 4 7 5
9 4 9 5 6
9 9 7 9 6
9 5 9 4 8
8 5 6 9 8
7 5 6 7 4
out =
4
9
This question already has answers here:
Matlab: How to sort data for each row/column, loop or formula?
(3 answers)
Closed 8 years ago.
Sorry for so many question i post.
this is my another problem, how to sort rows from smallest to largest, (see picture below):
so here's my data and code:
A=
[a b c
1 2 3
4 5 1
0 1 0
2 1 2]
i used sort function:
B = [sort(A(1:end,:), 'ascend')]
but it didn't display the output i want.
smallest no. display in 1st column and large no. display in last column, so guys please help.
You have two options.
Sortrows: http://www.mathworks.com/help/matlab/ref/sortrows.html
Sort with argument 2, e.g. sort(A, 2):
http://www.mathworks.com/help/matlab/ref/sort.html?refresh=true
I think you're looking for this
a =
1 2 3 4 5
3 6 2 4 7
9 6 5 8 4
B1 = sort(a,2,'ascend')
B1 =
1 2 3 4 5
2 3 4 6 7
4 5 6 8 9
More about this at MathWorks
I have two arrays
A=[1;2]
B= [5;6]
Now I want to have the matrix C = A:B such that
C = [1 2 3 4 5; 2 3 4 5 6]
How can I do this in matlab?
You can do this using arrayfun in combination with cell2mat like this:
A =
1 4 7 10
B =
5 8 11 14
cell2mat(arrayfun(#(n) (A(n):B(n)), 1:numel(A),'UniformOutput', false)')
ans =
1 2 3 4 5
4 5 6 7 8
7 8 9 10 11
10 11 12 13 14
You can shorten it down a bit using an abbreviation for UniformOutput, but I suggest writing it out since the abbreviations might no longer be unambiguous in future MATLAB versions. Check this question for a lengthy discussion on the topic.
Your question implicitly assumes that B(1)-A(1) equals B(2)-A(2) etc; otherwise the result is undefined.
You can do it quite generally and efficiently as follows: build the first row, and then use bsxfun to obtain all other rows:
C = bsxfun(#plus, A(1):B(1), A(:)-A(1));
C = [A(1,1):B(1,1);A(2,1):B(2,1)];
Try it:
C=[ A(1):1:B(1); A(2):1:B(2) ]