Let's say I have a table:
m:([] t: raze 3#'(2021.01.04+til 5); sym:15#`A`B`C; c: til 15)
t sym c
-----------------
2021.01.04 A 0
2021.01.04 B 1
2021.01.04 C 2
2021.01.05 A 3
2021.01.05 B 4
When I try to pivot it:
exec t!c by sym:sym from m
sym|
---| -----------------------------------------------------------------
A | 2021.01.04 2021.01.05 2021.01.06 2021.01.07 2021.01.08!0 3 6 9 12
B | 2021.01.04 2021.01.05 2021.01.06 2021.01.07 2021.01.08!1 4 7 10 13
C | 2021.01.04 2021.01.05 2021.01.06 2021.01.07 2021.01.08!2 5 8 11 14
I'd expect to get a table back, with columns sym, but I don't. What am I doing wrong?
if you're after a pivot with columns of sym you would want the following:
q)exec sym!c by t:t from m
t | A B C
----------| --------
2021.01.04| 0 1 2
2021.01.05| 3 4 5
2021.01.06| 6 7 8
2021.01.07| 9 10 11
2021.01.08| 12 13 14
It's because your column names have to be symbols:
q)exec(`$string t)!c by sym:sym from m
sym| 2021.01.04 2021.01.05 2021.01.06 2021.01.07 2021.01.08
---| ------------------------------------------------------
A | 0 3 6 9 12
B | 1 4 7 10 13
C | 2 5 8 11 14
These would be terrible column names though, so I would use .Q.id
q).Q.id exec(`$string t)!c by sym:sym from m
sym| a20210104 a20210105 a20210106 a20210107 a20210108
---| -------------------------------------------------
A | 0 3 6 9 12
B | 1 4 7 10 13
C | 2 5 8 11 14
It sounds like this isn't what you actually want though, so maybe Matthews answer is more relevant. My answer just explains why it didn't look like what you thought
Related
t: ([] a: til 10; b: til 10)
a b
---
0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
I'm trying to get it to sum a, and upsert it, leaving b in place. I don't want to use q-sql for it.
I think it should be something like:
({x[`a]: x`a + y`a}\) t
I keep getting 'type errors though, on the indexing operation.
What am I doing wrong?
EDIT:
An even simpler example,
({x[`a]: 3}\) t
Same error. Expected result:
q)
a b
---
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
What am I'm trying to achieve in pseudocode:
assign case:
for each row in t
row[`a]: 3
For the summation case:
for each row in t
row[`a]: row_prior[`a] + row[`a]
I finally figured out the answer; you have to return after the assignment:
({y[`a]: y[`a]+x[`a]; y}\) t
a b
----
0 0
1 1
3 2
6 3
10 4
15 5
21 6
28 7
36 8
45 9
Given a keyed table, e.g.:
q)\S 7 / seed random numbers for reproducibility
q)v:flip (neg[d 0]?`1)!#[;prd[d]?12] d:4 6 / 4 cols 6 rows
q)show kt:([]letter:d[1]#.Q.an)!v
letter| c g b e
------| ----------
a | 11 0 3 9
b | 11 8 10 0
c | 7 2 2 3
d | 8 4 9 6
e | 0 0 5 0
f | 1 0 0 11
How to calculate an average for each row --- e.g. (c+g+b+e)%4 --- for any number of columns?
Following on from your own solution, note that you have to be a little careful with null handling. Your approach won't ignore nulls in the way that avg normally would.
q).[`kt;("a";`g);:;0N];
q)update av:avg flip value kt from kt
letter| c g b e av
------| ---------------
a | 11 3 9
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
To make it ignore nulls you have to avg each row rather than averaging the flip.
q)update av:avg each value kt from kt
letter| c g b e av
------| -------------------
a | 11 3 9 7.666667
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
Solution 1: q-sql
q)update av:avg flip value kt from kt
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
Solution 2: functional q-sql
tl;dr:
q)!enlist(avg;)enlist,cols[`kt]except cols key`kt
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
let's start with a look how the parse tree of a non-general solution would look like:
q)parse"update av:avg (c;g;b;e) from kt"
!
`kt
()
0b
(,`av)!,(avg;(enlist;`c;`g;`b;`e))
(note that q is a wrapper implemented in k, so the , prefix operator in the above expression is the same as enlist keyword in q)
so all the below are equivalent (verify with ~). relying on projection: (x;y)~(x;)y, we can further improve the readability by reducing the distance between parens:
q)k)(!;`kt;();0b;(,`av)!,(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;(enlist`av)!enlist(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;(1#`av)!enlist(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;)(1#`av)!enlist(avg;)(enlist;`c;`g;`b;`e)
let's evaluate the parse tree to check:
q)eval(!;`kt;();0b;)(1#`av)!enlist(avg;)(enlist;`c;`g;`b;`e)
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
(enlist;`c;`g;`b;`e) in the general case is:
q)enlist,cols[`kt]except cols key`kt
enlist
`c
`g
`b
`e
so let's plug in and check:
q)eval(!;`kt;();0b;(1#`av)!enlist(avg;)enlist,cols[`kt]except cols key`kt)
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
also:
q)![`kt;();0b;(1#`av)!enlist(avg;)enlist,cols[`kt]except cols key`kt]
q)!enlist(avg;)enlist,cols[`kt]except cols key`kt
I have a function that takes 2 parameters: date and sym. I would like to do this for multiple dates and multiple sym. I have a list for each parameter. I can currently loop through 1 list using
raze function[2020.07.07;] peach symlist
How can I do something similar but looping through the list of dates too?
You may try following:
Create list of pairs of input parameters.
Write anonymous function which calls your function and use peachon list op paired parameters
For example
symlist: `A`B`C; // symlist defined for testing
function: {(x;y)}; // function defined for testing
raze {function . x} peach (2020.07.07 2020.07.08 2020.07.09) cross symlist
I think this could work:
raze function'[2020.07.07 2020.07.08 2020.07.09;] peach symlist
If not some more things to consider. Could you change your function to accept a sym list instead of individual syms by including an each/peach inside it? Then you could each the dates.
Also, you could create a new list of each date matched with the symlist and create a new function which takes this list and does whatever the initial function did by separating the elements of the list.
q)dates
2020.08.31 2020.09.01 2020.09.02
q)sym
`llme`obpb`dhca`mhod`mgpg`jokg`kgnd`nhke`oofi`fnca`jffe`hjca`mdmc
q)func
{[date;syms]string[date],/:string peach syms}
q)func2
{[list]func[list 0;list 1]}
q)\t res1:func[;sym]each dates
220
q)\t res2:func[;sym]peach dates
102
q)
q)func2
{[list]func[list 0;list 1]}
q)dateSymList:dates,\:enlist sym
q)\t res3:func2 peach dateSymList
80
q)res3~res2
1b
q)res3~res1
1b
Let us know if any of those solutions work, thanks.
Some possible ways to do this
Can project dyadic f as monadic & parallelise over list of argument pairs
q)a:"ABC";b:til 3;f:{(x;y)}
q)\s 4
q)(f .)peach l:raze a,\:/:b
"A" 0
"B" 0
"C" 0
"A" 1
"B" 1
"C" 1
"A" 2
"B" 2
"C" 2
Or could define function to take a dictionary argument & parallelise over a table
q)f:{x`c1`c2}
q)f peach flip`c1`c2!flip l
"A" 0
"B" 0
"C" 0
"A" 1
"B" 1
"C" 1
"A" 2
"B" 2
"C" 2
Jason
I'll generalize everything, if you have a given function foo which will operate on an atom dt with a vector s
q)foo:{[dt;s] dt +\: s}
q)dt:10?10
q)s:100?10
q)dt
8 1 9 5 4 6 6 1 8 5
q)s
4 9 2 7 0 1 9 2 1 8 8 1 7 2 4 5 4 2 7 8 5 6 4 1 3 3 7 8 2 1 4 2 8 0 5 8 5 2 8..
q)foo[;s] each dt
12 17 10 15 8 9 17 10 9 16 16 9 15 10 12 13 12 10 15 16 13 14 12 9 11 11 ..
5 10 3 8 1 2 10 3 2 9 9 2 8 3 5 6 5 3 8 9 6 7 5 2 4 4 ..
13 18 11 16 9 10 18 11 10 17 17 10 16 11 13 14 13 11 16 17 14 15 13 10 12 12 ..
9 14 7 12 5 6 14 7 6 13 13 6 12 7 9 10 9 7 12 13 10 11 9 6 8 8 ..
The solution is to project the symList over the function in question, then use each (or peach) for the date variable.
If your function requires an atomic date and sym, then you can just create a new function to implement this
q)bar:{[x;y] foo[x;] each y};
datelist:`date$10?10
symlist:10?`IBM`MSFT`GOOG
function:{0N!(x;y)}
{.[function;x]} each datelist cross symlist
code Table
code_grille code_grille_talend
s01 4 7 2 8
s02 5 2 8 9 6 3 7
s03 3 6 4 7 5 8 2
s04 2 6 4 8 5 2 8 0
s05 4 7 8 5 9 7 4 5 8
s06 2 4 7 8 9 3 6 5
s07 2 5 4 7 8
s08 2 3 4 5 6 7 8 9
s09 9 8 2 5 7 3 6 4
s10 2 4 5 2 8 7 9 3 6
s11 4 5 7 2 3 2 3 8
commande table
code_commande code_article taille
001 1 s
001 1 m
001 1 xl
001 1 x52
001 2 m
001 1 5566
001 2 x52
001 1 xl
002 1 s
002 2 m
001 3 xxl
code T table (result of the first query)
code
2
3
4
1
12
I have two queries which I need to use the result of the first query in the second query dynamically.
The first query returns much code which I need to put them in the second query to have resulted for each row.
I have load the result of the first query in a table but I have one result in the second query.
The first query is:
select [code_commande],[code_article],[code]
from [dbo].[conversion],[dbo].[commande]
where [dbo].[conversion].taille=[dbo].[commande].taille
and code_article=? and code_commande=?
The second query is:
select top 1 (G.[code_grille_talend]), count(C.code) as counter
from [dbo].[code] G
left join [dbo].[codeT] C
on G.code_grille_talend not like '%'+LTRIM(RTRIM(C.code))+'%'
group by G.code_grille_talend
having LEN(g.code_grille_talend)+count(C.code)<=40 or count(C.code)=0
order by len(g.code_grille_talend)desc
I have loaded the result of the first query in a table (codeT)
I have a table
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10)
ref a b c d e
0 5 3 3 9 1
1 1 9 0 0 0
2 5 9 4 1 7
3 0 0 5 1 3
4 2 6 8 9 3
5 3 2 0 6 6
6 7 6 4 9 8
7 4 8 9 7 2
8 7 0 8 8 3
9 7 9 0 4 8
how can I set all values in columns a,b,c ,.. to 0Ni if their value equals the value in column ref without having to do a single line update for every columns?
So something taht would look a bit like (which returns ERROR:type)
{update x:?[x=t;0Ni;x] from t} each `a`b`c`....
The previous answer involves working with strings, this can often get very messy. To avoid this it is possible to build up the query by passing column names as symbols instead. The dictionary for the functional select can be built up using the following function:
q){y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]
a| ({?[y=x;0Ni;x]};`ref) `a
b| ({?[y=x;0Ni;x]};`ref) `b
c| ({?[y=x;0Ni;x]};`ref) `c
The initial column is x and it allows to any number of columns to be passed as y for comparison.
This can then be added into the functional select:
q)![t;();0b;{y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]]
ref a b c d e
-------------
0 4 5 8 4 8
1 2 6 9 1
2 8 4 7 2 9
3 0 1 2 7 5
4 5 3 0 4
5 8 3 1 6
6 5 7 4 9 6
7 2 8 2 2 1
8 2 7 1 8
9 6 1 8 8 5
You may reuse the next code snippet
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10);
columns: `a`b`c`d`e;
![t;();0b;columns!{parse "?[",x,"=ref;0Ni;",x,"]" }each string columns]
Where updated columns are put in columns list.
And functional update, which maps every column X to value ?[x=ref;0Ni;x], is used