Has kdb Vertical output? In ClickHouse I can specify FORMAT VERTICAL
SELECT * FROM foo FORMAT VERTICAL;
Row 1:
──────
date: 2020-04-21
datetime: 2020-04-21 09:00:01
Row 2:
──────
date: 2020-04-21
datetime: 2020-04-21 09:00:01
Kdb+ doesn't have a vertical formatter built in, but it's easy to create your own:
vformat:{
{-1 hdr:"Row ", string[x], ":"; -1 count[hdr]#"_"; -1 .Q.s y;}'[1+til count x; x:0!x];
}
q)vformat ([a:til 10]b:10?`4;long_col_name:10?.Q.A)
Row 1:
______
a | 0
b | `bghg
long_col_name| "M"
Row 2:
______
a | 1
b | `ifna
long_col_name| "I"
...
Kdb tables are lists of dictionaries so you could just "show" the dictionaries. Many ways to achieve this depending on your preferences:
q)show tab:([]col1:`a`b`c;col2:1 2 3)
col1 col2
---------
a 1
b 2
c 3
q)0N!'tab;
`col1`col2!(`a;1)
`col1`col2!(`b;2)
`col1`col2!(`c;3)
q){show x;show`}each tab;
col1| `a
col2| 1
`
col1| `b
col2| 2
`
col1| `c
col2| 3
`
but note that this will really only work in a terminal - a gui/IDE may not display this properly.
From a gui/IDE you could use something like
q)enlist each tab
+`col1`col2!(,`a;,1)
+`col1`col2!(,`b;,2)
+`col1`col2!(,`c;,3)
/or
q)raze string[til count tab],'.Q.s2 each tab
"0"
"col1| `a"
"col2| 1"
"1"
"col1| `b"
"col2| 2"
"2"
"col1| `c"
"col2| 3"
Related
My table looks something like this:
id | data
1 | A=1000 B=2000
2 | A=200 C=300
In kdb is there a way to normalize the data such that the final table is as follows:
id | data.1 | data.2
1 | A | 1000
1 | B | 2000
2 | A | 200
2 | C | 300
One option would be to make use of 0: & it's key-value parsing functionality, documented here https://code.kx.com/q/ref/file-text/#key-value-pairs e.g.
q)ungroup delete data from {x,`data1`data2!"S= "0:x`data}'[t]
id data1 data2
---------------
1 A "1000"
1 B "2000"
2 A "200"
2 C "300"
assuming you want data2 to be long datatype (j), can do
update "J"$data2 from ungroup delete data from {x,`data1`data2!"S= "0:x`data}'[t]
You could use a combination of vs (vector from scalar), each-both ' and ungroup:
q)t:([]id:1 2;data:("A=1000 B=2000";"A=200 C=300"))
q)t
id data
------------------
1 "A=1000 B=2000"
2 "A=200 C=300"
q)select id, dataA:`$data[;0], dataB:"J"$data[;1] from
ungroup update data: "=" vs '' " " vs ' data from t
id dataA dataB
--------------
1 A 1000
1 B 2000
2 A 200
2 C 300
I wouldn't recommend naming the columns with . e.g. data.1
In a q session I've made a keyed table t:
q)/KDB+ 3.6 2018.05.17
q)f:flip (`a`b)!(1 2 3;4 5 6)
q)k:flip (enlist `k)!(enlist 101 102 103)
q)t:k!f;t
k | a b
---| ---
101| 1 4
102| 2 5
103| 3 6
Then I've tried to make a query and it gives a nice results:
q)select a,b from t where k=101
a b
---
1 4
q)flip select a,b from t where k=101
a| 1
b| 4
q)flip flip select a,b from t where k=101
a b
---
1 4
But without select-syntax this gives an error:
q)t[101]
a| 1
b| 4
q)flip t[101]
'rank
[0] flip t[101]
^
Why can't I just make a simple flip on the same result as from select of the same data types?
q)type flip select a,b from t where k=101
99h
q)type t[101]
99h
Because the elements of dictionary t[101] aren't lists, but atoms. So flip on a list of atoms fails.
Appending each element to an empty list first will work.
q)(),/:t[101]
a| 1
b| 4
Not necessarily something you want to do. For a given dictionary output, the solution you probably want is enlist
q)enlist t[101]
a b
---
1 4
An alternative approach would be to lookup using a table rather than a lookup using an atom:
q)t[([]k:(),101)]
a b
---
1 4
That would be the equivalent of select a,b from t where k=101
I'd like to be able to select rows in batches from a very large keyed table being stored remotely on disk. As a toy example to test my function I set up the following tables t and nt...
t:([sym:110?`A`aa`Abc`B`bb`Bac];px:110?10f;id:1+til 110)
nt:0#t
I select from the table only records that begin with the character "A", count the number of characters, divide the count by the number of rows I would like to fetch for each function call (10), and round that up to the nearest whole number...
aRec:select from t where sym like "A*"
counter:count aRec
divy:counter%10
divyUP:ceiling divy
Next I set an idx variable to 0 and write an if statement as the parameterized function. This checks if idx equals divyUP. If not, then it should select the first 10 rows of aRec, upsert those to the nt table, increment the function argument, x, by 10, and increment the idx variable by 1. Once the idx variable and divyUP are equal it should exit the function...
idx:0
batches:{[x]if[not idx=divyUP;batch::select[x 10]from aRec;`nt upsert batch;x+:10;idx+::1]}
However when I call the function it returns a type error...
q)batches 0
'type
[1] batches:{[x]if[not idx=divyUP;batch::select[x 10]from aRec;`nt upsert batch;x+:10;idx+::1]}
^
I've tried using it with sublist too, though I get the same result...
batches:{[x]if[not idx=divyUP;batch::x 10 sublist aRec;`nt upsert batch;x+:10;idx+::1]}
q)batches 0
'type
[1] batches:{[x]if[not idx=divyUP;batch::x 10 sublist aRec;`nt upsert batch;x+:10;idx+::1]}
^
However issuing either of those above commands outside of the function both return the expected results...
q)select[0 10] from aRec
sym| px id
---| ------------
A | 4.236121 1
A | 5.932252 3
Abc| 5.473628 5
A | 0.7014928 7
Abc| 3.503483 8
A | 8.254616 9
Abc| 4.328712 10
A | 5.435053 19
A | 1.014108 22
A | 1.492811 25
q)0 10 sublist aRec
sym| px id
---| ------------
A | 4.236121 1
A | 5.932252 3
Abc| 5.473628 5
A | 0.7014928 7
Abc| 3.503483 8
A | 8.254616 9
Abc| 4.328712 10
A | 5.435053 19
A | 1.014108 22
A | 1.492811 25
The issue is that in your example, select[] and sublist requires a list as an input but your input is not a list. Reason for that is when there is a variable in items(which will form a list), it is no longer considered as a simple list meaning blank(space) cannot be used to separate values. In this case, a semicolon is required.
q) x:2
q) (1;x) / (1 2)
Select command: Change input to (x;10) to make it work.
q) t:([]id:1 2 3; v: 3 4 5)
q) {select[(x;2)] from t} 1
`id `v
---------
2 4
3 5
Another alternative is to use 'i'(index) column:
q) {select from t where i within x + 0 2} 1
Sublist Command: Convert left input of the sublist function to a list (x;10).
q) {(x;2) sublist t}1
You can't use the select[] form with variable input like that, instead you can use a functional select shown in https://code.kx.com/q4m3/9_Queries_q-sql/#912-functional-forms where you input as the 5th argument the rows you want
Hope this helps!
I specified the columns to be of type String. Why am I getting the following error:
q)test: ([key1:"s"$()] col1:"s"$();col2:"s"$();col3:"s"$())
q)`test upsert(`key1`col1`col2`col3)!(string "999"; string "693"; string "943";
string "249")
'type
[0] `test upsert(`key1`col1`col2`col3)!(string "999"; string "693"; string "9
43"; string "249")
To do exactly this, you can remove the types of the list you defined in test:
q)test: ([key1:()] col1:();col2:();col3:())
q)test upsert (`key1`col1`col2`col3)!("999";"693";"943";"249")
key1 | col1 col2 col3
-----| -----------------
"999"| "693" "943" "249"
The reason you are getting a type error is because "s" corresponds to a list of symbols, not a list of characters. you can check this by using .Q.ty:
q).Q.ty `symbol$()
"s"
q).Q.ty `char$()
"c"
It is (generally) not a great idea to set the keys as nested list of chars, you might find it better to set them as integers ("i") or longs ("j") as in:
test: ([key1:"j"$()] col1:"j"$();col2:"j"$();col3:"j"$())
Having the keys as integers/longs will make the upsert function behave nicely. Also note that a table is a list of dictionaries, so each dictionary can be upserted inidividually as well as a table being upserted:
q)`test upsert (`key1`col1`col2`col3)!(9;4;6;2)
`test
q)test
key1| col1 col2 col3
----| --------------
9 | 4 6 2
q)`test upsert (`key1`col1`col2`col3)!(8;6;2;3)
`test
q)test
key1| col1 col2 col3
----| --------------
9 | 4 6 2
8 | 6 2 3
q)`test upsert (`key1`col1`col2`col3)!(9;1;7;4)
`test
q)test
key1| col1 col2 col3
----| --------------
9 | 1 7 4
8 | 6 2 3
q)`test upsert ([key1: 8 7] col1:2 4; col2:9 3; col3:1 9)
`test
q)test
key1| col1 col2 col3
----| --------------
9 | 1 7 4
8 | 2 9 1
7 | 4 3 9
You have a few issues:
an array of chars in quotes is a string so no need to write string "abc"
string "aaa" will split the string out in strings of strings
your initial defined types are symbols "s" and not strings
This will allow you to insert as symbols:
q)test: ([key1:"s"$()] col1:"s"$();col2:"s"$();col3:"s"$())
q)`test upsert(`key1`col1`col2`col3)!`$("999"; "693"; "943"; "249")
`test
This will keep them as strings:
q)test: ([key1:()] col1:();col2:();col3:())
q)`test upsert(`key1`col1`col2`col3)!("999"; "693"; "943"; "249")
`test
Have a look at the diffs in metas of the two
HTH,
Sean
Given a table that contains a number of null entries how can I create a summary table that describes the number of nulls per column? Can this be done on a general table where the number of columns and column names are not known beforehand?
q)t: ([] a: 1 2 3 4; b: (2018.10.08; 0Nd; 2018.10.08; 2018.10.08); c: (0N;0N;30;40); d: `abc`def``jkl)
q)t
a b c d
-------------------
1 2018.10.08 abc
2 def
3 2018.10.08 30
4 2018.10.08 40 jkl
Expected result:
columnName nullCount
--------------------
a 0
b 1
c 2
d 1
While sum null t is the simplest solution in this example, it doesn't handle string (or nested) columns. To handle string or nested columns for example you would need something like
q)t: ([] a: 1 2 3 4; b: (2018.10.08; 0Nd; 2018.10.08; 2018.10.08); c: (0N;0N;30;40); d: `abc`def``jkl;e:("aa";"bb";"";()," "))
q){sum$[0h=type x;0=count#'x;null x]}each flip t
a| 0
b| 1
c| 2
d| 1
e| 1
You can make such a table using
q)flip `columnName`nullCount!(key;value)#\:sum null t
columnName nullCount
--------------------
a 0
b 1
c 2
d 1
where sum null t gives a dictionary of the null values in each column
q)sum null t
a| 0
b| 1
c| 2
d| 1
and we apply the column names as keys and flip to a table.
To produce a table with the columns as the headers and number of nulls and the values you can use:
q)tab:enlist sum null t
Which enlists a dictionary with the number of nulls as the values and the columns names as keys:
a b c d
-------
0 1 2 1
If you then wanted this in your given format you could then use:
result:([]columnNames:cols tab; nullCount:raze value each tab)