separating a string column, converting sub values into float and assigning column labels - kdb

Question

I have a table t and vector times.
sym vals
-------------------
A "3.6, 2.1, 1.8"
B "2.1, 1.8, 1.6"
C "2.2, 1.9, 1.6"
D "1.9, 1.5, 1.3"
E "2.6, 2.1, 1.9"
times: `0`1`2
I want to separate the comma separated string in each row and convert each value into float. The column labels then need to be times. I also want to then drop the column vals. The following statement does this.
t_out: delete vals from t, 'flip exec times!("FFF";",")0:vals from t
sym 1 2 3
---------------
A 3.6 2.1 1.8
B 2.1 1.8 1.6
C 2.2 1.9 1.6
D 1.9 1.5 1.3
E 2.6 2.1 1.9
why does exec times!("FFF";",")0:vals from t transpose the table after converting the values to float? Why do we need 'flip and not just flip? I appreciate your help. Are there alternative methods to achieve this?

Answer 1

(This is a bit of a guess here, since you're asking about q's design choices.)
0: is often used to read in CSV files, where you would store a row per line, separated by a comma. This row won't always have the same types, for instance, this could be a CSV file (building on your example):
3.6, 2.1, 1.8, 10
2.1, 1.8, 1.6, 20
2.2, 1.9, 1.6, 30
1.9, 1.5, 1.3, 40
2.6, 2.1, 1.9, 50
So we have four columns, the first three are float columns, and the fourth is a long (int) column.
When you use 0: to read in the CSV (or in your case, just a list of strings that resemble a CSV), q will transpose the table so that you have a list containing four lists.
q)vals: ("3.6, 2.1, 1.8, 10"; "2.1, 1.8, 1.6, 20"; "2.2, 1.9, 1.6, 30"; "1.9, 1.5, 1.3, 40"; "2.6, 2.1, 1.9, 50")
q)vals
"3.6, 2.1, 1.8, 10"
"2.1, 1.8, 1.6, 20"
"2.2, 1.9, 1.6, 30"
"1.9, 1.5, 1.3, 40"
"2.6, 2.1, 1.9, 50"
q)("FFFJ"; ",") 0: vals
3.6 2.1 2.2 1.9 2.6
2.1 1.8 1.9 1.5 2.1
1.8 1.6 1.6 1.3 1.9
10 20 30 40 50
Each of the four lists in this list will be correctly typed:
q)first ("FFFJ"; ",") 0: vals
3.6 2.1 2.2 1.9 2.6
q)type first ("FFFJ"; ",") 0: vals
9h
q)last ("FFFJ"; ",") 0: vals
10 20 30 40 50
q)type last ("FFFJ"; ",") 0: vals
7h
This makes it easier to work with, as you don't have a list of mixed lists. The alternative would be:
q)flip ("FFFJ"; ",") 0: vals
3.6 2.1 1.8 10
2.1 1.8 1.6 20
2.2 1.9 1.6 30
1.9 1.5 1.3 40
2.6 2.1 1.9 50
q)first flip ("FFFJ"; ",") 0: vals
3.6
2.1
1.8
10
q)type first flip ("FFFJ"; ",") 0: vals
0h
I'm guessing the reasoning for this is performance, as under the hood, tables are in fact column dictionaries, so actually look something like this:
q)`1`2`3`4 ! ("FFFJ"; ",") 0: vals
1| 3.6 2.1 2.2 1.9 2.6
2| 2.1 1.8 1.9 1.5 2.1
3| 1.8 1.6 1.6 1.3 1.9
4| 10 20 30 40 50
But again, you're asking about q's design choices, so I'm just guessing.
The reason you need to use 'flip instead of just flip, is because you want to join each element of two tables (which are treated as lists of dictionaries) to each other, so you are using the each iterator. You can read about it here (if you scroll down just a bit to the "Advanced part, just above the each-left header, it explains it a bit better).
Just to make it clear that the iterator ' is changing , and not flip, I would write your query as:
... from t ,' flip exec ...

Answer 2

This is more of a curiosity but you can effectively force the "flip" to occur naturally by using a by grouping (even if the by grouping is meaningless)
q)exec times!raze("FFF";",")0:vals by sym:sym from t
sym| 0 1 2
---| -----------
A | 3.6 2.1 1.8
B | 2.1 1.8 1.6
C | 2.2 1.9 1.6
D | 1.9 1.5 1.3
E | 2.6 2.1 1.9
This also does away with the need to append-each (,') to sideways-join the result