I have a query that returns data for a bunch of syms for a date from a table.
dataFromTab:delete date from select from table where date=2015.01.01,any (sym like) each ("sym1";"sym2";"sym3";"sym4")
dataFromTab:`sym xasc dataFromTab;
#[`dataFromTab;`sym;`p#];
type var is 98h and sorting by sym is working fine but when adding p# using functional amend is giving me a 'type error.
Any inputs on where i am going wrong would be appreciated.
Thanks!
Are you calling all of this from within a function or from the global namespace?
If dataFromTab is a local variable then applying the p# to the global referencedataFromTab will not work.
q){data:select from tab;#[`data;`sym;`p#]}[]
'type
[1] {data:select from tab;#[`data;`sym;`p#]}
^
q))\
You will need to reassign dataFromTab locally for this to work;
q){data:select from tab;data:#[data;`sym;`p#]}[]
You could also use 'set' to create your table globally, which would allow your original syntax to work. However you probably do not want to create unnecessary globals;
q){`data set select from tab;#[`data;`sym;`p#]}[]
`data
Is sym a string column? I get that error if I convert to string:
q)trades:update string sym from trades
q)trades:`sym xasc trades
q)#[`trades;`sym;`p#]
'type
[0] #[`trades;`sym;`p#]
^
Can you try converting it to a symbol and see if that works?
Related
I want to be able to run something like the following:
f:{[dt] syms:`sym1;eval parse"select from tbl where date = dt, sym=syms"}
f[.z.D]
Given the following :
tbl:([] date:2022.01.01 2022.01.01; Id:1000000 2000000; sym:`sym1`sym2;price:10 20;qty:3 4)
f:{[dt] syms:`sym1; ?[tbl;((=;`date;`dt);(=;`sym;`syms));0b;()]}
f1:{[dt] syms:`sym1; (?) . (tbl;((=;`date;`dt);(=;`sym;`syms));0b;())}
f2:{[dt] syms:`sym1; value (?;tbl;((=;`date;`dt);(=;`sym;`syms));0b;())}
f[.z.D] // works
f1[.z.D] // Gives Error - dt not recognized/out of scope
f2[.z.D] // Gives Error - dt not recognized/out of scope
Value within a function seems to not be able to detect the local variables and surprisingly (?) . also fails. (maybe because this in itself is a function and dt is not defined here?)
Is there any work around for this?
For context, I have a function that takes a select string/functional select, parses it, does some checks and manipulations on the functional form and returns a modified functional form.
I want users to be able to call this function from their own functions and that parameters they have defined in their function can be in the outputted functional form and that functional form can be valued some how.
I don't want users to be forced to pass more variables into my function etc.
What you need to do here is remove the backtick for dt and syms
I would also recommend using a backtick when calling your table name.
Further, you should make sure syms is enlisted if it is only one symbol.
So your function should be:
f:{[dt] syms:(),`sym1; ?[`tbl;((=;`date;dt);(=;`sym;syms));0b;()]}
If you parse your select statement you can see the correct form for functional selects:
q)parse "select from tbl where date=2022.01.01,sym=`sym1"
?
`tbl
,((=;`date;2022.01.01);(=;`sym;,`sym1)) // comma in front of `sym1 means enlist
0b
()
The backtick is not needed as this is a variable, defined in your function, it would be the same as doing:
?[`tbl;((=;`date;2022.01.01);(=;`sym;enlist `sym1));0b;()]
This should allow you to use your function correctly:
q)f[2022.01.01]
date Id sym price qty
---------------------------------
2022.01.01 1000000 sym1 10 3
For more information, see the kx documentation
I need to be able to use variables in table names - I basically have the same set of tables used for different types of data, so I would like to just have one dashboard and swapping between all types instead of always having to set up multiple identical dashboards.
My query is something like:
select * from table_$variable_name;
Where my list of possible variable is something like cat, dog, bird
I can seem to make this work, if I only put the variable as shown above I get the following error
Error 1146: Table 'table_$variable_name' doesn't exist
If I enclose it in curly brackets, I get this error instead.
Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '{bird}' at line 1
(i.e. with the selected variable actually being visible this time)
I'm not sure if the issue is having underscores in the table names, I tried putting underscores around my variables too to check and I had no luck with that.
Another thing I tried was gradually adding on to the table name, so e.g.
select * from table_$variable;
Still returns an error, but I can see the table name starting to form correctly
Error 1146: Table 'table_bird_' doesn't exist
However, as soon as I add another underscore, the variable is not picked up abymore
```Error 1146: Table 'table_$variable_' doesn't exist``
I'm sure it's something silly I am missing in the syntax of the query - anyone has any suggestions?
Using this https://grafana.com/docs/grafana/latest/variables/templates-and-variables/ for reference
As #arturomp suggests, use
${var:raw}
At least in my case, this was the solution that worked.
I found double square brackets work. e.g.
Rather than
select * from table_$variable_name;
use
select * from table_[[variable_name]];
I'm trying to work out how to expand a JSONB field with a jsonb_to_recordset and a custom type.
Postgres 11.4.
This is just a test case, but for the minute I'm trying to work out the syntax. I've got a table named data_file_info with a JSONB field named table_stats. The JSONB field always includes a JSON array with the same structure:
[
{"table_name":"Activity","record_count":0,"table_number":214},
{"table_name":"Assembly","record_count":1,"table_number":15},
{"table_name":"AssemblyProds","record_count":0,"table_number":154}
]
The following code works correctly:
from data_file_info,
jsonb_to_recordset(table_stats) as table_stats_splat (
table_name text,
record_count integer,
table_number integer
)
What I would like to do is pass in a custm type definition instead of the long-form column definition list shown above. Here's the matching type:
create type data.table_stats_type as (
table_name text,
record_count integer,
table_number integer)
Some examples I've seen, and the docs say, that you can supply a type name using a null:row_type casting in the first parameter to jsonb_to_recordset. The examples that I've found use in-line JSON, while I'm trying to pull stored JSON. I've made a few attempts, all have failed. Below are two of the trials, with errors. Can someone point me towards the correct syntax?
FAIL:
select table_stats_splat.*
from data_file_info,
jsonb_populate_recordset(null::table_stats_type, data_file_info) as table_stats_splat;
-- ERROR: function jsonb_populate_recordset(table_stats_type, data_file_info) does not exist
-- LINE 4: jsonb_populate_recordset(null::table_stats_type, dat...
^
-- HINT: No function matches the given name and argument types. You might need to add explicit type casts. (Line 4)
FAIL:
select *
from jsonb_populate_recordset(NULL::table_stats_type, (select table_stats from data_file_info)) as table_stats_splat;
-- ERROR: more than one row returned by a subquery used as an expression. (Line 2)
I'm doubtlessly missing something pretty obvious, and am hoping someone can suggest what that is.
Use the column as the second parameter:
select table_stats_splat.*
from data_file_info,
jsonb_populate_recordset(null::table_stats_type, table_stats) as table_stats_splat;
I would like to pass a column name into a Q function to query a loaded table.
Example:
getDistinct:{[x] select count x from raw}
getDistinct "HEADER"
This doesn't work as the Q documentation says I cannot pass column as arguments. Is there a way to bypass this?
When q interprets x it will treat it as a string, it has no reference to the column, so your output would just be count "HEADER".
If you want to pass in the column as a string you need to build the whole select statement then use value
{value "select count ",x," from tab"} "HEADER"
However, the recommended method would be to use a functional select. Below I use parse to build the functional select equivalent using the parse tree.
/Create sample table
tab:([]inst:10?`MSFT`GOOG`AAPL;time:10?.z.p;price:10?10f)
/Generate my parse tree to get my functional form
.Q.s parse "select count i by inst from tab"
/Build this into my function
{?[`tab;();(enlist x)!enlist x;(enlist `countDistinct)!enlist (#:;`i)]} `inst
Note that you have to pass the column in as a symbol. Additionally the #:i is just the k equivalent to count i.
Update for multiple columns
tab:([]inst:10?`MSFT`GOOG`AAPL;time:10?.z.p;price:10?10f;cntr:10`HK`SG`UK`US)
{?[`tab;();(x)!x;(enlist `countDistinct)!enlist (#:;`i)]} `inst`cntr
To get the functional form of a select statement, I recommend using buildSelect. Also, reduce the scope of parenthesis, i.e. use enlist[`countDistinct] instead of (enlist `countDistinct).
The following lines of Q code all throw an error, because when the statement "local" is parsed, the local variable is not in the correct scope.
{local:1; value "local"}[]
{[local]; value "local"}[1]
{local:1; eval parse "local"}[]
{[local]; eval parse "local"}[1]
Is there a way to reach the local variable from inside the parsed string?
Note: This is a simplification of the actual problem I'm grappling with, which is to write a function that executes a query, accepting a list of columns which it should return. I imagine the finished product looking something like this:
getData:{[requiredColumns, condition]
value "select ",(", " sv string[requiredColumns])," from myTable where someCol=condition"
}
The condition parameter in this query is the one that isn’t recognised and I do realise I could append it’s value rather than reference it inside a string, but the real query uses lots of local variables including tables etc, so it’s not as easy as just pulling all the variables out of the string before calling value on it.
I'm new to KDB and Q, so if anyone has a better way to achieve the same effect I'm happy to be schooled on the proper way to achieve this outcome in Q. Would still be interested to know in the variable access thing is possible though.
In the first example, you are right that local is not within the correct scope, as value is looking for the global variable local.
One way to get around this is to use a namespace, which will define the variable globally, but can only be accessed by calling that namespace. In the modified example below I have defined local in the .ns namespace
{.ns.local:1; value ".ns.local"}[]
For the problem you are facing with selecting, if requiredColumns is a symbol list of columns you can just use the take operator # to select them.
getData:{[requiredColumns] requiredColumns#myTable}
For more advanced queries using variables you may have to use functional select form, explained here. This will allow you to include variables in the where and by clause of the select statement
The same example in functional form would be (no by clause, only select and where):
getData:{[requiredColumns;condition] requiredColumns:(), requiredColumns;
?[myTable;enlist (=;`someCol;condition);0b;requiredColumns!requiredColumns]}
The first line ensures that requiredColumns is a list even if the user enters a single column name
value will look for a variable in the global scope that's why you are getting an error. You can directly use local variables like you are doing that in your function.
Your function is mostly correct, just need a slight correction to append condition(I have mentioned that below). However, a better approach would be to use functional select in this case.
Using functional select:
q) t:([]id:`a`b; val:3 4)
q) gd: {?[`t;enlist (=;`val;y);0b;((),x)!(),x]}
q) gd[`id;3] / for single column
Output:
id
-
1
q) gd[`id`val;3] / for multiple columns
In case your condition column is of type symbol, then enlist your condition value like:
q) gd: {?[`t;enlist (=;`id;y);0b;((),x)!(),x]}
q) gd[`id;enlist `a]
You can use parse to get a functional form of qsql queries:
q) parse " select id,val from t where id=`a"
?
`t
,,(=;`id;,`a)
0b
`id`val!`id`val
Using String concat(your function):
q)getData:{[requiredColumns;condition] value "select ",(", " sv string[requiredColumns])," from t where id=", .Q.s1 condition}
q) getData[enlist `id;`a] / for single column
q) getData[`id`val;`a] / for multi columns