We are trying to build functional queries using qPython. Strating with simple examples to build where conditions at the run time.
we defined a q function on out KDB server like
fn:{[c]
t: (select from tbl);
:?[t;c;0b;()];
}
in Python we open a connection and send the condition
c = [['=', numpy.string_('TradeId'), 123456]]
result = conn.sendSync('fn', c)
when I do this, in q console I see that = operator as "=".
the question is how to pass operators
#terrylynch answer works
for specifically qPython
from python sending works
c = [[ qtype.QLambda( '{x in y}'), numpy.string_('TradeId'), [123, 456,789]]]
You could value the string/char on the kdb side to convert it from a string/char to the underlying kdb operator. This works in your example but you might need some extra work/testing to generalise it to all possible operators that you might send:
q)tbl:([]TradeId:0 123456 123456 123456;col2:1 2 3 4)
q)fn:{[c] c:.[c;(::;0);value];t:(select from tbl); :?[t;c;0b;()];}
q)0(`fn;enlist("=";`TradeId;123456))
TradeId col2
------------
123456 2
123456 3
123456 4
q)0(`fn;(("=";`TradeId;123456);("in";`col2;2 4)))
TradeId col2
------------
123456 2
123456 4
Note - Im using 0() to send requests within kdb itself but this should be the equivalent of sending from qPython. The bit you need to change is the c:.[c;(::;0);value]; in your kdb function
Related
I'm trying to implement dynamic subscribers in a kdb-tick system whereby a subset of the events are passed to a given consumer based on a query supplied by the consumer.
For instance given a batch of events i.e.:
flip `source`channel`value!(10?`a`b`c;10?`a`b`c;10?10)
source channel value
--------------------
a a 4
b b 5
a c 4
b a 2
c c 7
c b 8
c a 5
a c 6
b a 4
b a 1
The tickerplant should only send the events without a channel of `c i.e.
source channel value
--------------------
a a 4
b b 5
b a 2
c b 8
c a 5
b a 4
b a 1
I have tried to implement this by parsing a dynamic conditional as follows:
q).tp.subscribers
hp | host isOpen port h subs
----------| --------------------------------------------------
:test:5000| test 0 5000 6 "enlist(not;(in;`channel;enlist`c))"
Whereby subs is a conditional argument to a functional select statement that is used in the following code:
.tp.send:{neg[x] y};
.tp.reval:{[batch;subscriber]
.tp.send[raze .subscriber`h] reval[parse["?[batch;",raze[subscriber`subs],";0b;()]"]]
};
// Called with event batch
.tp.broadcast:{[batch]
.tp.reval[batch]'[select from .tp.subscribers where isOpen]
};
This fails on account of batch not being addressable in a non global context through a functional select statement.
I was wondering how this functionality might be effectively achieved?
Could anyone advise me on or point me to information pertaining to a solution for this problem statement.
Your advice would very much appreciated.
Apologies if this is a newbie question.
Thanks
I think the fact that you're expecting a string form of a conditional argument is part of your problem (that in turn requires you to parse a stringified functional select and that parse assumes global).
Why not expect a list form of the conditional argument instead? Then there's no need to parse and you can create a local functional select. E.g.
.tp.subscribers:([hp:1#`:test:5000]subs:enlist(not;(in;`channel;1#`c)))
q){[batch] reval ?[batch;(0!.tp.subscribers)`subs;0b;()]}flip `source`channel`value!(10?`a`b`c;10?`a`b`c;10?10)
source channel value
--------------------
a a 4
b b 5
b a 2
c b 8
c a 5
b a 4
b a 1
Or have the user specify a lambda and run that (though I guess you would lose the ability to use reval in that case):
.tp.subscribers:([hp:1#`:test:5000]subs:enlist{select from x where not channel=`c})
q){[batch] #[first(0!.tp.subscribers)`subs;batch;()]}flip `source`channel`value!(10?`a`b`c;10?`a`b`c;10?10)
source channel value
--------------------
a b 9
c b 0
b b 0
b a 9
b a 3
b a 9
I would like to take an existing table that I deserialized from a binary file or obtained from a remote process and generate code that will create an empty copy of the table so that I can have a human-readable representation of the schema that I can use to easily re-create the table.
For example, assume I have a trade table in memory and I want to generate code that will return an empty table of the same schema.
q)show meta trade
c | t f a
-----| -----
time | n
sym | s g
price| f
size | i
stop | b
cond | c
ex | c
I'm aware I can obtain an empty copy of trade by running 0#trade. However, I'd like to have a general function (let's say it's called getSchema) that will behave something like this:
q) getSchema trade
"trade:([]time:`timespan$(); sym:`g#`symbol$(); price:`float$(); size:`int$(); stop:`boolean$(); cond:`char$(); ex:`char$())"
I think it would be straightforward to implement this by processing the result of meta trade, but I was wondering if there was was a more straightforward or publicly available implementation of this function. Thanks.
I haven't seen such function available in public. An example is below, but it does not cover all cases (this is left for an exercise)
getSchema: {
typeMapping: "nsfibc"!("timespan";"symbol";"float";"int";"boolean";"char");
c: exec c from meta x;
t: exec t from meta x;
statement: string[x],":([]";
statement,: "; " sv string[c],'": `",/:(typeMapping#t),\:"$()";
statement,: ")";
statement
};
//Expects table's name as symbol
getSchema`trade
What is not covered:
attributes: Attribute should go in the middle of "; " sv string[c],'": _code_for_attribute_ `",/:(typeMapping#t),\:"$()" statement
types: typeMapping must be enriched to cover the rest of Q types
keys. If table is keyed, then keyed columns are listed inside square brackets t: ([keyed_columns] other_columns)
foreign keys. To be fair they are seldom in use, so I would ignore them
The following should work nicely, however it will only render types atomic types, which will conform to how tables are normally defined. This method just utilizes the string representation of the empty schema that you can achieve via -3!
This should handle keyed tables and attributes but not foreign key.
getSchema:{[x]
/ Render the types with -3!
typs:ssr[-3!value flip 0!0#get x;"\"\"";"`char$()"];
/ Append the column names to the types
colNameType:(string[cols 0!get x],\:":"),'";" vs 1_-1_typs;
/ Ensure the correct keys are shown, add in ;
colnametyp:#[colNameType;count[keys x] - 1;,;"]"],\:";";
/ Combine
raze string[x],":([",(-1_raze colnametyp),")"
}
q)trade:2!([]time:`timespan$(); sym:`g#`symbol$(); price:`float$(); size:`int$(); stop:`boolean$(); cond:`char$(); ex:`char$());
q)getSchema `trade
"trade:([time:`timespan$();sym:`g#`symbol$()];price:`float$();size:`int$();stop:`boolean$();cond:`char$();ex:`char$())"
Consider the following example:
test:([] name:`symbol$(); secondColumn:`int$());
insert[`test;(`John;1)];
myvar:exec name from test;
Now myvar is now:
q)myvar
,`John
So to select the actual result, I have to do:
q)myvar[0]
`John
I understand this is because of the initialisation, so is there a way to make myvar contain the actual value immediately?
Array access with [0] or first is the correct way if you want an "atomic" variable.
myvar:first exec name from test;
A list with a single element in KDB can be created in multiple ways (which is something you are getting in myvar)
q)enlist `John
,`John
q)(),`John
,`John
A KDB table is basically a flip of a dictionary of lists.
`name`secondColumn!(`John`James;1 2) /Dictionary of lists
name | John James
secondColumn| 1 2
q)test2:flip `name`secondColumn!(`John`James;1 2)
name secondColumn
------------------
John 1
James 2
Both of the following commands achieve the same results :
q)exec name from test2
q)test2[`name]
`John`James
When you selected the test column using the exec command it returned all the elements of the list (a list with one element)
Apart from the ways explained in the accepted answer, there are few more ways (slightly different however) you can get the first element returned from the table.
q)exec name[0] from test
q)test[`name][0]
q)exec first name from test
I have a table like so:
Task 1 | $59,700
Task 2 | $59,700
Task 3 | $59,700
10% Off | $xx,xxx
-------------------
Total: $xx,xxx
I'd like to use formulas in Word to calculate this.
For the first value, I'm using =SUM(ABOVE)*0.10 I would expect this to yield $17,910.
For the second value, I'm using =SUM(ABOVE)-C5 I'd expect this to yield $161,190.
Unfortunately, the first value and second value both yield $179,100 and I'm not exactly sure why. Appreciate any help and thanks for reading.
For the discount use =SUM(ABOVE) * -0.1 to give you a negative discount. Then use =SUM(ABOVE) by itself for total. Numbers work out as expected.
I'm trying to write an expression for a variable (not parameter), so that I can use/reference it to do a calculation in another textbox. I have multiple datasets, and I need the SUM(SUM(Fields!amount.Value)) for each of these datasets. I am then going to use these numbers in another textbox, adding them with each other. I need some assistance with the syntax. I am able to use SUM by itself, without an issue. For example, this works fine:
=SUM(Fields!Amount.Value, "DataSet1")
But I get an error when trying to amend it to the following (which is what I actually need):
=SUM(SUM(Fields!amt.Value, "Acctrange_90300_90399_InterestExpenses"))
I get an error saying
"The variable expression for the report 'body' uses an aggregate
expression without a scope. A scope is required for all aggregates
used outside of a data region unless the report contains exactly one dataset."
I have a hunch that there's a problem with my syntax/parantheses placement. Any suggestions?
could you please provide a detailed example with maybe sample data and an expected solution?
Because I do not understand why you want to SUM(SUM()). The inner SUM() would result in a single integer value, why would you want to do another SUM on just a single value. even if it worked, it would just be the same value. I apologize if I understood the question wrongly, i can't comment, so i am answering, but I want to know clearly what you are looking for. I can understand if you are trying to do SUM( SUM(), SUM(), SUM()...). As in, sum of sums.
Sorry again for answering just to inquire more info, but I can't see a comment option.
UPDATE:
ok now i think you have something like
| a | b | c | d |
---------------------------
1 | * | * | * | * |
---------------------------
2 | * | * | * | * |
---------------------------
3 | * | * | * | * |
---------------------------
total | w | x | y | z |
so SUM(Fields!a.Value) would give you w, and SUM(Fields!b.Value) would give you x, and so on.
and you want w+x+y+z? If that is so, then you can do this:
add a calculated field to your dataset to calculate the row totals like
{ (a1+b1+c1+d1),(a2+b2+....), .... },
and then from your variable, call
SUM(Fields!CalculatedField.Value).
for above example, you can give the calculated field an expression as:
= CInt(Fields!a.Value)+CInt(Fields!b.Value)+CInt(Fields!c.Value)+CInt(Fields!d.Value)
this would make each entry in the calculated field as sum of each entry in all fields.
So sum of calculated field would give you your answer.
Hope thats what you wanted. Otherwise, well I tried understanding the problem. :)
Outer SUM needs a scope too. Try this:
=SUM(SUM(Fields!amt.Value,"Acctrange_90300_90399_InterestExpenses"),"Acctrange_90300_90399_InterestExpenses")
I expect this will also fail--in some new & different way--but that error will get you closer to a solution.
When SSRS wants to give me headaches like this, I cheat. I put the inner calculation in a hidden text box or name the box that displays it if I want it to show, then refer to it by name. So if I had the sum of the first column in a text box called txtFirstColumnSum, and the second column sum in the text box called txtSecondColumnSum, I'd use:
=cdec(ReportItems!txtFirstColumnSum.Value)+ cdec(ReportItems!txtSecondColumnSum.Value)...
Another way to do it is using the built-in scopes, but writing custom code to handle the math between the fields.
For example if I want the percentage of the current YTD sales over the same period the prior year, and also want to check for divide by zero, I can do it in an expression, but I use it on several levels, so instead I made custom code:
Public Function DeltaPercent(ByVal currentAmount as decimal,ByVal pastAmount as Decimal) as Decimal
Dim difference as decimal
Dim result as decimal
if pastAmount<>0 then
difference=currentAmount-pastAmount
result=difference/pastamount
else
result=0
end if
return result
End Function
Then to get the % change, I call it:
=Code.DeltaPercent(Sum(Fields!SalesYTD.Value, "SalesTerritory"),Sum(Fields!SalesPY1TD.Value, "SalesTerritory"))
One tip I wish I'd known sooner: If you use the syntax
ReportItems("txtFirstColumnSum").Value
you have to make sure you type it right. If you use the bang syntax (!) above, you get intellisense.