I have a table where I want to update few columns of a row based on a condition
q)t:([] id:10 20; l1:("Blue hor";"Antop"); l2:("Malad"; "KC"); pcd:("NCD";"FRB") )
When I used update statement, it throws 'length error
q)update l1:"Chin", l2:"Gor" from t where id=10
'length
q)update l1:"Chin", l2:"Gor" from `t where id=10
'length
I read below in Q for Mortals but is there any way to update few columns of a row based on a condition?
The actions in the Where phrase and the Update phrase are vector
operations on entire column lists. This is the Zen of update.
Please try statement below:
update l1:count[i]#enlist"Chin", l2:count[i]#enlist"Gor" from t where id=10
It works regardless to how many rows are matched to where clause.
On update, length of assigned list should be equal to number of updated rows. Q treats string as list of characters. This is why, when you assign "Chin" to l1, Q tries to assign list of length 4, when list of length 1 is expected. This causes 'length error.
count[i]#enlist"Chin" creates list of N repeated values: ("Chin";"Chin";...). Where N is number of updated rows. This fixes the issue
As you are dealing with Char-lists here (rather than symbol), you need use enlist:
q)update l1:enlist "Chin", l2:enlist "Gor" from t where id=10
id l1 l2 pcd
----------------------
10 "Chin" "Gor" "NCD"
20 "Antop" "KC" "FRB"
Otherwise you are trying to update a vector of length 1 (t where id=10) with a vector of length 4 ("Chin"), or 3 ("Gor").
To update the table like this, you need to add the enlist keyword:
q)update l1:enlist "Chin", l2:enlist "Gor" from t where id=10
id l1 l2 pcd
----------------------
10 "Chin" "Gor" "NCD"
20 "Antop" "KC" "FRB"
This is because you need to add lists of strings rather than just strings
Related
For a table t with a custom field c which is dictionary I could use select with limit expression, but simple select failes:
q)r1: `n`m`k!111b;
q)r2: `n`m`k!000b;
q)t: ([]a:1 2; b:10 20; c:(r1; r2));
q)t
a b c
----------------
1 10 `n`m`k!111b
2 20 `n`m`k!000b
q)select[2] c[`n] from t
x
-
1
0
q)select c[`n] from t
'type
[0] select c[`n] from t
^
Is it a bug, or am I missing something?
Upd:
Why does select [2] c[`n] from t work here?
Since c is a list, it does not support key indexing which is why it has returned a type
You need to index into each element instead of trying to index the column.
q)select c[;`n] from t
x
-
1
0
A list of confirming dictionaries outside of this context is equivalent to a table, so you can index like you were
q)c:(r1;r2)
q)type c
98h
q)c[`n]
10b
I would say that the way complex columns are represented in memory makes this not possible. I suspect that any modification that creates a copy of a subset of the elements will allow column indexing as the copy will be formatted as a table.
One example here is serialising and deserialising the column (not recommended to do this). In the case of select[n] it is selecting a subset of 2 elements
q)type exec c from t
0h
q)type exec -9!-8!c from t
98h
q)exec (-9!-8!c)[`n] from t
10b
I have a joined table which consists of list of list of characters.
q)t:([] a:`c`d; b:("";"fill"));
q)s:([] a:`b`c`c; b:("";"";""))
q)select from t lj select b by a from s
Output:
a b
---------
c ("";"") / This is the culprit want to replace it with null character
d "fill"
The output of join consists of a list of list of empty characters.
I want to replace that with empty character.
Expected output:
a b
---------
c ""
d "fill"
Tried: Few Unsuccessful attempts
q)update b:?[null in b;raze b;b]from select from t lj select b by a from s
q)update b:?["" in b;raze b;b]from select from t lj select b by a from s
To replace a list of list of empty strings with empty string, you can try below query:
q) select from t lj select (b;"")all""~/:b by a from s
Output:
a b
--------
c ""
d "fill"
Explanation:
Basically, empty strings list is coming from group command on the right table. So during the grouping stage, we can match if all the items in a grouped list (b column values) for particular a value are an empty string. And if they are just replacing them with a single empty string.
q) select (b;"")all""~/:b by a from s
a| b
-| --
b| ""
c| ""
For a = c , b grouped values are ("";""). Lets break down the command:
q) b:("";"")
q) ""~/:b / output 11b
q) all ""~/:b / output 1b
q)(b;"") all ""~/:b / output ""
The last command is list indexing. If the return from the previous command is 1b which means all items are empty strings, then return "" else return actual b.
Edit:
Based on the discussion in the comment section of TerryLynch's answer, it looks like your requirement is:
if all values of b list after grouping are empty strings then return a single empty string.
if values of b are a mixture of empty strings and non-empty strings, then remove all empty strings.
For that, you could use the below query:
q) select from t lj select b:raze ("";b except enlist "") by a from s
But that would result in different types for different values in b column. An empty string will be 10h and all non-empty string list will be 0h.
For consistent type, can use below query which returns enlist"" instead of "" but that will not be an empty string:
q) select from t lj select b:{(c;enlist "")()~c:x except enlist ""}b by a from s
Instead of trying to fix the adverse outcome I think you need to decide what you want to do with the duplicate c rows in the s table. You're grouping by the a column but it has duplicates so how should it behave .... should it take the first value, should it take the last value? Should it append the two strings together? If you solve that then you avoid this problem, for example:
q)t lj select last b by a from s
a b
--------
c ""
d "fill"
An alternative solution would be to simply raze all the results of b together. Less where clauses in use and less match (~) operations.
q)update raze'/[b] from (t lj select b by a from s)
a b
--------
c ""
d "fill"
Here I've used over to account for more an unknown level of enlistment, as a precaustion, and then applied it to each row from the lj. For your case, an even faster solution would be
update raze each b from (t lj select b by a from s)
This will give different results than Rahuls answers
q)update raze each b from (t lj select b by a from s)
a b
--------
c "str"
d "fill"
q) select from t lj select (b;"")all""~/:b by a from s
a b
------------
c ("";"str")
d "fill"
q)update raze each b from (t lj select b by a from s)
a b
--------
c "str"
d "fill"
Say I have a table called list, where there are items like these (the ids are random uuids):
id rank text
--- ----- -----
x 0 Hello
x 1 World
x 2 Foo
x 3 Bar
x 4 Baz
I want to maintain the property that rank column always goes from 0 to n-1 (n being the number of rows)---if a client asks to insert an item with rank = 3, then the pg server should push the current 3 and 4 to 4 and 5, respectively:
id rank text
--- ----- -----
x 0 Hello
x 1 World
x 2 Foo
x 3 New Item!
x 4 Bar
x 5 Baz
My current strategy is to have a dedicated insertion function add_item(item) that scans through the table, filter out items with rank equal or greater than that of the item being inserted, and increment those ranks by one. However, I think this approach will run into all sorts of problems---like race conditions.
Is there a more standard practice or more robust approach?
Note: The rank column is completely independent of rest of the columns, and insertion is not the only operation I need to support. Think of it as the back-end of a sortable to-do list, and the user can add/delete/reorder the items on the fly.
Doing verbatim what you suggest might be difficult or not possible at all, but I can suggest a workaround. Maintain a new column ts which stores the time a record is inserted. Then, insert the current time along with rest of the record, i.e.
id rank text ts
--- ----- ----- --------------------
x 0 Hello 2017-12-01 12:34:23
x 1 World 2017-12-03 04:20:01
x 2 Foo ...
x 3 New Item! 2017-12-12 11:26:32
x 3 Bar 2017-12-10 14:05:43
x 4 Baz ...
Now we can easily generate the ordering you want via a query:
SELECT id, rank, text,
ROW_NUMBER() OVER (ORDER BY rank, ts DESC) new_rank
FROM yourTable;
This would generate 0 to 5 ranks in the above sample table. The basic idea is to just use the already existing rank column, but to let the timestamp break the tie in ordering should the same rank appear more than once.
you can wrap it up to function if you think its worth of:
t=# with u as (
update r set rank = rank + 1 where rank >= 3
)
insert into r values('x',3,'New val!')
;
INSERT 0 1
the result:
t=# select * from r;
id | rank | text
----+------+----------
x | 0 | Hello
x | 1 | World
x | 2 | Foo
x | 3 | New val!
x | 4 | Bar
x | 5 | Baz
(6 rows)
also worth of mention you might have concurrency "chasing condition" problem on highly loaded systems. the code above is just a sample
You can have a “computed rank” which is a double precision and a “displayed rank” which is an integer that is computed using the row_number window function on output.
When a row is inserted that should rank between two rows, compute the new rank as the arithmetic mean of the two ranks.
The advantage is that you don't have to update existing rows.
The down side is that you have to calculate the displayed ranks before you can insert a new row so that you know where to insert it.
This solution (like all others) are subject to race conditions.
To deal with these, you can either use table locks or serializable transactions.
The only way to prevent a race condition would be to lock the table
https://www.postgresql.org/docs/current/sql-lock.html
Of course this would slow you down if there are lots of updates and inserts.
If can somehow limit the scope of your updates then you can do a SELECT .... FOR UPDATE on that scope. For example if the records have a parent_id you can do a select for update on the parent record first and any other insert who does the same select for update would have to wait till your transaction is done.
https://www.postgresql.org/docs/current/explicit-locking.html#:~:text=5.-,Advisory%20Locks,application%20to%20use%20them%20correctly.
Read the section on advisory locks to see if you can use those in your application. They are not enforced by the system so you'll need to be careful of how you write your application.
If this is the dictionary of constraint:
dictName:`region`Code;
dictValue:(`NJ`NY;`EEE213);
dict:dictName!dictValue;
I would like to pass the dict to a function and depending on how many keys there are and let the query react accordingly. If there is one key region, then I would like to put it as
select from table where region in dict`region;
The same thing is for code. But if I pass two keys, I would like the query knows and pass it as:
select form table where region in dict`region,Code in dict`code;
Is there any way to do this?
I came up this code:
funcForOne:{[constraint]?[`bce;enlist(in;constraint;(`dict;enlist constraint));0b;()]};
funcForAll[]
{[dict]$[(null dict)~1;select from bce;($[(count key dict)=1;($[`region in (key dict);funcForOne[`region];funcForOne[`Code]]);select from bce where region in dict`region,rxmCode in dict`Code])]};
It works for one and two constraint. but when I called funcForAll[] it gives type error. How should I change it? i think it is from null dict~1
I tried count too. but doesn't work too well.
Update
So I did this but I have some error
tab:([]code:`B90056`B90057`B90058`B90059;region:`CA`NY`NJ`CA);
dictKey:`region`Code;dictValue:(`NJ`NY;`B90057);
dict:dictKey!dictValue;
?[tab;f dict;0b;()];
and I got 'NY error. Do you know why? Also,if I pass a null dictionary it doesn't seem working.
As I said funtional form would be the better approach but if your requirement is very limited as you said then you can consider other solution as below:
Note: Assuming all dictionary keys will be in table columns list.
q) f:{[dict] if[0=count dict;:select from t];
select from t where (#[key dict;t]) in {$[any 0<=type each value x;flip ;enlist ]x}[dict] }
Explanation:
1. convert dict to table depending on the values type. Flip if any value is a general list else enlist.
$[any 0<=type each value dict;flip ;enlist ]dict
Get subset of table t which consists only of dictionary keys as columns.
#[key dict;t]
get rows where (2) in (1)
Basically we are using below form of querying and matching:
q)t1:([]id:1 2;s:`a`b);
q)t2:([]id:1 3 ;s:`a`b);
q)select from t1 where ([]id;s) in t2
If you're just using in, you can do something like:
f:{{[x;y](in),'key[y],'(),x}[;x]enlist each value[x]}
So that:
q)d
a| 10 1
b| ,`a
q)f d
in `a 10 1
in `b ,`a
q)t
a b c
------
1 a 10
2 b 20
3 c 30
q)?[t;f d;0b;()]
a b c
------
1 a 10
Note that because of the enlist each the resulting list is enlisted so that singletons work too:
q)d:enlist[`a]!enlist 1
q)d
a| 1
q)?[t;f d;0b;()]
a b c
------
1 a 10
Update to secondary question
This still works with empty dict, i.e. ()!(). I'm passing in the dictionary variable.
In your 2nd question your dictionary is not constructed correctly (also remember q is case sensitive). Also your values need to be enlisted. Look up functional select in the reference pages on the kx site, you'll see that you need to enlist the symbol lists to differentiate them from column name declarations
`region`code!(enlist `NY`NJ;enlist `B90057)
very simple, silly question. Consider the following table:
tt:([]Id:`6`7`12 ;sym:`A`B`C;symlist:((`A`B`M);(`X`Y`Z);(`H`F`C)))
Id sym symlist
---------------
6 A `A`B`M
7 B `X`Y`Z
12 C `H`F`C
I would like to select all rows in tt where the element in sym is contained in the list symlist. In this case, it means just the first and third rows. However, the following query gives me a type error.
select from tt where sym in symlist
(`type)
Whats the proper way to do this? Thanks
You want to use the ' (each-both) adverb, so that they "pair up" so to speak. Recall that sym is just list, and symlist is a list of lists. You want to check each element in sym with the respective sub-list in symlist. You do this by telling it to "pair up".
q)tt:([]id:6712; sym:`A`B`C; symlist:(`A`B`M;`X`Y`Z;`H`F`C))
q)select from tt where sym in'symlist
id sym symlist
----------------
6712 A A B M
6712 C H F C
It's not entirely clear to me why your query results in a type error, so I'd be interested in hearing other people's responses.
q)select from tt where sym in symlist
'type
in
`A`B`C
(`A`B`M;`X`Y`Z;`H`F`C)
q)select from tt where {x in y}[sym;symlist]
id sym symlist
--------------
In reponse to JPCs answer (couldn't format this as a comment)....
Type error possibly caused by applying "where" to a scalar boolean
q)(`a`b`c) in (`a`g`b;`u`i`o;`g`c`t)
0b
q)where (`a`b`c) in (`a`g`b;`u`i`o;`g`c`t)
'type
Also, the reason the {x in y} lambda doesn't cause the error is because the "in" is obscured and is not visible to the parser (parser doesn't look inside lambdas)
q)0N!parse"select from tt where {x in y}[sym;symlist]";
(?;`tt;,,({x in y};`sym;`symlist);0b;())
Whereas the parser can "see" the "in" in the first case
q)0N!parse"select from tt where sym in symlist";
(?;`tt;,,(in;`sym;`symlist);0b;())
I'm guessing the parser tries to do some optimisations when it sees the "in"