How to insert null in column array datatype in ksql - confluent-platform

I am creating a stream where data type is Array. a null value comes so handle this i am creating another stream on top of previous stream, and giving a case like if its null put []. but its not working.
i have tried [],{},[0],0 in cases.
CREATE STREAM stream1
(
id VARCHAR,
tags ARRAY<INT>,
feed_id VARCHAR,
status INT,
updated_at VARCHAR
)
WITH (kafka_topic='origin_topic', value_format='JSON');
Create Stream stream2 AS
select
id AS id,
case when tags is NULL THEN [] END ELSE tags END as tags,
case when feed_id is NULL THEN '0' ELSE feed_id END as feed_id,
case when status is NULL THEN 0 ELSE status END as status,
case when updated_at is NULL THEN '0' ELSE updated_at END as
updated_at
from stream1 PARTITION BY id;

ksqlDB supports an array constructor ARRAY[], though if you're running an old version this may not be available. Also, it does not yet support empty arrays, so you'll need to pass at least one parameter, e.g. ARRAY[1] will create an array with a single INT/BIGINT element.

Related

Get row number of row to be inserted in Postgres trigger that gives no collisions when inserting multiple rows

Given the following (simplified) schema:
CREATE TABLE period (
id UUID NOT NULL DEFAULT uuid_generate_v4(),
name TEXT
);
CREATE TABLE course (
id UUID NOT NULL DEFAULT uuid_generate_v4(),
name TEXT
);
CREATE TABLE registration (
id UUID NOT NULL DEFAULT uuid_generate_v4(),
period_id UUID NOT NULL REFERENCES period(id),
course_id UUID NOT NULL REFERENCES course(id),
inserted_at timestamptz NOT NULL DEFAULT now()
);
I now want to add a new column client_ref, which identifies a registration unique within a period, but consists of only a 4-character string. I want to use pg_hashids - which requires a unique integer input - to base the column value on.
I was thinking of setting up a trigger on the registration table that runs on inserting a new row. I came up with the following:
CREATE OR REPLACE FUNCTION set_client_ref()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
DECLARE
next_row_number integer;
BEGIN
WITH rank AS (
SELECT
period.id AS period_id,
row_number() OVER (PARTITION BY period.id ORDER BY registration.inserted_at)
FROM
registration
JOIN period ON registration.period_id = period.id ORDER BY
period.id,
row_number
)
SELECT
COALESCE(rank.row_number, 0) + 1 INTO next_row_number
FROM
period
LEFT JOIN rank ON (rank.period_id = period.id)
WHERE
period.id = NEW.period_id
ORDER BY
rank.row_number DESC
LIMIT 1;
NEW.client_ref = id_encode (next_row_number);
RETURN NEW;
END
$function$
;
The trigger is set-up like: CREATE TRIGGER set_client_ref BEFORE INSERT ON registration FOR EACH ROW EXECUTE FUNCTION set_client_ref();
This works as expected when inserting a single row to registration, but if I insert multiple within one statement, they end up having the same client_ref. I can reason about why this happens (the rows don't know about each other's existence, so they assume they're all just next in line when retrieving their row_order), but I am not sure what a way is to prevent this. I tried setting up the trigger as an AFTER trigger, but it resulted in the same (duplicated) behaviour.
What would be a better way to get the lowest possible, unique integer for the rows to be inserted (to base the hash function on) that also works when inserting multiple rows?

Postgres UPSERT row with null constraint value not working

Im trying to write a stored procedure in which I can upsert a row even if one of the values in the key is null. I've read the documentation and found that Postgres doesn't work with comparing equality of null values.
I've also read other forum posts and noticed that what I want to achieve can be done through a partial index. I'm able to successfully get a constraint violation, however my "on conflict" never gets hit when i pass in a value that has a null birthday.
I want to be able to pass in a null birthday and update an ID for a person even if their birthday is null.
(
id bigint not null,
name text,
birthday date
);
I create an index and partial index so that it allows birthday to be null
CREATE UNIQUE INDEX name_birthday_key on people (name, birthday);
CREATE UNIQUE INDEX birthday_null_key on people (name) where birthday is null;
create or replace procedure store_person(_identifier bigint, _name character varying, _birthday date)
language plpgsql
as
$$
begin
insert into people (
id, name, birthday
)
values (
_identifier, _name, _birthday
)
on conflict (name, birthday)
do update
set
id = _identifier
where people.birthday = _date and people.name = _name;
end
$$;
if I run:
call public.store_person(1, 'Bob', '1955-01-09')
call public.store_person(2, 'Bob', '1955-01-09')
i successfully see that the only row in the DB is Bob with an ID of 2.
however, if i run
call public.store_person(3, 'Joe', null)
call public.store_person(4, 'Joe', null)
the only row i get is ID 3. the second insert for ID 4 never updates the existing row. I do get a violation error but the "on conflict" update never is hit.
can someone point me in the right direction of how to do this?
The CONFLICT doesn't match because NULLis not equal to NULL. This is not a PostgreSQL thing, it's defined in SQL standard.
Use something like COALESCE(birthday, '0001-01-01') when inserting your data, that will match; and remove the partial index.
Your code has an error, in DO UPDATE...WHERE: there's nothing named _date, should be _birthday.

A view that shows the name of the server, the id of the instance and the number of active sessions (a session is active if the end timestamp is null)

CREATE TABLE instances(
ser_name VARCHAR(20) NOT NULL,
id INTEGER NOT NULL ,
ser_ip VARCHAR(16) NOT NULL,
status VARCHAR(10) NOT NULL,
creation_ts TIMESTAMP,
CONSTRAINT instance_id PRIMARY KEY(id)
);
CREATE TABLE characters(
nickname VARCHAR(15) NOT NULL,
type VARCHAR(10) NOT NULL,
c_level INTEGER NOT NULL,
game_data VARCHAR(40) NOT NULL,
start_ts TIMESTAMP ,
end_ts TIMESTAMP NULL ,
player_ip VARCHAR(16) NOT NULL,
instance_id INTEGER NOT NULL,
player_username VARCHAR(15),
CONSTRAINT chara_nick PRIMARY KEY(nickname)
);
ALTER TABLE
instances ADD CONSTRAINT ins_ser_name FOREIGN KEY(ser_name) REFERENCES servers(name);
ALTER TABLE
instances ADD CONSTRAINT ins_ser_ip FOREIGN KEY(ser_ip) REFERENCES servers(ip);
ALTER TABLE
characters ADD CONSTRAINT chara_inst_id FOREIGN KEY(instance_id) REFERENCES instances(id);
ALTER TABLE
characters ADD CONSTRAINT chara_player_username FOREIGN KEY(player_username) REFERENCES players(username);
insert into instances values
('serverA','1','138.201.233.18','active','2020-10-20'),
('serverB','2','138.201.233.19','active','2020-10-20'),
('serverE','3','138.201.233.14','active','2020-10-20');
insert into characters values
('characterA','typeA','1','Game data of characterA','2020-07-18 02:12:12','2020-07-18 02:32:30','192.188.11.1','1','nabin123'),
('characterB','typeB','3','Game data of characterB','2020-07-19 02:10:12',null,'192.180.12.1','2','rabin123'),
('characterC','typeC','1','Game data of characterC','2020-07-18 02:12:12',null,'192.189.10.1','3','sabin123'),
('characterD','typeA','1','Game data of characterD','2020-07-18 02:12:12','2020-07-18 02:32:30','192.178.11.1','2','nabin123'),
('characterE','typeB','3','Game data of characterE','2020-07-19 02:10:12',null,'192.190.12.1','1','rabin123'),
('characterF','typeC','1','Game data of characterF','2020-07-18 02:12:12',null,'192.188.10.1','3','sabin123'),
('characterG','typeD','1','Game data of characterG','2020-07-18 02:12:12',null,'192.188.13.1','1','nabin123'),
('characterH','typeD','3','Game data of characterH','2020-07-19 02:10:12',null,'192.180.17.1','2','bipin123'),
('characterI','typeD','1','Game data of characterI','2020-07-18 02:12:12','2020-07-18 02:32:30','192.189.18.1','3','dhiraj123'),
('characterJ','typeD','3','Game data of characterJ','2020-07-18 02:12:12',null,'192.178.19.1','2','prabin123'),
('characterK','typeB','4','Game data of characterK','2020-07-19 02:10:12','2020-07-19 02:11:30','192.190.20.1','1','rabin123'),
('characterL','typeC','2','Game data of characterL','2020-07-18 02:12:12',null,'192.192.11.1','3','sabin123'),
('characterM','typeC','3','Game data of characterM','2020-07-18 02:12:12',null,'192.192.11.1','2','sabin123');
here I need a view that shows the name of the server, the id of the instance and the number of active sessions (a session is active if the end timestamp is null). do my code wrong or something else? i am starting to learn so hoping for positive best answers.
my view
create view active_sessions as
select i.ser_name, i.id, count(end_ts) as active
from instances i, characters c
where i.id=c.instance_id and c.end_ts = null
group by i.ser_name, i.id;
This does not do what you want:
where i.id = c.instance_id and c.end_ts = null
Nothing is equal to null. You need is null to check a value against null.
Also, count(end_ts) will always produce 0, as we know already that end_ts is null, which count() does not consider.
Finally, I would highly recommend using a standard join (with the on keyword), rather than an implicit join (with a comma in the from clause): this old syntax from decades ago should not be used in new code. I think that a left join is closer to what you want (it would also take in account instances that have no character at all).
So:
create view active_sessions as
select i.ser_name, i.id, count(c.nickname) as active
from instances i
left join characters c on i.id = c.instance_id and c.end_ts is null
group by i.ser_name, i.id;

Block records from updating

I have the following tables
tbl_orders
CREATE TABLE tbl_orders (
id integer NOT NULL,
customer_name character varying NOT NULL,
is_archived boolean NOT NULL
);
tbl_order_items
CREATE TABLE tbl_order_items (
id integer NOT NULL,
product_name character varying NOT NULL,
quantity integer NOT NULL,
order_id int NOT NULL
);
In my application I have the possibility to archive an order, which set the boolean isArchived to true to that order record. The order record can have multiple order items which I want to prevent from being updated when the order has the boolean isArchived set to true. Do I have to set an isArchived boolean on the order items level?
Is this possible to prevent this on database level?
You can create a BEFORE UPDATE trigger on tbl_order_items FOR EACH ROW that throws an error when the order NEW.order_id is archived.
That is the most elegant and normalized solution, but it requires that a trigger runs whenever a row in tbl_order_items is updated.

Postgresql - retrieving referenced fields in a query

I have a table created like
CREATE TABLE data
(value1 smallint references labels,
value2 smallint references labels,
value3 smallint references labels,
otherdata varchar(32)
);
and a second 'label holding' table created like
CREATE TABLE labels (id serial primary key, name varchar(32));
The rationale behind it is that value1-3 are a very limited set of strings (6 options) and it seems inefficient to enter them directly in the data table as varchar types. On the other hand these do occasionally change, which makes enum types unsuitable.
My question is, how can I execute a single query such that instead of the label IDs I get the relevant labels?
I looked at creating a function for it and stumbled at the point where I needed to pass the label holding table name to the function (there are several such (label holding) tables across the schema). Do I need to create a function per label table to avoid that?
create or replace function translate
(ref_id smallint,reference_table regclass) returns varchar(128) as
$$
begin
select name from reference_table where id = ref_id;
return name;
end;
$$
language plpgsql;
And then do
select
translate(value1, labels) as foo,
translate(value2, labels) as bar
from data;
This however errors out with
ERROR: relation "reference_table" does not exist
All suggestions welcome - at this point a can still alter just about anything...
CREATE TABLE labels
( id smallserial primary key
, name varchar(32) UNIQUE -- <<-- might want this, too
);
CREATE TABLE data
( value1 smallint NOT NULL REFERENCES labels(id) -- <<-- here
, value2 smallint NOT NULL REFERENCES labels(id)
, value3 smallint NOT NULL REFERENCES labels(id)
, otherdata varchar(32)
, PRIMARY KEY (value1,value2,value3) -- <<-- added primary key here
);
-- No need for a function here.
-- For small sizes of the `labels` table, the query below will always
-- result in hash-joins to perform the lookups.
SELECT l1.name AS name1, l2.name AS name2, l3.name AS name3
, d.otherdata AS the_data
FROM data d
JOIN labels l1 ON l1.id = d.value1
JOIN labels l2 ON l2.id = d.value2
JOIN labels l3 ON l3.id = d.value3
;
Note: labels.id -> labels.name is a functional dependency (id is the primary key), but that doesn't mean that you need a function. The query just acts like a function.
You can pass the label table name as string, construct a query as string and execute it:
sql = `select name from ` || reference_table_name || `where id = ` || ref_id;
EXECUTE sql INTO name;
RETURN name;