KSQL query to display receiver and sender info - apache-kafka

I have a table called users which looks like this:
+----+----------------+
| id | name |
+----+----------------+
| 1 | Blake |
| 2 | Jenn |
+----+----------------+
And i have a STREAM called transactions which looks like this:
+----+----------------+----------------+
| id | sender | receiver |
+----+----------------+----------------+
| 1 | 1 | 2 |
| 2 | 2 | 1 |
+----+----------------+----------------+
So basicly what i want to end up with, looks kind of like this:
+----+----------------+----------------+----------------+----------------+
| id | sender | sender_name | receiver |receiver_name |
+----+----------------+----------------+----------------+----------------+
| 1 | 1 | Blake | 2 | Jenn |
| 2 | 2 | Jenn | 1 | Blake |
+----+----------------+----------------+----------------+----------------+
I've only managed to join the stream and the table partitioned by sender or either receiver, therefore i can only get either the sender info or the receiver info.

You can use two times JOIN with the users table to get the expected result:
SELECT TR.id, TR.sender, SE.name AS sender_name, TR.receiver, RE.name AS receiver_name
FROM transactions TR
JOIN users SE ON SE.id = TR.sender
JOIN users RE ON RE.id = TR.receiver

the point is to create a stream to display that, i've tried that method you described actually but i think it does not work on ksql as it expects a ';' after the first inner join.
Statement: create stream userstransactionsjoinedfinal as select t.txid,t.sender,
up1.firstname as senderfirstname,up1.lastname as senderlastname,up1.phonenumber as
senderphonenumber,up1.email as senderemail,t.receiver,up2.firstname as
receiverfirstname,up2.lastname as receiverlastname,up2.phonenumber as
receiverphonenumber,up2.email as receiveremail, t.SENDERWALLETID,
t.RECEIVERWALLETID,t.status,t.type,t.amount,t.totalfee from transactionsrekeyed
inner join usersnow up1 on up1.id=t.sender inner join usersnow up2 on up2.id =
t.receiver;
Caused by: line 1:483: mismatched input 'inner' expecting ';'
Caused by: org.antlr.v4.runtime.InputMismatchException

Related

Insert a record for evey row from one table into another using one field in postesql

I'm trying to fill a table with data to test a system.
I have two tables
User
+----+----------+
| id | name |
+----+----------+
| 1 | Majikaja |
| 2 | User 2 |
| 3 | Markus |
+----+----------+
Goal
+----+----------+---------+
| id | goal | user_id |
+----+----------+---------+
I want to insert into goal one record for every user only using their IDs (they have to exists) and some fixed or random value.
I was thinking in something like this:
INSERT INTO Goal (goal, user_id) values ('Fixed value', select u.id from user u)
So it will generate:
Goal
+----+-------------+---------+
| id | goal | user_id |
+----+-------------+---------+
| 1 | Fixed value | 1 |
| 2 | Fixed value | 2 |
| 3 | Fixed value | 3 |
+----+-------------+---------+
I could just write a simple PHP script to achieve it but I wonder if is it possible to do using raw SQL only.

Reset column with numeric value that represents the order when destroying a row

I have a table of users that has a column called order that represents the order in they will be elected.
So, for example, the table might look like:
| id | name | order |
|-----|--------|-------|
| 1 | John | 2 |
| 2 | Mike | 0 |
| 3 | Lisa | 1 |
So, say that now Lisa gets destroyed, I would like that in the same transaction that I destroy Lisa, I am able to update the table so the order is still consistent, so the expected result would be:
| id | name | order |
|-----|--------|-------|
| 1 | John | 1 |
| 2 | Mike | 0 |
Or, if Mike were the one to be deleted, the expected result would be:
| id | name | order |
|-----|--------|-------|
| 1 | John | 1 |
| 3 | Lisa | 0 |
How can I do this in PostgreSQL?
If you are just deleting one row, one option uses a cte and the returning clause to then trigger an update
with del as (
delete from mytable where name = 'Lisa'
returning ord
)
update mytable
set ord = ord - 1
from del d
where mytable.ord > d.ord
As a more general approach, I would really recommend trying to renumber the whole table after every delete. This is inefficient, and can get tedious for multi-rows delete.
Instead, you could build a view on top of the table:
create view myview as
select id, name, row_number() over(order by ord) ord
from mytable

SUM values from two tables with GROUP BY and WHERE

I have two tables below named sent_table and received_table. I am attempting to mash them together in a query to achieve output_table. All my attempts so far result in a huge amount of duplicates and totally bogus sum values.
I am assuming I would need to use GROUP BY and WHERE to achieve this goal. I want to be able to filter based on the users name.
sent_table
+----+------+-------+----------+
| id | name | value | order_id |
+----+------+-------+----------+
| 1 | dave | 100 | 1 |
| 2 | dave | 200 | 1 |
| 3 | dave | 300 | 2 |
+----+------+-------+----------+
received_table
+----+------+-------+----------+
| id | name | value | order_id |
+----+------+-------+----------+
| 1 | dave | 400 | 1 |
| 2 | dave | 500 | 2 |
| 3 | dave | 600 | 2 |
+----+------+-------+----------+
output table
+------+----------+----------+
| sent | received | order_id |
+------+----------+----------+
| 300 | 400 | 1 |
| 300 | 1100 | 2 |
+------+----------+----------+
I tried the following with no joy. This does not impose any restrictions on how I would desire to solve this problem. It is just how I attempted to do it.
SELECT *
FROM
( select SUM(value) as sent, order_id FROM sent_table WHERE name='dave' GROUP BY order_id) A
CROSS JOIN
( select SUM(value) as received, order_id FROM received_table WHERE name='dave' GROUP BY order_id) B
Any help would be greatly appreciated.
Do the sums on each table, grouping by order_id, then join the results. To get the rows even if one side is missing, do a FULL OUTER JOIN:
SELECT COALESCE(s.order_id, r.order_id) AS order_id, s.sent, r.received
FROM (
SELECT order_id, SUM(value) AS sent
FROM sent
GROUP BY order_id
) s
FULL OUTER JOIN (
SELECT order_id, SUM(value) AS received
FROM received
GROUP BY order_id
) r
USING (order_id)
ORDER BY 1
Result:
| order_id | sent | received |
| -------- | ---- | -------- |
| 1 | 300 | 400 |
| 2 | | 1100 |
Note the COALESCE on the order_id, so that if it's missing from sent it will be taken from recevied, so that that value will never be NULL.
If you want to have 0 in place of NULL (when e.g. there is no record for that order_id in either sent or received), you would do COALESCE(s.sent, 0) AS sent, COALESCE(r.received, 0) AS received.
https://www.db-fiddle.com/f/nq3xYrcys16eUrBRHT6xLL/2

Grouping fields from one to many relationship in postgres

I need to group fields in a child table in one query in postgres.
I have following data
Stores:
| id | name |
|----|------|
| 1 | abcd |
Features:
| id | store | name | other |
|----|-------|------|-------|
| 1 | 1 | door | metal |
| 2 | 1 | fork | green |
I've got to this query
SELECT
stores.id,
stores.name,
concate_ws(',', features.id, features.name, features.other)
FROM stores
LEFT JOIN features
ON(features.store=stores.id)
WHERE stores.id =1
GROUP BY stores.id, features.id;
This is best I've got so far but yields 2 tuples
1, abcd, (1,door,metal)
1, abcd, (2,fork,green)
I'd like to be able to get one row with the features '|' concatenated like so
1, abcd ,(1,door,metal|2,fork,green)
Use string_agg():
SELECT stores.id,
stores.name,
string_agg(concate_ws(',', features.id, features.name, features.other), '|')
FROM stores
LEFT JOIN features ON features.store=stores.id
WHERE stores.id =1
GROUP BY stores.id, stores.name;

postgres select latest record per entity except on field value

I have a log table that looks something like this:
---------------------------------------------
| id | company | type | date_created |notes|
---------------------------------------------
| 1 | co1 | | 2016-06-30 | ... |
| 2 | co2 | ERROR | 2016-06-30 | ... |
| 3 | co1 | | 2016-06-29 | ... |
| 4 | co2 | | 2016-06-29 | ... |
I have the following which selects the latest record per entity:
SELECT *
FROM import_data_log a
JOIN (SELECT company, max(date_created) date_created
FROM import_data_log
GROUP BY company) b
ON a.company = b.company AND a.date_created = b.date_created
which gives the result:
| 1 | co1 | | 2016-06-30 | ... |
| 2 | co2 | ERROR | 2016-06-30 | ... |
I need to add a condition that does not select the entry with type = ERROR and get the next highest date for that company, so it would give:
| 1 | co1 | | 2016-06-30 | ... |
| 4 | co2 | | 2016-06-29 | ... |
Any ideas? It's probably something simple but for the life of me I can't seem to get it to work.
UPDATE / FIX:
Ok so after a lot of hair pulling, for anyone running into this issue, apparently Postgres doesn't compare null fields with anything, so it completely ignores all rows with type = null.
The way I fixed it is this, there is probably a nicer solution out there but for now this works:
SELECT *
FROM import_data_log a
JOIN (SELECT company, max(date_created) date_created
FROM import_data_log
WHERE (type <> 'ERROR' OR type is null)
GROUP BY company) b
ON a.company = b.company AND a.date_created = b.date_created
Use the below Query
SELECT id,company,type,max(date_created),notes
FROM import_data_log
WHERE type != 'ERROR'
GROUP BY company
select distinct on (company) *
from import_data_log
where type is distinct from 'ERROR'
order by company, date_created desc
Check distinct on and is [not] distinct from:
Ordinary comparison operators yield null (signifying "unknown"), not true or false, when either input is null. For example, 7 = NULL yields null, as does 7 <> NULL. When this behavior is not suitable, use the IS [ NOT ] DISTINCT FROM constructs