remove overlapping entries from a table KDB+/Q - kdb

I have two tables, called table1 and table2. Both have date and id as columns, and table2 is a small subset of table1 entries. How do I remove all entries that exist in table2 from table1 by matching up date and id?

select from table1 where not([]date;id)in table2

/ Begin with example input of 10 entries
q)t1:([]date:.z.d+til 10;id:til 10;c:til 10)
/ Pick 3 random entries to form key table t2
q)t2:3?delete c from t1
/ Key t1 by date and id and drop entries matching t2
q)t2 _ `date`id xkey t1

Related

Merge Row values from different columns to one column on top of each other: MySQL

I have Table1 with columns:
Fname,Sname
Table 2 with columns:
Fname,Lname
Now,in a query I want to take all the values from these two tables (First names and last names [Sname in table 1 is Lname]) and return to one columns.
Basically I want to create column to get list of participants which include everyone from these two tables.
Is it possible?
Both the tables are joined indirectly via third table.
You can use UNION ALL will give all the rows from both tables.
SELECT
Fname,
Sname,
CONCAT(Fname,Sname) AS FSname
FROM table1
UNION ALL
SELECT
Fname,
Lname,
CONCAT(Fname,Lname)
FROM table2;
The column names are taken from the first SELECT.
If you use UNION and not UNION ALL rows in table2 which are duplicates of table1 will be omitted, but it can run slower as the values have to be compared.
You can use the third table and LEFT JOIN onto both tables and use COALESCE which returns the first argument which is not null.
SELECT
COALESCE(t1.Fname,t2.Fname),
COALESCE(t1.Sname,t2.Lname),
CONCAT(
COALESCE(t1.Fname,t2.Fname),
COALESCE(t1.Sname,t2.Lname)
) AS FSname
FROM third_table t3
LEFT JOIN table1 t1
ON t1.id = t3.id
LEFT JOIN table2 t2
ON t2.id = te.id;

Date in one table is before date in another table - Postgres

I have a table 1
and Table 2
I need to get the following table where the date from table 1 is the closest (i.e. before) to the date from table 2 by id.
I assume I need to join two table where table1.id=table2.id and table1.date<=table2.date and then, rank to get the 'last' record in that merged table? Is it correct? Is there a simpler way?
You can see structure and result in: dbfiddle
select
distinct on (t1.id)
t1.id,
last_value(t1.type) over (order by to_date(t1.date, 'mm/dd/yyyy') desc)
from
table1 t1 inner join table2 t2 on t1.id = t2.id
where
to_date(t1.date, 'mm/dd/yyyy') <= to_date(t2.date, 'mm/dd/yyyy');

Apply join, sort on date column and select the first row where one of the column value is not null

I have two tables(Table A and Table B) in a Postgres DB.
Both have "id" column in common. Table A has one column called "id" and Table B has three columns: "id, date, value($)".
For each "id" of Table A there exists multiple rows in Table B in the following format - (id, date, value).
For instance, for Table A with "id" as 1 if there exists following rows in Table B:
(1, 2018-06-21, null)
(1, 2018-06-20, null)
(1, 2018-06-19, 202)
(1, 2018-06-18, 200)
I would like to extract the most recent dated non-null value. For example for id - 1, the result should be 202. Please share your thoughts or let me know in case more info is required.
Here is the solution I went ahead with:
with mapping as ( select distinct table1.id, table2.value, table2.date, row_number() over (partition by table1.id order by table2.date desc nulls last) as row_number
from table1
left join table2 on table2.id=table1.id and table2.value is not null
)
select * from mapping where row_number = 1
Let me know if there is scope for improvement.
You may very well want an inner join, not an outer join. If you have an id in table1 that does not exist in table2 or that has only null values you will get NULL for both date and value. This is due to the how outer join works. What it says is if nothing in the right side table matches the ON condition then return NULL for each column in that table. So
with mapping as
(select distinct table1.id
, table2.value
, table2.date
, row_number() over (partition by table1.id order by table2.date desc nulls last) as row_number
from table1
join table2 on table2.id=table1.id and table2.value is not null
)
select *
from mapping
where row_number = 1;
See example of each here. Your query worked because all your test data satisfied the 1st condition of the ON condition. You really need test data that fails to see what your query does.
Caution: DATE and VALUE are very poor choice for a column names. Both are SQL standard reserved words, although not Postgres specifically. Further DATE is a Postgres data type. Having columns with names the same as datatype leads to confusion.

How to join vertical and horizontal table together table

I have two table with one of them is vertical i.e store only key value pair with ref id from table 1. i want to join both table and dispaly key value pair as a column in select. and also perform sorting on few keys.
T1 having (id,empid,dpt)
T2 having (empid,key,value)
select
T1.*,
t21.value,
t22.value,
t23.value,
t24.value
from Table1 t1
join Table2 t21 on t1.empid = t21.empid
join Table2 t22 on t1.empid = t22.empid
join Table2 t23 on t1.empid = t23.empid
where
t21.key = 'FNAME'
and t22.key = 'LNAME'
and t23.key='AGE'
The query you demonstrate is very inefficient (another join for each additional column) and also has a potential problem: if there isn't a row in T2 for every key in the WHERE clause, the whole row is excluded.
The second problem can be avoided with LEFT [OUTER] JOIN instead of [INNER] JOIN. But don't bother, the solution to the first problem is a completely different query. "Pivot" T2 using crosstab() from the additional module tablefunc:
SELECT * FROM crosstab(
'SELECT empid, key, value FROM t2 ORDER BY 1'
, $$VALUES ('FNAME'), ('LNAME'), ('AGE')$$ -- more?
) AS ct (empid int -- use *actual* data types
, fname text
, lname text
, age text);
-- more?
Then just join to T1:
select *
from t1
JOIN (<insert query from above>) AS t2 USING (empid);
This time you may want to use [INNER] JOIN.
The USING clause conveniently removes the second instance of the empid column.
Detailed instructions:
PostgreSQL Crosstab Query

Bulk update a column in Oracle 11G

I have two tables say Table1 and Table2 that contains the following column with which I should join and perform an update a column of Table1 with the value of the same column present in Table2.
Columns for Join condition:
Table1.mem_ssn and Table2.ins_ssn
Table1.sys_id and Table2.sys_id
Table1.grp_id and Table2.grp_id
Column to update:
Table1.dtofhire=Table2.dtofhire
I need a way to bulk update (using single update query without looping) the above mentioned column in Oracle 11G.
Table1 does not contain any key constraint specified since it will be used as a staging table for Data upload.
Please help me out to update the same.
You can use the MERGE statement.
It should look something like this:
MERGE INTO table1 D
USING (SELECT * FROM table2 ) S
ON (D.mem_ssn = S.ins_ssn and D.sys_id = S.sys_id and D.grp_id=S.grp_id)
WHEN MATCHED THEN
UPDATE SET D.dtofhire=S.dtofhire;
UPDATE:
Since you have more than one row in table2 with the same (ins_ssn,sys_id,grp_id) and you want the max dtofhire, you should change the query in the using clause:
MERGE INTO table1 D
USING (SELECT ins_ssn, sys_id, grp_id, max(dtofhire) m_dtofhire
FROM table2
GROUP BY ins_ssn,sys_id,grp_id) S
ON (D.mem_ssn = S.ins_ssn and D.sys_id = S.sys_id and D.grp_id=S.grp_id)
WHEN MATCHED THEN
UPDATE SET D.dtofhire=S.m_dtofhire;
The query that I used to arrive the functionality is seen below
UPDATE table1 T2
SET dtofhire = (SELECT Max(dtofhire) AS dtofhire
FROM table2 T1
WHERE T2.mem_ssn = T1.ins_ssn
AND T2.sys_id = T1.sys_id
AND T2.grp_id = T1.grp_id
GROUP BY ins_ssn,
sys_id,
grp_id)
WHERE ( mem_ssn, sys_id, grp_id ) IN (SELECT ins_ssn,
sys_id,
grp_id
FROM table2 );