I have a Script Activity with the following sql script in my data pipeline as follows:
#concat('ALTER TABLE tbl',replace(pipeline().RunId,'-',''),' ADD Depth int; WITH emp AS ( SELECT *, 1 AS d FROM tbl',replace(pipeline().RunId,'-',''),' WHERE Email = ','EMAIL',' UNION ALL SELECT e.*, emp.d + 1 FROM tbl',replace(pipeline().RunId,'-',''),' e INNER JOIN emp ON e.ReportsToPersonnelNbr = emp.PersonnelNumber), ForUpd as (SELECT PersonnelNumber, d FROM emp) UPDATE tbl',replace(pipeline().RunId,'-',''),' SET Depth = B.d FROM tbl',replace(pipeline().RunId,'-',''),' A JOIN ForUpd B ON A.PersonnelNumber = B.PersonnelNumber')
I see this error on running pipeline:
Operation on target Add Depth Column failed: Invalid column name 'EMAIL'.
What am I missing?
Converting the dynamic content that you have written to SQL script, the query would be as following:
ALTER TABLE <table_name> ADD Depth int; WITH emp AS ( SELECT *, 1 AS d FROM <table_name> WHERE Email =EMAIL UNION ALL SELECT e.*, emp.d + 1 FROM <table_name> e INNER JOIN emp ON e.ReportsToPersonnelNbr = emp.PersonnelNumber), ForUpd as (SELECT PersonnelNumber, d FROM emp) UPDATE <table_name> SET Depth = B.d FROM <table_name> A JOIN ForUpd B ON A.PersonnelNumber = B.PersonnelNumber
According to the error message, the problem is with Email = EMAIL. This is because EMAIL column does not exist. If EMAIL is on of the column values for Email column, then has to be enclosed in single quotes.
Instead of using #concat() and complicating the process to write a query, you can use string interpolation. Using #{...}, you can use dynamic content within a string. Look at the following example.
I am creating a table using string interpolation in the following demonstration.
create table #{variables('table_name')} (id int, name varchar(20))
So, first use set variable activity to store the table name in a my_table variable with the value
#concat('tbl',replace(pipeline().RunId,'-',''))
Now replace your dynamic content in script activity with the following:
ALTER TABLE #{variables('my_table')} ADD Depth int; WITH emp AS ( SELECT *, 1 AS d FROM #{variables('my_table')} WHERE Email ='EMAIL' UNION ALL SELECT e.*, emp.d + 1 FROM #{variables('my_table')} e INNER JOIN emp ON e.ReportsToPersonnelNbr = emp.PersonnelNumber), ForUpd as (SELECT PersonnelNumber, d FROM emp) UPDATE #{variables('my_table')} SET Depth = B.d FROM #{variables('my_table')} A JOIN ForUpd B ON A.PersonnelNumber = B.PersonnelNumber
Related
I need to do multiple inserts in the table that number depends on how many records we got from select. I need to iterate over records from select and then insert them to another table. I want to get all new Ids from insert to array to use them later in the following inserts, how can I do this?
I can't collect them using select after insert, because there can be old records.
for record in (select test, test1, test2
from public.a
join public.b on a.reg_id = b.id
where a.id = arg_id) loop
(INSERT into public.c
( a, b, c)
(select test, test1, test2
from record)--need to get ids from this
end loop;
---
some block where I have old_ids
---
--to insert them there
insert into public.d(d,e,f,g)values(..,..,old_id,(id from previous insert))
upd
Tried to make like this:
with a2 as(
INSERT INTO public.reg
(name_, code, state)
(select a.secondname, a.code, b.state_name--multiple rows from select
from public.client a
left join public.states b on a.state_id = b.id
where a.id = id_p) RETURNING id
)
INSERT INTO public.request
(phone, address, qty, prod_id, reg_id)
(select phone, address, qty, prod_id, (select id from a2)--maybe something wrong there, but error happend before
from public.shp a
where a.id = id_p);
but getting an error: more than one row returned by a subquery used as an expression
Demonstration of using the result of a query:
\i tmp.sql
CREATE TABLE aa(aa integer not null primary key);
CREATE TABLE bb(bb integer not null primary key);
CREATE TABLE cc(cc integer not null primary key);
WITH x0 AS (
INSERT INTO aa(aa) values (1),(2),(3)
returning aa
)
, x1 AS (
INSERT INTO bb(bb)
SELECT aa*aa from x0
returning bb
)
, x2 AS (
INSERT INTO cc(cc)
SELECT bb*bb from x1
returning cc
)
-- main query
SELECT *
FROM x2
;
-- Check cc
SELECT *
FROM cc
;
Output:
DROP SCHEMA
CREATE SCHEMA
SET
CREATE TABLE
CREATE TABLE
CREATE TABLE
cc
----
1
16
81
(3 rows)
cc
----
1
16
81
(3 rows)
I have two tables in postgresql
One table is of the form
Create table table1(
ID serial PRIMARY KEY,
Type []Text
)
Create table table2(
type text,
sellerID int
)
Now i want to get all the rows from table1 which are having type same that in table2 but the problem is that in table1 the type is an array.
In case the type in the table has an identifiable delimiter like ',' ,';' etc. you can rewrite the query as regexp_split_to_table(type,',') or versions later than 9.5 unnest function can be use too.
For eg.,
select * from
( select id ,regexp_split_to_table(type,',') from table1)table1
inner join
select * from table2
on trim(table1.type) = trim(table2.type)
Another good example can be found - https://www.dbrnd.com/2017/03/postgresql-regexp_split_to_array-to-split-string-using-different-delimiters/
SELECT
a[1] AS DiskInfo
,a[2] AS DiskNumber
,a[3] AS MessageKeyword
FROM (
SELECT regexp_split_to_array('Postgres Disk information , disk 2 , failed', ',')
) AS dt(a)
You can use the ANY operator in the JOIN condition:
select *
from table1 t1
join table2 t2 on t2.type = any (t1.type);
Note that if the types in the table1 match multiple rows in table2, you would get duplicates (from table1) because that's how a join works. Maybe you want an EXISTS condition instead:
select *
from table1 t1
where exists (select *
from table2 t2
where t2.type = any(t1.type));
I want to select from 2 table where 2nd id is equal with first 4 character from first id
SELECT a.*, b.*, substring(a.my_id from 1 for 4)::integer as number
FROM table1 as a
INNER Join table2 as b ON(b.id_2=number) where my_id = 101
^
It produces an error here |
ERROR: column "number" does not exist
SQL state: 42703
Character: 189
You can't use an alias like that, you need a derived table:
select *
from (
SELECT t1.*,
substring(t1.my_id::text from 1 for 4)::integer as number
FROM table1 t1
) as a
inner join table2 as b ON (b.id_2 = a.number)
where my_id = 101
Storing a number that is used as a foreign key as a part in a varchar column is a really, really ugly design. That number should be a column of its own in table1.
Lets say I have table with ID int, VALUE string:
ID | VALUE
1 abc
2 abc
3 def
4 abc
5 abc
6 abc
If I do select value, count(*) group by value I should get
VALUE | COUNT
abc 5
def 1
Now the tricky part, if there is count == 1 I need to get that ID from first table. Should I be using CTE? creating resultset where I will add ID string == null and run update b.ID = a.ID where count == 1 ?
Or is there another easier way?
EDIT:
I want to have result table like this:
ID VALUE count
null abc 5
3 def 1
If your ID values are unique, you can simply check to see if the max(id) = min(id). If so, then use either one, otherwise you can return null. Like this:
Select Case When Min(id) = Max(id) Then Min(id) Else Null End As Id,
Value, Count(*) As [Count]
From YourTable
Group By Value
Since you are already performing an aggregate, including the MIN and Max function is not likely to take any extra (noticeable) time. I encourage you to give this a try.
The way I would do it would indeed be a CTE:
using #group AS (SELECT value, Count(*) as count from MyTable GROUP BY value HAVING count = 1)
SELECT MyTable.ID, #group.value, #group.count from MyTable
JOIN #group ON #group.value = MyTable.value
When using group by, after the group by statement you can use a having clause.
So
SELECT [ID]
FROM table
GROUP BY [VALUE]
HAVING COUNT(*) = 1
Edit: with regards to your edited question: this uses some fun joins and unions
CREATE TABLE #table
(ID int IDENTITY,
VALUE varchar(3))
INSERT INTO #table (VALUE)
VALUES('abc'),('abc'),('def'),('abc'),('abc'),('abc')
SELECT * FROM (
SELECT Null as ID,VALUE, COUNT(*) as [Count]
FROM #table
GROUP BY VALUE
HAVING COUNT(*) > 1
UNION ALL
SELECT t.ID,t.VALUE,p.Count FROM
#table t
JOIN
(SELECT VALUE, COUNT(*) as [Count]
FROM #table
GROUP BY VALUE
HAVING COUNT(*) = 1) p
ON t.VALUE=p.VALUE
) a
DROP TABLE #table
maybe not the most efficient but something like this works:
SELECT MAX(Id) as ID,Value FROM Table WHERE COUNT(*) = 1 GROUP BY Value
How can I extract the values from a record as individual comuns in postgresql
SELECT
p.*,
(SELECT ROW(id,server_id,format,product_id) FROM products_images pi WHERE pi.product_id = p.id LIMIT 1) AS image
FROM products p
WHERE p.company = 1 ORDER BY id ASC LIMIT 10
Instead of
image
(3, 4, "jpeg", 7)
I would like to have
id | server_id | format | product_id
3 | 4 | jpeg | 7
Is there any way of selecting only one image for each product and return the columns directly instead of a record?
Try this:
create type xxx as (t varchar, y varchar, z int);
with a as
(
select row(table_name, column_name, (random() * 100)::int) x
from information_schema.columns
)
-- cannot cast directly to xxx, should cast to text first
select (x::text::xxx).t, (x::text::xxx).y, (x::text::xxx).z
from a
Alternatively, you can do this:
with a as
(
select row(table_name, column_name, (random() * 100)::int) x
from information_schema.columns
),
-- cannot cast directly to xxx, should cast to text first
b as (select x::text::xxx as w from a)
select
(w).t, (w).y, (w).z
from b
To select all fields:
with a as
(
select row(table_name, column_name, (random() * 100)::int) x
from information_schema.columns
),
-- cannot cast directly to xxx, should cast to text first
b as (select x::text::xxx as w from a)
select
(w).*
from b
You can do this too, but this makes the whole exercise of using ROW a pointless one when you can just remove the ROW function and re-pick it up from outside of cte/derived table. I surmised the OP's ROW came from a function; for which he should use the codes above, not the following:
with a as
(
select row(table_name, column_name, (random() * 100)::int)::xxx x
from information_schema.columns
)
select
(x).t, (x).y, (x).z
from a
Just specify the components of your struct:
SELECT a,b,c,(image).id, (image).server_id, ...
FROM (
SELECT
p.*,
(SELECT ROW(id,server_id,format,product_id) FROM products_images pi WHERE pi.product_id = p.id LIMIT 1) AS image
FROM products p
WHERE p.company = 1 ORDER BY id ASC LIMIT 10
) as subquery
But anyway, I would rework the query and use a join instead of a subclause.
SELECT DISTINCT ON (p.*) p.*,
p.id,pi.server_id,pi.format,pi.product_id
FROM products p
LEFT JOIN product_images pi ON pi.product_id = p.id
WHERE p.company = 1
ORDER BY id ASC
LIMIT 10
But I believe you have to specify all the p-fields in the distinct separately to ensure just one image is loaded per product.
Try this, will work on your existing code with minimal modification(if creating a type is a minimal modification for you ;-)
create type image_type as (id int, server_id int, format varchar, product_id int);
SELECT
p.*,
( (SELECT ROW(id,server_id,format,product_id)
FROM products_images pi
WHERE pi.product_id = p.id LIMIT 1)::text::image_type ).*
FROM products p
WHERE p.company = 1 ORDER BY id ASC LIMIT 10
Proof-of-concept code:
Create type first:
create type your_type_here as (table_name varchar, column_name varchar)
Actual code:
select
a.b,
( (select row(table_name, column_name)
from information_schema.columns limit 1)::text::your_type_here ).*
from generate_series(1,10) as a(b)
But I guess you should tackle it with GROUP BY' andMAXcombo or useDISTINCT ON` like what Daniel have posted
every table has an associated composite type of the same name
https://www.postgresql.org/docs/current/plpgsql-declarations.html#PLPGSQL-DECLARATION-ROWTYPES
So, this code
drop table if exists "#typedef_image"
;
create temp table "#typedef_image"(
id int,
server_id int,
format text,
product_id int
)
;
select (row(3, 4, 'jpeg', 7)::"#typedef_image").*
will work