UPDATE:
I am using the CTE because I am using a LOOP to loop in batches of 10000.
I am already using a CTE expression within a plpgsql Procedure to grab some Foreign Keys from (1) specific table, we can call it master_table. I created a brand new table, we can call this table table_with_fks, in my DDL statements so this table holds the FKs I am fetching and saving.
I later take these FKs from my table_with_fks and JOIN on my other tables in my database to get the entire original record (the full record with all columns from its corresponding table) and insert it into an archive table.
I have an awesome lucid chart I drew that might make what I say down below make much more sense:
My CTE example:
LOOP
EXIT WHEN some_condition;
WITH fk_list_cte AS (
SELECT mt.fk1, mt.fk2, mt.fk3, mt.fk4
FROM master_table mt
WHERE mt.created_date < now() - interval '365' // archive record if >= 1 year old
LIMIT 10000
)
INSERT INTO table_with_fks (SELECT * FROM fk_list_cte);
commit;
END LOOP;
Now, I have (4) other Procedures that JOIN on each FK in this table_with_fks with its parent table that it references. I do this because as I said, I only got the FK at first, and I don't have all the original columns for the record. So I will do something like
LOOP
EXIT WHEN some_condition;
WITH full_record_cte AS (
SELECT *
FROM table_with_fks fks
JOIN parent_table1 pt1
ON fks.fk1 = pt1.id
LIMIT 10000),
INSERT INTO (select * from full_record_cte);
commit;
END LOOP;
NOW, what I want to do, is instead of having to RE-JOIN 4 times later on these FK's that are found in my table_with_fks, I want to use the first CTE fk_list_cte to JOIN on the parent tables right away and grab the full record from each (4) tables and put it in some TEMP postgres table. I think I will need (4) unique TEMP tables, as I don't know how it would work if I combine all their data into one BIG table, because each table has different data/different columns.
Is there a way to use the original CTE fk_list_cte and call it multiple times in succession and CREATE 4 TEMP tables right after, that all use the original CTE? example:
LOOP
EXIT WHEN some_condition;
WITH fk_list_cte AS (
SELECT mt.fk1, mt.fk2, mt.fk3, mt.fk4
FROM master_table mt
WHERE mt.created_date < now() - interval '365' // archive record if >= 1 year old
LIMIT 10000
),
WITH fetch_fk1_original_record_from_parent AS (
SELECT *
FROM fk_list_cte cte
JOIN parent_table1 pt1
ON cte.fk1 = pt1.id
),
WITH fetch_fk2_original_record_from_parent AS (
SELECT *
FROM fk_list_cte cte
JOIN parent_table2 pt2
ON cte.fk2 = pt2.id
),
WITH fetch_fk3_original_record_from_parent AS (
SELECT *
FROM fk_list_cte cte
JOIN parent_table3 pt3
ON cte.fk3 = pt3.id
),
WITH fetch_fk4_original_record_from_parent AS (
SELECT *
FROM fk_list_cte cte
JOIN parent_table4 pt4
ON cte.fk4 = pt4.id
),
CREATE TEMPORARY TABLE fk1_tmp_tbl AS (
SELECT *
FROM fetch_fk1_original_record_from_parent
)
CREATE TEMPORARY TABLE fk2_tmp_tbl AS (
SELECT *
FROM fetch_fk2_original_record_from_parent
)
CREATE TEMPORARY TABLE fk3_tmp_tbl AS (
SELECT *
FROM fetch_fk3_original_record_from_parent
)
CREATE TEMPORARY TABLE fk4_tmp_tbl AS (
SELECT *
FROM fetch_fk4_original_record_from_parent
);
END LOOP;
I know the 4 CREATE TEMPORARY TABLE statements definitely won't work, (can I create 4 temp tables simultaneously/at once?) . Does anyone see the logic of what I am trying to do here and can help me?
I want to create a procedure in which I insert data into several tables. I need to get the inserted ID's so I create temp table in which I catch them. The problem is that I receive an error "Invalid column name 'app_guid'" and "Invalid column name 'app_nazwa_pliku'" but I create temp tables with such columns. Do you happen to know what's wrong with my code?
create procedure p_paseczek_przenies
as
declare #new_nr_sprawy varchar(50)
if object_id('tempdb..##paseczki') is not null drop table ##paseczki
select
top 1 with ties
s.sp_numer as SprawaGlowna_sp_numer,
s.sp_id as SprawaGlowna_sp_id
,Paseczek.max_ak_id as Paseczek_max_ak_id
,apisp_data_przyjscia
,app_guid
,app_nazwa_pliku
into ##paseczki
from sprawa as s
join akcja as a on a.ak_sp_id=s.sp_id and ak_akt_id=111
join sprawa_powiazania as sp on s.sp_id=sp.sp_id and rodzaj_powiazania='SPRAWY POLUBOWNE'
join (select max(ak_id) max_ak_id,ak_sp_id from akcja
where ak_akt_id=1089
group by ak_sp_id) as Paseczek on Paseczek.ak_sp_id=sp.sp_id_powiazana
join akcja_pismo on apis_ak_id=max_ak_id
join akcja_pismo_przychodzace on apis_apisp_id=apisp_id
join akcja_pismo_plik on app_apis_id=apis_id
where s.sp_numer=#new_nr_sprawy
order by ROW_NUMBER() over (partition by s.sp_id order by paseczek.max_ak_id desc)
if exists (select * from ##paseczki)
begin
if object_id('tempdb..##akcja') is not null drop table ##akcja
create table ##akcja (
ak_id int
,apisp_data_przyjscia datetime
,app_guid varchar(max)
,app_nazwa_pliku varchar(max)
)
merge akcja as target using (
select * from ##paseczki) as source on 1=0
when not matched then insert
(ak_akt_id, ak_sp_id, ak_kolejnosc, ak_interwal, ak_zakonczono, ak_pr_id, ak_publiczna)
values (1089,SprawaGlowna_sp_id,1,1,getdate(),5,1)
output inserted.ak_id,source.apisp_data_przyjscia,source.app_guid,source.app_nazwa_pliku
into ##akcja;
insert into rezultat
(re_ak_id, re_ret_id, re_data_planowana, re_us_id_planujacy, re_data_wykonania, re_us_id_wykonujacy, re_konczy)
select ak_id,309,getdate(),5,getdate(),5,1 from ##akcja
if object_id('tempdb..##akcja_pismo_przychodzace') is not null drop table ##akcja_pismo_przychodzace
create table ##akcja_pismo_przychodzace (
apisp_id int
,ak_id int
,app_guid varchar(max)
,app_nazwa_pliku varchar(max)
)
merge akcja_pismo_przychodzace as target using (
select * from ##akcja) as source on 1=0
when not matched then insert
(apisp_data_przyjscia)
values (apisp_data_przyjscia)
output inserted.apisp_id,source.ak_id,source.app_guid,source.app_nazwa_pliku
into ##akcja_pismo_przychodzace;
if object_id('tempdb..##akcja_pismo') is not null drop table ##akcja_pismo
create table ##akcja_pismo (
apis_id int
,app_guid varchar(max)
,app_nazwa_pliku varchar(max)
)
merge akcja_pismo as target using (
select * from ##akcja_pismo_przychodzace) as source on 1=0
when not matched then insert
(apis_ak_id, apis_apisp_id, apis_data_stworzenia,[apis_us_id_tworzacy])
values (ak_id,apisp_id,getdate(),5)
output inserted.apis_id,source.app_guid,source.app_nazwa_pliku
into ##akcja_pismo;
alter table [dm_data_bps].[dbo].[akcja_pismo_plik] disable trigger [tr_akcja_pismo_plik_ins]
insert into akcja_pismo_plik
([app_guid],[app_apis_id],[app_nazwa_pliku])
select [app_guid],[apis_id],[app_nazwa_pliku] from ##akcja_pismo
alter table [dm_data_bps].[dbo].[akcja_pismo_plik] enable trigger [tr_akcja_pismo_plik_ins]
end
SQL Server compiles the procedure at creation and when it is first executed, verifying the entire procedure based on the context at that time.
For example, try the following query:
CREATE PROCEDURE P
AS
IF OBJECT_ID('tempdb..#T') IS NOT NULL DROP TABLE #T
SELECT 1 Y INTO #T
SELECT Y FROM #T
GO
CREATE TABLE #T (X INT)
GO
EXEC P
You will get an error ("Invalid column name 'Y'."), because when the procedure is compiled the table #T has only the column X.
To avoid this problem, you should make sure that the table #T either does not exist or has the right columns, before the procedure is executed.
One way would be to have another stored procedure (a wrapper):
CREATE PROCEDURE P1
AS
SELECT 1 Y INTO #T
SELECT Y FROM #T
GO
CREATE PROCEDURE P2
AS
IF OBJECT_ID('tempdb..#T') IS NOT NULL DROP TABLE #T
EXEC P1
GO
CREATE TABLE #T (X INT)
GO
EXEC P2
GO
DROP PROCEDURE P1, P2
--DROP TABLE #T
Another way would be to use dynamic SQL, because that code is compiled separately, as if it would be another stored procedure.
A better way would be to make sure that temp tables are uniquely named in each stored procedure, unless sharing data between them is desired. For the later case, you can read http://www.sommarskog.se/share_data.html#temptables for more insights.
This error is also encountered when a stored procedure creates a #temp table and then fires a trigger which creates a #temp table with the same name. The SP #temp table is referenced by the trigger when the column names are explicit, (like SELECT id FROM #temp;), but the local trigger #temp table is referenced when SELECT * FROM #temp; is used.
Microsoft, if you are listening, could you kindly attend to it and retrofit existing supported versions with a maintenance update?
I am using Postgresql 9.3 and what I am trying to do is Passing column names as string into my query. For newtable my column number can be dynamic sometimes it might be 3 or more for which I am trying to select column from another table and pssing relut of my query as string in the existing query
Please help how can i do this
select * from crosstab (
'select "TIMESTAMP_S","VARIABLE","VALUE" from archieve_export_db_a3 group by 1,2,3 order by 1,2',
'select distinct "VARIABLE" From archieve_export_db_variables order by 1'
) AS newtable (TIMESTAMP_S int,_col1 integer,_col2 integer);
The end result of what I am after is a query that calls a function and that function returns a set of records that are in their own separate fields. I can do this but the results of the function are all in one field.
ie: http://i.stack.imgur.com/ETLCL.png and the results I am after are: http://i.stack.imgur.com/wqRQ9.png
Here's the code to create the table
CREATE TABLE tbl_1_hm
(
tbl_1_hm_id bigserial NOT NULL,
tbl_1_hm_f1 VARCHAR (250),
tbl_1_hm_f2 INTEGER,
CONSTRAINT tbl_1_hm PRIMARY KEY (tbl_1_hm_id)
)
-- do that for a few times to get some data
INSERT INTO tbl_1_hm (tbl_1_hm_f1, tbl_1_hm_f2)
VALUES ('hello', 1);
CREATE OR REPLACE FUNCTION proc_1_hm(id BIGINT)
RETURNS TABLE(tbl_1_hm_f1 VARCHAR (250), tbl_1_hm_f2 int AS $$
SELECT tbl_1_hm_f1, tbl_1_hm_f2
FROM tbl_1_hm
WHERE tbl_1_hm_id = id
$$ LANGUAGE SQL;
--And here is the current query I am running for my results:
SELECT t1.tbl_1_hm_id, proc_1_hm(t1.tbl_1_hm_id) AS t3
FROM tbl_1_hm AS t1
Thanks for having a read. Please if you want to haggle about the semantics of what I am doing by hitting the same table twice or my naming convention --> this is a simplified test.
When a function returns a set of records, you should treat it as a table source:
SELECT t1.tbl_1_hm_id, t3.*
FROM tbl_1_hm AS t1, proc_1_hm(t1.tbl_1_hm_id) AS t3;
Note that functions are implicitly using a LATERAL join (scroll down to sub-sections 4 and 5) so you can use fields from tables listed previously without having to specify an explicit JOIN condition.
I am using IN clause for column "job_no". In this in clause i checking 1000 values, query retreiving the values but some of the job number are not existed, then how to find unmattched values in the in clause.
assuming you really are using Oracle:
create type table_of_integers is table of integer;
/
select * from table(table_of_integers(1, 2, 3))
where column_value not in (select job_no from my_table);
or you should be able to achieve the same thing using an outer join, such as this example for postgres:
select *
from (select unnest(array[1, 2, 3]) as job_no) j
left outer join my_table using(job_no)
where my_table.job_no is null;
Insert the values into a temporary table instead and do a LEFT OUTER JOIN to join with your data.