Related
I am attempting to allow insert statements with a returning clause into a view in Postgres v9.4, but am struggling with the syntax. This is how I want to call the insert statement:
CREATE VIEW MyView AS SELECT a.*, b.someCol1
FROM tableA a JOIN tableB b USING(aPrimaryKey);
INSERT INTO MyView (time, someCol) VALUES (someTime, someValue) RETURNING *;
INSERT INTO MyView (someCol) VALUES (someValue) RETURNING *;
Note that the default for time is NOW(). This is what I have so far:
CREATE RULE MyRuleName AS ON INSERT TO MyView DO INSTEAD (
INSERT INTO tableA (time) VALUES COALESCE(NEW.time, NOW());
INSERT INTO tableB (aPrimaryKey, someCol)
VALUES (CURRVAL('tableA_aPrimaryKey_seq'), NEW.someValue);
);
The above works to insert the value, but I am struggling to try and figure out how to add the returning statement. I have tried the following without success:
CREATE RULE MyRuleName AS ON INSERT TO MyView DO INSTEAD (
INSERT INTO tableA (time) VALUES COALESCE(NEW.time, NOW())
RETURNING *, NEW.someValue;
INSERT INTO tableB (aPrimaryKey, someCol)
VALUES (CURRVAL('tableA_aPrimaryKey_seq'), NEW.someValue);
);
-- ERROR: invalid reference to FROM-clause entry for table "new"
CREATE RULE MyRuleName AS ON INSERT TO MyView DO INSTEAD (
WITH a AS (INSERT INTO tableA (time)
VALUES COALESCE(NEW.time, NOW()) RETURNING *)
INSERT INTO tableB (aPrimaryKey, someCol)
SELECT aPrimaryKey, NEW.someValue FROM a RETURNING *;
);
-- ERROR: cannot refer to NEW within WITH query
Argh! Does anyone know of a way to add a returning statement that gets the primary key (SERIAL) and time (TIMESTAMP WITH TIME ZONE) added to the database in the first insert, along with the value of someCol in the second insert? Thanks!
You are much better off using an INSTEAD OF INSERT trigger here:
CREATE FUNCTION MyFuncName() RETURNS trigger AS $$
DECLARE
id integer;
BEGIN
INSERT INTO tableA (time) VALUES COALESCE(NEW.time, NOW()) RETURNING aPrimaryKey INTO id;
INSERT INTO tableB (aPrimaryKey, someCol1) VALUES (id, NEW.someValue);
RETURN NEW;
END; $$ LANGUAGE PLPGSQL;
CREATE TRIGGER MyView_on_insert INSTEAD OF INSERT ON MyView
FOR EACH ROW EXECUTE PROCEDURE MyFuncName();
Checking the current value of a sequence to see what was inserted in another table is bad bad bad practice. Even while you are here in a single transaction, don't do it.
You are confused about the issue of RETURNING information, because I am confused too when I read your question. Inside of a function use the INTO clause to populate locally declared variables to hold record values which you can then use in subsequent statements. Outside of a function, use the RETURNING clause as you do in your top-most code snippet.
I don't agree with the the hint ("use triggers instead of rules"), because triggers don't allow RETURNING. As written in the Postgresql docu it is a little bit tedious to write the right return list. If you keep the following in mind, it works:
You can only use columns from the original table to form a list which returns columns for the view (!). This means that you have to repeat the view expressions including all subqueries. (using WHERE instead of JOIN ... ON). Additionally you have to replace the NEW table by the original table name.
I have a table in for which I have provided default values for some of its columns. I want to create a function with arguments corresponding to the columns that will insert a record in the table after modifying any null value to the column's default value.I dont want to programmatically construct the query based on which arguments are null. Essentially I would like something like
INSERT into Table (c1, c2, c3, c4)
Values (coalesce(somevar, DEFAULT(c1)), ...)
Is this possible? I ve seen that mysql can do this. Does postgres offer anything similar?
I am using version 9.1
UPDATE: This question provides some interesting solution but unfortunately the results are always text. I would like to get the default value as its true datatype so that I can use it for inserting it. I have tried to find a solution that will cast the default value from text to its datatype (which is provided as text) but I can't find a way:
SELECT column_name, column_default, data_type
FROM information_schema.columns
WHERE (table_schema, table_name) = ('public', 'mytable')
AND column_name = 'mycolumn'
ORDER BY ordinal_position;
The above returns the column_default and data_type as text so how can I cast the column_default to the value of data_type? If I could do this, then my problem would be solved.
If the table definition accepts an INSERT with the default values for all columns, the two-steps method below may work:
CREATE FUNCTION insert_func(c1 typename, c2 typename, ...)
RETURNS VOID AS $$
DECLARE
r record;
BEGIN
INSERT into the_table default values returning *,ctid INTO r;
UPDATE the_table set
col1=coalesce(c1,r.col1),
col2=coalesce(c2,r.col2),
...
WHERE the_table.ctid=r.ctid;
END;
$$ language plpgsql;
The trick is to get all the default values into the r record variable while inserting and use them subsequently in an UPDATE to replace any non-default value. ctid is a pseudo-column that designates the internal unique ID of the row that has just been inserted.
Caveat: this method won't work if some columns have a default null value AND a non-null check (or any check that implies that the default value is not accepted), since the first INSERT would fail.
I ve worked around my problem with a solution similar to Daniel's by creating a temp table with LIKE and INCLUDING DEFAULTS clauses in order to match my rowtype, then i use
INSERT INTO temp_table (c1, c2, ...) VALUES(x1, DEFAULT, ..)
using the default keyword for whatever column i am interested in. Then I insert to the real table by selecting from the temporary and using
VALUES( x1, coalesce(x2, temp_table.c2), ...).
I dont like it, but it works ok: I can select which on which columns I would like to do this "null-replace-with-default" check and it could work for many rows with one pass if I overload my function to accept a record array.
So I have 2 tables that I need to insert similar data into. They are employee DBs that different applications access. They are:
dbo_Employees
dbo_EmpDefaultSchedules
dbo_EmpUsers
dbo_EmpDefaultLocation
So on dbo_Employees, when you insert a row, the primary key is auto-created. That column is called EmpID. There are a total of around 22 different columns being added to these different tables. They are things like FirstName, LastName, Address, Phone, etc., etc.
I'm trying to create a script where I can take a list of variables, enter them in once, and run that script to add it to the 4 tables at once (eventually a web page that HR will enter the info and create it themselves)
The only one that is giving me trouble is dbo_EmpDefaultSchedules. This is because I need the primary key from dbo_Employees (EmpID) to insert into dbo_EmpDefaultSchedules, and obviously it's not created until the first part of the script runs. My insert statement for the second part is this:
INSERT INTO [Database].[dbo].[EmpDefaultSchedules] (StaffCode, EmpID, LastName, FirstName, Dept, MgrStaffCode, IsMgr, PayrollStatus, PayFrequency, StdHrsWk, EmailAddress )
VALUES (#StaffCode, (select EmpID from [Database].[dbo].[Employees] WHERE StaffCode = #StaffCode), #LastName, #FirstName, #Dept, #MgrStaffCode, #IsMgr, #PayrollStatus, #PayrollFrequency, #StdHrsWk, #EmailAddress)
When I do this I get this error:
Subqueries are not allowed in this context. Only scalar expressions
are allowed.
I have seen/read this:
Subqueries are not allowed in this context. Only scalar expressions are allowed
And this:
MYSQL inserting into multiple tables
And this:
How do I store a value from a sql query into a variable?
And this:
How do I combine a SELECT + WHERE query with an INSERT query?
But still cannot get this to work.
You retrieve the identity to a variable using SCOPE_IDENTITY just after you insert into employees
SCOPE_IDENTITY (Transact-SQL)
What you can do is the following.
insert into Employees values (x,y,z)
insert into EmpDefaultSchedules (#staffcode, ##identity,....)
The ##identity is filled with the inserted EmpID, which you can use later on. If you insert another record after the Employees, and before the EmpDefaultschedules, make sure you put the ##identity in a variable
declare #EmpId int
insert into Employees values (x,y,z)
set #EmpId = ##identity
insert into EmpDefaultSchedules (#staffcode, #EmpId,....)
Im working with Postgres, using SERIAL as my primary key. After I insert a row I can get the generated key either by using 'RETURNING' or CURRVAL().
Now my problem is that I want to do a batch insert inside a transaction and get ALL the generated keys.
All I get with RETURNING and CURRVAL is the last generated id, the rest of the result get discarded.
How can I get it to return all of them?
Thanks
You can use RETURNING with multiple values:
psql=> create table t (id serial not null, x varchar not null);
psql=> insert into t (x) values ('a'),('b'),('c') returning id;
id
----
1
2
3
(3 rows)
So you want something more like this:
INSERT INTO AutoKeyEntity (Name,Description,EntityKey) VALUES
('AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a','Testing 5/4/2011 8:59:43 AM',DEFAULT)
returning EntityKey;
INSERT INTO AutoKeyEntityListed (EntityKey,Listed,ItemIndex) VALUES
(CURRVAL('autokeyentity_entityKey_seq'),'Test 1 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 0),
(CURRVAL('autokeyentity_entityKey_seq'),'Test 2 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 1),
(CURRVAL('autokeyentity_entityKey_seq'),'Test 3 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 2)
returning EntityKey;
-- etc.
And then you'll have to gather the returned EntityKey values from each statement in your transaction.
You could try to grab the sequence's current value at the beginning and end of the transaction and use those to figure out which sequence values were used but that is not reliable:
Furthermore, although multiple sessions are guaranteed to allocate
distinct sequence values, the values might be generated out of
sequence when all the sessions are considered. For example, with a
cache setting of 10, session A might reserve values 1..10 and return
nextval=1, then session B might reserve values 11..20 and return
nextval=11 before session A has generated nextval=2. Thus, with a
cache setting of one it is safe to assume that nextval values are
generated sequentially; with a cache setting greater than one you
should only assume that the nextval values are all distinct, not
that they are generated purely sequentially. Also, last_value will
reflect the latest value reserved by any session, whether or not
it has yet been returned by nextval.
So, even if your sequences have cache values of one you can still have non-contiguous sequence values in your transaction. However, you might be safe if the sequence's cache value matches the number of INSERTs in your transaction but I'd guess that that's going to be too large to make sense.
UPDATE: I just noticed (thanks to the questioner's comments) that there are two tables involved, got a bit lost in the wall of text.
In that case, you should be able to use the current INSERTS:
INSERT INTO AutoKeyEntity (Name,Description,EntityKey) VALUES
('AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a','Testing 5/4/2011 8:59:43 AM',DEFAULT)
returning EntityKey;
INSERT INTO AutoKeyEntityListed (EntityKey,Listed,ItemIndex) VALUES
(CURRVAL('autokeyentity_entityKey_seq'),'Test 1 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 0),
(CURRVAL('autokeyentity_entityKey_seq'),'Test 2 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 1),
(CURRVAL('autokeyentity_entityKey_seq'),'Test 3 AutoKey 254e3c64-485e-42a4-b1cf-d2e1e629df6a', 2);
-- etc.
And grab the EntityKey values one at a time from the INSERTs on AutoEntityKey. Some sort of script might be needed to handle the RETURNING values. You could also wrap the AutoKeyEntity and related AutoKeyEntityListed INSERTs in a function, then use INTO to grab the EntityKey value and return it from the function:
INSERT INTO AutoKeyEntity /*...*/ RETURNING EntityKey INTO ek;
/* AutoKeyEntityListed INSERTs ... */
RETURN ek;
you can pre-assign consecutive ids using this:
SELECT setval(seq, nextval(seq) + num_rows - 1, true) as stop
it should be a faster alternative to calling nextval() gazillions of times.
you could also store ids in a temporary table:
create temporary blah (
id int
) on commit drop;
insert into table1 (...) values (...)
returning id into blah;
in postgres 9.1, can able to use CTEs:
with
ids as (
insert into table1 (...) values (...)
returning id
)
insert into table2 (...)
select ...
from ids;
In your application, gather values from the sequence :
SELECT nextval( ... ) FROM generate_series( 1, number_of_values ) n
Create your rows using those values, and simply insert (using a multiline insert). It's safe (SERIAL works as you'd expect, no reuse of values, concurrent proof, etc) and fast (you insert all the rows at once without many client-server roundtrips).
Replying to Scott Marlowe's comment in more detail :
Say you have a tree table with the usual parent_id reference to itself, and you want to import a large tree of records. Problem is you need the parent's PK value to be known to insert the children, so potentially this can need lots of individual INSERT statements.
So a solution could be :
build the tree in the application
grab as many sequence values as nodes to insert, using "SELECT nextval( ... ) FROM generate_series( 1, number_of_values ) n" (the order of the values does not matter)
assign those primary key values to the nodes
do a bulk insert (or COPY) traversing the tree structure, since the PKs used for relations are known
There are three ways to do this. Use currval(), use returning, or write a stored procdure to wrap either of those methods in a nice little blanket that keeps you from doing it all in half client half postgres.
Currval method:
begin;
insert into table a (col1, col2) values ('val1','val2');
select currval('a_id_seq');
123 -- returned value
-- client code creates next statement with value from select currval
insert into table b (a_fk, col3, col4) values (123, 'val3','val4');
-- repeat the above as many times as needed then...
commit;
Returning method:
begin;
insert into table a (col1, col2) values ('val1','val2'), ('val1','val2'), ('val1','val2') returning a_id; -- note we inserted three rows
123 -- return values
124
126
insert into table b (a_fk, col3, col4) values (123, 'val3','val4'), (124, 'val3','val4'), (126, 'val3','val4');
commit;
Perform a FOR LOOP and process records one by one. It might be less performant but it is concurrency safe.
Example code:
DO $$
DECLARE r record;
BEGIN
FOR r IN SELECT id FROM {table} WHERE {condition} LOOP
WITH idlist AS (
INSERT INTO {anotherTable} ({columns}) VALUES ({values})
RETURNING id
UPDATE {table} c SET {column} = (SELECT id FROM idlist) WHERE c.id = {table}.id;
END LOOP;
END $$;
In PostgreSQL, how do I get the last id inserted into a table?
In MS SQL there is SCOPE_IDENTITY().
Please do not advise me to use something like this:
select max(id) from table
( tl;dr : goto option 3: INSERT with RETURNING )
Recall that in postgresql there is no "id" concept for tables, just sequences (which are typically but not necessarily used as default values for surrogate primary keys, with the SERIAL pseudo-type).
If you are interested in getting the id of a newly inserted row, there are several ways:
Option 1: CURRVAL(<sequence name>);.
For example:
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John');
SELECT currval('persons_id_seq');
The name of the sequence must be known, it's really arbitrary; in this example we assume that the table persons has an id column created with the SERIAL pseudo-type. To avoid relying on this and to feel more clean, you can use instead pg_get_serial_sequence:
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John');
SELECT currval(pg_get_serial_sequence('persons','id'));
Caveat: currval() only works after an INSERT (which has executed nextval() ), in the same session.
Option 2: LASTVAL();
This is similar to the previous, only that you don't need to specify the sequence name: it looks for the most recent modified sequence (always inside your session, same caveat as above).
Both CURRVAL and LASTVAL are totally concurrent safe. The behaviour of sequence in PG is designed so that different session will not interfere, so there is no risk of race conditions (if another session inserts another row between my INSERT and my SELECT, I still get my correct value).
However they do have a subtle potential problem. If the database has some TRIGGER (or RULE) that, on insertion into persons table, makes some extra insertions in other tables... then LASTVAL will probably give us the wrong value. The problem can even happen with CURRVAL, if the extra insertions are done intto the same persons table (this is much less usual, but the risk still exists).
Option 3: INSERT with RETURNING
INSERT INTO persons (lastname,firstname) VALUES ('Smith', 'John') RETURNING id;
This is the most clean, efficient and safe way to get the id. It doesn't have any of the risks of the previous.
Drawbacks? Almost none: you might need to modify the way you call your INSERT statement (in the worst case, perhaps your API or DB layer does not expect an INSERT to return a value); it's not standard SQL (who cares); it's available since Postgresql 8.2 (Dec 2006...)
Conclusion: If you can, go for option 3. Elsewhere, prefer 1.
Note: all these methods are useless if you intend to get the last inserted id globally (not necessarily by your session). For this, you must resort to SELECT max(id) FROM table (of course, this will not read uncommitted inserts from other transactions).
Conversely, you should never use SELECT max(id) FROM table instead one of the 3 options above, to get the id just generated by your INSERT statement, because (apart from performance) this is not concurrent safe: between your INSERT and your SELECT another session might have inserted another record.
See the RETURNING clause of the INSERT statement. Basically, the INSERT doubles as a query and gives you back the value that was inserted.
Leonbloy's answer is quite complete. I would only add the special case in which one needs to get the last inserted value from within a PL/pgSQL function where OPTION 3 doesn't fit exactly.
For example, if we have the following tables:
CREATE TABLE person(
id serial,
lastname character varying (50),
firstname character varying (50),
CONSTRAINT person_pk PRIMARY KEY (id)
);
CREATE TABLE client (
id integer,
CONSTRAINT client_pk PRIMARY KEY (id),
CONSTRAINT fk_client_person FOREIGN KEY (id)
REFERENCES person (id) MATCH SIMPLE
);
If we need to insert a client record we must refer to a person record. But let's say we want to devise a PL/pgSQL function that inserts a new record into client but also takes care of inserting the new person record. For that, we must use a slight variation of leonbloy's OPTION 3:
INSERT INTO person(lastname, firstname)
VALUES (lastn, firstn)
RETURNING id INTO [new_variable];
Note that there are two INTO clauses. Therefore, the PL/pgSQL function would be defined like:
CREATE OR REPLACE FUNCTION new_client(lastn character varying, firstn character varying)
RETURNS integer AS
$BODY$
DECLARE
v_id integer;
BEGIN
-- Inserts the new person record and retrieves the last inserted id
INSERT INTO person(lastname, firstname)
VALUES (lastn, firstn)
RETURNING id INTO v_id;
-- Inserts the new client and references the inserted person
INSERT INTO client(id) VALUES (v_id);
-- Return the new id so we can use it in a select clause or return the new id into the user application
RETURN v_id;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
Now we can insert the new data using:
SELECT new_client('Smith', 'John');
or
SELECT * FROM new_client('Smith', 'John');
And we get the newly created id.
new_client
integer
----------
1
you can use RETURNING clause in INSERT statement,just like the following
wgzhao=# create table foo(id int,name text);
CREATE TABLE
wgzhao=# insert into foo values(1,'wgzhao') returning id;
id
----
1
(1 row)
INSERT 0 1
wgzhao=# insert into foo values(3,'wgzhao') returning id;
id
----
3
(1 row)
INSERT 0 1
wgzhao=# create table bar(id serial,name text);
CREATE TABLE
wgzhao=# insert into bar(name) values('wgzhao') returning id;
id
----
1
(1 row)
INSERT 0 1
wgzhao=# insert into bar(name) values('wgzhao') returning id;
id
----
2
(1 row)
INSERT 0
The other answers don't show how one might use the value(s) returned by RETURNING. Here's an example where the returned value is inserted into another table.
WITH inserted_id AS (
INSERT INTO tbl1 (col1)
VALUES ('foo') RETURNING id
)
INSERT INTO tbl2 (other_id)
VALUES ((select id from inserted_id));
See the below example
CREATE TABLE users (
-- make the "id" column a primary key; this also creates
-- a UNIQUE constraint and a b+-tree index on the column
id SERIAL PRIMARY KEY,
name TEXT,
age INT4
);
INSERT INTO users (name, age) VALUES ('Mozart', 20);
Then for getting last inserted id use this for table "user" seq column name "id"
SELECT currval(pg_get_serial_sequence('users', 'id'));
SELECT CURRVAL(pg_get_serial_sequence('my_tbl_name','id_col_name'))
You need to supply the table name and column name of course.
This will be for the current session / connection
http://www.postgresql.org/docs/8.3/static/functions-sequence.html
For the ones who need to get the all data record, you can add
returning *
to the end of your query to get the all object including the id.
You can use RETURNING id after insert query.
INSERT INTO distributors (id, name) VALUES (DEFAULT, 'ALI') RETURNING id;
and result:
id
----
1
In the above example id is auto-increment filed.
The better way is to use Insert with returning. Though there are already same answers, I just want to add, if you want to save this to a variable then you can do this
insert into my_table(name) returning id into _my_id;
Postgres has an inbuilt mechanism for the same, which in the same query returns the id or whatever you want the query to return.
here is an example. Consider you have a table created which has 2 columns column1 and column2 and you want column1 to be returned after every insert.
# create table users_table(id serial not null primary key, name character varying);
CREATE TABLE
#insert into users_table(name) VALUES ('Jon Snow') RETURNING id;
id
----
1
(1 row)
# insert into users_table(name) VALUES ('Arya Stark') RETURNING id;
id
----
2
(1 row)
Try this:
select nextval('my_seq_name'); // Returns next value
If this return 1 (or whatever is the start_value for your sequence), then reset the sequence back to the original value, passing the false flag:
select setval('my_seq_name', 1, false);
Otherwise,
select setval('my_seq_name', nextValue - 1, true);
This will restore the sequence value to the original state and "setval" will return with the sequence value you are looking for.
I had this issue with Java and Postgres.
I fixed it by updating a new Connector-J version.
postgresql-9.2-1002.jdbc4.jar
https://jdbc.postgresql.org/download.html:
Version 42.2.12
https://jdbc.postgresql.org/download/postgresql-42.2.12.jar
Based on #ooZman 's answer above, this seems to work for PostgreSQL v12 when you need to INSERT with the next value of a "sequence" (akin to auto_increment) without goofing anything up in your table(s) counter(s). (Note: I haven't tested it in more complex DB cluster configurations though...)
Psuedo Code
$insert_next_id = $return_result->query("select (setval('"your_id_seq"', (select nextval('"your_id_seq"')) - 1, true)) +1");