I have a simple question, suppose we have a table:
id A B
1 Jon Doe
2 Foo Bar
Is there a way to know, which is the next id's increment, in this case 3 ?
Database is PostgreSQL!
Tnx alot!
If you want to claim an ID and return it, you can use nextval(), which advances the sequence without inserting any data.
Note that if this is a SERIAL column, you need to find the sequence's name based on the table and column name, as follows:
Select nextval(pg_get_serial_sequence('my_table', 'id')) as new_id;
There is no cast-iron guarantee that you'll see these IDs come back in order (the sequence generates them in order, but multiple sessions can claim an ID and not use it yet, or roll back an INSERT and the ID will not be reused) but there is a guarantee that they will be unique, which is normally the important thing.
If you do this often without actually using the ID, you will eventually use up all the possible values of a 32-bit integer column (i.e. reach the maximum representable integer), but if you use it only when there's a high chance you will actually be inserting a row with that ID it should be OK.
To get the current value of a sequence without affecting it or needing a previous insert in the same session, you can use;
SELECT last_value FROM tablename_fieldname_seq;
An SQLfiddle to test with.
Of course, getting the current value will not guarantee that the next value you'll get is actually last_value + 1 if there are other simultaneous sessions doing inserts, since another session may have taken the serial value before you.
SELECT currval('names_id_seq') + 1;
See the docs
However, of course, there's no guarantee that it's going to be your next value. What if another client grabs it before you? You can though reserve one of the next values for yourself, selecting a nextval from the sequence.
I'm new so here's the process I use having little to no prior knowledge of how Postgres/SQL work:
Find the sequence for your table using pg_get_serial_sequence()
SELECT pg_get_serial_sequence('person','id');
This should output something like public.person_id_seq. person_id_seq is the sequence for your table.
Plug the sequence from (1) into nextval()
SELECT nextval('person_id_seq');
This will output an integer value which will be the next id added to the table.
You can turn this into a single command as mentioned in the accepted answer above
SELECT nextval(pg_get_serial_sequence('person','id'));
If you notice that the sequence is returning unexpected values, you can set the current value of the sequence using setval()
SELECT setval(pg_get_serial_sequence('person','id'),1000);
In this example, the next call to nextval() will return 1001.
Related
I have an app which does have a maintenance menu (which is of course used in only very rare cases). In this menu I also display the next number that a certain sequence will generate next and give the user an option to reset the sequence.
I use the following query to show the next number:
select case when last_value is not null then last_value+increment_by else start_value end
from pg_sequences
where sequencename = 'my_sequence'
And if the user changes the sequence I run:
alter sequence my_sequence restart with $NEW_NUMBER
This usually works EXCEPT right after resetting the sequence with the query above and before any new number was pulled from the sequence. The my query to find out what the next number would be shows "1" which is not necessarily correct.
What can I do to reliably determine the next number that nextval would produce if called without actually calling nextval (to not actually alter the sequence)?
Use setval() instead of alter sequence..., e.g.
select setval('my_sequence', 110)
The last sequence value written to disk (last_value of pg_sequences) can be set only by nontransactional functions nextval() and setval(). After the sequence is restarted with alter sequence ... the last value is not saved until it is actually used by nextval() or set by setval(). This is because alter sequence ... may be rolled back.
I have a PostgreSQL table that can be simplified as this:
CREATE TABLE car(
garage_id uuid,
number integer
);
I need that in each garage, each car has a unique integer number (Car N°1, Car N°2).
First idea with sequence
My first idea was to create a PostgreSQL sequence for each garage thanks to a trigger, and then when I insert a car, I just need to call nextval on the sequence created for the garage. (Like seen here)
The problem is that for each garage, I need to create a sequence. My garage table will soon grow up to 1M rows, and according to this thread: Maximum number of sequences that can be created, too much PostgreSQL sequence can really hurt performance and make pgdump fails.
Second Idea
I found another idea on stackoverflow.
The second idea was to create another table like
CREATE TABLE garage_number(
garage_id uuid,
max_number integer
);
This table would keep the max car number, and then at each insert we select the max_number of the garage, and update the max_number of the garage.
Problem: For the moment it does not work in terms of concurrency (2 cars can have the same number if inserted at the same time).
Do you have any idea of how I can make this be transaction-proof and work at scale? Maybe locking the garage_number table?
Thank you very much for your help,
"how I can make this be transaction-proof and work at scale" - you can't. The best bet is to minimize the locking that is needed when you generate the numbers via the non-sequence approach.
Your garage_number table can be used for that, however I wouldn't actually use the max() on the actual garage table, but rather something like this:
create or replace function next_car_number(p_garage uuid)
returns integer
as
$$
update garage_number
set max_number = max_number + 1
where garage_id = p_garage
returning max_number;
$$
language sql;
Of course you would need to initialize the garage_number once, and you also need to insert a new row into that table each time a new garage is created (with a value of 0 for max_number)
Then to assign a new car number, you can use something like this:
insert into car
(garage, number)
values
('a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11', next_car_number('a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11'));
The row for that garage will be locked until that insert is committed and no other insert using the same function can finish until then. Once that is committed any waiting transaction will see the new value.
You need to generate (and know) the UUID for the garage before you insert a new car - but I guess as there is another table named garage that is referenced by the car table, the garage PK is already know when inserting a new car.
This still serializes access to the "generator" for a single garage, but will be faster then running a max() query for each insert
If you don't insert several cars in one transaction, maybe it's worth of trying the txid_current()?.. The number would be isolated from other transactions and would be unique. The only obvious but here - it would not be sequential (you cant predict the step, as it depends on amount of transactions while you perform yours).
smth like:
insert into car
(garage, number)
values
('a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11', txid_current());
Of course it will not work if you run the above statement several times in same transaction, as txid_current() would be same. But if you separate each insert - this might work
https://www.postgresql.org/docs/current/static/functions-info.html
In PostgreSQL (9.3) I have a table defined as:
CREATE TABLE charts
( recid serial NOT NULL,
groupid text NOT NULL,
chart_number integer NOT NULL,
"timestamp" timestamp without time zone NOT NULL DEFAULT now(),
modified timestamp without time zone NOT NULL DEFAULT now(),
donotsee boolean,
CONSTRAINT pk_charts PRIMARY KEY (recid),
CONSTRAINT chart_groupid UNIQUE (groupid),
CONSTRAINT charts_ichart_key UNIQUE (chart_number)
);
CREATE TRIGGER update_modified
BEFORE UPDATE ON charts
FOR EACH ROW EXECUTE PROCEDURE update_modified();
I would like to replace the chart_number with a sequence like:
CREATE SEQUENCE charts_chartnumber_seq START 16047;
So that by trigger or function, adding a new chart record automatically generates a new chart number in ascending order. However, no existing chart record can have its chart number changed and over the years there have been skips in the assigned chart numbers. Hence, before assigning a new chart number to a new chart record, I need to be sure that the "new" chart number has not yet been used and any chart record with a chart number is not assigned a different number.
How can this be done?
Consider not doing it. Read these related answers first:
Gap-less sequence where multiple transactions with multiple tables are involved
Compacting a sequence in PostgreSQL
If you still insist on filling in gaps, here is a rather efficient solution:
1. To avoid searching large parts of the table for the next missing chart_number, create a helper table with all current gaps once:
CREATE TABLE chart_gap AS
SELECT chart_number
FROM generate_series(1, (SELECT max(chart_number) - 1 -- max is no gap
FROM charts)) chart_number
LEFT JOIN charts c USING (chart_number)
WHERE c.chart_number IS NULL;
2. Set charts_chartnumber_seq to the current maximum and convert chart_number to an actual serial column:
SELECT setval('charts_chartnumber_seq', max(chart_number)) FROM charts;
ALTER TABLE charts
ALTER COLUMN chart_number SET NOT NULL
, ALTER COLUMN chart_number SET DEFAULT nextval('charts_chartnumber_seq');
ALTER SEQUENCE charts_chartnumber_seq OWNED BY charts.chart_number;
Details:
How to reset postgres' primary key sequence when it falls out of sync?
Safely and cleanly rename tables that use serial primary key columns in Postgres?
3. While chart_gap is not empty fetch the next chart_number from there.
To resolve possible race conditions with concurrent transactions, without making transactions wait, use advisory locks:
WITH sel AS (
SELECT chart_number, ... -- other input values
FROM chart_gap
WHERE pg_try_advisory_xact_lock(chart_number)
LIMIT 1
FOR UPDATE
)
, ins AS (
INSERT INTO charts (chart_number, ...) -- other target columns
TABLE sel
RETURNING chart_number
)
DELETE FROM chart_gap c
USING ins i
WHERE i.chart_number = c.chart_number;
Alternatively, Postgres 9.5 or later has the handy FOR UPDATE SKIP LOCKED to make this simpler and faster:
...
SELECT chart_number, ... -- other input values
FROM chart_gap
LIMIT 1
FOR UPDATE SKIP LOCKED
...
Detailed explanation:
Postgres UPDATE ... LIMIT 1
Check the result. Once all rows are filled in, this returns 0 rows affected. (you could check in plpgsql with IF NOT FOUND THEN ...). Then switch to a simple INSERT:
INSERT INTO charts (...) -- don't list chart_number
VALUES (...); -- don't provide chart_number
In PostgreSQL, a SEQUENCE ensures the two requirements you mention, that is:
No repeats
No changes once assigned
But because of how a SEQUENCE works (see manual), it can not ensure no-skips. Among others, the first two reasons that come to mind are:
How a SEQUENCE handles concurrent blocks with INSERTS (you could also add that the concept of Cache also makes this impossible)
Also, user triggered DELETEs are an uncontrollable aspect that a SEQUENCE can not handle by itself.
In both cases, if you still do not want skips, (and if you really know what you're doing) you should have a separate structure that assign IDs (instead of using SEQUENCE). Basically a system that has a list of 'assignable' IDs stored in a TABLE that has a function to pop out IDs in a FIFO way. That should allow you to control DELETEs etc.
But again, this should be attempted, only if you really know what you're doing! There's a reason why people don't do SEQUENCEs themselves. There are hard corner-cases (for e.g. concurrent INSERTs) and most probably you're over-engineering your problem case, that probably can be solved in a much better / cleaner way.
Sequence numbers usually have no meaning, so why worry? But if you really want this, then follow the below, cumbersome procedure. Note that it is not efficient; the only efficient option is to forget about the holes and use the sequence.
In order to avoid having to scan the charts table on every insert, you should scan the table once and store the unused chart_number values in a separate table:
CREATE TABLE charts_unused_chart_number AS
SELECT seq.unused
FROM (SELECT max(chart_number) FROM charts) mx,
generate_series(1, mx(max)) seq(unused)
LEFT JOIN charts ON charts.chart_number = seq.unused
WHERE charts.recid IS NULL;
The above query generates a contiguous series of numbers from 1 to the current maximum chart_number value, then LEFT JOINs the charts table to it and find the records where there is no corresponding charts data, meaning that value of the series is unused as a chart_number.
Next you create a trigger that fires on an INSERT on the charts table. In the trigger function, pick a value from the table created in the step above:
CREATE FUNCTION pick_unused_chart_number() RETURNS trigger AS $$
BEGIN
-- Get an unused chart number
SELECT unused INTO NEW.chart_number FROM charts_unused_chart_number LIMIT 1;
-- If the table is empty, get one from the sequence
IF NOT FOUND THEN
NEW.chart_number := next_val(charts_chartnumber_seq);
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_charts_cn
BEFORE INSERT ON charts
FOR EACH ROW EXECUTE PROCEDURE pick_unused_chart_number();
Easy. But the INSERT may fail because of some other trigger aborting the procedure or any other reason. So you need a check to ascertain that the chart_number was indeed inserted:
CREATE FUNCTION verify_chart_number() RETURNS trigger AS $$
BEGIN
-- If you get here, the INSERT was successful, so delete the chart_number
-- from the temporary table.
DELETE FROM charts_unused_chart_number WHERE unused = NEW.chart_number;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tr_charts_verify
AFTER INSERT ON charts
FOR EACH ROW EXECUTE PROCEDURE verify_chart_number();
At a certain point the table with unused chart numbers will be empty whereupon you can (1) ALTER TABLE charts to use the sequence instead of an integer for chart_number; (2) delete the two triggers; and (3) the table with unused chart numbers; all in a single transaction.
While what you want is possible, it can't be done using only a SEQUENCE and it requires an exclusive lock on the table, or a retry loop, to work.
You'll need to:
LOCK thetable IN EXCLUSIVE MODE
Find the first free ID by querying for the max id then doing a left join over generate_series to find the first free entry. If there is one.
If there is a free entry, insert it.
If there is no free entry, call nextval and return the result.
Performance will be absolutely horrible, and transactions will be serialized. There'll be no concurrency. Also, unless the LOCK is the first thing you run that affects that table, you'll face deadlocks that cause transaction aborts.
You can make this less bad by using an AFTER DELETE .. FOR EACH ROW trigger that keeps track of entries you delete by INSERTing them into a one-column table that keeps track of spare IDs. You can then SELECT the lowest ID from the table in your ID assignment function on the default for the column, avoiding the need for the explicit table lock, the left join on generate_series and the max call. Transactions will still be serialized on a lock on the free IDs table. In PostgreSQL you can even solve that using SELECT ... FOR UPDATE SKIP LOCKED. So if you're on 9.5 you can actually make this non-awful, though it'll still be slow.
I strongly advise you to just use a SEQUENCE directly, and not bother with re-using values.
How is it possible to get the current sequence value in postgresql 8.4?
Note: I need the value for the some sort of statistics, just retrieve and store. Nothing related to the concurrency and race conditions in case of manually incrementing it isn't relevant to the question.
Note 2: The sequence is shared across several tables
Note 3: currval won't work because of:
Return the value most recently obtained by nextval for this sequence in the current session
ERROR: currval of sequence "<sequence name>" is not yet defined in this session
My current idea: is to parse DDL, which is weird
You may use:
SELECT last_value FROM sequence_name;
Update:
this is documented in the CREATE SEQUENCE statement:
Although you cannot update a sequence directly, you can use a query
like:
SELECT * FROM name;
to examine the parameters and current state of a
sequence. In particular, the last_value field of the sequence shows
the last value allocated by any session. (Of course, this value might
be obsolete by the time it's printed, if other sessions are actively
doing nextval calls.)
If the sequence is being used for unique ids in a table, you can simply do this:
select max(id) from mytable;
The most efficient way, although postgres specific, is:
select currval('mysequence');
although technically this returns the last value generated by the call to nextval('mysequence'), which may not necessarily be used by the caller (and if unused would leave gaps in an auto increments id column).
As of some point in time, a secret function (which is undocumented ... still), does exactly this:
SELECT pg_sequence_last_value('schema.your_sequence_name');
We have an Oracle application that uses a standard pattern to populate surrogate keys. We have a series of extrinsic rows (that have specific values for the surrogate keys) and other rows that have intrinsic values.
We use the following Oracle trigger snippet to determine what to do with the Surrogate key on insert:
IF :NEW.SurrogateKey IS NULL THEN
SELECT SurrogateKey_SEQ.NEXTVAL INTO :NEW.SurrogateKey FROM DUAL;
END IF;
If the supplied surrogate key is null then get a value from the nominated sequence, else pass the supplied surrogate key through to the row.
I can't seem to find an easy way to do this is T-SQL. There are all sorts of approaches, but none of which use the notion of a sequence generator like Oracle and other SQL-92 compliant DBs do.
Anybody know of a really efficient way to do this in SQL Server T-SQL? By the way, we're using SQL Server 2008 if that's any help.
You may want to look at IDENTITY. This gives you a column for which the value will be determined when you insert the row.
This may mean that you have to insert the row, and determine the value afterwards, using SCOPE_IDENTITY().
There is also an article on simulating Oracle Sequences in SQL Server here: http://www.sqlmag.com/Articles/ArticleID/46900/46900.html?Ad=1
Identity is one approach, although it will generate unique identifiers at a per table level.
Another approach is to use unique identifiers, in particualr using NewSequantialID() that ensues the generated id is always bigger than the last. The problem with this approach is you are no longer dealing with integers.
The closest way to emulate the oracle method is to have a separate table with a counter field, and then write a user defined function that queries this field, increments it, and returns the value.
Here is a way to do it using a table to store your last sequence number. The stored proc is very simple, most of the stuff in there is because I'm lazy and don't like surprises should I forget something so...here it is:
----- Create the sequence value table.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[SequenceTbl]
(
[CurrentValue] [bigint]
) ON [PRIMARY]
GO
-----------------Create the stored procedure
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE procedure [dbo].[sp_NextInSequence](#SkipCount BigInt = 1)
AS
BEGIN
BEGIN TRANSACTION
DECLARE #NextInSequence BigInt;
IF NOT EXISTS
(
SELECT
CurrentValue
FROM
SequenceTbl
)
INSERT INTO SequenceTbl (CurrentValue) VALUES (0);
SELECT TOP 1
#NextInSequence = ISNULL(CurrentValue, 0) + 1
FROM
SequenceTbl WITH (HoldLock);
UPDATE SequenceTbl WITH (UPDLOCK)
SET CurrentValue = #NextInSequence + (#SkipCount - 1);
COMMIT TRANSACTION
RETURN #NextInSequence
END;
GO
--------Use the stored procedure in Sql Manager to retrive a test value.
declare #NextInSequence BigInt
exec #NextInSequence = sp_NextInSequence;
--exec #NextInSequence = sp_NextInSequence <skipcount>;
select NextInSequence = #NextInSequence;
-----Show the current table value.
select * from SequenceTbl;
The astute will notice that there is a parameter (optional) for the stored proc. This is to allow the caller to reserve a block of ID's in the instance that the caller has more than one record that needs a unique id - using the SkipCount, the caller need make only a single call for however many IDs are needed.
The entire "IF EXISTS...INSERT INTO..." block can be removed if you remember to insert a record when the table is created. If you also remember to insert that record with a value (your seed value - a number which will never be used as an ID), you can also remove the ISNULL(...) portion of the select and just use CurrentValue + 1.
Now, before anyone makes a comment, please note that I am a software engineer, not a dba! So, any constructive criticism concerning the use of "Top 1", "With (HoldLock)" and "With (UPDLock)" is welcome. I don't know how well this will scale but this works OK for me so far...