I would like to use sequences to create friendly IDs for some objects in my database. My problem is that I don't want the sequence to be global to the table, instead I want to increment the value based on a foreign key.
For example, my table is defined as:
CREATE TABLE foo (id numeric PRIMARY KEY, friendly_id SERIAL, bar_id numeric NOT NULL)
And I would like friendly_id to increment separately for each bar_id such that the following statements:
INSERT INTO foo (123, DEFAULT, 345)
INSERT INTO foo (124, DEFAULT, 345)
INSERT INTO foo (125, DEFAULT, 346)
INSERT INTO foo (126, DEFAULT, 345)
Would result in (desired behavior):
id | friendly_id | bar_id
-----------+------------------+-----------------
123 | 1 | 345
124 | 2 | 345
125 | 1 | 346
126 | 3 | 345
Instead of (current behavior):
id | friendly_id | bar_id
-----------+------------------+-----------------
123 | 1 | 345
124 | 2 | 345
125 | 3 | 346
126 | 4 | 345
Is this possible using sequences or is there a better way to achieve this?
create table foo (
id serial primary key,
friendly_id integer not null,
bar_id integer not null,
unique(friendly_id, bar_id)
);
At the application wrap the insertion in a exception catching loop to retry if a duplicate key exception is raised
insert into foo (friendly_id, bar_id)
select
coalesce(max(friendly_id), 0) + 1,
346
from foo
where bar_id = 346
Related
In a Postgres 9.3 database I have a table in which one column contains JSON, as in the test table shown in the example below.
test=# create table things (id serial PRIMARY KEY, details json, other_field text);
CREATE TABLE
test=# \d things
Table "public.things"
Column | Type | Modifiers
-------------+---------+-----------------------------------------------------
id | integer | not null default nextval('things_id_seq'::regclass)
details | json |
other_field | text |
Indexes:
"things_pkey" PRIMARY KEY, btree (id)
test=# insert into things (details, other_field)
values ('[{"json1": 123, "json2": 456},{"json1": 124, "json2": 457}]', 'nonsense');
INSERT 0 1
test=# insert into things (details, other_field)
values ('[{"json1": 234, "json2": 567}]', 'piffle');
INSERT 0 1
test=# select * from things;
id | details | other_field
----+-------------------------------------------------------------+-------------
1 | [{"json1": 123, "json2": 456},{"json1": 124, "json2": 457}] | nonsense
2 | [{"json1": 234, "json2": 567}] | piffle
(2 rows)
The JSON is always an array containing a variable number of hashes. Each hash always has the same set of keys. I am trying to write a query which returns a row for each entry in the JSON array, with columns for each hash key and the id from the things table. I'm hoping for output like the following:
thing_id | json1 | json2
----------+-------+-------
1 | 123 | 456
1 | 124 | 457
2 | 234 | 567
i.e. two rows for entries with two items in the JSON array. Is it possible to get Postgres to do this?
json_populate_recordset feels like an essential part of the answer, but I can't get it to work with more than one row at once.
select id,
(details ->> 'json1')::int as json1,
(details ->> 'json2')::int as json2
from (
select id, json_array_elements(details) as details
from things
) s
;
id | json1 | json2
----+-------+-------
1 | 123 | 456
1 | 124 | 457
2 | 234 | 567
My table permission looks like:
id serial,
person_id integer,
permission_id integer,
valid_from date,
valid_to date
I'd like to prevent creating permissions which overlaps valid_from, valid_to date
eg.
1 | 1 | 1 | 2010-10-01 | 2999-12-31
2 | 1 | 2 | 2010-10-01 | 2020-12-31
3 | 2 | 1 | 2015-10-01 | 2999-12-31
this can be added:
4 | 1 | 3 | 2011-10-01 | 2999-12-31 - because no such permission
5 | 2 | 1 | 2011-10-10 | 2999-12-31 - because no such person
6 | 1 | 2 | 2021-01-01 | 2999-12-31 - because doesn't overlaps id:2
but this can't
7 | 1 | 1 | 2009-10-01 | 2010-02-01 - because overlaps id:1
8 | 1 | 2 | 2019-01-01 | 2022-12-31 - because overlaps id:2
9 | 2 | 1 | 2010-01-01 | 2016-12-31 - beacuse overlaps id:3
I can do outside checking but wonder if possible to do it on database
A unique constraint is based on an equality operator and cannot be used in this case, but you can use an exclude constraint. The constraint uses btree operators <> and =, hence you have to install btree_gist extension.
create extension if not exists btree_gist;
create table permission(
id serial,
person_id integer,
permission_id integer,
valid_from date,
valid_to date,
exclude using gist (
person_id with =,
permission_id with =,
daterange(valid_from, valid_to) with &&)
);
These inserts are successful:
insert into permission values
(1, 1, 1, '2010-10-01', '2999-12-31'),
(2, 1, 2, '2010-10-01', '2020-12-31'),
(3, 2, 1, '2015-10-01', '2999-12-31'),
(4, 1, 3, '2011-10-01', '2999-12-31'),
(5, 3, 1, '2011-10-10', '2999-12-31'), -- you meant person_id = 3 I suppose
(6, 1, 2, '2021-01-01', '2999-12-31'),
(7, 1, 1, '2009-10-01', '2010-02-01'); -- ranges do not overlap!
but this one is not:
insert into permission values
(8, 1, 2, '2019-01-01', '2022-12-31');
ERROR: conflicting key value violates exclusion constraint "permission_person_id_permission_id_daterange_excl"
DETAIL: Key (person_id, permission_id, daterange(valid_from, valid_to))=(1, 2, [2019-01-01,2022-12-31)) conflicts with existing key (person_id, permission_id, daterange(valid_from, valid_to))=(1, 2, [2010-10-01,2020-12-31)).
Try it in db<>fiddle.
I need to optimize the performance of a commom WITH RECURSIVE query... We can limit the depth of the tree and decompose in many updates, and can also change representation (use array)... I try some options but perhaps there are a "classic optimization solution" that I'm not realizing.
All details
There are a t_up table, to be updated, with a composit primary key (pk1,pk2), one attribute attr and an array of references to primary keys... And a unnested representation t_scan, with the references; like this:
pk1 | pk2 | attr | ref_pk1 | ref_pk2
n | 123 | 1 | |
n | 456 | 2 | |
r | 123 | 1 | w | 123
w | 123 | 5 | n | 456
r | 456 | 2 | n | 123
r | 123 | 1 | n | 111
n | 111 | 4 | |
... | ...| ... | ... | ...
There are no loops.
UPDATE t_up SET x = pairs
FROM (
WITH RECURSIVE tree as (
SELECT pk1, pk2, attr, ref_pk1, ref_pk2,
array[array[0,0]]::bigint[] as all_refs
FROM t_scan
UNION ALL
SELECT c.pk1, c.pk2, c.attr, c.ref_pk1, c.ref_pk2
,p.all_refs || array[c.attr,c.pk2]
FROM t_scan c JOIN tree p
ON c.ref_pk1=p.pk1 AND c.ref_pk2=p.pk2 AND c.pk2!=p.pk2
AND array_length(p.all_refs,1)<5 -- 5 or 6 avoiding endless loops
)
SELECT pk1, pk2, array_agg_cat(all_refs) as pairs
FROM (
SELECT distinct pk1, pk2, all_refs
FROM tree
WHERE array_length(all_refs,1)>1 -- ignores initial array[0,0].
) t
GROUP BY 1,2
ORDER BY 1,2
) rec
WHERE rec.pk1=t_up.pk1 AND rec.pk2=t_up.pk2
;
To test:
CREATE TABLE t_scan(
pk1 char,pk2 bigint, attr bigint,
ref_pk1 char, ref_pk2 bigint
);
INSERT INTO t_scan VALUES
('n',123, 1 ,NULL,NULL),
('n',456, 2 ,NULL,NULL),
('r',123, 1 ,'w' ,123),
('w',123, 5 ,'n' ,456),
('r',456, 2 ,'n' ,123),
('r',123, 1 ,'n' ,111),
('n',111, 4 ,NULL,NULL);
Running only rec you will obtain:
pk1 | pk2 | pairs
-----+-----+-----------------
r | 123 | {{0,0},{1,123}}
r | 456 | {{0,0},{2,456}}
w | 123 | {{0,0},{5,123}}
But, unfortunately, to appreciate the "Big Data performance problem", you need to see it in a real database... I am preparing a public Github that run with OpenStreetMap Big Data.
I tried executing the same UPDATE query twice like below.
First time the transaction has no lock but I can see a row lock after second query.
Schema:
test=# \d t1
Table "public.t1"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
i | integer | | not null |
j | integer | | |
Indexes:
"t1_pkey" PRIMARY KEY, btree (i)
Referenced by:
TABLE "t2" CONSTRAINT "t2_j_fkey" FOREIGN KEY (j) REFERENCES t1(i)
test=# \d t2
Table "public.t2"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
i | integer | | not null |
j | integer | | |
k | integer | | |
Indexes:
"t2_pkey" PRIMARY KEY, btree (i)
Foreign-key constraints:
"t2_j_fkey" FOREIGN KEY (j) REFERENCES t1(i)
Existing data:
test=# SELECT * FROM t1 ORDER BY i;
i | j
---+---
1 | 1
2 | 2
(2 rows)
test=# SELECT * FROM t2 ORDER BY i;
i | j | k
---+---+---
3 | 1 |
4 | 2 |
(2 rows)
UPDATE queries and row lock status:
test=# BEGIN;
BEGIN
test=# UPDATE t2 SET k = 123 WHERE i = 3;
UPDATE 1
test=# SELECT * FROM t1 AS t, pgrowlocks('t1') AS p WHERE p.locked_row = t.ctid;
i | j | locked_row | locker | multi | xids | modes | pids
---+---+------------+--------+-------+------+-------+------
(0 rows)
test=# UPDATE t2 SET k = 123 WHERE i = 3;
UPDATE 1
test=# SELECT * FROM t1 AS t, pgrowlocks('t1') AS p WHERE p.locked_row = t.ctid;
i | j | locked_row | locker | multi | xids | modes | pids
---+---+------------+--------+-------+----------+-------------------+------
1 | 1 | (0,1) | 107239 | f | {107239} | {"For Key Share"} | {76}
(1 row)
test=#
Why does postgres try to get a row lock only on second time?
By the way, queries updating column t2.j create new lock (ForKeyShare) on t1 row at once. This behavior make sense because t2.j has foreign key constraint references t1.i. But the queries above seems not.
Does anyone can explain this lock?
PostgreSQL version: 9.6.3
Okay, I got it.
http://blog.nordeus.com/dev-ops/postgresql-locking-revealed.htm
This is optimization that exists in Postgres. If locking manager can figure out from the first query that foreign key is not changed (it is not mentioned in update query or is set to same value) it will not lock parent table. But in second query it will behave as it is described in documentation (it will lock parent table in ROW SHARE locking mode and referenced row in FOR SHARE mode)
It seems MySQL is wiser about foreign key locks because the same UPDATE query doesn't make such locks on MySQL.
I have a table with a list of names and indices. For example like this:
ID | Name | Index
1 | Value 1 | 3
2 | Value 2 | 4
...
N | Value N | NN
I need to create a new table, where every value from field "Name" will be repeat repeated as many times as the "Index" field is specified. For example like this:
ID | Name_2 | ID_2
1 | Value 1 | 1
2 | Value 1 | 2
3 | Value 1 | 3
4 | Value 2 | 1
5 | Value 2 | 2
6 | Value 2 | 3
7 | Value 2 | 4
...
N | Value N | 1
N+1| Value N | 2
...
I have no idea how to write a cycle to get such result. Please, give me an advice.
Here is solution to repeat the rows based on a column value
declare #order table ( Id int, name varchar(20), indx int)
Insert into #order
(Id, name, indx)
VALUES
(1,'Value1',3),
(2,'Value2',4),
(3,'Value3',2)
;WITH cte AS
(
SELECT * FROM #order
UNION ALL
SELECT cte.[ID], cte.name, (cte.indx - 1) indx
FROM cte INNER JOIN #order t
ON cte.[ID] = t.[ID]
WHERE cte.indx > 1
)
SELECT ROW_NUMBER() OVER(ORDER BY name ASC) AS Id, name as [name_2], 1 as [Id_2]
FROM cte
ORDER BY 1