Go pq and Postgres appropriate error handling for constraints - postgresql

I'm currently using the pq lib for Go to communicate with my PostgreSQL database. Error checking is proving to be a little more difficult than anticipated. The easiest way to describe my question is through an example scenario.
Imagine a web form:
Username ________
Email ________
Voucher ________
Password ________
A rough schema:
username VARCHAR(255) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
voucher VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL
Ignore the presumed plain text password for now. If a person submits the form, I can do all of my validation to verify constraints such as length/allowed characters/etc.
Now it comes to putting it in the database, so we write a prepared statement and execute it. If the validation was done correctly, the only thing that can really go wrong is the UNIQUE constraints. In the event that someone attempts to enter an existing username, database/sql is going to fire back an error.
My problem is that I have no idea what to do with that error and recover from (what should be) a recoverable error. pq provides some support for this, but there still appears to be come ambiguity to what's returned.
I can see two solutions, neither of which sound particularly appealing to me:
A SERIALIZABLE transaction which checks every single form value prior to insertion. Alternatively, some form of parsing on the pq error struct.
Is there a common pattern for implementing such a system? I'd like to be able to say to a user Sorry that username exists rather than Sorry something bad happened
As a sidenote, the PostgreSQL documentation states:
The fields for
schema name, table name, column name, data type name, and constraint
name are supplied only for a limited number of error types; see
Appendix A.
but the linked page isn't very helpful with respect to values returned in the database object.

If the validation was done correctly, the only thing that can really go wrong is the UNIQUE constraints.
No, the client could lack sufficient privileges, the client might have entered a valid password that's not the right password, the client might have entered a valid voucher that belongs to a different client, etc.
Using "A SERIALIZABLE transaction which checks every single form value prior to insertion" doesn't make sense. Just insert data, and trap errors.
At the very least, your code needs to examine and respond to the C (Code) field, which is always present in the error struct. You don't need to parse the error struct, but you do need to read it.
If you violate a unique constraint, PostgreSQL will return SQL state 23505 in the Code field. It will also return the name of the first constraint that's violated. It doesn't return the column name, probably because a unique constraint can include more than one column.
You can select the column(s) the constraint refers to by querying the information_schema views.
Here's a simple version of your table.
create table test (
username VARCHAR(255) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
voucher VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL
);
insert into test values ('msherrill', 'me#example.com', 'a', 'wibble');
This quick and dirty go program inserts the same row again. It violates every unique constraint.
package main
import (
"github.com/lib/pq"
"database/sql"
"fmt"
"log"
)
func main() {
db, err := sql.Open("postgres", "host=localhost port=5435 user=postgres password=xxxxxxxx dbname=scratch sslmode=disable")
if err != nil {
log.Fatal(err)
}
rows, err := db.Exec("insert into public.test values ('msherrill', 'me#example.com', 'a', 'wibble');")
if err, ok := err.(*pq.Error); ok {
fmt.Println("Severity:", err.Severity)
fmt.Println("Code:", err.Code)
fmt.Println("Message:", err.Message)
fmt.Println("Detail:", err.Detail)
fmt.Println("Hint:", err.Hint)
fmt.Println("Position:", err.Position)
fmt.Println("InternalPosition:", err.InternalPosition)
fmt.Println("Where:", err.Where)
fmt.Println("Schema:", err.Schema)
fmt.Println("Table:", err.Table)
fmt.Println("Column:", err.Column)
fmt.Println("DataTypeName:", err.DataTypeName)
fmt.Println("Constraint:", err.Constraint)
fmt.Println("File:", err.File)
fmt.Println("Line:", err.Line)
fmt.Println("Routine:", err.Routine)
}
fmt.Println(rows)
}
Here's the output.
Severity: ERROR
Code: 23505
Message: duplicate key value violates unique constraint "test_username_key"
Detail: Key (username)=(msherrill) already exists.
Hint:
Position:
InternalPosition:
Where:
Schema: public
Table: test
Column:
DataTypeName:
Constraint: test_username_key
File: nbtinsert.c
Line: 406
Routine: _bt_check_unique
You have the schema, table, and constraint names. You presumably know the database (catalog) name, too. Use these values to select the schema, table, and column names from information_schema views. You're lucky; in this case you need only one view.
select table_catalog, table_schema, table_name, column_name
from information_schema.key_column_usage
where
table_catalog = 'scratch' and -- Database name
table_schema = 'public' and -- value returned by err.Schema
table_name = 'test' and -- value returned by err.Table
constraint_name = 'test_username_key' -- value returned by err.Constraint
order by constraint_catalog, constraint_schema, constraint_name, ordinal_position;

Related

PostgreSQL two-table constaint not working properly

I run the following code:
-- Table describing messages
CREATE TABLE messages
(
id serial PRIMARY KEY NOT NULL,
text TEXT -- Message can have or not have text
);
-- Table describing media attached to messages
CREATE TABLE messages_attachments
(
message_id integer NOT NULL REFERENCES messages,
-- Messages can have any number of attachments, including 0
attachment_id TEXT NOT NULL
);
-- Messages must have either text or at least one attachment
CREATE FUNCTION message_has_text_or_attachments(integer) RETURNS bool STABLE
AS
$$
SELECT
EXISTS(SELECT 1 FROM messages_attachments WHERE message_id = $1)
OR
(SELECT text IS NOT NULL FROM messages WHERE id = $1);
$$ LANGUAGE SQL;
ALTER TABLE messages ADD CONSTRAINT nonempty_message CHECK ( message_has_text_or_attachments(id) );
-- Insert a message with no text and no attachments. Should fail, but it does not
INSERT INTO messages(text) VALUES (NULL);
SELECT *, message_has_text_or_attachments(id) FROM messages;
I expected it to fail on the INSERT line because the row being inserted violates the check constraint (we are inserting a message which's text is NULL and there are no attachments for that message), but it runs successfully and the next query returns (1, NULL, false) (here is an example with slightly modified function definition (apostrophes instead of dollar symbols because of the database version).
One more interesting thing is that if I change the order of the commands and INSERT the row before adding the CONSTRAINT, then PostgreSQL fails to ALTER the table, because "check constraint "nonempty_message" is violated by some row".
Why does PostgreSQL allow inserting the row, which violates the constraint? Am I mistaken somewhere in the function definition? Is there some limitation on how constraints can be applied and which tables can they depend on? Is it a PostgreSQL bug?
From the docs:
PostgreSQL does not support CHECK constraints that reference table data other than the new or updated row being checked. While a CHECK constraint that violates this rule may appear to work in simple tests, it cannot guarantee that the database will not reach a state in which the constraint condition is false (due to subsequent changes of the other row(s) involved).

Cross table constraints in PostgreSQL

Using PostgreSQL 9.2.4, I have a table users with a 1:many relation to the table user_roles. The users table stores both employees and other kinds of users.
Table "public.users"
Column | Type | Modifiers
-----------------+-------------------+-----------------------------------------------------
uid | integer | not null default nextval('users_uid_seq'::regclass)
employee_number | character varying |
name | character varying |
Indexes:
"users_pkey" PRIMARY KEY, btree (uid)
Referenced by:
TABLE "user_roles" CONSTRAINT "user_roles_uid_fkey" FOREIGN KEY (uid) REFERENCES users(uid)
Table "public.user_roles"
Column | Type | Modifiers
-----------+-------------------+------------------------------------------------------------------
id | integer | not null default nextval('user_roles_id_seq'::regclass)
uid | integer |
role | character varying | not null
Indexes:
"user_roles_pkey" PRIMARY KEY, btree (id)
Foreign-key constraints:
"user_roles_uid_fkey" FOREIGN KEY (uid) REFERENCES users(uid)
I want to ensure that the column users.employee_number cannot be NULL if there is a related row where user_roles.role_name contains an employee role name. That is, I want the database to enforce the constraint that for some roles, users.employee_number must have a value, but not for others.
How can I accomplish this, preferably without user-defined functions or triggers? I found (blog post, SO Answer) that SQL Server supports indexed views, which sounds like it would serve my purpose. However, I assume that materialized views will not work in my case, since they are not dynamically updated.
Clarifications
The formulation of this requirement leaves room for interpretation:
where UserRole.role_name contains an employee role name.
My interpretation:
with an entry in UserRole that has role_name = 'employee'.
Your naming convention is was problematic (updated now). User is a reserved word in standard SQL and Postgres. It's illegal as identifier unless double-quoted - which would be ill-advised. User legal names so you don't have to double-quote.
I am using trouble-free identifiers in my implementation.
The problem
FOREIGN KEY and CHECK constraint are the proven, air-tight tools to enforce relational integrity. Triggers are powerful, useful and versatile features but more sophisticated, less strict and with more room for design errors and corner cases.
Your case is difficult because a FK constraint seems impossible at first: it requires a PRIMARY KEY or UNIQUE constraint to reference - neither allows NULL values. There are no partial FK constraints, the only escape from strict referential integrity are NULL values in the referencing columns due to the default MATCH SIMPLE behavior of FK constraints. Per documentation:
MATCH SIMPLE allows any of the foreign key columns to be null; if any
of them are null, the row is not required to have a match in the referenced table.
Related answer on dba.SE with more:
Two-column foreign key constraint only when third column is NOT NULL
The workaround is to introduce a boolean flag is_employee to mark employees on both sides, defined NOT NULL in users, but allowed to be NULL in user_role:
Solution
This enforces your requirements exactly, while keeping noise and overhead to a minimum:
CREATE TABLE users (
users_id serial PRIMARY KEY
, employee_nr int
, is_employee bool NOT NULL DEFAULT false
, CONSTRAINT role_employee CHECK (employee_nr IS NOT NULL = is_employee)
, UNIQUE (is_employee, users_id) -- required for FK (otherwise redundant)
);
CREATE TABLE user_role (
user_role_id serial PRIMARY KEY
, users_id int NOT NULL REFERENCES users
, role_name text NOT NULL
, is_employee bool CHECK(is_employee)
, CONSTRAINT role_employee
CHECK (role_name <> 'employee' OR is_employee IS TRUE)
, CONSTRAINT role_employee_requires_employee_nr_fk
FOREIGN KEY (is_employee, users_id) REFERENCES users(is_employee, users_id)
);
That's all.
These triggers are optional but recommended for convenience to set the added tags is_employee automatically and you don't have to do anything extra:
-- users
CREATE OR REPLACE FUNCTION trg_users_insup_bef()
RETURNS trigger AS
$func$
BEGIN
NEW.is_employee = (NEW.employee_nr IS NOT NULL);
RETURN NEW;
END
$func$ LANGUAGE plpgsql;
CREATE TRIGGER insup_bef
BEFORE INSERT OR UPDATE OF employee_nr ON users
FOR EACH ROW
EXECUTE PROCEDURE trg_users_insup_bef();
-- user_role
CREATE OR REPLACE FUNCTION trg_user_role_insup_bef()
RETURNS trigger AS
$func$
BEGIN
NEW.is_employee = true;
RETURN NEW;
END
$func$ LANGUAGE plpgsql;
CREATE TRIGGER insup_bef
BEFORE INSERT OR UPDATE OF role_name ON user_role
FOR EACH ROW
WHEN (NEW.role_name = 'employee')
EXECUTE PROCEDURE trg_user_role_insup_bef();
Again, no-nonsense, optimized and only called when needed.
SQL Fiddle demo for Postgres 9.3. Should work with Postgres 9.1+.
Major points
Now, if we want to set user_role.role_name = 'employee', then there has to be a matching user.employee_nr first.
You can still add an employee_nr to any user, and you can (then) still tag any user_role with is_employee, irregardless of the actual role_name. Easy to disallow if you need to, but this implementation does not introduce any more restrictions than required.
users.is_employee can only be true or false and is forced to reflect the existence of an employee_nr by the CHECK constraint. The trigger keeps the column in sync automatically. You could allow false additionally for other purposes with only minor updates to the design.
The rules for user_role.is_employee are slightly different: it must be true if role_name = 'employee'. Enforced by a CHECK constraint and set automatically by the trigger again. But it's allowed to change role_name to something else and still keep is_employee. Nobody said a user with an employee_nr is required to have an according entry in user_role, just the other way round! Again, easy to enforce additionally if needed.
If there are other triggers that could interfere, consider this:
How To Avoid Looping Trigger Calls In PostgreSQL 9.2.1
But we need not worry that rules might be violated because the above triggers are only for convenience. The rules per se are enforce with CHECK and FK constraints, which allow no exceptions.
Aside: I put the column is_employee first in the constraint UNIQUE (is_employee, users_id) for a reason. users_id is already covered in the PK, so it can take second place here:
DB associative entities and indexing
First, you can solve this using a trigger.
But, I think you can solve this using constraints, with just a little weirdness:
create table UserRoles (
UserRoleId int not null primary key,
. . .
NeedsEmployeeNumber boolean not null,
. . .
);
create table Users (
. . .
UserRoleId int,
NeedsEmployeeNumber boolean,
EmployeeNumber,
foreign key (UserRoleId, NeedsEmployeeNumber) references UserRoles(UserRoleId, NeedsEmployeeNumber),
check ((NeedsEmployeeNumber and EmployeeNumber is not null) or
(not NeedsEmployeeNumber and EmployeeNumber is null)
)
);
This should work, but it is an awkward solution:
When you add a role to an employee, you need to add the flag along with the role.
If a role is updated to change the flag, then this needs to be propagated to existing records -- and the propagation cannot be automatic because you also need to potentially set EmployeeNumber.
New Answer:
This( SQL Sub queries in check constraint ) seems to answer your question, and the language is still in the 9.4 documentation( http://www.postgresql.org/docs/9.4/interactive/sql-createtable.html ).
Old Answer:
SELECT
User.*
, UserRole1.*
FROM
User
LEFT JOIN UserRole UserRole1
ON User.id = UserRole1.UserId
AND (
(
User.employee_number IS NOT NULL AND UserRole1.role_name IN (enumerate employee role names here)
)
OR
(User.employee_number IS NULL)
)
The above query selects all fields from User and all fields from UserRole(aliased as UserRole1). I assumed that the key field between the two fields is known as User.id and UserRole1.UserId, please change these to whatever the real values are.
In the JOIN part of the query there is an OR that on the left side requires an employee number not be NULL in the user table and that UserRole1.role_name be in a list that you must supply to the IN () operator.
The right part of the JOIN is the opposite, it requires that User.employee_number be NULL(this should be your non-employee set).
If you require a more exact solution then please provide more details on your table structures and what roles must be selected for employees.

How to tell PostgreSQL not to verify datatype

I have a table like:
CREATE TABLE test(
id integer not null default nextval('test_id_seq'::regclass),
client_name_id integer not null
);
Foreign-key constraints:
"test_client_name_id_fkey" FOREIGN KEY (client_name_id) REFERENCES company(id) DEFERRABLE INITIALLY DEFERRED
and company table:
CREATE TABLE company(
id integer not null default nextval('company_id_seq'::regclass),
company_name character varying(64) not null
)
Now I have trigger on test table which fetch id from company table using provided value client_name_id which is string by matching it with company_name. but when I insert record PostgreSQL return error that client_name_id is string and int required which is true.
How can I tell PostgreSQL not to verify inserted row as I have taken care of it in my triggers.
What you are trying to do is very unorthodox. Are you sure, this is what you want? Of course, you cant enter a string (with non-digits) into an integer column. No surprise there, right? If you want to enter the text instead, you'd have to add a text column instead - with a fk-constraint to company(company_name) if you want to match your current layout.
ALTER TABLE test ALTER DROP COLUMN client_name_id; -- drops fk constraint, too
ALTER TABLE test ADD COLUMN client_name REFERENCES company(company_name);
You would need a UNIQUE constraint on company.company_name to allow this.
However, I would advise to rethink your approach. Your table layout seems proper as it is. The trigger is the unconventional element. Normally, you would reference the primary key, just like you have it now. No trigger needed. To get the company name, you would join the table in a SELECT:
SELECT *
FROM test t
JOIN company c ON t.client_name_id = c.id;
Also, these non-standard modifiers should only be there if you need them: DEFERRABLE INITIALLY DEFERRED. Like, when you have to enter values in table test before you enter the referenced values in table company (in the same transaction).

PostgreSQL ON INSERT CASCADE

I've got two tables - one is Product and one is ProductSearchResult.
Whenever someone tries to Insert a SearchResult with a product that is not listed in the Product table the foreign key constrain is violattet, hence i get an error.
I would like to know how i could get my database to automatically create that missing Product in the Product Table (Just the ProductID, all other attributes can be left blank)
Is there such thing as CASCADE ON INSERT? If there is, i was not able not get it working.
Rules are getting executed after the Insert, so because we get an Error beforehand there are useless if you USE an "DO ALSO". If you use "DO INSTEAD" and add the INSERT Command at the End you end up with endless recursion.
I reckon a Trigger is the way to go - but all my attempts to write one failed.
Any recommendations?
The Table Structure:
CREATE TABLE Product (
ID char(10) PRIMARY KEY,
Title varchar(150),
Manufacturer varchar(80),
Category smallint,
FOREIGN KEY(Category) REFERENCES Category(ID) ON DELETE CASCADE);
CREATE TABLE ProductSearchResult (
SearchTermID smallint NOT NULL,
ProductID char(10) NOT NULL,
DateFirstListed date NOT NULL DEFAULT current_date,
DateLastListed date NOT NULL DEFAULT current_date,
PRIMARY KEY (SearchTermID,ProductID),
FOREIGN KEY (SearchTermID) REFERENCES SearchTerm(ID) ON DELETE CASCADE,
FOREIGN KEY (ProductID) REFERENCES Product ON DELETE CASCADE);
Yes, triggers are the way to go. But before you can start to use triggers in plpgsql, you
have to enable the language. As user postgres, run the command createlang with the proper parameters.
Once you've done that, you have to
Write function in plpgsql
create a trigger to invoke that function
See example 39-3 for a basic example.
Note that a function body in Postgres is a string, with a special quoting mechanism: 2 dollar signs with an optional word in between them, as the quotes. (The word allows you to quote other similar quotes.)
Also note that you can reuse a trigger procedure for multiple tables, as long as they have the columns your procedure uses.
So the function has to
check if the value of NEW.ProductID exists in the ProductSearchResult table, with a select statement (you ought to be able to use SELECT count(*) ... INTO someint, or SELECT EXISTS(...) INTO somebool)
if not, insert a new row in that table
If you still get stuck, come back here.
In any case (rules OR triggers) the insert needs to create a new key (and new values for the attributes) in the products table. In most cases, this implies that a (serial,sequence) surrogate primary key should be used in the products table, and that the "real world" product_id ("product number") should default to NULL, and be degraded to a candidate key.
BTW: a rule can be used, rules just are tricky to implement correctly for N:1 relations (they need the same kind of EXISTS-logic as in Bart's answer above).
Maybe cascading on INSERT is not such a good idea after all. What do you want to happen if someone inserts a ProductSearchResult record for a not-existing product? [IMO a FK is always a domain; you cannot just extend a domain just by referring to a not-existant value for it; that would make the FK constraint meaningless]

Getting right row while using inheritance of tables in PostgreSQL

I'm recently trying to drop Doctrine due to performance and problems with abstraction and move to database driven logic. I'm using mostly PostgreSQL.
Doctrine
One thing that I liked about Doctrine was inheritance, which I used for multiple roles in web app. There is base table/class Person and every role (for example admin, developer, user) extends this class. All users share one base table, so it helps to keep unique login/idenfiticator (in my case it's email). But getting person's info from doctrine resulted in final class, with all it's properties. For example:
$user = $em->getRepository('Entities\Person')->findOneBy(array('email' => 'john.doe#example.com'));
if ( $user instanceof Entities\Developer) {
...
}
Nice feature, but when having many roles, resulting SQL query was very ineffective, selecting from base class left joining all roles and then by defined discriminator mapper builded final class from base table and final table.
PostgreSQL
I discovered that postgres has implemented table inheritance and it works well. But I'd like to simulate Doctrine's behaviour, getting role from db (without knowing it's role and therefore it's final table).
For better examples, my tables look like this:
--
-- base people table
--
CREATE TABLE people
(
id serial NOT NULL,
first_name character varying(25) NOT NULL,
last_name character varying(25) NOT NULL,
email character varying(50) NOT NULL,
"password" character varying(150),
CONSTRAINT people_pkey PRIMARY KEY (id)
);
--
-- role developer (does not have any role specific info)
--
CREATE TABLE developer
(
-- Inherited from table people: id integer NOT NULL DEFAULT nextval('people_id_seq'::regclass),
-- Inherited from table people: first_name character varying(25) NOT NULL,
-- Inherited from table people: last_name character varying(25) NOT NULL,
-- Inherited from table people: email character varying(50) NOT NULL,
-- Inherited from table people: "password" character varying(150),
CONSTRAINT developer_pkey PRIMARY KEY (id)
)
INHERITS (people);
--
-- role user
--
CREATE TABLE installer
(
-- Inherited from table people: id integer NOT NULL DEFAULT nextval('people_id_seq'::regclass),
-- Inherited from table people: first_name character varying(25) NOT NULL,
-- Inherited from table people: last_name character varying(25) NOT NULL,
-- Inherited from table people: email character varying(50) NOT NULL,
client character varying(50),
-- Inherited from table people: "password" character varying(150),
CONSTRAINT installer_pkey PRIMARY KEY (id)
)
INHERITS (people);
Solution 1 -> 2 queries
It's quite simple to find out role from base table people and then select directly from role's table:
-- returns name of table (and role) 'developer'
SELECT pg.relname
FROM people p, pg_class pg
WHERE pg.oid=p.tableoid and p.email = 'john.doe#example.com';
-- getting roles full info
SELECT *
FROM developer
WHERE email = 'kracmar#dannax.sk';
This solution is fine, but I was searching for nicer solution.
Solution 2 -> 1 query using procedure
It would be nice to get info about user in just single query. I went deep into docs of functions and dig something out, but couldn't reach finish. I thought that using return query would be way, but my problem is that I need to specify type of result to function, but it can change depending on user's role (different table with number of columns and types).
This is one of results, function returns record but it's not query, single column with all fields in it separated with comma.
CREATE OR REPLACE FUNCTION get_person_by_email(person_email VARCHAR)
RETURNS record
LANGUAGE plpgsql
STABLE STRICT AS
$BODY$
DECLARE
role varchar;
result record;
BEGIN
SELECT pg.relname
INTO role
FROM people p,
pg_class pg
WHERE pg.oid=p.tableoid
AND p.email = person_email;
IF NOT FOUND THEN
RAISE exception 'Person with email % does not exists.', person_email;
END IF;
CASE
WHEN role = 'developer' THEN
SELECT *
INTO result
FROM developer
WHERE email = person_email;
WHEN ROLE = 'installer' THEN
SELECT *
INTO result
FROM installer
WHERE email = person_email;
END CASE;
RETURN result;
END;
$BODY$;
Selecting from this function is no go as there is missing columns definition. Maybe I complicate things and should use solution 1, but that way I won't learn anything. Any help will be appreciated.
I suppose you might be interested in zyxist's Doctrine2 fork:
Fork of Doctrine 2 Object Relational Mapper aiming to create real table inheritance support for PostgreSQL
You can read more on author's blog.