Laravel 5.3 Eloquent transactions and foreign key restrictions - postgresql

Am working on bigger project where we have multiple schemas in one Postgres DB. We have created foreign keys between schemas. Here is an example >
We have company schema and user schema. Company schema has company_users table which have foreign key restriction on user.users table
CREATE TABLE company.company_user
(
id serial NOT NULL,
company_id integer NOT NULL,
user_id integer NOT NULL,
created_at timestamp(0) without time zone,
updated_at timestamp(0) without time zone,
deleted_at timestamp(0) without time zone,
CONSTRAINT company_user_pkey PRIMARY KEY (id),
CONSTRAINT company_user_company_id_foreign FOREIGN KEY (company_id)
REFERENCES company.companies (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION,
CONSTRAINT company_user_user_id_foreign FOREIGN KEY (user_id)
REFERENCES "user".users (id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
Following queries run in Postgres without issue
BEGIN;
insert into "db"."user"."users" (id,"surname", "firstname", "email", "position", "language_id", "is_super_admin", "updated_at", "created_at") values (156,'Mueller', 'Julianne', 'julianne.mueller1#example.org', 'Nuclear Power Reactor Operator', 41, false, '2017-01-13 12:35:10', '2017-01-13 12:35:10') returning "id";
insert into "db"."company"."company_user" ("company_id", "user_id", "updated_at", "created_at") values (4445, 156, '2017-01-13 12:35:10', '2017-01-13 12:35:10') returning "id";
COMMIT;
However if i perform same queries via Eloquent in Laravel
\DB::beginTransaction();
$user = new User(["surname" => 'Mueller',
"firstname" => 'Julianne',
"email" => 'julianne.mueller1#example.org',
"position" => 'Nuclear Power Reactor Operator',
"language_id" => 41,
"is_super_admin" => false]
);
if (!$user->save()) {
\DB::rollBack();
return false;
}
\Log::error($user->id);
$company_user = new CompanyUser([
"company_id" => 4445,
"user_id" => $user->id
]);
if (!$company_user->save()) {
\DB::rollBack();
return false;
}
\DB::commit();
is throwing folloing error (it seems that it cannot find id of user in the table)
PDOException: SQLSTATE[23503]: Foreign key violation: 7 ERROR: insert or update on table "company_user" violates foreign key constraint "company_user_user_id_foreign"
Would anyone can say why this is not working? \Log::error($user->id) is printing id of inserted user. I tried to print out queries from Laravel with DB listener, all queries are executed in correct order, but am still getting this error.

Ok so we found a solution. It seems that we need to start transaction for each of schemas separately + each foreign key that are referencing different schema than their own should be created as deferred.

Make sure CompanyUser has $fillable:
$fillable = ['user_id', 'company_id'];
Also, make sure there is a user with this ID is already in users table. Maybe you'll need to get rid of transaction.

Related

SQLAlchemy + asyncpg doesn't catch deferred exception

I am now using SQLAlchemy 1.4, asyncpg and FastAPI, and I wrote the code like this:
try:
cr: sa.engine.CursorResult = await conn.execute(stmt)
return schemas.UserGroup(
**user_group_dict,
id=cr.inserted_primary_key[0],
)
except sa.exc.IntegrityError:
raise exceptions.conflict_exception()
UserGroup table references User and Group tables and has a unique key constraint (user, group)
user_groups = sa.Table(
"auth_user_groups",
metadata,
sa.Column(
"id",
sa.BigInteger,
primary_key=True,
index=True,
),
sa.Column(
"user_id",
sa.BigInteger,
sa.ForeignKey("auth_user.id"),
),
sa.Column(
"group_id",
sa.BigInteger,
sa.ForeignKey("auth_group.id"),
),
sa.UniqueConstraint("user_id", "group_id"),
)
CREATE TABLE IF NOT EXISTS public.auth_user_groups
(
id bigint NOT NULL DEFAULT nextval('auth_user_groups_id_seq'::regclass),
user_id integer NOT NULL,
group_id integer NOT NULL,
CONSTRAINT auth_user_groups_pkey PRIMARY KEY (id),
CONSTRAINT auth_user_groups_user_id_group_id_94350c0c_uniq UNIQUE (user_id, group_id),
CONSTRAINT auth_user_groups_group_id_97559544_fk_auth_group_id FOREIGN KEY (group_id)
REFERENCES public.auth_group (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
DEFERRABLE INITIALLY DEFERRED,
CONSTRAINT auth_user_groups_user_id_6a12ed8b_fk_auth_user_id FOREIGN KEY (user_id)
REFERENCES public.auth_user (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
DEFERRABLE INITIALLY DEFERRED
)
When I try to insert a duplicated record, it works fine.
However, when I try to insert a record with fk which does not exist in User and Group, I cannot catch the exception.
2022-06-14 01:02:50,978 INFO sqlalchemy.engine.Engine COMMIT
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/test/projects/fastapi-from-dj/venv/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 366, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/home/test/projects/fastapi-from-dj/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
...
File "/home/test/projects/fastapi-from-dj/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 685, in do_commit
dbapi_connection.commit()
File "/home/test/projects/fastapi-from-dj/venv/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 741, in commit
self._handle_exception(error)
File "/home/test/projects/fastapi-from-dj/venv/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 682, in _handle_exception
raise translated_error from error
sqlalchemy.exc.IntegrityError: (sqlalchemy.dialects.postgresql.asyncpg.IntegrityError) <class 'asyncpg.exceptions.ForeignKeyViolationError'>: insert or update on table "auth_user_groups" violates foreign key constraint "auth_user_groups_user_id_6a12ed8b_fk_auth_user_id"
DETAIL: Key (user_id)=(15) is not present in table "auth_user".
(Background on this error at: https://sqlalche.me/e/14/gkpj)
Of course, I tried to catch exception with asyncpg.exceptions.ForeignKeyViolationError and Exception as e, but I failed.
Thank you.
Thanks to #Gord Thompson's comment
I was able to solve my problem by altering the table FK constraints:
ALTER TABLE IF EXISTS public.auth_user_groups
DROP CONSTRAINT auth_user_groups_user_id_6a12ed8b_fk_auth_user_id;
ALTER TABLE IF EXISTS public.auth_user_groups
ADD CONSTRAINT auth_user_groups_user_id_6a12ed8b_fk_auth_user_id FOREIGN KEY (user_id)
REFERENCES public.auth_user (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
NOT DEFERRABLE;
As a result, now I have a table like this:
CREATE TABLE IF NOT EXISTS public.auth_user_groups
(
id bigint NOT NULL DEFAULT nextval('auth_user_groups_id_seq'::regclass),
user_id integer NOT NULL,
group_id integer NOT NULL,
CONSTRAINT auth_user_groups_pkey PRIMARY KEY (id),
CONSTRAINT auth_user_groups_user_id_group_id_94350c0c_uniq UNIQUE (user_id, group_id),
CONSTRAINT auth_user_groups_group_id_97559544_fk_auth_group_id FOREIGN KEY (group_id)
REFERENCES public.auth_group (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION,
CONSTRAINT auth_user_groups_user_id_6a12ed8b_fk_auth_user_id FOREIGN KEY (user_id)
REFERENCES public.auth_user (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
)
It does not have the line: DEFERRABLE INITIALLY DEFERRED
I am now using the tables which were migrated by Django so that FastAPI uses them as a REST API.
I believe that the following answer link will be useful for Django users:
How can I set a table constraint "deferrable initially deferred" in django model?
Thank you.

F# SqlProvider: System.Exception: Error - you cannot update an entity that does not have a primary key. (public.users)

I'm using the SQLProvider type-provider project: https://fsprojects.github.io/SQLProvider/
As the documentation states on CRUD usage, I'm trying to update a row like this:
let UpdateGpsLocation(gpsLocationDetails: UpdateGpsLocation) =
let ctx = SQL.GetDataContext()
let maybeFoundUser =
query {
for user in ctx.Public.Users do
where(user.UserId = gpsLocationDetails.UserId)
select(Some user)
exactlyOneOrDefault
}
match maybeFoundUser with
| Some user ->
user.GpsLatitude <- Some gpsLocationDetails.Latitude
user.GpsLongitude <- Some gpsLocationDetails.Longitude
ctx.SubmitUpdates()
| None -> failwithf "User %i not found" gpsLocationDetails.UserId
But it is failing with this:
System.Exception: Error - you cannot update an entity that does not have a primary key. (public.users)
at FSharp.Data.Sql.Providers.PostgresqlProvider.createUpdateCommand(IDbConnection con, StringBuilder sb, SqlEntity entity, FSharpList`1 changedColumns)
at <StartupCode$FSharp-Data-SqlProvider>.$Providers.Postgresql.FSharp-Data-Sql-Common-ISqlProvider-ProcessUpdates#1205-8.Invoke(SqlEntity e)
at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in D:\a\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 497
at FSharp.Data.Sql.Providers.PostgresqlProvider.FSharp-Data-Sql-Common-ISqlProvider-ProcessUpdates(IDbConnection con, ConcurrentDictionary`2 entities, TransactionOptions transactionOptions, FSharpOption`1 timeout)
at <StartupCode$FSharp-Data-SqlProvider>.$SqlRuntime.DataContext.FSharp-Data-Sql-Common-ISqlDataContext-SubmitPendingChanges#110.Invoke(Unit unitVar0)
at FSharp.Data.Sql.Runtime.SqlDataContext.FSharp-Data-Sql-Common-ISqlDataContext-SubmitPendingChanges()
at BackendDataLayer.Access.UpdateGpsLocation(UpdateGpsLocation gpsLocationDetails) in C:\Users\knocte\Documents\Code\RunIntoMeMASTER\src\BackendDataLayer\Access.fs:line 34
But the table has a primary key! In particular, it's using the SERIAL feature of PostgreSQL, this way:
CREATE TABLE users (
user_id SERIAL PRIMARY KEY
, gps_latitude FLOAT8
, gps_longitude FLOAT8
);
Is this a bug in SqlProvider project? Or maybe just in its PostgreSQL component?
I was able to reproduce this problem, there seems to be an issue with using user_id as the column name here.
This is just a workaround but changing the column name to "UserId" (double-quotes included!) fixed the issue for me.
See the new schema:
CREATE TABLE "Users" (
"UserId" SERIAL PRIMARY KEY
, "GpsLatitude" FLOAT8
, "GpsLongitude" FLOAT8
);
(If you don't use the double-quotes, it will convert UserId into Userid and your F# code won't compile.)

postgresql transactions violates foreign key constraint

I have the following nodejs code (simplified):
// tables
CREATE TABLE admins (
admin_id bigint DEFAULT nextval('admins_seq') NOT NULL,
username character varying(25) NOT NULL,
password character varying(150) NOT NULL
);
CREATE TABLE admin_notifications (
admin_notification_id bigint DEFAULT nextval('admin_notifications_seq') NOT NULL,
admin_id bigint NOT NULL,
type character varying(150) NOT NULL
);
ALTER TABLE admin_notifications ADD CONSTRAINT admin_notifications_to_admins_fk FOREIGN KEY (admin_id) REFERENCES admins(admin_id) ON DELETE CASCADE;
await client.query('BEGIN');
// create a new array in the database and return the newly created admin id
const query = 'INSERT INTO admins (username, password) VALUES($1,$2) RETURNING admin_id';
const values = [];
const result = await client.query(query, values);
// insert a welcome notification for the newly created admin
const query2 = 'INSERT INTO admin_notifications (admin_id, type) VALUES ($1,$2)';
const values2 = [result.rows[0].admin_id, 'welcome'];
const result2 = await client.query(query2, values2);
await client.query('COMMIT');
A little explanation: i create a new admin in the table admins. When a new admin is created, I want to insert a new welcome notification in the table admin_notifications. The column admin_id in the table admin_notifications is a foreign key to the table admins (admin_id).
I get the following error:
Error: insert or update on table "admin_notifications" violates foreign key constraint "admin_notifications_to_admins_fk"
When I leave out the transactions it works?!
Does it has something to do with deferring constraints? And how to handle this in my code?

Knex : Altering Primary Key Id column Error

The database is built on Knex / PostgreSQL.
I would like to alter a table to add 'unique()' type to primary key Id column.
But Migration failed with an error message below.
alter table "users" alter column "id" drop not null - column "id" is
in a primary key
exports.up = knex =>
knex.schema
.raw('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"')
.createTable('users', table => {
table
.uuid('id')
.primary()
.notNullable()
.defaultTo(knex.raw('uuid_generate_v4()'));
table
.string('companyName')
.unique()
.notNullable();
table
.string('username')
.unique()
.notNullable();
table.string('password').notNullable();
table.string('contactNo').notNullable();
table.string('email').unique();
table.string('address');
table
.boolean('isAdmin')
.notNullable()
.defaultTo(false);
table
.enum('businessType', ['catering', 'restaurant'])
.defaultTo('catering');
table.integer('lunchQty').defaultTo(null);
table.integer('dinnerQty').defaultTo(null);
table
.uuid('bankAccountId')
.references('id')
.inTable('bank_account')
.onDelete('SET NULL')
.onUpdate('RESTRICT')
.index();
table.string('resetPasswordToken');
table.timestamps(true, true);
});
exports.down = knex => knex.schema.dropTable('users');
exports.up = knex =>
knex.schema.alterTable('users', table => {
table
.uuid('id')
.unique()
.primary()
.notNullable()
.defaultTo(knex.raw('uuid_generate_v4()'))
.alter();
});
exports.down = knex =>
knex.schema.table('users', table => {
table.dropColumn('id').alter();
});
PostgreSQL version : 11.1
Knex version : 0.19.2
I have searched here and there but couldn't find an answer for this issue.
Thanks for taking your time to help me out !
------------------------------ EDITION -----------------------------------
Qustion ) when I created delivery table like below. The error below accurred. I thought this was caused because I didn't set primary key unique.
migration failed with error: alter table "delivery" add constraint
"delivery_userid_foreign" foreign key ("userId") references "users"
("id") on update RESTRICT on delete CASCADE - there is no unique
constraint matching given keys for referenced table "users"
exports.up = knex =>
knex.schema
.raw('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"')
.createTable('delivery', table => {
table
.uuid('id')
.primary()
.notNullable()
.defaultTo(knex.raw('uuid_generate_v4()'));
table
.uuid('routeId')
.references('id')
.inTable('routes')
.onDelete('CASCADE')
.onUpdate('RESTRICT')
.index();
table
.uuid('userId')
.references('id')
.inTable('users')
.onDelete('CASCADE')
.onUpdate('RESTRICT')
.index();
table.timestamps(true, true);
});
exports.down = knex => knex.schema.dropTable('delivery');
```
Primary keys are already unique and not null: there is no need for your alteration. See documentation. Knex is trying to do what you ask, but in order to alter the table it would have to drop id which is in a PRIMARY KEY constraint.
resolved the issue by removing alter users file !!
the 'unique key()' was the one that was causing the problem.

Foreign Key could not be found with SqlAlchemy and PostgreSQL

I just started a Flask - SqlAlchemy project and am having some trouble with Foreign Keys.
I have the tables User and Portfolio. Portfolio has a foreign key to user, using username. I set up my model like this.
class User(db.Model):
__tablename__ = 'portfolio_users'
__table_args__ = {"schema":"keldan"}
username = Column(String(), primary_key=True)
date_added = Column(DateTime())
class Portfolio(db.Model):
__tablename__ = 'portfolios'
__table_args__ = {"schema":"keldan"}
id = Column('pid', Integer(), Sequence('portfolios_pid_seq'), primary_key=True)
date_added = Column(DateTime())
name = Column(String())
username = Column(String(), ForeignKey('portfolio_users.username'))
user = relationship('User', backref=backref('portfolios', cascade='save-update, merge, delete, delete-orphan'))
The error I get when I try to run a simple select all query is:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'portfolios.username' could not find table 'portfolio_users' with which to generate a foreign key to target column 'username'
The tables are created like this:
CREATE TABLE keldan.portfolio_users
(
username text NOT NULL,
date_added date NOT NULL,
CONSTRAINT users_pk PRIMARY KEY (username)
)
WITH (
OIDS=FALSE
);
CREATE TABLE keldan.portfolios
(
pid serial NOT NULL,
username text NOT NULL,
date_added date NOT NULL,
name text NOT NULL,
CONSTRAINT portfolios_pk PRIMARY KEY (pid),
CONSTRAINT portfolios_fk FOREIGN KEY (username)
REFERENCES keldan.portfolio_users (username) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
WITH (
OIDS=FALSE
);
I have spent the better part of a day trying to figure this out or making workarounds using primaryjoin but nothing seems to work.
I finally found the answer I was looking for here
If you are not using the default schema (public) then it's not enough to specify the schema for each class, but I need to specify it in the foreign key as well.
username = Column(String(), ForeignKey('keldan.portfolio_users.username'))