I have a table called Person with three fields - ID, city and name. City can be null, so I have two partial unique indexes - one on ID and city where city is not null, and one on ID where city is null.
Now I want to have an upsert statement using SQLAlchemy, with those partial indexes, but I can't figure out the syntax. Currently I have:
table = models.Person.__table__
insert_statement = insert(table, upsert_values)
update_dict = {c.name: c for c in insert_statement.excluded if c.name != "id"}
upsert_statement = insert_statement.on_conflict_do_update(
index_elements=[table.c['id'], table.c['city']],
set_=update_dict
)
but when I try to execute I get
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidColumnReference) there is no unique or exclusion constraint matching the ON CONFLICT specification
How can this work? Is there a way other than running pure SQL?
Related
Our system uses postgres for its database.
We have queries that can select rows from a database table where an array field in the table contains a specific value, e.g.:
Find which employee manages the employee with ID 123.
staff_managed_ids is a postgres array field containing an array of the employees that THIS employee manages.
This query works as expected:
select *
from employees
where 123=any(staff_managed_ids)
We now need to query where an array field contains a postgres NULL. We tried the following query, but it doesn't work:
select *
from employees
where NULL=any(staff_managed_ids)
We know the staff_managed_ids array field contains NULLs from other queries.
Are we using NULL wrongly?
NULL can not be compared using =. The only operators that work with that are IS NULL and IS NOT NULL.
To check for nulls, you need to unnest the elements:
select e.*
from employees e
where exists (select *
from unnest(e.staff_managed_ids) as x(staff_id)
where x.staff_id is null);
if all your id values are positive, you could write something like this:
select *
from employees
where (-1 < all(staff_managed_ids)) is null;
how this works is that -1 should be less than all values, however comparison with null will make the whole array comparison expression null.
Suppose I have the following table structure:
create table PEOPLE (
ID integer not null primary key,
NAME varchar(100) not null
);
create table CHILDREN (
ID integer not null primary key,
PARENT_ID_1 integer not null references PERSON (id),
PARENT_ID_2 integer not null references PERSON (id)
);
and that I want to generate a list of the names of each person who is a parent. In slick I can write something like:
for {
parent <- people
child <- children if {
parent.id === child.parent_id_1 ||
parent.id === child.parent_id_2
}
} yield {
parent.name
}
and this generates the expected SQL:
select p.name
from people p, children c
where p.id = c.parent_id_1 or p.id = c.parent_id_2
However, this is not optimal: the OR part of the expression can cause horrendously slow performance in some DBMSes, which end up doing full-table scans to join on p.id even though there's an index there (see for example this bug report for H2). The general problem is that the query planner can't know if it's faster to execute each side of the OR separately and join the results back together, or simply do a full table scan [2].
I'd like to generate SQL that looks something like this, which then can use the (primary key) index as expected:
select p.name
from people p, children c
where p.id in (c.parent_id_1, c.parent_id_2)
My question is: how can I do this in slick? The existing methods don't seem to offer a way:
ColumnExtensionMethods.in takes Query as a parameter, but I don't have a Query I have a number or Rep[Long] for each of my ID columns
ColumnExtensionMethods.inSet is for binding existing (known) literal arrays, not for joining to sets of columns
What I'd like to be able to write is something like this:
for {
parent <- people
child <- children
if parent.id in (child.parent_id_1, child.parent_id_2)
} yield {
p.name
}
but that's not possible right now.
[1] My actual design is a little more complex than this, but it boils down to the same problem.
[2] Some DBMSes do have this optimisation for simple cases, e.g. OR-expansion in Oracle.
Turns out this isn't currently (as at slick 3.2.3) possible, so I've raised an issue on github and submitted a pull request to add this functionality.
Here is what I have so far:
INSERT INTO Tenants (LeaseStartDate, LeaseExpirationDate, Rent, LeaseTenantSSN, RentOverdue)
SELECT CURRENT_DATE, NULL, NewRentPayments.Rent, NewRentPayments.LeaseTenantSSN, FALSE from NewRentPayments
WHERE NOT EXISTS (SELECT * FROM Tenants, NewRentPayments WHERE NewRentPayments.HouseID = Tenants.HouseID AND
NewRentPayments.ApartmentNumber = Tenants.ApartmentNumber)
So, HouseID and ApartmentNumber together make up the primary key. If there is a tuple in table B (NewRentPayments) that doesn't exist in table A (Tenants) based on the primary key, then it needs to be inserted into Tenants.
The problem is, when I run my query, it doesn't insert anything (I know for a fact there should be 1 tuple inserted). I'm at a loss, because it looks like it should work.
Thanks.
Your subquery was not correlated - It was just a non-correlated join query.
As per description of your problem, you don't need this join.
Try this:
insert into Tenants (LeaseStartDate, LeaseExpirationDate, Rent, LeaseTenantSSN, RentOverdue)
select current_date, null, p.Rent, p.LeaseTenantSSN, FALSE
from NewRentPayments p
where not exists (
select *
from Tenants t
where p.HouseID = t.HouseID
and p.ApartmentNumber = t.ApartmentNumber
)
update Claim
set first_name = random_name(7),
Last_name = random_name(6),
t2.first_name=random_name(7),
t2.last_name=random_name(6)
from Claim t1
inner join tbl_ecpremit t2
on t1.first_name = t2.first_name
I am getting below error
column "t2" of relation "claim" does not exist
You can do this with a so-called data-modifying CTE:
WITH c AS (
UPDATE claim SET first_name = random_name(7), last_name = random_name(6)
WHERE <put your condition here>
RETURNING *
)
UPDATE tbl_ecpremit SET last_name = c.last_name
FROM c
WHERE first_name = c.first_name;
This assumes that random_name() is a function you define, it is not part of PostgreSQL afaik.
The nifty trick here is that the UPDATE in the WITH query returns the updated record in the first table using the RETURNING clause. You can then use that record in the main UPDATE statement to have exactly the same data in the second table.
This is all very precarious though, because you are both linking on and modifying the "first_name" column with some random value. In real life this will only work well if you have some more logic regarding the names and conditions.
I have recently started using Entity Framework and have run into a problem.
I have 2 simple tables mapped with Entity Framework in my solution:
Employees:
emp_id INT
first_name VARCHAR
last_name VARCHAR
department INT ( FOREIGN KEY MAPPED TO departments.dept_id )
and
Departments:
dept_id INT
department_name VARCHAR
Using the code below, I want to write to the database.
var record = db.employees.Create();
string test = "test";
record.first_name = test;
record.last_name = test;
record.department = 1;
db.employees.Add(record);
db.SaveChanges();
I get an error the error:
Entities in "'DBContextContainer.employees' participate in the 'employeedepartment' relationship. 0 related 'department' were found. 1 'department' is expected."
at the db.SaveChanges() method. Can someone please explain to me how I could resolve or troubleshoot this?
Update: There is a record in the departments table with a dept_id of 1 and I am still getting the error.
You'll need to add a field to the Departments table first since Departments is the parent table (Employees depend on Departments as per your table structure). You cant add an employee with department that doesn't have a corresponding entry in the Departments table.