Using Qt Creator 4.1.0 based on Qt 5.6.
I have a postgresql table that looks like this:
CREATE TABLE shelf (
bookid sequence,
title character varying(200),
author character varying(200),
publisher character varying(200),
isbn character varying(200),
genre character varying(200)
);
I have just done a successful insert
QSqlQuery que;
que.exec("insert into shelf(title) values('blah') returning bookid;");
The insert worked fine, how to I get returning bookid?
try:
while (que.next()) {
QString bookid = que.value(0).toString();
qDebug() << bookid;
}
?..
You have two ways to get the last insert id when using Qt and PostgreSQL:
Using lastInsertId(): This has the advantage of being portable (i.e. your code keeps working if you decide to change to another DBMS later on), but it might be unsafe if you had triggers in your database that fire upon insertion in your table (e.g. shelf) and insert other values in other tables (note that it is implemented by issuing the query SELECT lastval(); on PostgreSQL). When using this approach, your code should look something like this:
QSqlQuery query;
query.prepare("INSERT INTO some_table(col_name) VALUES(?);");
query.addBindValue("sth");
if(!query.exec()) //insert statement failed
qWarning() << "insert statement failed with error: "
<< query.lastError().databaseText();
int insertId = query.lastInsertId().toInt();
//do anything you want with the id
qDebug() << "id: " << insertId;
Using the RETURNING clause: This has the advantage of being safer in case you had the weird triggers mentioned above, but it is not portable (i.e. the insert statement won't execute if you ever decide to change to another DBMS because it is not standard SQL anymore). This way, your code should look something like this:
QSqlQuery query;
query.prepare("INSERT INTO some_table(col_name) VALUES(?) RETURNING id_col_name;");
query.addBindValue("sth");
if(!query.exec()) //insert statement failed
qWarning() << "insert statement failed with error: "
<< query.lastError().databaseText();
if(!query.next()) //returning clause failed
qWarning() << "returning clause did not return any data";
int insertId = query.value(0).toInt();
//do anything you want with the id
qDebug() << "id: " << insertId;
Related
We are benchmarking some queries to see if they will still work reliably for "a lot of" data. (1 million isn't that much to be honest, but Postgres already fails here, so it evidently is.)
Our Java code to call this queries looks something like that:
#PersistenceContext
private EntityManager em;
#Resource
private UserTransaction utx;
for (int i = 0; i < 20; i++) {
this.utx.begin();
for (int inserts = 0; inserts < 50_000; inserts ++) {
em.createNativeQuery(SQL_INSERT).executeUpdate();
}
this.utx.commit();
for (int parameter = 0; parameter < 25; parameter ++)
long time = System.currentTimeMillis();
Assert.assertNotNull(this.em.createNativeQuery(SQL_SELECT).getResultList());
System.out.println(i + " iterations \t" + parameter + "\t" + (System.currentTimeMillis() - time) + "ms");
}
}
Or with plain JDBC:
Connection connection = //...
for (int i = 0; i < 20; i++) {
for (int inserts = 0; inserts < 50_000; inserts ++) {
try (Statement statement = connection.createStatement();) {
statement.execute(SQL_INSERT);
}
}
for (int parameter = 0; parameter < 25; parameter ++)
long time = System.currentTimeMillis();
try (Statement statement = connection.createStatement();) {
statement.execute(SQL_SELECT);
}
System.out.println(i + " iterations \t" + parameter + "\t" + (System.currentTimeMillis() - time) + "ms");
}
}
The queries we tried were a simple INSERT into a table with JSON and a INSERT over two tables with about 25 lines. The SELECT has one or two JOINs and is pretty easy. One set of queries is (I had to anonymize the SQL else I wouldn't have been allowed to post it):
CREATE TABLE ts1.p (
id integer NOT NULL,
CONSTRAINT p_pkey PRIMARY KEY ("id")
);
CREATE TABLE ts1.m(
pId integer NOT NULL,
mId character varying(100) NOT NULL,
a1 character varying(50),
a2 character varying(50),
CONSTRAINT m_pkey PRIMARY KEY (pI, mId)
);
CREATE SEQUENCE ts1.seq_p;
/*
* SQL_INSERT
*/
WITH p AS (
INSERT INTO ts1.p (id)
VALUES (nextval('ts1.seq_p'))
RETURNING id AS pId
)
INSERT INTO ts1.m(pId, mId, a1, a2)
VALUES ((SELECT pId from p), 'M1', '11', '12'),
((SELECT pId from p), 'M2', '13', '14'),
/* ... about 20 to 25 rows of values */
/*
* SQL_SELECT
*/
WITH userInput (mId, a1, a2) AS (
VALUES
('M1', '11', '11'),
('M2', '12', '15'),
/* ... about "parameter" rows of values */
)
SELECT m.pId, COUNT(m.a1) AS matches
FROM userInput u
LEFT JOIN ts1.m m ON (m.mId) = (u.mId)
WHERE (m.a1 IS NOT DISTINCT FROM u.a1) AND
(m.a2 IS NOT DISTINCT FROM u.a2) OR
(m.a1 IS NULL AND m.a2 IS NULL)
GROUP BY m.pId
/* plus HAVING, additional WHERE clauses etc. according to the use case, but that just speeds up the query */
When executing, we get the following output (the values are supposed to rise steadly and linearly):
271ms
414ms
602ms
820ms
995ms
1192ms
1396ms
1594ms
1808ms
1959ms
110ms
33ms
14ms
10ms
11ms
10ms
21ms
8ms
13ms
10ms
As you can see, after some value (usually at around 300,000 to 500,000 inserts) the time needed for the query drops significantly. Sadly we can't really debug what the result is at that point (other than that it's not null), but we assume it's an empty list, because the database tables are empty.
Let me repeat that: After half a million INSERTS, Postgres clears tables.
Of course that's not acceptable at all.
We tried different queries, all of easy to medium difficulty, and all produced this behavior, so we assume it's not the queries.
We thought that maybe the sequence returned a value too high for a column integer, so we droped and recreated the sequence.
Once there was this exception:
org.postgresql.util.PSQLException : FEHLER: Verklemmung (Deadlock) entdeckt
Detail: Prozess 1620 wartet auf AccessExclusiveLock-Sperre auf Relation 2001098 der Datenbank 1937678; blockiert von Prozess 2480.
Which I'm entirely unable to translate. I guess it's something like:
org.postgresql.util.PSQLException : ERROR: Jamming? Clamping? Constipation? (Deadlock) found
But I don't think this error has anything to do with the clearing of the table. We just tested against the wrong database, so multiple queries were run on the same table. Normally we have one database per benchmark test.
Of course it's important that we find out what the error is, so that we can decide if there is any risk to our customers losing their data (because again, on error the database empties some table of its choice).
Postgres version: PostgreSQL 10.6, compiled by Visual C++ build 1800, 64-bit
We tried PostgreSQL 9.6.11, compiled by Visual C++ build 1800, 64-bit, too. And we never had the same problem there (even though that could just be luck, since it's not 100% reproducible).
Do you have any idea what the error is? Or how we could debug it? The entire benchmark test runs for an hour, so there is no immediate feedback.
I have a query which contains two part. First part call function which creates a temporary table, second part select data from this table.
SELECT create_data_slice(15962, NULL, ARRAY[[15726]]);
SELECT
AK."15962_15726" as AK_NAME
FROM
t15962 AK
GROUP BY
AK."15962_15726;"
If I execute this query in PgAdmin, it turns right result with data. But if I execute it in Qt:
QSqlDatabase db = store->get_db();
QSqlQuery query(db);
query.exec(sql);
it executes only the first part (create temporary table), but do not execute second part and do not return data.
You can use a transaction like this:
QSqlDatabase::database().transaction();
QSqlQuery query;
query.exec("SELECT create_data_slice(15962, NULL, ARRAY[[15726]]);");
if (query.next())
{
int employeeId = query.value(0).toInt();
query.exec("SELECT AK."15962_15726" as AK_NAME FROM t15962 AK GROUP BY AK."15962_15726;");
while(query.next())
{
qDebug() << query.value().toString(); ///or what you want to do with data
}
}
QSqlDatabase::database().commit();
I searching for help. I have to map my Postgres 9.4 Database (DB) with Hibernate 5.2, of course it's an study task. The biggest Problem is, that I'm no brain in Hibernate, Java and coding itself XD
It's an SozialNetwork DB. To map the DB with Hibernate doing fine.
Now I should map a stored produce. This Produce should find the shortest friendship path between two persons. In Postgres the produce working fine.
That are the relevant DB-Tables:
For Person:
CREATE TABLE Person (
PID bigint NOT NULL,
firstName varchar(50) DEFAULT NULL,
lastName varchar(50) DEFAULT NULL,
(some more...)
PRIMARY KEY (PID)
);
And for the Relationship between to Persons:
CREATE TABLE Person_knows_Person (
ApID bigint NOT NULL,
BpID bigint REFERENCES Person (PID) (..)
knowsCreationDate timestamp,
PRIMARY KEY (ApID,BpID));
And that is the Stored Produce in short:
CREATE OR REPLACE FUNCTION ShortFriendshipPath(pid bigint, pid2 bigint)
RETURNS TABLE (a_pid bigint, b_pid bigint, depth integer, path2 bigint[], cycle2 boolean)
AS $$
BEGIN
RETURN QUERY
SELECT * FROM (
WITH RECURSIVE FriendshipPath(apid, bpid, depth, path, cycle) AS(
SELECT pkp.apid, pkp.bpid,1,
ARRAY[pkp.apid], false
FROM person_knows_person pkp
WHERE apid=$1 --OR bpid=$1
UNION ALL
SELECT pkp.apid, pkp.bpid, fp.depth+1, path || pkp.apid,
pkp.apid = ANY(path)
FROM person_knows_person pkp, FriendshipPath fp
WHERE pkp.apid = fp.bpid AND NOT cycle)
SELECT *
FROM FriendshipPath WHERE bpid=$2) AS OKOK
UNION
SELECT * FROM (
WITH RECURSIVE FriendshipPath(apid, bpid, depth, path, cycle) AS(
SELECT pkp.apid, pkp.bpid,1,
ARRAY[pkp.apid], false
FROM person_knows_person pkp
WHERE apid=$2 --OR bpid=$1
UNION ALL
SELECT pkp.apid, pkp.bpid, fp.depth+1, path || pkp.apid,
pkp.apid = ANY(path)
FROM person_knows_person pkp, FriendshipPath fp
WHERE pkp.apid = fp.bpid AND NOT cycle)
SELECT *
FROM FriendshipPath WHERE bpid=$1) AS YOLO
ORDER BY depth ASC LIMIT 1;
END;
$$ LANGUAGE 'plpgsql' ;
(Sorry for so much code, but it's for both directions, and before I post some copy+reduce misttakes^^)
The Call in Postgre for example:
SELECT * FROM ShortFriendshipPath(10995116277764, 94);
gives me this Output:
enter image description here
I use the internet for help and find 3 solutions for calling:
direct SQL call
call with NamedQuery and
map via XML
(fav found here)
I faild with all of them XD
I favorite the 1. solution with this call in session:
Session session = HibernateUtility.getSessionfactory().openSession();
Transaction tx = null;
try {
tx = session.beginTransaction();
System.out.println("Please insert a second PID:");
Scanner scanner = new Scanner(System.in);
long pid2 = Long.parseLong(scanner.nextLine());
// **Insert of second ID*/
Query query2 = session.createQuery("FROM " + Person.class.getName() + " WHERE pid = :pid ");
query2.setParameter("pid", pid2);
List<Person> listB = ((org.hibernate.Query) query2).list();
int cnt1 = 0;
while (cnt1 < listB.size()) {
Person pers1 = listB.get(cnt1++);
pid2 = pers1.getPid();
}
// Query call directly:
Query querySP = session.createSQLQuery("SELECT a_pid,path2 FROM ShortFriendshipPath(" + pid + "," + pid2 + ")");
List <Object[]> list = ((org.hibernate.Query) querySP).list();
for (int i=0; i<list.size();i++){
Personknowsperson friendship = (Personknowsperson)result.get(i);
}
} catch (Exception e) { (bla..)}
} finally { (bla....) }
Than I get following Error:
javax.persistence.PersistenceException:
org.hibernate.MappingException: No Dialect mapping for JDBC type: 2003
(..blabla...)
I understand why. Because my output is not of type Personknowsperson. I found an answer: that I have to say Hibernate what is the correct formate. And should use 'UserType'. So I try to find some explanations for how I create my UserType. But I found nothing, that I understand. Second Problem: I'm not sure what I should use for the bigint[] (path2). You see I'm expert -.-
Than I got the idea to try the 3.solution. But the first problem I had was where should I write the xml stuff. Because my Output is no table. So I try in the .cfg.xml but than Hibernate say that
Caused by: java.lang.IllegalArgumentException: org.hibernate.internal.util.config.ConfigurationException: Unable to perform unmarshalling at line number -1 and column -1 in RESOURCE hibernate.cfg.xml. Message: cvc-complex-type.2.4.a: Ungültiger Content wurde beginnend mit Element 'sql-query' gefunden. '{some links}' wird erwartet.
translation:
invalid content found starts with 'sql-query'
Now I'm a nervous wreck. And ask you.
Could someone explain what I have to do and what I did wrong (for dummies please). If more code need (java classes or something else) please tell me. Critic for coding also welcome, cause I want improve =)
Ok, I'm not an expert in postgressql, not hibernate, nor java. (I'm working with C#, SQL Server, NHibernate so ...) I still try to give you some hints.
You probably can set the types of the columns using addXyz methods:
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(...)")
.addScalar("a_pid", LongType.INSTANCE)
...
// add user type?
You need to create a user type for the array. I don't know how and if you can add it to the query. See this answer here.
You can also add the whole entity:
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(...)")
.addEntity(Personknowsperson.class)
...;
I hope it takes the mapping definition of the corresponding mapping file, where you can specify the user type.
Usually it's much easier to get a flat list of values, I mean a separate row for each different value in the array. Like this:
Instead of
1 | 2 | (3, 4, 5) | false
You would get:
1 | 2 | 3 | false
1 | 2 | 4 | false
1 | 2 | 5 | false
Which seems denormalized, but is actually the way how you build relational data.
In general: use parameters when passing stuff like ids to queries.
Query querySP = session
.createSQLQuery("SELECT * FROM ShortFriendshipPath(:pid1, :pid2)")
.setParameter("pid1", pid1)
.setParameter("pid2", pid2)
...
DROP TABLE IF EXISTS Pose ;
results in the error
Error code -1, SQL state 42X01: Syntax error: Encountered "EXISTS" at line 1, column 15.
I'm running this from inside NetBeans 7.3 using the default Derby sample db.
Derby does not currently support IF EXISTS
Are you trying to create a table? If yes, this is what you should do:
public void createTables() throws SQLException {
Statement statement = getConnection().createStatement();
System.out.println("Checking database for table");
DatabaseMetaData databaseMetadata = getConnection().getMetaData();
ResultSet resultSet = databaseMetadata.getTables(null, null, "PATIENT", null);
if (resultSet.next()) {
System.out.println("TABLE ALREADY EXISTS");
} else {
//language=MySQL
statement.execute("CREATE TABLE Patient (" +
"CardNumber CHAR(10) NOT NULL PRIMARY KEY, " +
" FirstName CHAR(50)," +
" MiddleName CHAR(50)," +
" LastName CHAR(50) )");
}
}
Remember to use all caps for the table name you pass into databaseMetadata.getTables(...)
The MySQL 6.0 syntax for declaring a table is this:
CREATE TABLE [IF NOT EXISTS] tableName ...
and the MySQL syntax for removing a table is this:
DROP TABLE [IF EXISTS] tableName ...
These clauses are MySQL extensions which are not part of the ANSI/ISO SQL Standard. This functionality may be peculiar to MySQL: I can't find anything similar documented for Derby, Postgres, Oracle, or DB2.
The best alternative I can find is to query the system tables to see if the table exists.
select count(*) from sys.systables where tablename = 'YOUR_TABLE_NAME'"
I had a similar issue dropping stored procedures. They can be queried using this statement.
select count(*) from sys.sysaliases where alias = 'YOUR_STORED_PROCEDURE_NAME'
If someone is looking to drop and create a table in an sql file that is Run with Spring test framework, Check https://stackoverflow.com/a/47459214/3584693 for an answer that ensures that no exception is thrown when drop table is invoked when said table doesn't exist.
I'm trying to do an insert or update in a postgres database using node.js with pg extension (version 0.5.4).
So far I have this code:
(...)
client.query({
text: "update users set is_active = 0, ip = $1 where id=$2",
values: [ip,id]
}, function(u_err, u_result){
debug(socket_id,"update query result: ",u_result);
debug(socket_id,"update query error: ",u_err);
date_now = new Date();
var month = date_now.getMonth() + 1;
if(!u_err){
client.query({
text: 'insert into users (id,first_name,last_name,is_active,ip,date_joined) values' +
'($1,$2,$3,$4,$5,$6)',
values: [
result.id,
result.first_name,
result.last_name,
1,
ip,
date_now.getFullYear() + "-" + month + "-" + date_now.getDate() + " " + date_now.getHours() + ":" + date_now.getMinutes() + ":" + date_now.getSeconds()
]
}, function(i_err, i_result){
debug(socket_id,"insert query result: ",i_result);
debug(socket_id,"insert query error: ",i_err);
});
}
});
The problem is that, although both queries work the problem is always running both instead of only running the insert function if the update fails.
The debug functions in code output something like:
UPDATE
Object { type="update query result: ", debug_value={...}}
home (linha 56)
Object { type="update query error: ", debug_value=null}
home (linha 56)
Object { type="insert query result: "}
home (linha 56)
Object { type="insert query error: ", debug_value={...}}
Insert
Object { type="update query result: ", debug_value={...}}
home (linha 56)
Object { type="update query error: ", debug_value=null}
home (linha 56)
Object { type="insert query result: ", debug_value={...}}
home (linha 56)
Object { type="insert query error: ", debug_value=null}
** EDIT **
ANSWER FROM node-postgres developer:
It's possible to retrieve number of rows affected by an insert and
update. It's not fully implemented in the native bindings, but does
work in the pure javascript version. I'll work on this within the
next week or two. In the mean time use pure javascript version and
have a look here:
https://github.com/brianc/node-postgres/blob/master/test/integration/client/result-metadata-tests.js
** END EDIT **
Can anyone help?
The immediate answer to your question is to use a stored procedure to do an upsert.
http://www.postgresql.org/docs/current/static/plpgsql-control-structures.html#PLPGSQL-UPSERT-EXAMPLE
Something like this works fine with the pg module.
client.query({
text: "SELECT upsert($1, $2, $3, $4, $5, $6)"
values: [ obj.id,
obj.first_name,
obj.last_name,
1,
ip,
date_now.getFullYear() + "-" + month + "-" + date_now.getDate() + " " + date_now.getHours() + ":" + date_now.getMinutes() + ":" + date_now.getSeconds()
]
}, function(u_err, u_result){
if(err) // this is a real error, handle it
// otherwise your data is updated or inserted properly
});
Of course this assumes that you're using some kind of model object that has all the values you need, even if they aren't changing. You have to pass them all into the upsert. If you're stuck doing it the way you've shown here, you should probably check the actual error object after the update to determine if it failed because the row is already there, or for some other reason (which is real db error that needs to be handled).
Then you've gotta deal with the potential race condition between the time your update failed and the time your insert goes through. If some other function tries to insert with the same id, you've got a problem. Transactions are good for that. That's all I got right now. Hope it helps.
I had this issue when connecting to a PG instance using the JDBC. The solution I ended up using was:
UPDATE table SET field='C', field2='Z' WHERE id=3;
INSERT INTO table (id, field, field2)
SELECT 3, 'C', 'Z'
WHERE NOT EXISTS (SELECT 1 FROM table WHERE id=3);
The update does nothing if the record doesn't exist and the insert does nothing if the record does exist. It works pretty well and is an SQL based solution vs a stored procedure.
Here's the initial question:
Insert, on duplicate update in PostgreSQL?
I have an electronic component database to which I add components that I either salvage from e-waste or buy as new, and the way I did it was:
const upsertData = (request, response) => {
const {
category, type, value, unit, qty,
} = request.body;
pool.query(`DO $$
BEGIN
IF EXISTS
( SELECT 1
FROM elab
WHERE category='${category}'
AND type='${type}'
AND value='${value}'
AND unit='${unit}'
)
THEN
UPDATE elab
SET qty = qty + ${qty}
WHERE category='${category}'
AND type='${type}'
AND value='${value}'
AND unit='${unit}';
ELSE
INSERT INTO elab
(category, type, value, unit, qty)
values ('${category}', '${type}', '${value}', '${unit}', ${qty});
END IF ;
END
$$ ;`, (error, results) => {
if (error) {
throw error;
}
response.status(201).send('Task completed lol');
});
};
The reason for this was that the only unique column any entry had was the ID, which is automatically updated, none of the other columns are unique only the whole entry is e.g. you can have a 100 kOhm resistor as a potentiometer or a "normal" one - and you can have a potentiometer with different values than 100 kOhm so only the whole entry is unique.