Psycopg2 functions in parameters - postgresql

I'm wonder how I can pass a Postgres function to a field in an insert statement in Psycopg2. Looked everywhere in the doc but can't find the answer.
The code:
From the DBconn class:
def insert(self, table, data):
fields = data.keys()
sql = "INSERT INTO " + table + "("
sql += ",".join(fields) + ") VALUES (" + ",".join(["%s"]*len(fields)) + ");"
self.cur.execute(sql, data.values())
And when calling this function:
db.insert("tblpeople", {"person_name":"john", "org_id":"currval('tblorganisations_org_id_seq')"})
So the question is how can I tell Psycopg2 that the org_id value is a Postgres function?
Tnx!

Your code is producing the statement:
INSERT INTO tblpeople (org_id,person_name) VALUES (%s,%s);
and you are passing 2 arguments. That won't work. You want to produce this code:
INSERT INTO tblpeople (org_id,person_name)
VALUES (currval('tblorganisations_org_id_seq'),%s);
and then pass a single argument. I don't see any way of doing that with your current data structure. If you add another piece of data to your dictionary, like:
{
"text":{"person_name":"john"},
"function":{"org_id":"currval('tblorganisations_org_id_seq')"}
}
and change your constructor to produce the proper query based upon the 'function' key, or, you could 'hard code' your query builder to recognize the 'currval(' being at the beginning of the value. Anyway, the basic problem is you are passing the data as a string argument which won't be handled correctly.
-g

Related

Postgres: Python: TypeError: SQL.__init__() takes 2 positional arguments but 3 were given

Hello I am getting error from my code, can someone help me please?
def query_builder(self, field_name, table_name, pkey, id):
queryx=sql.SQL("select {field} from {table} where {pkey} = %s",(id)).format(
field=sql.Identifier(field_name),
table=sql.Identifier(table_name),
pkey=sql.Identifier(pkey))
self.cur.execute(queryx.as_string(self.conn))
I'm going to assume you are using psycopg2.
If so the issues are, first:
"select {field} from {table} where {pkey} = %s",(id) ..."
Do not include the argument (id) in the string. Also this is not proper form for a single value in a tuple. Python requires it be (id,), note the comma.
Second:
self.cur.execute(queryx.as_string(self.conn))
Should be:
self.cur.execute(queryx, (id,))
The execute is where you supply the argument. Also the composable sql.SQL(...) can be passed directly to execute without being run through as_string. See here sql for more examples.
UPDATE
To use "*" there are two ways:
cur.execute(sql.SQL("select * from {table} where {pkey} = %s).format(table.sql.Identifier(table_name), pkey=sql.Identifier(pkey))
--OR
cur.execute(sql.SQL("select {field} from {table} where {pkey} = %s).format(field=sql.SQL("*"), table=sql.Identifier(table_name), pkey=sql.Identifier(pkey))
Warning, the second does allow for SQL injection as sql.SQL() does not escape values.
As to multiple fields the sql section of the docs has multiple examples. For instance:
If part of your query is a variable sequence of arguments, such as a comma-separated list of field names, you can use the SQL.join() method to pass them to the query:
query = sql.SQL("select {fields} from {table}").format(
fields=sql.SQL(',').join([
sql.Identifier('field1'),
sql.Identifier('field2'),
sql.Identifier('field3'),
]),
table=sql.Identifier('some_table'))

Is there a way to make Delphi's FireDAC recognize PostgreSQL positional parameters that FireDAC generated?

I am executing queries with named parameters from FireDAC to PostgreSQL 11 using the native FireDAC Postgres driver. During the prepare statement FireDAC converts the named parameters to positional parameters, which is correct. However, if I then attempt to assign values to those parameters FireDAC throws an "Argument out of range" exception. It appears that FireDAC does not recognize the positional parameters that it generated. For example, if the original SQL text looked something like this:
SELECT * FROM account WHERE accountid = :accid;
upon calling the Prepare method of the FDQuery, FireDAC converted this query into this:
SELECT * FROM account WHERE accountid = $1;
But when I try to assign the value to the parameter, I get the error. The assignment looks something like this:
FDQuery1.Params[0].AsString = strID;
where strID is a string value and accountid is a text field. Furthermore, if I use something like the following, it returns 0.
ShowMessage( IntToStr( FDQuery1.Params.Count ) );
I've simplified this code significantly, but the issues are the same. How do I get FireDAC to recognize the positional parameters that it generated?
Update : As I mentioned, the above code is greatly simplified. What is actually happening is that in our framework we have one set of routines that assign values to FireDAC macros, and then we generate the SQL statement by preparing the query and then reading the FDQuery's Text property. That SQL statement then gets assigned to the SQL.Text property of another FDQuery (also dynamically created), and it is there that the query fails. So, here is a very simple example of what is happening internally in the code:
var
Query: TFDQuery;
begin
Query := TFDQuery.Create( nil );
Query.Connection := PGConnection;
// In reality, the SQL statement below was generated earlier,
// from a function call where the SQL was created by the FireDAC
// SQL preprocessor, as opposed to being a literal expression
Query.SQL.Text := 'SELECT * FROM tablename WHERE field2 = $1;';
Query.Params[0].AsString := '4'; // BANG! Argument out of range
I thought it might be due to FireDAC macro expansion, so I added the following two lines after instantiating the FDQuery:
Query.ResourceOptions.MacroCreate := False;
Query.ResourceOptions.MacroExpand := False;
Nope. That did help either. I am guessing that FireDAC simply doesn't recognize that $1 is a valid positional parameter in PostgreSQL.
You need to put the symbolic parameter identifier (in your case :accid) in the SQL.Text string, not the positional code ($1)
I just tested these two variations. The first works, the second doesn't.
var
MyQ : tfdquery;
begin
MyQ := Tfdquery.Create(nil);
MyQ.Connection := dm1.dbMain;
MyQ.SQL.Text := 'Select * from person where lastname = :lname;';
MyQ.Params[0].AsString := 'Brodzinsky';
MyQ.Open();
ShowMessage('Records found = '+MyQ.RecordCount.ToString);
MyQ.Close;
MyQ.Free;
end;
The next tries to use positional. Of course, FireDac doesn't see a colon, so doesn't know there is a parameter to create
var
MyQ : tfdquery;
begin
MyQ := Tfdquery.Create(nil);
MyQ.Connection := dm1.dbMain;
MyQ.SQL.Text := 'Select * from person where lastname = $1;';
MyQ.Params[0].AsString := 'Brodzinsky';
MyQ.Open();
ShowMessage('Records found = '+MyQ.RecordCount.ToString);
MyQ.Close;
MyQ.Free;
end;
In the first, 11 records in the person table are returned with my last name; in the second, the Argument Out Of Range error is generated, since there is no parameter specified in the SQL text. Note: I am accessing a MySQL database, but the issue here is Delphi & FireDac preprocessing of the code to send to the db server.
Ok, there may be a way to fix this issue using FireDAC properties, but I haven't found it yet. However, for this particular situation, where the SQL is prepared in one method, and then assigned to a different FDQuery from within another method, I have found an answer. Since PostgreSQL permits named parameters to use numerals, such as :1, what I did was to replace the $ characters with : characters. As in
var
SQLStmt: String;
FDQuery: TFDQuery;
begin
SQLStmt := 'SELECT * FROM tablename WHERE field2 = $1;';
FDQuery := TFDQuery.Create( nil );
FDQuery.Connection := FDConnection1;
FDQuery.SQL.Text := SQLStmt.Replace( '$', ':' );
FDQuery.Params[0].AsString := 'SomeValue'); // Works!
...
And, yes, if your query includes more than one instance of a named parameter, FireDAC replaces it with the same numeral.
If I find another solution I will post it. If someone else comes up with a solution using FireDAC properties, I will select that answer as the correct answer.

How to write where Between query in Yii2 with integer as a second parameter

I have this query in PostgreSQL that I want to implement in my model in Yii2:
SELECT *
FROM some_table
WHERE 1492257600 BETWEEN start AND end
start and end are attributes of some_table.
In my model, I tried the following but, no matter how I write it, it keeps throwing the same error.
$results = static::find()->where(['between', 1492257600, 'start', 'end'])
->all();
This is the error I'm getting:
Undefined column: 7 ERROR: column \"1492257600\" does not exist
The SQL being executed was: SELECT * FROM \"some_table\" WHERE \"1492257600\" BETWEEN 'start' AND 'end'"
When the query is built, these symbols " " are added to the integer, so PostgreSQL thinks it is a column of the table.
Can anyone tell me how to write the where between correctly?
There is dedicated BetweenColumnsCondition expression for such cases:
use yii\db\conditions\BetweenColumnsCondition;
$results = static::find()
->where(new BetweenColumnsCondition(1492257600, 'BETWEEN', 'start', 'end'))
->all();
It automatically quotes column names and escapes value, so it should be more convenient than simple yii\db\Expression.
Also note that WHERE 1492257600 BETWEEN start AND end may be slower than WHERE 1492257600 >= start AND 1492257600 <= end, I suggest to do some performance test if your table may grow big and usage indexes is crucial.
I don't have experience with postgresql but this will work same as between,
$results = self::find()
->where(['>=', 'start', 1492257600])
->andWhere(['<=', 'end', 1492257600])
->all();
Refer to this: Yii2 Doc
When an Expression object is embedded within a SQL statement or fragment, it will be replaced with the $expression property value without any DB escaping or quoting.
so You'll have something like this:
$expression = new \yii\db\Expression('1492257600 BETWEEN start AND end');
$results = self::find()->where($expression)
->all();

How to send multiple variables using birt report?

Using the following query in db2:
select * from table where num in ('1a2334','1a43432','1a34243','1b34325','1b4545')
Now whenever I get data to report I get the rows like from the users:
1a23344
1a43432
1a34243
1b34325
1b45454
Then I use notepad++ to replace rf with ',' so it becomes
'1a2334','1a43432','1a34243','1b34325','1b4545'
What are my options for creating a report that accepts input easy enough for the average user?
This specific user has an excel sheet with multiple columns, I only use the first column (the mentioned examples above are rows from the first column).
A good solution provided by #Simulant, but I need this to get values from an excel file (preferably by copy paste). I noticed his/her solution uses static values, so I think I need dynamic values.
For the record I got the following error using the script:
Error evaluating Javascript expression. Script engine error:
TypeError: Cannot call method "replace" of null
(/report/data-sets/script-data-set[#id="12"]/method[#name="beforeOpen"]#3)
Script source:
/report/data-sets/script-data-set[#id="12"]/method[#name="beforeOpen"],
line: 0, text:
__bm_beforeOpen(). (Element ID:1) Error.ScriptEvaluationError ( 1 time(s) ) detail : org.eclipse.birt.report.engine.api.EngineException:
There are errors evaluating script "var parameters =
params["multiSelectParameter"].value; var replacesPart = "'" +
parameters.join("', '") + "'"; this.queryText =
this.queryText.replace("replaceMe", replacesPart);":
Create a Report with a Multi-Select Parameter. Create a List Box Parameter and allow multiple values. You can add static values or select dynamic and display the result of another query.
Write your query as following SQL-Statement:
select * from table where num in (replaceMe);
Select your Data-Set and select the script Tab. Enter for the beforeOpen the following script. This replaces the placeholder replaceMe in your SQL-Statement with the concatinated values of your Multi-Select Parameter enclosed with single quotes ' and separated with commas , as you need it:
var parameters = params["multiSelectParameter"].value;
var replacesPart = "'" + parameters.join("', '") + "'";
this.queryText = this.queryText.replace("replaceMe", replacesPart);

Firebird 2.5.x. Extract column names and column datatypes of a result from stored procedure

I have a Firebird 2.5 database .As an example I have stored procedure with a name QRESULT which expected return is:
Parameter - DATATYPE
a - date
b - numeric(18,0)
c - integer
d - varchar(50)
and so on....
I use PHP - PDO to query the firebird database using the procedure QRESULT like this:
SELECT a,b,d from QRESULT() where a = "some value"
I need to run some query before QRESULT procedure and i need it to return the datatype of all the columns that QRESULT would return if it was ran. So i can help user to type proper value for my "where" clause.I know i can set that manually in the user interface, but in the real project there are lots of procedures and if there is a way i can make my filter interface generate dynamically i would be happy about that.If this is not possible for a stored procedure i can make it with select statements.I just need some lead.
The information you want is in the RDB$PROCEDURE_PARAMETERS table, basically what you need is query
SELECT r.RDB$PARAMETER_NAME ParName, F.RDB$FIELD_TYPE ParType
FROM RDB$PROCEDURE_PARAMETERS r
JOIN RDB$FIELDS F ON(F.RDB$FIELD_NAME = R.RDB$FIELD_SOURCE)
WHERE r.RDB$PROCEDURE_NAME = 'QRESULT'
AND r.RDB$PARAMETER_TYPE = 1
ORDER BY r.RDB$PARAMETER_TYPE, r.RDB$PARAMETER_NUMBER
Note that the SP name should be in upper case as this is how it is stored into system tables (unless you use quoted identifiers). If you want to get both input and output parameters the delete the r.RDB$PARAMETER_TYPE = 1 predicate from the WHERE (type 0 is input parameters and 1 is output).
The type returned by this query is integer id for the type, quick googling found this:
14,"TEXT "
7,"SHORT "
8,"LONG "
9,"QUAD "
10,"FLOAT "
27,"DOUBLE "
35,"TIMESTAMP "
37,"VARYING "
261,"BLOB "
40,"CSTRING "
45,"BLOB_ID "
12,"DATE "
13,"TIME "
16,"INT64 "
but if you want to have more precise type then see this SO post.