Escaping Special Characters in PostgreSQL Insert Statement - postgresql

I am writing a function that requires me to store some JavaScript code in a PostgreSQL database table (this is required). I am looking for a "Lazy" way of doing this without modifying my PostgreSQL insert statement to escape the special characters at every instance it might occur within the JavaScript code. I primarily want to avoid doing the escapes in the event that the JavaScript code were to get lengthier. Since this might get a bit messy quickly. PostgreSQL seems to offer the following functions:
quote_literal()
quote_ident()
Reference: PostgreSQL String Formatting Functions
Having tested both of these a common error I am running into is the following error:
Error: unterminated quoted identifier at or near "": true
At a quick glance it appears that my issue lies in the formatted JavaScript text itself.
Is there a "Lazy" way for me to avoid escaping all these special characters without having to do this manually? Essentially, I would like to dump this code into a variable and perform the insert using the stored variable without (ideally) or with minimal modifications to the stored JScript text.
Below is an example of what my code looks like:
CREATE OR REPLACE FUNCTION abc.my_function(text, text, text, text, text[])
RETURNS void AS $body$
DECLARE
-- Variable Declarations here
jscript TEXT := quote_ident('/* JScript Comments Here*/ $(document).ready(function(){
// Initialize Datatable ...
$('#Index').Datatable({
"paging": true, // comment here
"responsive": true, // comment here
"pageLength": 25, // comment here
"columnDefs": [ {
...
...
...
}]
});
');
BEGIN
...
...
...
-- Insert static HTML
execute 'Insert into abc.my_table(file, data, gen_flag) values('||'''main.js||','||jscript||','|| '''N''' || ')';
...
...
...
END;
$body$
LANGUAGE 'plpgsql' VOLATILE;

The main problem is, that you are using quote_ident which is for quoting identifiers (column names, table names, ...) where you should be using quote_literal which is used to properly quote literal values.
To declare your variable and assign the value you can use a second level of dollar quoting so you don't need to worry about single quotes inside the value:
declare
jscript TEXT := $js$
/* JScript Comments Here*/
$(document).ready(function(){
// Initialize Datatable ...
$(#Index').Datatable({
"paging": true, // comment here
"responsive": true, // comment here
"pageLength": 25, // comment here
"columnDefs": [ {
...
...
...}]
});
$js$;
Dynamic SQL is better created using the format() function which takes care of quoting properly:
execute format('Insert into abc.my_table(file, data, gen_flag) values(%L, %L, %L), 'main.js',jscript, 'N')';
But you do not need dynamic SQL at all, you can just write:
insert into abc.my_table(file, data, gen_flag) values('main.js', jscript, 'N');
Online demo: https://dbfiddle.uk/?rdbms=postgres_10&fiddle=81b36bb1d3f246637186a419a9b337d4

Related

Porting Oracle associative arrays to Postgres and wrapping the data access using npgsql

It is great to get prompt replies on the npgsql queries. Thanks to its owner! I am trying to port a sproc that took in array valued parameters to postgres with similar abstraction/semantics. I am wondering how one would shape the pgsql sproc and use npgsql api to call that. I have another layer of application abstraction on top of our dal abstraction. We use data adapters to go to sql server, assoc arrays to oracle and trying to figure out what we could map this to for postgres using npgsql. We have some room in shaping the sproc but still keep the number of input params the same. we could certainly build this sproc much different but we still need it behind the same app api which supplies some set of typed arrays as shown below
public static void Flush2OraWithAssocArrayInsnetworkdatabatch(string dbKey ,int?[] ENDPOINTID,DateTime?[] INSERTEDDATETIME,int?[] RECORDTYPEID,long?[] RECORDVALUE,int?[] PACKETSIZE)
{
Database db = Helper.GetDatabase(dbKey);
using (DbConnection con = db.CreateConnection()){
con.Open();
using (DbCommand cmd = con.CreateCommand()){
cmd.CommandText = "Insnetworkdatabatch";
Helper.InitializeCommand(cmd, 300, "Insnetworkdatabatch");
BuildInsnetworkdatabatchOracleAssocArrayCommandParameters(cmd ,ENDPOINTID,INSERTEDDATETIME,RECORDTYPEID,RECORDVALUE,PACKETSIZE);
try {
Helper.ExecuteNonQuery(cmd, cmd.CommandText);
con.Close();
} catch (DALException ) {
throw;
}
}
}
}
I have a oracle sproc written as follows
create or replace PROCEDURE InsNetworkDataBatch2
(
-- Add the parameters for the stored procedure here
v_endPointID IN arrays.t_number ,
v_insertedDateTime IN arrays.t_date ,
v_recordTypeID IN arrays.t_number ,
v_recordValue IN arrays.t_number ,
v_packetSize IN arrays.t_number )
AS
BEGIN
DECLARE
BEGIN
FORALL i IN v_endpointID.FIRST..v_endpointID.LAST SAVE EXCEPTIONS
INSERT
INTO STGNETWORKSTATS
(
INSERTEDDATE,
ENDPOINTID,
RECORDTYPEID,
RECORDVALUE,
PACKETSIZE
)
VALUES
(
v_insertedDateTime(i),
v_endPointID(i),
v_recordTypeID(i),
v_recordValue(i),
v_packetSize(i)
);
END;
END;
-- END PL/SQL BLOCK (do not remove this line) ----------------------------------
Here is the assoc array package in oracle
create or replace PACKAGE Arrays AS
type t_number is table of number index by binary_integer;
type t_date is table of date index by binary_integer;
END Arrays;
Here is how we build the oracle parm and wondering what its equivalency if at all possible in postgres and trying to see how npgsql will support it
public override void CreateAssociativeArrayParameter(DbCommand cmd, string parameterName, object parameterValue, string dbType, ParameterDirection direction)
{
OracleDbType oracleDbType = dbSpecificTypesMap[dbType];
OracleParameter param = new OracleParameter(parameterName, oracleDbType, direction);
param.CollectionType = OracleCollectionType.PLSQLAssociativeArray;
param.Value = parameterValue;
cmd.Parameters.Add(param);
}
I don't know anything about Oracle arrays or associative arrays. However, PostgreSQL has a rich support for complex types. PostgreSQL arrays are a good way to store an array of values in a column, and PostgreSQL even provides indexing and database-side functions to work with arrays.
If you're looking for a dictionary type (associative array?), take a look at hstore or json.
EDITED: If your associative array has a fixed schema (i.e. the fields don't change), you can also consider PostgreSQL composite.
Here is an attempt with Postgres stored procedure. This is now working. I got around some casting issues thrown from inside the npgsql which was a result of my .net type not being compatible with the sproc parameter data type in postgres.
Here is how i am trying to add the param value
create or replace FUNCTION InsNetworkDataBatch
(
-- Add the parameters for the stored procedure here
v_endPointID IN int[] ,
v_insertedDateTime IN timestamp[] ,
v_recordTypeID IN int[] ,
v_recordValue IN bigint[] ,
v_packetSize IN int[] ) RETURNS void
LANGUAGE 'plpgsql'
AS $$
BEGIN
DECLARE
BEGIN
FOR i IN array_lower(v_endPointID, 1) .. array_upper(v_endPointID, 1)
loop
INSERT INTO STGNETWORKSTATS
(
INSERTEDDATE,
ENDPOINTID,
RECORDTYPEID,
RECORDVALUE,
PACKETSIZE
)
VALUES
(
v_insertedDateTime[i],
v_endPointID[i],
v_recordTypeID[i],
v_recordValue[i],
v_packetSize[i]
);
end loop;
END;
END;
$$
Here is how i am trying to bind the app to the command params
public override void CreateAssociativeArrayParameter(DbCommand cmd, string parameterName, object parameterValue, string dbType, ParameterDirection direction)
{
NpgsqlDbType npgsqlDbType;
if (dbSpecificTypesMap.ContainsKey(dbType))
{
npgsqlDbType = dbSpecificTypesMap[dbType];
}
else
{
throw new ApplicationException($"The db type {dbType} could not be parsed into the target NpgsqlDbType. Please check the underlying type of the parameter");
}
NpgsqlParameter param = new NpgsqlParameter(parameterName.ToLower(), NpgsqlDbType.Array | npgsqlDbType);
param.Value = parameterValue;
cmd.Parameters.Add(param);
}

Stop jOOQ stripping characters from SQL field

I am implementing a function to get an estimate of the count as described in the PostgreSQL documentation here https://wiki.postgresql.org/wiki/Count_estimate
I'm using the function:
public static Field<Integer> countEstimate(final QueryPart query) {
final String sql = String.format("count_estimate(%s)", escape(query.toString()));
return field(sql(sql), PostgresDataType.INT);
}
Which looks fine until I pass it an IN clause array field in the query. When this happens jOOQ strips the array curly braces from within my SQL. e.g. Calling it with this java code:
final UUID[] ids = new UUID[]{UUID.randomUUID()};
return db.select(countEstimate(db.select(TABLE.ID)
.from(TABLE)
.where(overlaps(ids, TABLE.FILTER_IDS))));
Results in both the variable sql and DSL.sql(sql) in the above function rendering:
count_estimate(E'select "schema"."table"."id"
from "schema"."table"
where (
((\'{"75910f3b-83e6-41ed-bf57-085c225e0131"}\') && ("schema"."table"."filter_ids"))
)')
But field(sql(sql), PostgresDataType.INT) renders this:
count_estimate(E'select "schema"."table"."id"
from "schema"."table"
where (
((\'"75910f3b-83e6-41ed-bf57-085c225e0131"\') && ("schema"."table"."filter_ids"))
)')
Is there any way to work around this and to tell jOOQ to leave my query alone?
(jOOQ 3.8.3, PG 9.5.5, PG driver 9.4-1203-jdbc4)
It turns out it only strips '{}' style arrays. Replacing the code that turns the UUID[] into sql from
DSL.val(ids)
with
DSL.array(Arrays.stream(ids)
.map(UUID::toString)
.collect(Collectors.toList())
.toArray(new String[0]))
.cast(PostgresDataType.UUID.getArrayDataType()
results in it rendering cast(array[\'75910f3b-83e6-41ed-bf57-085c225e0131\'] as uuid[]) prevents it being stripped

Using procedure in structured data type as callback for C library (GTK+3)

I'm trying to use procedures within a structured data type as callback functions for a program using GTK+3 as its toolkit in FreePascal. (The GTK+3 bindings I have were generated by the gir2pascal tool (http://wiki.freepascal.org/gir2pascal))
In the example below, I use advanced records, but I would definitely consider classes or objects if it works better/at all with them.
The problem that occurs is that when the callback procedure is called, it cannot access anything else within its own record. It seems to "forget" where it comes from.
For instance, in the example below I have the integer myRecord.myInt, that I can set and retrieve happily by calling the procedure myRecord.testProcedure. However when testProcedure is used as a C callback (when I click the button), I will receive some number (e.g. 30976), but not 7.
{$MODESWITCH ADVANCEDRECORDS}
uses gobject2, gtk3, math;
type
myRecord=record
public
myInt: Integer;
procedure testProcedure; cdecl;
end;
procedure myRecord.testProcedure; cdecl;
begin
WriteLn(myInt);
end;
var
recordInstance: myRecord;
button, win: PGtkWidget;
begin
SetExceptionMask([exDenormalized, exInvalidOp, exOverflow,
exPrecision, exUnderflow, exZeroDivide]); {this is needed for GTK not to crash}
gtk_init(#argc, #argv);
win:=gtk_window_new(GTK_WINDOW_TOPLEVEL);
recordInstance.myInt:=7;
button:=gtk_button_new;
{The following does not work. The procedure will run when the button is
clicked; it will print some number, but not the content of recordInstance.myInt}
g_signal_connect_data(button, 'clicked',
TGCallback(#recordInstance.testProcedure), nil, nil, 0);
{add button to window}
gtk_container_add(PGtkContainer(win), button);
gtk_widget_show_all(win);
{Test call to recordInstance.testProcedure to see that it outputs
'7' correctly}
recordInstance.testProcedure;
gtk_main;
end.
When I try to use Classes or Objects instead of an Advanced Record, I receive error messages of the kind
"<procedure variable type of procedure of object;CDecl>" to "<procedure variable type of procedure;CDecl>"
What ways are there of using a structured data type with a procedure to use as a C callback as in the example above (if any)?
class static methods are compatible with procedures. But they also have the disadvantage that they don't have a reference to the data of the object.
{$mode delphi}
type
myRecord=record
public
myInt: Integer;
class procedure testProcedure; cdecl;static;
end;
tproctype = procedure; cdecl;
class procedure myrecord.testProcedure; cdecl;static;
begin
end;
var x : tproctype;
y : myrecord;
begin
x:=y.testprocedure;
end.
compiles, but the usage is sterile, since if it maps to plain C, it doesn't have (implicit) OO properties.

Pass a NULL value to numeric parameter via Yii in PostgreSQL

I have written a function in PostgreSQL with a single parameter of type NUMERIC. I am attempting to call that function from a webapp developed using the Yii framework via the SqlDataProvider component. However, each time the parameter value is left empty, I keep getting the following error:
Ivalid input syntax for type 'numeric'.
Whenever I try to execute the function directly via PhpPgAdmin, everything seems to work flawlessly.
Below is the code for the discussed PL/pgSQL function search(..):
search("itemcode" character varying DEFAULT NULL, "taxpercentage" numeric DEFAULT NULL)
LANGUAGE plpgsql
AS $$
BEGIN
SELECT * FROM item WHERE (itemcode IS NULL OR item.item_code LIKE itemcode||'%')
AND (taxpercentage IS NULL OR item.tax_percentage = taxpercentage);
END;
$$;
In the Yii model, I have also created a search function which in turn calls the database-level PL/pgSQL function:
public function search()
{
$count=Yii::app()->db->createCommand("SELECT COUNT(*) FROM search('$this->item_code','$this->tax_percentage')")->queryScalar();
$sql="SELECT * FROM search('$this->item_code','$this->tax_percentage')";
$dataProvider=new CSqlDataProvider($sql, array(
'totalItemCount'=>$count,
'pagination'=>array(
'pageSize'=>10,
),
));
return $dataProvider;
}
How can I pass a NULL value to parameter of type NUMERIC?
All help will be much appreciated. Thanks in advance.
I solved it by checking for empty in the search function and using parameterized query. So the modified function is
public function search()
{
if(empty($this->tax_percentage))
$this->tax_percentage=null;
$count=Yii::app()->db->createCommand("SELECT COUNT(*) FROM search(:item_code,:tax_percentage)")->queryScalar();
$sql="SELECT * FROM search('$this->item_code','$this->tax_percentage')";
$dataProvider=new CSqlDataProvider($sql, array(
'params' => array(':item_code' => $this->item_code,':tax_percentage' => $this->tax_percentage),
'totalItemCount'=>$count,
'pagination'=>array(
'pageSize'=>10,
),
));
return $dataProvider;
}

Free Pascal multiple interfaces problem

This may prove difficult -- I'm using multiple CORBA interfaces on an object, so it looks somehow like this:
TBaseObject = class(TSuperBaseObject, IInterfaceA)
function Afunction; // implemented from IInterfaceA
end;
TOtherObject = class(TBaseObject, IInterfaceB);
function Bfunction; // implemented from IInterfaceB
end;
Now I have a function that takes a variant, and in case that variant is an object, it assumes that object to be a IInterfaceA object:
case var.vtype of
...
vtObject : begin
Something := (var.vObject as IInterfaceA).AFunction; (1)
end;
end;
Now once I run that code, and pass a TOtherObject to the function, in line (1) BFunction gets called with forced parameters!
Am I doing something wrong or is it a bug in the compiler? Also, any sane way to circumvent that without changing the class structure?
Full code for a EAccessViolation if anyone wants to try - http://pastebin.com/D7sDpDHx
Reported this as a bug to the FPC bugtracker - http://bugs.freepascal.org/view.php?id=20076
It turned out that FPC doesn't identify CORBA interfaces internally. To solve the problem one needs to identify them by himself:
type IInterfaceA = interface['interface_a']
function AFunction;
end;
Then the as keyword will work.
Not sure about FreePascal, but in Delphi you would use the supports function to query the interface.
var
IntfA : IInterfaceA;
IntfB : IInterfaceB;
begin
case var.vtype of
...
vtObject : begin
if supports(var.vObject,IInterfaceA,IntfA) then
Something := IntfA.AFunction
else if supports(var.vObject,IInterfaceB,IntfB) then
Something := IntfB.BFunction;
end;
end;
end;