My web-application allows for token replacements and therefore my SQL INSERT query looks something like this:
INSERT INTO mytable (Col1, Col2, Col3)
VALUES ('[Col1Value]', '[Col2Value]', '[Col3Value]')
The web app is replacing whats inside the brackets [Col1Value] with the input entered into the form field.
Problem is when an input field is left empty this query is still inserting ' ' just an empty space so the field is not considered null
I'm trying to use SQL's default value/binding section so that all columns that are null have a default value of -- but having my query insert a blank space ' ' is making it so SQL does not trigger the default value action and it still shows blank rather than my desired default value of --
Any ideas as to how I can solve this and make sure ' ' is inserted as a null rather than a space or empty space so it will trigger SQL replacing null with my default value of --
There is no easy going...
How are you inserting the values? If you create these statements literally you are stumbling on the dangerous fields of SQL injection... Use parameters!
One approach might be an insert through a Stored Procedure, another approach is an Instead Of TRIGGER and the third uses the fact, that the string-length does not calculate trailing blanks:
SELECT LEN('') --returns 0
,LEN(' ') --returns 0 too
You can use this in an expression like this:
CASE WHEN LEN(#YourInputValue)=0 THEN NULL ELSE #YourInputValue END
Related
in Postgres (Supabase) I am trying to automatically generate a column from another column which contains a text array of short title variants.
The tsvector works just fine and as expected. The other possibility which would be to use array_to_tsvector is not an option as short title text array contains not just single words but variants of short titles (sentences).
alter table "MOVIES"
add column fts_short_title
tsvector GENERATED ALWAYS AS (
to_tsvector('simple',
array_to_string( title_short,' '::text))
) STORED;
but I get this error
Failed to run sql query: generation expression is not immutable
On the other hand I was successful when adding such a column for JSONB of full titles for different languages
alter table "MOVIES"
add column fts
tsvector GENERATED ALWAYS AS (
to_tsvector('simple',
coalesce(title->>'en', '') || ' ' ||
coalesce(title->>'de', '') || ' ' ||
coalesce(title->>'it', '') || ' ' ||
coalesce(title->>'fr', ''))
) STORED;
Thank you very much for any tip and help.
.. SQL is rather new to me, have used only from MongoDB previously, so sorry for my question.
You could define immutable wrappers for otherwise non-immutable functions. online demo
create or replace function array_to_string_immutable (
arg text[],
separator text,
null_string text default null)
returns text immutable parallel safe language sql as $$
select array_to_string(arg,separator,null_string) $$;
alter table "MOVIES"
add column fts_short_title
tsvector GENERATED ALWAYS AS (
to_tsvector('simple',
array_to_string_immutable( title_short,' '::text))
) STORED;
table "MOVIES";
While the textcat() function behind || operator is immutable, I'm pretty sure array_to_string() is only stable for the same reason concat() is so you need to be reasonably careful with where you use this workaround.
You could do the same for the other column to use a concat_ws() and avoid the repeated ||' '||coalesce():
create or replace function concat_ws_immutable (
separator text,
variadic arg text[])
returns text immutable parallel safe language sql as $$
select concat_ws(separator,variadic arg) $$;
alter table "MOVIES"
add column fts
tsvector GENERATED ALWAYS AS (
to_tsvector('simple',concat_ws_immutable(' ',title->>'en',title->>'de',title->>'it',title->>'fr'))
) STORED;
You are also free to do pretty much whatever you want, however you want to the column in a plpgsql function used by a trigger after insert or update on "MOVIES".
I am using PostgreSQL 11.9
I have a table containing a jsonb column with arbitrary number of key-values. There is a requirement when we perform a search to include all values from this column as well. Searching in jsonb is quite slow so my plan is to create a trigger which will extract all the values from the jsonb column:
select t.* from app.t1, jsonb_each(column_jsonb) as t(k,v)
with something like this. And then insert the values in a newly created column in the same table so I can use this column for faster searches.
My question is what type would be most suitable for storing the keys and then searchin within them. Currently the search looks like this:
CASE
WHEN something IS NOT NULL
THEN EXISTS(SELECT value FROM jsonb_each(column_jsonb) WHERE value::text ILIKE search_term)
END
where the search_term is what the user entered from the front end.
This is not going to be pretty, and normalizing the data model would be better.
You can define a function
CREATE FUNCTION jsonb_values_to_string(
j jsonb,
separator text DEFAULT ','
) RETURNS text LANGUAGE sql IMMUTABLE STRICT
AS 'SELECT string_agg(value->>0, $2) FROM jsonb_each($1)';
Then you can query like
WHERE jsonb_values_to_string(column_jsonb, '|') ILIKE 'search_term'
and you can define a trigram index on the left hand side expression to speed it up.
Make sure that you choose a separator that does not occur in the data or the pattern...
I am trying to write a function that will dynamically create the sql statement, but I am facing problems with typecasts so, how can I identify using the type of field if it needs to be quoted
-- using this I can recover the types of each field
-- but I do not have a simple way to express that for a specific type it need
-- to quote
create table test (
id serial not null,
employee_name text,
salary decimal(12,2),
hire_date date,
active boolean
);
select column_name,data_type, null as need_to_be_quoted
from information_schema.columns
where table_name = 'table_name';
column type need to be quoted (this is a missing information)
-------------------------------------
id integer false
employee_name text true
salary decimal false
hire_date date true
active boolean false
quote_ident docs says:
Return the given string suitably quoted to be used as an identifier in an SQL statement string. Quotes are added only if necessary
But it is not what I was expecting:
insert into test (employee_name, salary, hire_date, active)
values (quote_identy('John Doe'), quote_identy(100000), quote_identy(current_date), quote_identy(true));
This is kind of necessary because I am trying to generate the statement string dinamically.
I have values to be inserted in some table, I can discover the type of each value, but to generated the insert string statement, I should know if a specific value type should be quoted or not for example
text: type should be quoted in the string statement
boolean: should not be quoted
numeric: should not be quoted
date: should be quoted
Don't quote. Quoting adds complexity and if you don't get it just right it has syntax and security issues.
Instead use bind parameters. The details depend on what database library you're working with, but the basic idea is always the same. First prepare the statement with placeholders for your values. Then execute it passing in the values. The database takes care of putting the values into the statement. You can execute the same prepared statement multiple times optimizing your coding and database calls.
Here's how you'd do it in PL/pgSQL with EXECUTE. The linked documentation has lots of information about safely executing dynamic queries.
do $$
begin
execute '
insert into test (employee_name, salary, hire_date, active)
values ($1, $2, $3, $4)
' using 'John Doe', 100000, current_date, true;
end;
$$;
Furthermore, while writing a SQL builder is a good exercise, it's also very complicated and very easy to get subtly wrong. There are plenty of libraries which will build SQL for you.
Here's your statement using PHP's Laravel.
DB::table('test')->insert([
'employee_name' => 'John Doe',
'salary' => 100000,
'hire_date' => current_date,
'active' => true
);
If you're curious about how SQL builders work, you can dig inside Laravel for ideas. Writing one in PL/pgSQL is ambitious, but I don't know of one that exists. Good luck!
I would like to know how I can insert regular expression in a table column in a PostgreSQl table.
For example I have column called "rule" in a table where I need to store the expression ^[0-9]+$. I tried:
insert into rule_master(rule)
values('^[0-9]+$') where rule_id='7'
But I am getting error syntax near where is wrong. I tried this with and with out single quotes. Please suggest me a solution.
It appears you want to UPDATE an existing record. In that case you should do:
UPDATE rule_master
SET rule = '^[0-9]+$'
WHERE rule_id = '7';
But if this is indeed a new record and you want to INSERT that regex with the value of "rule_id" then do:
INSERT INTO rule_master(rule_id, rule)
VALUES ('7', '^[0-9]+$');
I have two columns and i need to copy of data from column VISITSAUTHORIZED to NEWVISITS, When i use below command to copy data i am getting an error message "invalid number".Can anyone correct this ?
VISITSAUTHORIZED VARCHAR2(10)
NEWVISITS NUMBER(8)
SQL> update patientinsurance set NEWVISITS=VISITSAUTHORIZED ;
ERROR at line 1:
ORA-01722: invalid number
It depends what kind of data you have in your old column. If it is all consistently formatted then you might be able to do:
update patientinsurance
set newvisits = to_number(visitsauthorized, '<format model>')
But it sounds more likely that you have something less easy to deal with. (The joys of storing data as the wrong datatype, which I assume is what you're now correcting). If there are rogue characters then you could use translate to get rid of them, perhaps, but you'd have to wonder about the integrity of the data and the values you end up with.
You can do something like this to display all the values that can't be converted, which may give you an idea of the best way to proceed - if there are only a few you might be able to correct them manually before re-running your update:
set serveroutput on
declare
newvisits number;
number_format_exception exception;
pragma exception_init(number_format_exception, -6502);
begin
for r in (select id, visitsauthorized from patientinsurance) loop
begin
newvisits := to_number(r.visitsauthorized);
exception
when number_format_exception then
dbms_output.put_line(sqlcode || ' ID ' || r.id
|| ' value ' || r.visitsauthorized);
end;
end loop;
end;
/
This is guessing you have a unique identifier field called ID, but change that as appropriate for your table, obviously.
Another approach is to convert the numbers that are valid and skip over the rest, which you can do with an error logging table:
exec dbms_errlog.create_error_log(dml_table_name => 'PATIENTINSURANCE');
merge into patientinsurance target
using (select id, visitsauthorized from patientinsurance) source
on (target.id = source.id)
when matched then
update set target.newvisits = source.visitsauthorized
log errors into err$_patientinsurance reject limit unlimited;
You can then query the error table to see what failed:
select id, visitsauthorized, ora_err_number$
from err$_patientinsurance;
Or see which records in your main table have newvisits still null. Analysing your data should probably be the first step though.
If you want to strip out all non-numeric characters and treat whatever is left as a number then you can change the merge to do:
...
update set target.newvisits = regexp_replace(source.visitsauthorized,
'[^[:digit:]]', null)
But then you probably don't need the merge, you can just do:
update patientinsurance set newvisits = regexp_replace(visitsauthorized,
'[^[:digit:]]', null);
This will strip out any group or decimal separators as well, which might not be an issue, particularly as you're inserting into a number(8) column. But you could preserve those if you wanted to, by changing the pattern to '[^[:digit:].,]'... though that could give you other problems still, potentially.
You can also do this with translate if the regex is too slow.