Postgres: Concat multiple columns, while including nulls - postgresql

I have a table with four columns that I need to concat. Two of the columns contain some NULL values.
I need the result to contain separators indicating all four columns, like so:
colA,colB,colC,colD
or if one column (here colB) was null,
colA,,colC,colD
I can't seem to find a clean way to do this. The best I have come up with is:
concat_ws(colA, COALESCE(colB, ''), COALESCE(colC, ''), colD, ',')
This feels cumbersome (especially because I need to do this repeatedly). Is there a better way?

Since the final value can't be NULL, you don't need to worry about preserving them. Just use an empty string. This is how you'd write it in standard SQL.
select coalesce(cola, '') || ', ' ||
coalesce(colb, '') || ', ' ||
coalesce(colc, '') || ', ' ||
coalesce(cold, '')
from your_table_name;
The concat_ws() function doesn't skip empty strings, but it does skip null columns. That means you still have to use coalesce().
select concat_ws(', ', coalesce(cola, ''),
coalesce(colb, ''),
coalesce(colc, ''),
coalesce(colb, ''))
from your_table_name;

I think you should use a cursor and then a loop also, in order to show null values you must use a nvl() function, like this:
PROCEDURE TODO IS
vAux in VARCHAR2;
CURSOR CUR_TEST IN
SELECT NVL(colb, 'replace_with'), colA, colD
FROM DUAL;
Begin
for x in CUR_TEST loop
vAux := x.colb||','||x.colA||','||x.colD;
end loop;
end TODO;
For checking what nvl() does: http://www.techonthenet.com/oracle/functions/nvl.php

Related

Dynamic SQL with EXECUTE and nested format()

I tried doing this with just a normal CTE (no EXECUTE ... FORMAT) and I was able to do it.
I basically have a table that has some 5 columns and some data in each column that I wanted to concatenate and such to manifest/generate some data in a new column.
I can do something like this and it works:
WITH cte AS (SELECT *, case
when var1 = '' then ''
when var2 = '' then ''
else '' end, 'adding_dummy_text_column'
FROM some_other_table sot
WHERE sot.type = 'java')
INSERT INTO my_new_table
SELECT *, 'This is the new column I want to make with some data from my CTE' || c.type
FROM cte c;
So this works as I said. I'll end up getting a new table that has an extra column which a hardcoded, concatenated string This is the new column I want to make with some data from my CTE Java
Of course, whatever is in the c.type column for the corresponding row in the CTE as it loads the SELECT is what gets concatenated to that string.
The problem is, as soon as I start using the EXECUTE...FORMAT to make it cleaner and have more power to concatenate/combined different data pieces from my different columns (I have data kind of scattered around in bad formats and I'm populating a fresh new table), it's as if the FORMAT arguments or the variables cannot detect the CTE table.
This is how I'm doing it
EXECUTE FORMAT ('WITH cte AS (SELECT *, case
when var1 = %L then %L
when var2 = '' '' then '' ''
else '' '' end, ''adding_dummy_text_column''
FROM some_other_table sot
WHERE sot.type = ''java'')
INSERT INTO my_new_table
SELECT *, ''This is the new column I want to make with some data from my CTE %I''
FROM cte c', 'word1', 'word2', c.type
);
OK, so I know I used the empty string '' '' in this example and the %L but i just wanted to show I had no issues with any of that. Its when I try to reference my CTE columns, so you can see I'm trying to do the same concatenation but by leveraging the EXECUTE...FORMAT and using the %I identifiers. So, the first 2 args are just fine, its the c.type that just no matter what column I try, doesn't work. Also, I removed the c alias and didn't get any better luck. It's 100% anytime I reference the columns on the CTE though, as I have removed all that code and it runs just fine without that.
But yeah, is there any work around? I really want to transform some data and now have to do the || for concatenation.
This should do it:
EXECUTE format($f$WITH cte AS (SELECT *, CASE
WHEN var1 = %L THEN %L
WHEN var2 = ' ' THEN ' '
ELSE ' ' END, 'adding_dummy_text_column'
FROM some_other_table sot
WHERE sot.type = 'java')
INSERT INTO my_new_table
SELECT *, format('This is the new column I want to make with some data from my CTE %%I', c.type)
FROM cte c$f$
, 'word1', 'word2');
There are two levels. You need a second format() that's executed by the dynamic SQL string that's been concatenated by the first format().
I simplified with dollar-quoting. See:
Insert text with single quotes in PostgreSQL
The nested % character must be escaped by doubling it up: %%I. See:
What does %% in PL/pgSQL mean?
Generates and executes this SQL statement:
WITH cte AS (SELECT *, CASE
WHEN var1 = 'word1' THEN 'word2'
WHEN var2 = ' ' THEN ' '
ELSE ' ' END, 'adding_dummy_text_column'
FROM some_other_table sot
WHERE sot.type = 'java')
INSERT INTO my_new_table
SELECT *, format('This is the new column I want to make with some data from my CTE %I', c.type)
FROM cte c
Which could be simplified and improved to this equivalent:
INSERT INTO my_new_table(co1, col2, ...) -- provide target column list!
SELECT *
, CASE WHEN var1 = 'word1' THEN 'word2'
WHEN var2 = ' ' THEN ' '
ELSE ' ' END
, 'adding_dummy_text_column'
, format('This is the new column I want to make with some data from my CTE %I', sot.type)
FROM some_other_table sot
WHERE sot.type = 'java'
About the missing target column list:
Cannot create stored procedure to insert data: type mismatch for serial column
You can one command, create and insert into new table!
create table my_new_table as
select *
, CASE WHEN var1 = 'hello' THEN 'word2'
WHEN var2 = ' ' THEN 'var2 is empty'
ELSE ' ' END adding_dummy_text_column
, format(
'This is the new column I want to make with some data from my CTE %I', sot.type)
from some_other_table sot
where sot.type ='java';

Why is T-SQL WHERE NOT IN Returning one of the filtered elements

In my below query I among others filter on specific values with NOT IN, which for most part works like it should, but one of the filtered values (not the others) still returns in the view, and I cannot figure out why this is happening .. the value that is returned is '76751219' .. Can anyone help me figure out why?
The query:
DECLARE #json NVARCHAR(MAX)
SET #json = (SELECT PHONE_DATA FROM EFP_JSON WHERE ID = 1)
SELECT PhoneNumber
FROM EFP_PhoneNumberSeries
WHERE REPLACE(PhoneNumber, ' ', '') NOT IN (
SELECT PhoneNumbers.Number
FROM OPENJSON(#json)
WITH (
Localnumber VARCHAR(50) '$.Localnumber',
Name VARCHAR(50) '$.Name',
Email VARCHAR(50) '$.Email',
PhoneNumbers nvarchar(max) '$.PhoneNumbers' AS JSON,
Phones nvarchar(max) '$.Phones' AS JSON
) as UserInfo
CROSS APPLY OPENJSON(PhoneNumbers)
WITH(
Number nvarchar(100) '$.Number',
LineName nvarchar(100) '$.LineName',
GotoLocalNumber nvarchar(100) '$.GotoLocalNumber',
BelongsTo nvarchar(100) '$.BelongsTo'
) as PhoneNumbers
CROSS APPLY OPENJSON(Phones)
WITH(
description nvarchar(100) '$.description'
) as description
) AND REPLACE(PhoneNumber, ' ', '') LIKE '767512%' OR REPLACE(PhoneNumber, ' ', '') LIKE '767520%'
AND LTRIM(REPLACE(PhoneNumber, ' ', '')) NOT IN ('76752000','76752005','76752006','76752007','76752008','76752010','76752011','76752012','76751219','76751221','76752022','76752058','76752060','76752063','76752097','76752098');
76751219 is being returned because it matches the REPLACE(PhoneNumber, ' ', '') LIKE '767512%' criteria regardless of the NOT IN filter.
To resolve this, you need to fix the parenthesis in the last few lines as you appear to be missing some. Based on what you've said, I think those lines should read
AND ( -- add this round opening bracket
REPLACE(PhoneNumber, ' ', '') LIKE '767512%'
OR REPLACE(PhoneNumber, ' ', '') LIKE '767520%'
) -- add this round closing bracket
AND LTRIM(REPLACE(PhoneNumber, ' ', '')) NOT IN ('76752000','76752005','76752006','76752007','76752008','76752010','76752011','76752012','76751219','76751221','76752022','76752058','76752060','76752063','76752097','76752098');
but you should review this yourself to make sure it meets your requirements for the AND & OR logic.

Postgres: how to use LIKE on every word of user input, AND-ing the results

In postgresql (9.6), given a variable length user input of the type 'alice chaplin' or 'alice' or 'alice chaplin meyer' but also 'lic chapl', I would like to search for records that contain 'alice' in column firstname OR column lastname (AND contain 'chaplin' in firstname OR lastname (AND contain 'meyer' in firstname OR lastname)), etc.
I had decided to use ILIKE %searchterm% for the matching, so the query would presumably be along the lines of:
... where
((lastname ILIKE '%' || SEARCHTERM1 || '%') OR (firstname ILIKE '%' || SEARCHTERM1 || '%'))
AND ((lastname ILIKE '%' || SEARCHTERM2 || '%') OR (firstname ILIKE '%' || SEARCHTERM2 || '%'))
AND etc.
After lots of attempts and searching, nothing comes up that resolves this... As a last resort I'll write a very procedural pgplsql function that loops over a split search string, intersecting the ILIKE results, but there has to be some more idiomatic SQL way of resolving such a run of the mill problem.
You can use string_to_array to convert an input string into an array of words. You can then use unnest to convert the array into a (virtual) table, and operate on the words to add '%' before and after. And finally, you can use the ALL comparison using ILIKE ALL (SELECT ...). This ALL will actually be AND-ing the results, as desired.
WITH q AS
(
SELECT 'Alice Chaplin Meyer'::text AS q
)
, words AS
(
SELECT
'%' || word || '%' AS wordish
FROM
q
JOIN LATERAL unnest(string_to_array(q, ' ')) AS a(word) ON true
)
SELECT
*
FROM
t
WHERE
concat_ws(' ', first_name, last_name) ILIKE ALL(SELECT wordish FROM words)
You can check it all at http://rextester.com/LNB38296
References:
string_to_array and unnest
Using ALL
NOTE: This can probably be simplified, but I've prefered a step-by-step approach.

Make index use using calculative field T-SQL

We do have one query which is running very frequently but as we are using function on both side of column, it's not doing index seek and this query turn out to be one of the most expensive query. Is there any way i can make this column calculative?
SELECT TOP 1 column1
FROM table1
WHERE replace(replace(replace(column2, char(10), ''), char(13), ''), ' ', '') =
replace(replace(replace(#var1, char(10), ''), char(13), ''), ' ', '')
To expand on dfundako's comment, you can make an index on a persisted calculated column like this:
CREATE TABLE table1 (
column1 INT,
column2 VARCHAR(50),
column3 AS REPLACE(replace(replace(column2, char(10), ''), char(13), ''), ' ', '') PERSISTED
)
CREATE INDEX index1 ON table1 (column3) INCLUDE (column1)
DECLARE #var1 VARCHAR(50)
SELECT column1 FROM dbo.table1 WHERE column3=replace(replace(replace(#var1, char(10), ''), char(13), ''), ' ', '')

Why do two seemingly identical where clauses involving nulls produce different results

I'm trying to select all records that don't have a null in a particular column and it's value isn't in another table.
So this particular situation I want to get all 'Instructors' from an import table that aren't already in the Individuals table. Obviously I don't want any blank instructors. My first attempt I tried using in the where clause:
(Instructor IS NOT NULL OR Instructor <> '')
However the results still included all blank records. When I tried using
ISNULL(Instructor, '') <> ''
I got the desired result. I can't see how these two where clauses could possibly produce different results. To me it seems like ISNULL converting the value to empty string for comparison should have exactly the same outcome as comparing the column to null then to empty string. What am I missing here? I'm guessing it's to do with the oddness of null values.
Below are the full queries
SELECT * FROM [tempimporttblTrainingLog]
LEFT JOIN tblIndividual I ON [Instructor] = I.FirstName + ' ' + I.Surname
WHERE (I.FirstName + ' ' + I.Surname IS NULL) AND (Instructor IS NOT NULL OR Instructor <> '')
SELECT * FROM [tempimporttblTrainingLog]
LEFT JOIN tblIndividual I ON [Instructor] = I.FirstName + ' ' + I.Surname
WHERE (I.FirstName + ' ' + I.Surname IS NULL) AND (ISNULL(Instructor, '') <> '')
ISNULL(Instructor, '') <> '' (1)
is equivalent to
(Instructor IS NOT NULL AND Instructor <> '') (2)
not
(Instructor IS NOT NULL OR Instructor <> '') (3)
If Instructor IS NULL, (1) and (2) will return FALSE, when (3) returns TRUE.
Same for when Instructor = ''.
ISNULL(Instructor, '') <> ''
won't return blank records, and you have an OR in the
(Instructor IS NOT NULL OR Instructor <> '')
line, meaning it will return anything which isn't null as well as non blanks, did you mean to put an AND instead?