The use case seems pretty simple....
Produce a sql statement as part of a copy activity that includes a hard coded column listing and also concatenated to a parameter-provided database and table name (since the database and table names can change across environments such as dev/test/prod).
The problem is....If you use concat function it treats every comma as a new value to be concatenated. I was hoping for a way to escape the comma and treat it as a value but nothing I've tried works.
For example....concatenate the following string ....
SELECT event_date, event_timestamp,
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'page_title') AS page_title
FROM '
to... pipeline().parameters.Database_Nm + . + pipeline().parameters.Table_Nm
The workaround has been to quote the beginning and end of every line so the comma is treated as data so every column/line is a separate concatenation such as this....
#concat('SELECT event_date,',
'(SELECT value.string_value FROM UNNEST(event_params) WHERE key = ''page_title'') AS page_title,',
'from ', pipeline().parameters.Database_Nm, '.', pipeline().parameters.Table_Nm
That works...but I have over a hundred columns so this is just a bit silly as a solution. Am I missing a simpler method? TIA!
When most of your string is hard coded and not expressions you can use the following string interpolation expression format:
SELECT event_date,
(SELECT value.string_value FROM UNNEST(event_params) WHERE key = 'page_title') AS page_title
from #{pipeline().parameters.Database_Nm}.#{pipeline().parameters.Table_Nm}
Related
I have a Lookup that retrieves a few records from a MS SQL table containing schema, table name and a whole where clause. These values are passed to a copy data (within a ForEach) In the copy data i use a Dynamic query statement like:
#concat('select a.*, current_date as crt_tms from ',item().shm_nam,'.',item().tab_nam,
item().where_clause )
This construction works fine without the where_clause or with a where clause with an integer. But it goes wrong with strings like:
'a where a.CODSYSBRN ='XXX' ;'
it's about the quote (')
How can i pass it through?
I know that the where clause as a fixed string in the dynamic query works when i use double quotes (to escape the single quote):
'a where a.CODSYSBRN =''XXX'' ;'
Point is i need the where clause to be completely dynamic because it differ per table
whatever i try i get this kind of error:
Syntax error or access violation;257 sql syntax error: incorrect syntax near "where a"
ps i also tested this, but with the same result:
select a.*, current_date as crt_tms from #{item().shm_nam}.#{item().tab_nam} a #{item().where_clause}
As you have mentioned you are getting whole where clause from the lookup table, the query must have included the column values in where clause for string and integer types separately.
Example lookup table:
In your copy activity, you can use Concat() function as you were already doing it, to combine static values & parameters.
#concat('select * from ',item().schma_name,'.',item().table_name,' ',item().where_clause)
For debugging purposes, I have added the expression in set variable activity, to see the value of the expression.
Iteration1:
Iteration2:
i have table having below records.The below product description are given by user in textbox in
front end(asp.net).The product description will come with single quotes or double quotes.i want to insert in the table.so how can we check whether single quotes or double quotes are exists in the
the input and insert value in the table.please help.
String s=Textbook
CREATE TABLE Product_Details(Product_Description varchar(50))
Required Output
Product_Description
-------------------
STORE('COVERED)
STEEL("ROOFED)
Insert statement will be differ for above two string?.
I'm not sure what exactly what you are looking for. You should be able to store text with quotes or double quotes without any trouble (note, I'm testing on Postgresql 9.4, don't have 9.2).
The problem is sometimes creating the text with the single quotes. In those cases it is common to have two single quotes like this insert into product_details values ('STORE(''COVERED)') The double quotes (") should not be a problem. You can use the syntax E'STORE(\'COVERED)' instead of the two quotes. Sometimes more readable.
If you just want to check if there are ' or "" in the input, this check is convenient:
select length(replace(product_description,'''',''))!=length(product_description),
length(replace(product_description,'"',''))!=length(product_description)
which return true/false columns telling if single-quote exists in string in first column and double-quote in the latter.
To delete the quotes in string you can do:
select replace(replace(product_description,'"',''),'''','')
Best regards,
Bjarni
My web-application allows for token replacements and therefore my SQL INSERT query looks something like this:
INSERT INTO mytable (Col1, Col2, Col3)
VALUES ('[Col1Value]', '[Col2Value]', '[Col3Value]')
The web app is replacing whats inside the brackets [Col1Value] with the input entered into the form field.
Problem is when an input field is left empty this query is still inserting ' ' just an empty space so the field is not considered null
I'm trying to use SQL's default value/binding section so that all columns that are null have a default value of -- but having my query insert a blank space ' ' is making it so SQL does not trigger the default value action and it still shows blank rather than my desired default value of --
Any ideas as to how I can solve this and make sure ' ' is inserted as a null rather than a space or empty space so it will trigger SQL replacing null with my default value of --
There is no easy going...
How are you inserting the values? If you create these statements literally you are stumbling on the dangerous fields of SQL injection... Use parameters!
One approach might be an insert through a Stored Procedure, another approach is an Instead Of TRIGGER and the third uses the fact, that the string-length does not calculate trailing blanks:
SELECT LEN('') --returns 0
,LEN(' ') --returns 0 too
You can use this in an expression like this:
CASE WHEN LEN(#YourInputValue)=0 THEN NULL ELSE #YourInputValue END
I am facing a strange problem while using postgresql. I have a table with columns as id, data, day_data. I am firing a simple query
select * from tablename where id = 'someid';
However, when I am modifying the query to
select * from tablename where day_data = 'somedata';
Both the columns are primary key of the table and both have a data type of chracter varying (255). This, is a very strange behavior and I am not able to make any head or tail out of it. Any help will be appreciated.
My guess is that you've got trailing spaces in your data values. TRIM() removes trailing whitespace. One way to find out would be SELECT '"' || day_data || '"' FROM tablename;, which will enclose each value including leading and trailing whitespace with quotes.
I have that procedure which returns rows associated by ID with passed argument, i.e 1,5,7,9
ALTER PROCEDURE [dbo].[get_data]
#MyCodes as varchar(max) = ''
AS
BEGIN
DECLARE #query as nvarchar(max)
set #query = 'SELECT name FROM user WHERE id IN (#p_MyCodes)'
exec SP_EXECUTESQL #query,
N'#p_MyCodes varchar(max)',
#p_MyCodes = #MyCodes
END
That procedure generates an error : Error converting data type varchar to numeric. when I pass as an argument e.g. 3,7,5
What's wrong ?
I don't think this is going to accomplish what you are expecting it to. The error you are getting is because it can't convert the string '3,7,5' to a number (note that it is NOT trying to parse out your individual values).
Two ways to get what you want:
1) Create a table value function that takes a CSV string and returns the results (I'm sure there are many on the web; Here's a related question: Split function equivalent in T-SQL?). This is nice because you can get rid of the SP_EXECUTESQL sproc call entirely; Your query becomes:
SELECT name FROM user where id IN (SELECT value FROM dbo.f_Split(#p_MyCodes))
2) Change your set to something like:
set #query = 'SELECT name FROM user WHERE id in (' + #p_MyCodes + ')'
I don't recommend #2, it offers a SQL injection hole.
You cannot pass the ID list as parameter. You could create the SQL statement by concatenating:
set #query = 'SELECT name FROM user WHERE id IN (' + #MyCodes + ')'
exec SP_EXECUTESQL #query
Though, this disables any kind of execution plan re-usage and enables SQL injection
A better solution would be to split the list into a temp table (or table variable) and using a JOIN. Last year, I wrote a blog post about different ways to split strings in T-SQL:
http://florianreischl.blogspot.com/2009/09/high-performance-string-split-functions.html
You can't use a comma separated string with the in operator, you have to use the actual values. So, you either have to split the string up and put the values in a temporary table, or concatenate the string into the query:
set #query = 'SELECT name FROM user WHERE id IN (' + #p_MyCodes + ')'
Note that this opens up a potential security hole for SQL injection. You should not do this if you don't have full control over where the string comes from.