I can do this:
SELECT id, name, data FROM public.app
WHERE data=ANY(string_to_array('25,2,3,15', ',')::character varying[])
this is work correctly. But data field varchar and it will hold data which comma separated. In summary,
searching data= 25,2,3,15
db.field data=3,0,2,15,3,15 etc. both side include comma separated data.
I try this:
SELECT id, name, data FROM public.app
WHERE string_to_array(data,',')=ANY(string_to_array('25,2,3,15', ',')::character varying[])
but it is not working.
You are currently comparing two arrays using ANY when that is incorrect.
If you want to see if the two arrays are identical in their contents, you can simply remove ANY:
string_to_array(data,',') = (string_to_array('25,2,3,15', ',')::character varying[])
If you want to see if there is overlap between the two arrays, you use &&:
string_to_array(data,',') && (string_to_array('25,2,3,15', ',')::character varying[])
Additional operators are available here: https://www.postgresql.org/docs/current/static/functions-array.html
Related
I have an Excel sheet that has two seperate columns of data that I need combined for my table in Qlik. I know it would be easy to combine the two in the Excel document, but because this is a data feed I would prefer not to do any extra work. One column has the first name and the other the last. Thank you.
I tried to concat but that did not work.
It sounds like what you're trying to achieve is string concatenation, where you'd combine two or more strings together. It'd be the same concept for fields, as long as their values can be coerced to a string type. In Qlik, this is done very simply by using the ampersand & operator. You can use this in three places:
Data Manager
If done in the Data Manager, you are creating a new calculated field. This can be done by editing the table that you're loading in from Excel, selecting Add field >> Calculated field, and then using an expression like this:
first_name & ' ' & last_name
What that expression is doing is taking the field first_name and concatenating it's values with a space ' ' and then concatenating the values of the last_name field.
So your new field, which I'm naming full_name here, would look like this:
first_name
last_name
full_name
Chris
Schaeuble
Chris Schaeuble
Stefan
Stoichev
Stefan Stoichev
Austin
Spivey
Austin Spivey
Here's what the data manager would look like:
Then after you load the data, you will have a new field with the combined names:
Data Load Editor
Doing this in the Data Load Editor will also result in a new field and is the exact same expression (see line 6):
Chart expression
The other option you have is to use this expression "on-the-fly" in a chart without creating a new column in the app data model like the first two options. Just use that same expression from above in a chart field expression and you'll get the same result:
I have a mapping data flow with a derived column, where I want to use a column pattern for matching against an array of columns using in()
The data flow is executed in a pipeline, where I set the parameter $owColList_md5 based on a variable that I've populated from a single-line CSV file containing a comma-separated string
If I have a single column name in the CSV file/variable encapsuled in single quotes and have the "Expression" checkbox ticked, it works.
The problem is to get it to work with multiple columns. There seems to be parsing problems having multiple items in the variable each encapsuled in single-quotes, or potentially with the comma separating them. This often causes errors executing the data flow with messages like "store is not defined" etc
I've tried having ''col1'',''col2'' and "col1","col2" (2x single quotes and double quotes) in the CSV file. I've also tried having the file without quotes, trying to replace the comma with escaped quotes (using ) in the derived column pattern expression with no luck.
How do you populate this array in the derived column based on the data flow parameter which is based on the comma-separated string in the CSV file / variable from the pipeline with column names in a working way?
While array types are not supported as data flow parameters, passing in a comma-separated string can work if you use the instr() function to match.
Say you have two columns, col1 and col2. Pass in a parameter with value '"col1","col2"'.
Then use instr($<yourparamname>, '"' + name + '"') > 0 to see if the column name exists within the string you pass in. Note: You do not need double quotes, but the can be useful if you have column names that are subsets of other columns names such as id1 and id11.
Hope this helps!
The field name is message, table name is log.
Data Examples:
Values for message:
"(wsname,cmdcode,stacode,data,order_id) values (hyd-l904149,2,1,,1584425657892);"
"(wsname,cmdcode,stacode,data,order_id) values (hyd-l93mt54,2,1,,1584427657892);"
(command_execute,order_id,workstation,cmdcode,stacode,application_to_kill,application_parameters) values (kill, 1583124192811, hyd-psag314, 10, 2, tsws.exe, -u production ); "
and in log table i need to get separated column wsname with values as hyd-l904149 and hyd-l93mt54 and hyd-psag314, column cmdcode with values as 2,2 and 10 and column stacode with values as 1,1 and 2, e.g.:
wsname cmdcode stacode
hyd-l904149 2 1
hyd-l93mt54 2 1
hyd-psag314 10 2
Use regexp_matches to extract left and right part of values clause, then regexp_split_to_array to split these parts by commas, then filter rows containing wsname using = any(your_array) construct, then select required columns from array.
Or - alternative solution - fix data to be syntactically valid part of insert statement, create auxiliary tables, insert data into them and then just select.
As in comment section I mentioned about inbuilt function in posgressql
split_part(string,delimiter, field_number)
http://www.sqlfiddle.com/#!15/eb1df/1
As the json capabilities of the un-supported version 9.3 are very limited, I would install the hstore extension, and then do it like this:
select coalesce(vals -> 'wsname', vals -> 'workstation') as wsname,
vals -> 'cmdcode' as cmdcode,
vals -> 'stacode' as stacode
from (
select hstore(regexp_split_to_array(e[1], '\s*,\s*'), regexp_split_to_array(e[2], '\s*,\s*')) as vals
from log l,
regexp_matches(l.message, '\(([^\)]+)\)\s+values\s+\(([^\)]+)\)') as x(e)
) t
regexp_matches() splits the message into two arrays: one for the list of column names and one for the matching values. These arrays are used to create a key/value pair so that I can access the value for each column by the column name.
If you know that the positions of the columns are always the same, you can remove the use of the hstore type. But that would require quite a huge CASE expression to test where the actual columns appear.
Online example
With a modern, supported version of Postgres, I would use jsonb_object(text[], text[]) passing the two arrays resulting from the regexp_matches() call.
I have a table which looks like below
date | tags | name
------------+-----------------------------------+--------
2018-10-08 | 100.21.100.1, cpu, del ZONE1
2018-10-08 | 100.21.100.1, mem, blr ZONE2
2018-10-08 | 110.22.100.3, cpu, blr ZONE3
2018-10-09 | 110.22.100.3, down, hyd ZONE2
2018-10-09 | 110.22.100.3, down, del ZONE1
I want to select the name for those rows which have certain strings in the tags column
Here column tags has values which are strings containing comma separated values.
For example I have a list of strings ["down", "110.22.100.3"]. Now if I do a look up into the table which contains the strings in the list, I should get the last two rows which have the names ZONE2, ZONE1 respectively.
Now I know there is something called in operator but I am not quite sure how to use it here.
I tried something like below
select name from zone_table where 'down, 110.22.100.3' in tags;
But I get syntax error.How do I do it?
You can do something like this.
select name from zone_table where
string_to_array(replace(tags,' ',''),',')#>
string_to_array(replace('down, 110.22.100.3',' ',''),',');
1) delete spaces in the existing string for proper string_to_array separation without any spaces in the front using replace
2)string_to_array converts your string to array separated by comma.
3) #> is the contains operator
(OR)
If you want to match as a whole
select name from zone_table where POSITION('down, 110.22.100.3' in tags)!=0
For separate matches you can do
select name from zone_table where POSITION('down' in tags)!=0 and
POSITION('110.22.100.3' in tags)!=0
More about position here
We can try using the LIKE operator here, and check for the presence of each tag in the CSV tag list:
SELECT name
FROM zone_table
WHERE ', ' || tags LIKE '%, down,%' AND ', ' || tags LIKE '%, 110.22.100.3,%';
Demo
Important Note: It is generally bad practice to store CSV data in your SQL tables, for the very reason that it is unnormalized and makes it hard to work with. It would be much better design to have each individual tag persisted in its own record.
demo: db<>fiddle
I would do a check with array overlapping (&& operator):
SELECT name
FROM zone_table
WHERE string_to_array('down, 110.22.100.3', ',') && string_to_array(tags,',')
Split your string lists (the column values and the compare text 'down, 110.22.100.3') into arrays with string_to_array() (of course if your compare text is an array already you don't have to split it)
Now the && operator checks if both arrays overlap: It checks if one array element is part of both arrays (documentation).
Notice:
"date" is a reserved word in Postgres. I recommend to rename this column.
In your examples the delimiter of your string lists is ", " and not ",". You should take care of the whitespace. Either your string split delimiter is ", " too or you should concatenate the strings with a simple "," which makes some things easier (aside the fully agreed thoughts about storing the values as string lists made by #TimBiegeleisen)
Is there a way to set the filter in p6spy, such that it only logs "insert/delete/update" and NOT "select" SQL statements?
Documentation of p6spy mentions:
"P6Spy allows you to monitor specific tables or specific statement types"
An example they gave was the following:
An example showing capture of all
select statements, except the orders
table follows:
filter = true
# comma separated list of tables to include
include = select
# comma separated list of tables to exclude
exclude = orders
So I thought, there must be a way to include insert, delete, updates and exclude select... hence, I prepared my properties file like so:
filter = true
# comma separated list of tables to include
include = insert,update,delete
# comma separated list of tables to exclude
exclude = select
but that does not seem to work. Anyone with any suggestions??
The key to the answer is in the comments
# comma separated list of tables to include
include = select
select is a name of a table, not the type of a statement.
It seems impossible to filter by statement types (at least by select/update/delete) easily. You'll be able to do it by using
# sql expression to evaluate if using regex filtering
sqlexpression=
#allows you to use a regex engine or your own matching engine to determine
#which statements to log
stringmatcher=com.p6spy.engine.common.GnuRegexMatcher
The definition has changed. In short, you can use what the OP used i.e:
filter=true
# comma separated list of *******STRINGS******* to include
include=insert,update,delete
# comma separated list of *******STRINGS******* to exclude
exclude=select
Or if you want to have more control, you can use sqlexpression like so:
filter=true
sqlexpression=^(.*(from\\scustomers).*)$