Elixir: How to search database data with double spaces - postgresql

In my front end, the data display has only single spaces. However, when I filter my data in search bar, it yields no result. It turns out the data actually has double spaces. But when its being repo.all, the double spaces becomes single spaces. I need to search that data.
I have a regexp which i found which i tried in pgadmin which works. I need to replicate it in my elixir datatable, for specifically, integrating it in ilike function.
The regexp is
SELECT trim(regexp_replace(name, '\s+', ' ', 'g')) as col_name
FROM table where col_name = 'TEST DATA'

I think a better approach might be to use % in place of spaces when searching, I find that this gives me generally better results. Here are the docs for ILIKE in Postgres, but I'll explain with an example. The % character used in ILIKE matches any string of characters (including the empty string).
SELECT * FROM t1 WHERE col_name ILIKE '%test%data%';
The query above will match the example that you've given. Granted, it will also match strings like "testdata", "test data", "something test something data", but I think that in general this approach will return results that a user would expect when entering a search like "test data".
A helper function to do this transformation from user input to a string suitable for Ecto's ilike would look like this:
defmodule SearchHelpers do
def to_ilike_search_string(search_string) do
joined_string =
search_string
|> String.split()
|> Enum.join("%")
"%#{joined_string}%"
end
end
# SearchHelpers.to_ilike_search_string("test data")
# => "%test%data%"
And you could use it in a query like this:
ilike_search_string = SearchHelpers.to_ilike_search_string("test data")
query = from p in Post,
where: ilike(p.content, ^ilike_search_string)
results = Repo.all(query)
Good luck, hope this helps.

Related

Is there a ts (text search) function would return found string instead of boolean?

I am using PostgreSQL to find out the matched string in the article by using tsvector and tsquery.
I read the PostgreSQL manual 12.3 Controlling Text Search but nothing could help me to get the exact output I wanted.
Query:
SELECT ts_headline('english',
'The most common type of search
is to find all documents containing given query terms
and return them in order of their similarity to the
query.',
to_tsquery('query & similarity'),
'StartSel = <, StopSel = >');
ts_headline output
The most common type of search
is to find all documents containing given <query> terms
and return them in order of their <similarity> to the
<query>.
I'm looking for the only string as mentioned below:
query, similarity
If you pick delimiters for StartSel and StopSel that you are sure do not exist elsewhere in the string, then it is pretty easy to do this with a regexp.
SELECT distinct regexp_matches[1] from
regexp_matches(
ts_headline('english',
'The most common type of search
is to find all documents containing given query terms
and return them in order of their similarity to the
query.',
to_tsquery('query & similarity'),
'StartSel = <, StopSel = >'
),
'<(.*?)>','g'
);

Firebird query with string concatenation and like in the where clause doesn't work

I have a users table with first name and last name. I want to find a user for a string like 'clark kent'.
I do this
SELECT * FROM user WHERE fisrt_name || ' ' || last_name LIKE '%clark kent%'
but I don't have any result.
Why? If I search for 'clark' or 'kent' it works.
Solution:
SELECT *
FROM user
WHERE TRIM("first_name") || ' ' || TRIM("last_name") LIKE '%clark kent%';
Per the OP's comments, the difficulty was that first_name and last_name were CHAR and not VARCHAR fields, meaning that the fields are space-padded to their length. That is, the string 'clark' stored as CHAR(8) is 'clark[sp][sp][sp]'.
When concatenating the fields together to form a "full name", then, the resulting string had unexpected spaces: 'clark[sp][sp][sp][sp]kent[sp][sp][sp][sp]'.
(As an aside, in strict SQL, CHAR columns are rather counter-intuitive. They compare without regard to space padding, but the LIKE operator does not ignore space padding. See the Firebird FAQ.)
Try
SELECT *
FROM user
WHERE upper(fisrt_name) LIKE '%CLARK%'
and upper(last_name) LIKE '%KENT%'

Scala Play Framework Anorm SQL.on disable wrapping replacements with ' '

Whenever I replace placeholders in the SQL query using on it surrounds the replacement with '', is there a way to prevent this?
It means I can't do things like
SQL("SELECT * FROM {table} blah").on("table" -> tabletouse)
because it wraps the table name with '' which causes an SQL syntax error.
you could certainly combine both approaches, using the format function for data you don't want to be escaped
SQL(
"""
select %s from %s
where
name = {name} and
date between {start} and {end}
order by %s
""".format(fields, table, order)
).on(
'name -> name,
'start -> startDate,
'end -> endDate
)
Just take into account that the data you are sending using the format function should NOT come from user input, otherwise it should be properly sanitized
You cannot do what you are trying. Anorm's replacement is based on PreparedStatements. Meaning all data will automatically be escaped, meaning you cannot use replacement for :
table names,
column names,
whatever operand, SQL keyword, etc.
The best you can do here is a String concatenation (and what is really a bad way in my opinion) :
SQL("SELECT * FROM " + tabletouse + " blah").as(whatever *)
PS : Checkout this question about table names in PreparedStatements.

Select from any of multiple values from a Postgres field

I've got a table that resembles the following:
WORD WEIGHT WORDTYPE
a 0.3 common
the 0.3 common
gray 1.2 colors
steeple 2 object
I need to pull the weights for several different words out of the database at once. I could do:
SELECT * FROM word_weight WHERE WORD = 'a' OR WORD = 'steeple' OR WORD='the';
but it feels ugly and the code to generate the query is obnoxious. I'm hoping that there's a way I can do something like (pseudocode):
SELECT * FROM word_weight WHERE WORD = 'a','the';
You are describing the functionality of the in clause.
select * from word_weight where word in ('a', 'steeple', 'the');
If you want to pass the whole list in a single parameter, use array datatype:
SELECT *
FROM word_weight
WHERE word = ANY('{a,steeple,the}'); -- or ANY('{a,steeple,the}'::TEXT[]) to make explicit array conversion
If you are not sure about the value and even not sure whether the field will be an empty string or even null then,
.where("column_1 ILIKE ANY(ARRAY['','%abc%','%xyz%']) OR column_1 IS NULL")
Above query will cover all possibility.

Postgres INSERT INTO query bug?

What's wrong with the following Postgres query?
INSERT INTO kayak.airports(name, x, y, city) VALUES( $name, $x, $y, $city)
WHERE airport_id='$airport_id
EDIT (thanks Donnie for helping me make progress) :
I tried:
$query="UPDATE kayak.airports SET name=$name, x = $x, y = $y, city = $city
WHERE airports.airport_id='$airport_id'";
It said "column 'Brisbane' doesn't exist" (Brisbane is the first city to be inserted. ) I took out everything between SET and WHERE except for "x=$x" and those were successfully inserted. Ditto for "y=$y". When only leaving in name=$name it says
"Query failed: ERROR: syntax error at or near "International" LINE 1: UPDATE kayak .airports SET name=Brisbane International WHERE... ^"
Your query string is not quoted. Do not use PHP variable interpolation for building SQL queries, because this will leave your script or application vulnerable to an SQL injection attack.
Instead, use parameterized queries. Thus, your query above becomes:
$query = 'UPDATE kayak.airports SET name = $1, x = $2, y = $3, city = $4'.
'WHERE airports.airport_id = $5';
Then, you will use the parameterized query calling function pg_query_paramsto pass the required parameters:
$result = pg_query_params($query, $parameters)
Where $parameters is an array of parameters.
Also note that the $query string is single-quoted, because the $n placeholders are not there for interpolation. This prevents any mistakes (such as typing a real variable name by bumping a letter first) and eliminates any possibility of SQL injection.
You're attempting to insert literal values. A where clause makes no sense.
For insert, you can only use where in an insert ... select to limit what the select is returning.
Perhaps you actually want to update an existing record?
For me, if I get an error that a column doesn't exist, it's usually a tipoff that I've quoted something incorrectly (or not at all).
This is borne out by the error message from your attempt to update only the name field:
ERROR: syntax error at or near "International" LINE 1:
(The carat should point right to the problem area in the query.)
The value you are passing to the name field in your UPDATE statement needs to be quoted, just like the value you're passing to airport_id. (I'm going to take a wild guess that x and y are integers, which wouldn't require quoting, which is why you don't get an error when you try to update just those field.) (I'm going to take another wild guess that the value you pass to city will need to be quoted too, but you will probably figure that out shortly. :) )
The end result expanded UPDATE should look something like this:
UPDATE kayak.airports
SET name='Brisbane International', x = 123, y = 456, city = 'Brisbane'
WHERE ...