Querying a table with UUID using Ecto.Query gives Ecto.Query.CastError - postgresql

So I've got the following migration:
create table(:things, primary_key: false) do
add :id, :uuid, primary_key: true
add :state, :string
timestamps()
end
Which has the following schema:
#primary_key {:id, Ecto.UUID, autogenerate: true}
#derive {Phoenix.Param, key: :id}
schema "things" do
field :state, :string
timestamps()
end
And upon trying the following query in the REPL, I get an Ecto.Query.CastError:
iex(8)> q = from s in Thing, where: s.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: s
#Ecto.Query<from o in App.Thing,
where: o.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: o>
iex(9)> Repo.get!(Thing, q)
** (Ecto.Query.CastError) /project/deps/ecto/lib/ecto/repo/queryable.ex:341: value `#Ecto.Query<from o in App.Thing, where: o.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: o>` in `where` cannot be cast to type Ecto.UUID in query:
from o in App.Thing,
where: o.id == ^#Ecto.Query<from o in App.Thing, where: o.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: o>,
select: o
(elixir) lib/enum.ex:1811: Enum."-reduce/3-lists^foldl/2-0-"/3
(elixir) lib/enum.ex:1357: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
(elixir) lib/enum.ex:1811: Enum."-reduce/3-lists^foldl/2-0-"/3
(ecto) lib/ecto/repo/queryable.ex:124: Ecto.Repo.Queryable.execute/5
(ecto) lib/ecto/repo/queryable.ex:37: Ecto.Repo.Queryable.all/4
(ecto) lib/ecto/repo/queryable.ex:78: Ecto.Repo.Queryable.one!/4
(stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
(iex) lib/iex/evaluator.ex:219: IEx.Evaluator.handle_eval/5
(iex) lib/iex/evaluator.ex:200: IEx.Evaluator.do_eval/3
(iex) lib/iex/evaluator.ex:178: IEx.Evaluator.eval/3
I'm not sure this is the proper way of using UUID's with Ecto, I've looked around but there are several people doing it differently. I'm using Ecto 2.1 and Postgres 10.0 with postgrex.
Any tips on how to get Ecto querying correctly and casting the UUID?
EDIT: The ID I'm giving in the query is an actual existing ID I copied from the database.

I find the problem you're experiencing is a bit confusing at first, but I think it works quite correctly - the problem is in how you're querying the database.
If you have your query ready, you need to use Repo.all(query), so your code should be:
q = from s in Thing, where: s.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: s
Repo.all(q)
You can query the database to look for specific record by primary key, and this is when you can use Repo.get!/3.
Try this:
Repo.get!(Thing, "ba34d9a0-889f-4999-ac23-f04c7183f2ba")
When you pass your q as a second argument to get!, it will try to cast it to UUID, but in fact, query is not cast-able, which is explained in the exception:
** (Ecto.Query.CastError) /project/deps/ecto/lib/ecto/repo/queryable.ex:341: value `#Ecto.Query<from o in App.Thing, where: o.id == "ba34d9a0-889f-4999-ac23-f04c7183f2ba", select: o>` in `where` cannot be cast to type Ecto.UUID in query:
Can you see the whole #Ecto.Query<> is enclosed in ``?
You can find more in the documentation:
Repo#all/2
Repo#get!/3
Hope that helps!

Related

Expression for Lookup activity result when stored procedure returns scalar value

I'm using a Lookup activity that returns an scalar result and I want to use that result to build a dinamic query using a concat expression. However, I'm receiving an error complaining that my SQL query is not well formated. This is how I'm building the query:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow, ') AND e.EndDate IS NOT NULL;')
I expect that to return a query like this:
SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId WHERE v.CscadaEventId IN(2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,2378594,2379357) AND e.EndDate IS NOT NULL;
I had see some examples where the lookup return multiple columns, and the right expression is activity('LookupUnfinishedAlarms').output.firstRow.myColumnName but what about when the lookup activity return an scalar value, as in my case?
This is the full error so far:
You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near
&apos;\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859&apos;
at line
6,Source=Microsoft.DataTransfer.Runtime.GenericOdbcConnectors,''Type=System.Data.Odbc.OdbcException,Message=ERROR
[42000] [Microsoft][MariaDB] You have an error in your SQL syntax;
check the manual that corresponds to your MariaDB server version for
the right syntax to use near
&apos;\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859&apos;
at line 6,Source=MariaDBODBC_sb64.dll
Ok, just for the records, I found the solution. The expression must be:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow.output, ') AND e.EndDate IS NOT NULL;')
So, the default column becomes output

sqlalchemy to create temporary table

I created a temporary table with sqlalchemy (with an underlying postgres database) that is going to be joined with a database table. However, in some cases when a value is empty '' then postgres throws the error:
failed to find conversion function from unknown to text
SqlAlchemy assembles everything to the following context
[SQL: 'WITH temp_table AS \n(SELECT %(param_1)s AS id, %(param_2)s AS email, %(param_3)s AS phone)\n SELECT campaigns_contact.id, campaigns_contact.email, campaigns_contact.phone \nFROM campaigns_contact JOIN temp_table ON temp_table.id = campaigns_contact.id AND temp_table.email = campaigns_contact.email AND temp_table.phone = campaigns_contact.phone'] [parameters: {'param_1': 83, 'param_2': '', 'param_3': '+1234567890'}]
I assemble the temporary table as follows
stmts = []
for row in import_data:
row_values = [literal(row[value]).label(value) for value in values]
stmts.append(select(row_values))
subquery = union_all(*stmts)
subquery = subquery.cte(name="temp_table")
The problem seems to be the part here
...%(param_2)s AS email...
which after replacing the param_2 results in
...'' AS email...
which will cause the error mentioned above.
One way to solve the issue is to perform a cast
...''::text AS email...
However, I don't know how to perform ::text cast with sqlalchemy!?

group by error with postgres and pomm orm

I want to execute the following SQL query :
SELECT date, COUNT(id_customers)
FROM event
WHERE event_id = 3
GROUP BY date
When I try this query in my database, it works perfectly. But in my code I get an error which I can't resolve.
I use symfony2 with the orm pomm. It's Postgresql.
Here is my code :
$sql = "SELECT e.date, COUNT(id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
return $this->query($sql, [$eventId])->extract();
Here is the error :
request.CRITICAL: Uncaught PHP Exception InvalidArgumentException:
"No such field 'id'. Existing fields are {date, count}"
at /home/vagrant/sourcefiles/vendor/pomm-project/model-manager/sources/lib/Model/FlexibleEntity/FlexibleContainer.php line 64
{"exception":" [object] (InvalidArgumentException(code: 0): No such field 'id'.
Existing fields are {date, count}
at /home/vagrant/sourcefiles/vendor/pomm-project/model-manager/sources/lib/Model/FlexibleEntity/FlexibleContainer.php:64)"} []
So I tried to had the id in my select, by I get this error :
request.CRITICAL: Uncaught PHP Exception
PommProject\Foundation\Exception\SqlException: "
SQL error state '42803' [ERROR] ==== ERROR: column "e.id" must appear in the GROUP BY clause or be used in an aggregate function LINE 1: SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE ... ^
==== «PREPARE === SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE event_id = $1 GROUP BY e.date ===»." at /home/vagrant/sourcefiles/vendor/pomm-project/foundation/sources/lib/Session/Connection.php line 327 {"exception":"[object] (PommProject\Foundation\Exception\SqlException(code: 0): \nSQL error state '42803' [ERROR]\n====\nERROR: column \"e.id\" must appear in the GROUP BY clause or be used in an aggregate function\nLINE 1: SELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE ...\n ^\n\n====\n«PREPARE ===\nSELECT e.id, e.date, COUNT(id_customers) FROM event e WHERE event_id = $1 GROUP BY e.date\n ===». at /home/vagrant/sourcefiles/vendor/pomm-project/foundation/sources/lib/Session/Connection.php:327)"} []
The only thing that works is when I had the id in the group by, but this is not the result I want.
Someone can explain me why this is working in the database and not in the php ?
this is because you are fetching flexible entities without their primary key. There is an identity mapper behind the scene that ensure fetching twice the same entity will return the same instance.
In this case, you do not need to fetch entities (hence the extract after the query). So you can just use the QueryManager pooler to return converted arrays like the following:
$sql = "SELECT e.date, COUNT(id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
// Return an iterator that fetches converted arrays on demand:
return $this
->getSession()
->getQueryManager()
->query($sql, [$eventId])
;
i think its because the alias,
try this
$sql = "SELECT e.date, COUNT(e.id_customers) FROM event e WHERE event_id = $* GROUP BY e.date";
return $this->query($sql, [$eventId])->extract();
This is exactly how GROUP BY works in PostgreSQL:
When GROUP BY is present, it is not valid for the SELECT list
expressions to refer to ungrouped columns except within aggregate
functions, since there would be more than one possible value to return
for an ungrouped column.
It means that each field in your query either must be present in GROUP BY statement or handled by any of the aggregation functions. This is one of the differences between GROUP BY in MySQL and PostreSQL.
In other words you can add id at GROUP BY statement and do not worry about it ;)

Error when ignoring accents in full text search using Ecto and PostgreSQL

I have this search function in my controller:
def search(query, search_term) do
(from u in query,
where: fragment("(to_tsvector('portuguese', unaccent(?)) ## plainto_tsquery(?))", u.name, ^search_term),
order_by: fragment("ts_rank((to_tsvector('portuguese', unaccent(?)), plainto_tsquery(?))) DESC", u.name, ^search_term))
end
I'm getting this error:
ERROR (undefined_function): function unaccent(character varying) does not exist
I've also added the unaccented extension to PostgreSQL with:
CREATE EXTENSION unaccent;
How to make this work?

How to query Ecto 2.0 with UUID

I'm using Ecto 2.0.0-rc.4 And I have done this query that didn't work
def users do
Repo.all(
from u in User,
where: u.id == "93fd15fb-fe21-4a59-813d-f80447417a23",
select: u
)
end
The id is one in the database the error it shows is
** (Postgrex.Error) ERROR (character_not_in_repertoire): invalid byte sequence for encoding "UTF8": 0x93
[debug] QUERY ERROR db=8.2ms queue=0.2ms
SELECT u0."id", u0."full_name", u0."email", u0."encrypted_password", u0."settings", u0."organizations", u0."inserted_at", u0."updated_at" FROM "users" AS u0 WHERE (u0."id" = '��^U��!JY�=�^DGAz#') []
(ecto) lib/ecto/adapters/sql.ex:395: Ecto.Adapters.SQL.execute_and_cache/7
(ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
(ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4
Also have tried converting {:ok, id} = Ecto.UUID.dump "93fd15fb-fe21-4a59-813d-f80447417a23" to bitstring and query u.id == ^id but did not work-
Is this and Ecto issue.
It was my mistake; just needed to add the following import to my model (i.e: import Ecto.Query):
def all(org_id) do
Repo.all(
from l in Ledger,
where: l.organization_id == ^org_id
)
end