Ecto: How to update all records with a different random number - postgresql

I have a table in postgresql and i want to update the value of column "points" for all the records with a random number. In other languages we could loop over all the db records but how can i do it with ecto? I tried this:
Repo.all(from u in User, update: User.changeset(u, %{points: :rand.uniform(100)}))
but it outputs the following error:
== Compilation error in file lib/hello_remote/user.ex ==
** (Ecto.Query.CompileError) malformed update `User.changeset(u, %{points: :rand.uniform(100)})` in query expression, expected a keyword list with set/push/pop as keys with field-value pairs as values
expanding macro: Ecto.Query.update/3
lib/hello_remote/user.ex:30: HelloRemote.User.update_points/0
expanding macro: Ecto.Query.from/2
lib/hello_remote/user.ex:30: HelloRemote.User.update_points/0
(elixir 1.10.4) lib/kernel/parallel_compiler.ex:304: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7
I've also tried this:
from(u in User)
|> Repo.update_all(set: [points: Enum.random(0..100)])
but it updates all the records with the same value

You can use fragment/1 with update_all/3, calling a PostgreSQL function to calculate the random values, for example:
update(User, set: [points: fragment("floor(random()*100)")])
|> Repo.update_all([])

I don't think it is possible with the update_all function. The function was created to update many rows with the same value.
You can create a loop and update each record separately, it's not nice but a working solution.
User
|> Repo.all()
|> Enum.each(fn user -> update_user(user) end)
def update_user(user) do
user
|> Ecto.Changeset.cast(%{"points" => Enum.random(0..100)}, [:points])
|> Repo.update!()
end
If you would like to do it one call you can construct a raw SQL query and use that.

Related

psycopg2 - TypeError: not all arguments converted during string formatting

I'm using python 3.8 and psycopg2
I'm trying to insert a registry in the database.
I have a function that formats a query and send as result a list with 2 values, one is the query and the other the values.
I made a test and put a fixed value with the exact value of the result list query[1] and worked without error, but when I use the query[1] as values instead the value by itself I got this error:
TypeError: not all arguments converted during string formatting
At my log I have these values for the query list, result of my query construction function.
['INSERT INTO country (code, name, flag, update_time) VALUES(%s,%s,%s,%s)', "('US', 'USA', 'https://example.com/flags/us.svg', 1596551810)"]
query[0]
INSERT INTO country (code, name, flag, update_time) VALUES(%s,%s,%s,%s)
query[1]
('US', 'USA', 'https://example.com/flags/us.svg', 1596551810)
This is the code snipet
`
cursor = connection.cursor()
query_insert = query[0]
query_values = tuple(query[1])
cursor.execute(query_insert,(query_values))
I tried to put it as tuple, use parentheses, but error persists.
If I put the value of the query[1] at my code,as values, work well, so I suppose that the error is at the values part of the cursor.execute parameters.
Any help is welcome !

In Elixir with Postgres, how can I have the database return the enum values which are NOT in use?

I have an EctoEnum.Postgres:
# #see: https://en.wikipedia.org/wiki/ISO_4217
defmodule PricingEngine.Pricing.CurrencyEnum do
#options [
:AED,
:AFN,
# snip...
:ZWL
]
use EctoEnum.Postgres,
type: :currency,
enums: #options
def values, do: #options
end
This enum has been included in our Postgres database
We also have a structure:
defmodule PricingEngine.Pricing.Currency do
use Ecto.Schema
import Ecto.Changeset
schema "currencies" do
field(:currency, PricingEngine.Pricing.CurrencyEnum)
timestamps()
end
#doc false
def changeset(currency, attrs) do
currency
|> cast(attrs, [:currency])
|> validate_required([:currency])
|> unique_constraint(:currency)
end
end
We can currently successfully use the following functions to figure out which currencies are active/used:
def active_currency_isos do
Repo.all(select(Currency, [record], record.currency))
end
defdelegate all_currency_isos,
to: CurrencyEnum,
as: :values
def inactive_currency_iso do
Pricing.all_currency_isos() -- Pricing.active_currency_isos()
end
This works, but I'm led to believe this could be more efficient if we just asked the database for this information.
Any idea(s) how to do this?
If you want to get a list of all the used enums you should just do a distinct on the currency field. This uses the Postgres DISTINCT ON operator:
from(c in Currency,
distinct: c.currency,
select: c.currency
)
This will query the table, unique by the currency column, and return only the currency column values. You should get an array of all of the enums that exist in the table.
There are some efficiency concerns with doing it this way which could be mitigated by materialized views, lookup tables, in-memory cache etc. However, if your data set isn't extremely large, you should be able to use this for a while.
Edit:
Per the response, I will show how to get the unused enums.
There are 2 ways to do this.
Pure SQL
This query will get all of the used ones and do a difference from the entire set of available enums. The operator we use to do this is EXCEPT and you can get a list of all available enums with enum_range. I will use unnest to turn the array of enumerated types into individual rows:
SELECT unnest(enum_range(NULL::currency)) AS unused_enums
EXCEPT (
SELECT DISTINCT ON (c.name) c.name
FROM currencies c
)
You can execute this raw SQL in Ecto by doing this:
Ecto.Adapters.SQL.query!(MyApp.Repo, "SELECT unnest(...", [])
From this you'll get a Postgresx.Result that you'll have to get the values out of:
result
|> Map.get(:rows, [])
|> List.flatten()
|> Enum.map(&String.to_existing_atom/1)
I'm not really sure of a way to code this query up in pure Ecto, but let me know if you figure it out.
In Code
You can do the first query that I posted before with distinct then do a difference in the code.
query = from(c in Currency,
distinct: c.currency,
select: c.currency
)
CurrencyEnum.__enums__() -- Repo.all(query)
Either way is probably negligible in terms of performance so it's up to you.

Ecto query to grab all values that satisfy all values in array_aggregator not just any?

Wonder if someone can help me with an array aggregator issue
I’ve got a query that does a join using a joining table then it filters down all values that are inside a given array and filters out values that are in another array.
The code looks like this:
Product
|> join(:inner, [j], jt in "job_tech", on: j.id == jt.product_id)
|> join(:inner, [j, jt], t in Tech, on: jt.ingredient_id == t.id)
|> group_by([j], j.id)
|> having_good_ingredients(good_ingredients)
|> not_having_bad_ingredients(bad_ingredients)
With having_good_ingredients look like this:
def having_good_ingredients(query, good_ingredients) do
if Enum.count(good_ingredients) > 0 do
query
|> having(fragment("array_agg(t2.name) && (?)::varchar[]", ^good_ingredients))
else
query
end
end
This works but it’ll grab all values that satisfy any of the values in the good_stacks array where I want them to only satisfy if all of the stacks work
i.e. if I’ve got [A, C] in my array, I want to return values that have A AND C, not just A and not just C.
Anyone have any ideas?
I believe you want to use the #> operator, instead of the overlap && operator:
having(fragment("array_agg(t2.name) #> (?)::varchar[]", ^good_ingredients))
Reference: https://www.postgresql.org/docs/current/functions-array.html#ARRAY-OPERATORS-TABLE

Using Postgres UPDATE ... FROM in Ecto without raw SQL

Based on an Elixir thread from last year, I was able to write a raw SQL query to bulk update records with the values from an unrelated table. However, I would like to be able to generate this query using Ecto.
In the example below, assume there are two tables, cats and dogs, and the cats table has a foreign key (dog_id). I want to link a dog to a cat.
The code below is how I'm doing this manually with Elixir and raw SQL:
cat_ids = [1,2,3] # pretend these are uuids
dog_ids = [4,5,6] # ... uuids
values =
cat_ids
|> Enum.zip(dog_ids)
|> Enum.map(fn {cat_id, dog_id} ->
"('#{cat_id}'::uuid, '#{dog_id}'::uuid)"
end)
|> Enum.join(", ")
sql = """
UPDATE cats as a
SET dog_id = c.dog_id
from (values #{values}) as c(cat_id, dog_id)
where c.cat_id = a.id;
"""
Repo.query(sql)
Is there a way to move this to Repo.update_all or some use of fragments so I'm not manually building the query?
Sure, you can use Ecto syntax, but it ain't much different in my opinion, tho you have to use a Schema, for example in my application I have an user authentication and that's how we update the token:
def update_token(user_id, token) do
Repo.transaction(fn ->
from(t in UserAuthentication, where: t.user_id == ^to_string(user_id))
|> Repo.update_all(set: [token: token])
end
and the UserAuthentication schema looks more or less like:
defmodule MyApp.UserAuthentication do
use Ecto.Schema
import Ecto.Changeset
schema "user_authentication" do
field(:user_id, :integer)
field(:token, :string)
timestamps()
end
def changeset(%__MODULE__{} = user, attrs) do
user
|> cast(attrs, [:user_id, :token])
|> validate_required([:user_id, :token])
end
end
This is good for data validation and works in any DB you attach.

Elixir Ecto - PostgreSQL jsonb Functions

I am in the process of converting a Ruby on Rails API over to Elixir and Phoenix. In my Postgres database, I have a table with a jsonb column type. One of the keys in the json is an array of colors. For example:
{"id": 12312312, "colors": ["Red", "Blue", "White"]}
What I am trying to do from Ecto is query my table for all records that contain the colors Red or Blue. Essentially, recreate this query:
select * from mytable where data->'colors' ?| array['Red', 'Blue']
I'm having some difficulties constructing this query with Ecto. Here is what I have:
Note: "value" will be a pipe delimited list of colors
def with_colors(query, value) do
colors = value
|> String.split("|")
|> Enum.map(fn(x) -> "'#{x}'" end)
|> Enum.join(", ")
# colors should look like "'Red', 'Blue'"
from c in query,
where: fragment("data->'colors' \\?| array[?]", ^colors))
end
This is currently not working as expected. I am having issues with the replacement question mark, as it seems to wrap additional quotes around my field. What is the proper way to do this use fragment? Or maybe there is a better way?
I'm going to run into this problem again because I'm also going to have to recreate this query:
select * from mytable where data->'colors' #> '["Red", "Blue"]'
I have found a solution to my problem.
def with_colors(query, value) do
colors = value
|> String.split("|")
from c in query,
where: fragment("data->'colors' \\?| ?", ^colors))
end