Ecto insert_or_update creating multiple inserts? - postgresql

My code is resulting in a rare double or triple insert into the database and I am at a loss as to why. It is very difficult to reproduce but I can look at the timestamps to see the created at time is basically the same when it happens. I believe it only occurs when the CardMeta is not already found.
I figure I need to add a unique key or wrap it in a transaction.
def get_or_create_meta(user, card) do
case Repo.all(from c in CardMeta, where: c.user_id == ^user.id,
where: c.card_id == ^card.id) do
[] ->
%CardMeta{}
metas ->
hd metas
end
end
def bury(user, card) do
get_or_create_meta(user, card)
|> Repo.preload([:card, :user])
|> CardMeta.changeset(%{last_seen: DateTime.utc_now(), user_id: user.id, card_id: card.id,
learning: false, known: false, prev_interval: 0})
|> Repo.insert_or_update
end
Edit: adding changeset source
def changeset(struct, params \\ %{}) do
struct
|> cast(params, [:last_seen, :difficulty, :prev_interval, :due, :known, :learning,
:user_id, :card_id])
|> assoc_constraint(:user)
|> assoc_constraint(:card)
end
Calling bury from the controller
def update(conn, %{"currentCardId" => card_id, "command" => command}) do
# perform some update on card
card = Repo.get!(Card,card_id)
user = Guardian.Plug.current_resource(conn)
case command do
"fail" ->
SpacedRepetition.fail(user, card)
"learn" ->
SpacedRepetition.learn(user, card)
_ ->
SpacedRepetition.bury(user, card)
end
sendNextCard(conn, user)
end
Edit:
I noticed the last_seen field is microseconds different between duplicated rows, whereas the create_at field does not have that resolution. Thus I suspect the insert_or_update call is fine, but the controller is firing twice before the DB updates. This could be something on the client side, which I don't want to think about. So I am just going to add a unique key.

As an alternative to #aliCna's answer, if you don't want to change the primary key on CardMeta, you can put a unique index constraint in the database with a migration:
defmodule YourApp.Repo.Migrations.AddCardMetaUniqueIndex do
use Ecto.Migration
def change do
create unique_index(
:card_meta,
[:card_id, :user_id],
name: :card_meta_unique_index)
end
end
Which you can then handle in your changeset to produce nice errors if conflicts occur:
def changeset(struct, params \\ %{}) do
struct
|> cast(params, [:last_seen, :difficulty, :prev_interval, :due, :known, :learning,
:user_id, :card_id])
|> assoc_constraint(:user)
|> assoc_constraint(:card)
|> unique_constraint(:user_id, name: :card_meta_unique_index)
end

I believe you can solve this by adding a composite primary key on user_id and card_id
defmodule Anything.CardMeta do
use Anything.Web, :model
#primary_key false
schema "card_meta" do
field :user_id, :integer, primary_key: true
field :card_id, :integer, primary_key: true
. . .
timestamps()
end
end
If this does't solve your problem please add your data model here!

Related

composite unique constraint error while updating the changeset

I have a schema two_fa_details where answer and question_id are the fields and both are unique together..
Now when I am trying to insert data into it first it gets inserted but updating it next time isn't working..
It says constraint error.
I have a function set_two_factor_details written for updating table..
The function works fine for inserting the data very firsat time..but when iam updating it...its not working..i have a PUT API for this function.
this is my migration file for schema two_fa_details
def change do
create table(:two_fa_details) do
add :answer, :string
add :userprofile_id, references(:user_profile, on_delete: :nothing)
add :question_id, references(:questions, on_delete: :nothing)
timestamps()
end
create index(:two_fa_details, [:userprofile_id])
create index(:two_fa_details, [:question_id])
create unique_index(:two_fa_details, [:userprofile_id, :question_id], name: :user_twofa_detail)
end
here is a snippet of code
def set_twofactor_details(client_id, twofa_records) do
user = Repo.get_by(UserProfile, client_id: client_id)
twofa_records = Enum.map(twofa_records, &get_twofa_record_map/1)
Enum.map(twofa_records, fn twofa_record ->
Ecto.build_assoc(user, :two_fa_details)
|> TwoFaDetails.changeset(twofa_record)
end)
|> Enum.zip(0..Enum.count(twofa_records))
|> Enum.reduce(Ecto.Multi.new(), fn {record, id}, acc ->
Ecto.Multi.insert_or_update(acc, String.to_atom("twfa_record_#{id}"), record)
end)|>IO.inspect()
|> Ecto.Multi.update(
:update_user,
Ecto.Changeset.change(user, two_factor_authentication: true, force_reset_twofa: false)
)
|> Repo.transaction()|>IO.inspect()
|> case do
{:ok, _} ->
{:ok, :updated}
{:error, _, changeset, _} ->
error_string = get_first_changeset_error(changeset)
Logger.error("Error while updating TWOFA: #{error_string}")
{:error, 41001, error_string}
end
end
the output should be basically updating the table and returning two fa details updated message.
but in the logs its showing constraint error.please help me with this..Iam new to elixir.
{:error, :twfa_record_0,
#Ecto.Changeset<
action: :insert,
changes: %{answer: "a", question_id: 1, userprofile_id: 1},
errors: [
unique_user_twofa_record: {"has already been taken",
[constraint: :unique, constraint_name: "user_twofa_detail"]}
],
data: #Accreditor.TwoFaDetailsApi.TwoFaDetails<>,
valid?: false
>, %{}}
[error] Error while updating TWOFA: `unique_user_twofa_record` has already been taken
You wrote:
the output should be basically updating the table and returning two fa details updated message.
But the code returns:
#Ecto.Changeset<
action: :insert,
changes: %{answer: "a", question_id: 1, userprofile_id: 1},
errors: [
unique_user_twofa_record: {"has already been taken",
[constraint: :unique, constraint_name: "user_twofa_detail"]}
],
data: #Accreditor.TwoFaDetailsApi.TwoFaDetails<>,
valid?: false
>
Look how it says action: :insert. So you are not updating, but inserting, which explain the error.
insert_or_update will only update a record if the record was loaded from the database. In your code, you are building records from scratch, and therefore they will always be an insert. You need to use Repo.get or similar to fetch them before passing them to the changeset so you can finally call insert_or_update.
I tried doing it by using upserts for ecto
and it worked.
here is a snippet of code to refer
Ecto.Multi.insert_or_update(acc, String.to_atom("twfa_record_#{id}"), record,
on_conflict: :replace_all_except_primary_key,
conflict_target: [:userprofile_id, :question_id] )

Self referencing association in Phoenix using Ecto

So I've just started fiddeling around with Phoenix, and Elixir. So I have reached the point were I am trying to get a working rest-api endpoint to work with a prerequisite JSON.
So I have this module:
defmodule MyApp.Housing.Part do
use Ecto.Schema
import Ecto.Changeset
#primary_key {:id, :integer, []}
schema "parts" do
field :level, :integer
field :title, :string
belongs_to :parent, MyApp.Housing.Part
has_many :children, MyApp.Housing.Part, foreign_key: :parent_id
timestamps()
end
def changeset(part, params \\ %{}) do
part
|> cast(params, [:title, :level, :id, :parent_id])
|> put_assoc(:children, required: false)
|> put_assoc(:parent, required: false)
|> validate_required([:title, :level, :id])
end
end
And the module in which the table is created
defmodule MyApp.Repo.Migrations.CreateParts do
use Ecto.Migration
def change do
create table(:parts, primary_key: false) do
add :id, :integer, primary_key: true
add :title, :string
add :level, :integer
add :parent_id, references(:parts)
add :children, references(:parts)
timestamps()
end
create index(:parts, [:children])
create index(:parts, [:parent_id])
end
end
The inteded functionallity is for a part to be able to have multiple children but only one parent. And these are defined in a JSON like this:
{"id": 10,
"title": "Matt",
"level": 0,
"children": [],
"parent_id": null}
So my problem is the following:
"changeset" requires that the incoming object looks like {"part":{}}otherwise the ActionClauseError gets thrown.
When defining the object as above I get the error children":["is invalid"]. And I can't figure out how to get a valid one, if I did I probably could figure out the problem.
I might take the wrong approach here but would gladly accept any help.
As mentioned by #steve-pallen, it's not necessary to store any references to children in the database. Determining whether or not a Part is a parent or child, as well as which Parts are its children or which Part is its parent can be determined fully by the parent_id field.
You described in your question that each Part "can only have one parent, but multiple children". It's not explicit in your question how many levels the relationship allows: i.e. can a Part be both a parent and a child? In which case, there would be potentially infinite levels of nesting:
part1
|- part2
|- part3
|- part4
In this case, part1 is the parent of part2, part2 is itself the parent of part3, etc. I'm going to assume for my answer that there is no limit to the amount of nesting.
Given this case, your schema definition is 100% correct:
belongs_to :parent, MyApp.Housing.Part
has_many :children, MyApp.Housing.Part, foreign_key: :parent_id
I think the primary issue is with your changeset function. Remember that with put_assoc/3, it's expected that all the models referenced by parent and children already exist in the DB (see the docs for cast_assoc/3). For simplicity I suggest that you don't use put_assoc or cast_assoc and instead manage each model in isolation. If you change your changeset function to this (I've removed id since it's not necessary):
def changeset(part, params \\ %{}) do
part
|> cast(params, [:title, :level, :parent_id])
|> validate_required([:title, :level])
end
Then you can build the nested relationship I showed above by doing 4 inserts in isolation (much easier to reason about, and probably more in line with how you'd handle DB updates from a form or script):
part1 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part1", level: 0, parent_id: nil})
|> Repo.insert!()
part2 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part2", level: 0, parent_id: part1.id})
|> Repo.insert!()
part3 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part3", level: 0, parent_id: part2.id})
|> Repo.insert!()
part4 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part4", level: 0, parent_id: part3.id})
|> Repo.insert!()
Assuming we want to get part2, we can load it along with its parent and children like this:
part2 = Repo.preload(part2, [:parent, :children])
# part2.parent == %MyApp.Housing.Part{title: "part1", ...}
# part2.children == [%MyApp.Housing.Part{title: "part3", ...}]
Hope this helps!
The first thing you need to fix is the migration. You don't want the children field since that is a has_many relationship and is handled by the parent_id field in the children. It should look like this:
defmodule MyApp.Repo.Migrations.CreateParts do
use Ecto.Migration
def change do
create table(:parts, primary_key: false) do
add :id, :integer, primary_key: true
add :title, :string
add :level, :integer
add :parent_id, references(:parts)
timestamps()
end
create index(:parts, [:parent_id])
end
end
Handling the children in the changeset depends on a couple things.
What does the incoming payload look like when there are children?
Will there be new children in the children list or just existing children?

On conflict not updating back to default values

I have a users table like this
create table(:users) do
add(:email, :citext)
add(:email_confirmation_token, :string)
add(:email_confirmed_at, :utc_datetime)
add(:deleted_at, :utc_datetime)
timestamps()
end
and I have User schema as
schema "users" do
field(:email, :string)
field(:deleted_at, :utc_datetime)
field(:email_confirmed_at, :utc_datetime)
field(:email_confirmation_token, :string, default: ExApi.Utils.Random.string())
timestamps()
end
Now if I insert record with on_conflict option I want to reset deleted_at but deleted_at is never passed to changeset. I used following line of code
attrs = %{"email" => "tanweer.shahzaad#gmail.com", "deleted_at" => nil}
cs = %User{} |> User.changeset(attrs)
ExApi.Repo.insert(cs, on_conflict: :replace_all, conflict_target: :email)
Any workaround which will reset deleted_at will be very much appreciated.
Problem: if deleted_at is already set, I want to reset it back to nil on insert

How to perform atom update in embedded schema model with `jsonb_set`?

How to update only one key in map, I would like to perform it by jsonb_set like here: stackoverflow example or in transaction to avoid potential conflicts in database, is it possible with Ecto?
defmodule MySuperApp.Profile do
use MySuperApp.Model
schema "profiles" do
field :name, :string
embeds_one :settigns, MySuperApp.Settigns
end
def changeset(struct, params) do
struct
|> change
|> put_embed(:settigns, MySuerApp.Settigns.changeset(model, params))
end
end
defmodule MySuperApp.Settigns do
use MySuperApp.Model
#settigns %{socket: true, page: true, android: false, ios: false}
embedded_schema do
field :follow, :boolean
field :action, :map, default: #settigns
end
def changeset(struct, _params) do
# I would like to update only web key and leave old keys
model |> change(action: %{web: false}) # this will override old map -> changes: %{action: %{web: false}
end
end
No. Ecto currently does not support partial updates of the embeds with the high-level API (like changesets).
You could achieve this by using raw SQL queries through Ecto.Adapters.SQL.query/4 or in more recent versions Repo.query/3.

How to update one sub-document in an embedded list with Ecto?

I have a document with an embedded list of sub-docs. How do I update/change one particular document in the embedded list with Ecto?
defmodule MyApp.Thing do
use MyApp.Model
schema "things" do
embeds_many :users, User
end
end
defmodule MyApp.User do
use MyApp.Model
embedded_schema do
field :name, :string
field :email, :string
field :admin, :boolean, default: false
end
end
defmodule MyApp.Model do
defmacro __using__(_) do
quote do
use MyApp.Web, :model
#primary_key {:id, :binary_id, autogenerate: true}
#foreign_key_type :binary_id # For associations
end
end
end
My solution so far is to generate a list of all users except the one I want to update and make a new list of the one user's changeset and the other users and then put_embed this list on the thing. It works but it feels like there must be a more elegant solution to this.
user = Enum.find(thing.users, fn user -> user.id == user_id end)
other_users = Enum.filter(thing.users, fn user -> user.id != user_id end)
user_cs = User.changeset(user, %{email: email})
users = [user_cs | other_users]
thing
|> Ecto.Changeset.change
|> Ecto.Changeset.put_embed(:users, users)
|> Repo.update
EDIT: I just discovered a serious pitfall with this "solution". The untouched users get updated as well which can be a problem with concurring calls (race condition). So there has to be another solution.