I'm not clear on how to index embedded structs stored as JSONB with Ecto2/Postgres 9.4+
I have a schema with two embedded structs using embeds_one and embeds_many. They are ecto :map fields represented in Postgres as JSONB. I am wondering how I can be sure they are indexed (using Gin?) for speedy queries? I am not sure if this happens automagically, if I need to add an index to my migration or if I need to do it manually using psql etc..
Just looking for clarification on how this works.
Thanks!
defmodule App.Repo.Migrations.CreateClient
def change do
create table(:clients) do
add :name, :string
add :settings, :map
add :roles, {:array, :map}, default: []
timestamps()
end
// This works for normal schema/model fields
create index(:clients, [:name], unique: true, using: :gin)
// BUT CAN I INDEX MY EMBEDS HERE?
// GUESS:
create index(:clients, [:settings], using: :gin)
end
end
defmodule App.Client do
schema "client" do
field :name, :string
embeds_one :settings, Settings // single fixed schema "Settings" model
embeds_many :roles, Role // array of "Role" models
end
end
defmodule Settings do
use Ecto.Model
embedded_schema do // ALSO
field :name, :string // are types relevant?
field :x_count, :integer // stored as strings (INDEXED?)
field :is_active, :boolean // deserialized via cast?
end
end
defmodule Role do
use Ecto.Model
embedded_schema do
field :token
field :display_english
field :display_spanish
end
end
I think you just need to add this:
create index(:clients, [:name], unique: true, using: :gin)
to your migration file.
Or if the index sql statement is gonna be complicated, you could do it with execute so it would be something like this:
execute("CREATE INDEX clients_name_index ON clients USING GIN (name)")
I have not tested it but I believe it should work.
Related
So I've just started fiddeling around with Phoenix, and Elixir. So I have reached the point were I am trying to get a working rest-api endpoint to work with a prerequisite JSON.
So I have this module:
defmodule MyApp.Housing.Part do
use Ecto.Schema
import Ecto.Changeset
#primary_key {:id, :integer, []}
schema "parts" do
field :level, :integer
field :title, :string
belongs_to :parent, MyApp.Housing.Part
has_many :children, MyApp.Housing.Part, foreign_key: :parent_id
timestamps()
end
def changeset(part, params \\ %{}) do
part
|> cast(params, [:title, :level, :id, :parent_id])
|> put_assoc(:children, required: false)
|> put_assoc(:parent, required: false)
|> validate_required([:title, :level, :id])
end
end
And the module in which the table is created
defmodule MyApp.Repo.Migrations.CreateParts do
use Ecto.Migration
def change do
create table(:parts, primary_key: false) do
add :id, :integer, primary_key: true
add :title, :string
add :level, :integer
add :parent_id, references(:parts)
add :children, references(:parts)
timestamps()
end
create index(:parts, [:children])
create index(:parts, [:parent_id])
end
end
The inteded functionallity is for a part to be able to have multiple children but only one parent. And these are defined in a JSON like this:
{"id": 10,
"title": "Matt",
"level": 0,
"children": [],
"parent_id": null}
So my problem is the following:
"changeset" requires that the incoming object looks like {"part":{}}otherwise the ActionClauseError gets thrown.
When defining the object as above I get the error children":["is invalid"]. And I can't figure out how to get a valid one, if I did I probably could figure out the problem.
I might take the wrong approach here but would gladly accept any help.
As mentioned by #steve-pallen, it's not necessary to store any references to children in the database. Determining whether or not a Part is a parent or child, as well as which Parts are its children or which Part is its parent can be determined fully by the parent_id field.
You described in your question that each Part "can only have one parent, but multiple children". It's not explicit in your question how many levels the relationship allows: i.e. can a Part be both a parent and a child? In which case, there would be potentially infinite levels of nesting:
part1
|- part2
|- part3
|- part4
In this case, part1 is the parent of part2, part2 is itself the parent of part3, etc. I'm going to assume for my answer that there is no limit to the amount of nesting.
Given this case, your schema definition is 100% correct:
belongs_to :parent, MyApp.Housing.Part
has_many :children, MyApp.Housing.Part, foreign_key: :parent_id
I think the primary issue is with your changeset function. Remember that with put_assoc/3, it's expected that all the models referenced by parent and children already exist in the DB (see the docs for cast_assoc/3). For simplicity I suggest that you don't use put_assoc or cast_assoc and instead manage each model in isolation. If you change your changeset function to this (I've removed id since it's not necessary):
def changeset(part, params \\ %{}) do
part
|> cast(params, [:title, :level, :parent_id])
|> validate_required([:title, :level])
end
Then you can build the nested relationship I showed above by doing 4 inserts in isolation (much easier to reason about, and probably more in line with how you'd handle DB updates from a form or script):
part1 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part1", level: 0, parent_id: nil})
|> Repo.insert!()
part2 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part2", level: 0, parent_id: part1.id})
|> Repo.insert!()
part3 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part3", level: 0, parent_id: part2.id})
|> Repo.insert!()
part4 =
MyApp.Housing.Part.changeset(%MyApp.Housing.Part{}, %{title: "part4", level: 0, parent_id: part3.id})
|> Repo.insert!()
Assuming we want to get part2, we can load it along with its parent and children like this:
part2 = Repo.preload(part2, [:parent, :children])
# part2.parent == %MyApp.Housing.Part{title: "part1", ...}
# part2.children == [%MyApp.Housing.Part{title: "part3", ...}]
Hope this helps!
The first thing you need to fix is the migration. You don't want the children field since that is a has_many relationship and is handled by the parent_id field in the children. It should look like this:
defmodule MyApp.Repo.Migrations.CreateParts do
use Ecto.Migration
def change do
create table(:parts, primary_key: false) do
add :id, :integer, primary_key: true
add :title, :string
add :level, :integer
add :parent_id, references(:parts)
timestamps()
end
create index(:parts, [:parent_id])
end
end
Handling the children in the changeset depends on a couple things.
What does the incoming payload look like when there are children?
Will there be new children in the children list or just existing children?
How to update only one key in map, I would like to perform it by jsonb_set like here: stackoverflow example or in transaction to avoid potential conflicts in database, is it possible with Ecto?
defmodule MySuperApp.Profile do
use MySuperApp.Model
schema "profiles" do
field :name, :string
embeds_one :settigns, MySuperApp.Settigns
end
def changeset(struct, params) do
struct
|> change
|> put_embed(:settigns, MySuerApp.Settigns.changeset(model, params))
end
end
defmodule MySuperApp.Settigns do
use MySuperApp.Model
#settigns %{socket: true, page: true, android: false, ios: false}
embedded_schema do
field :follow, :boolean
field :action, :map, default: #settigns
end
def changeset(struct, _params) do
# I would like to update only web key and leave old keys
model |> change(action: %{web: false}) # this will override old map -> changes: %{action: %{web: false}
end
end
No. Ecto currently does not support partial updates of the embeds with the high-level API (like changesets).
You could achieve this by using raw SQL queries through Ecto.Adapters.SQL.query/4 or in more recent versions Repo.query/3.
I have a document with an embedded list of sub-docs. How do I update/change one particular document in the embedded list with Ecto?
defmodule MyApp.Thing do
use MyApp.Model
schema "things" do
embeds_many :users, User
end
end
defmodule MyApp.User do
use MyApp.Model
embedded_schema do
field :name, :string
field :email, :string
field :admin, :boolean, default: false
end
end
defmodule MyApp.Model do
defmacro __using__(_) do
quote do
use MyApp.Web, :model
#primary_key {:id, :binary_id, autogenerate: true}
#foreign_key_type :binary_id # For associations
end
end
end
My solution so far is to generate a list of all users except the one I want to update and make a new list of the one user's changeset and the other users and then put_embed this list on the thing. It works but it feels like there must be a more elegant solution to this.
user = Enum.find(thing.users, fn user -> user.id == user_id end)
other_users = Enum.filter(thing.users, fn user -> user.id != user_id end)
user_cs = User.changeset(user, %{email: email})
users = [user_cs | other_users]
thing
|> Ecto.Changeset.change
|> Ecto.Changeset.put_embed(:users, users)
|> Repo.update
EDIT: I just discovered a serious pitfall with this "solution". The untouched users get updated as well which can be a problem with concurring calls (race condition). So there has to be another solution.
I've just started working Elixir & Phoenix today, i am trying to add Ecto as a mapper, but i'm having some trouble using time.
This is my model.
schema "users" do
field :name, :string
field :email, :string
field :created_at, :datetime, default: Ecto.DateTime.local
field :updated_at, :datetime, default: Ecto.DateTime.local
end
I'm trying to set the created_at and updated_at per default, but when i try to compile this, i get the following error.
== Compilation error on file web/models/user.ex ==
** (ArgumentError) invalid default argument `%Ecto.DateTime{day: 13, hour: 19, min: 47, month: 2, sec: 12, year: 2015}` for `:datetime`
lib/ecto/schema.ex:687: Ecto.Schema.check_default!/2
lib/ecto/schema.ex:522: Ecto.Schema.__field__/4
web/models/board.ex:9: (module)
(stdlib) erl_eval.erl:657: :erl_eval.do_apply/6
There is not much help to get in the documentation, what would be the correct way to do this?
Defaults fields names are :inserted_at and :updated_at but you can merge with your own field names, passing a keyword list
schema "users" do
field :name, :string
field :email, :string
timestamps([{:inserted_at,:created_at}])
end
:datetime is the native Postgres data type for, well a datetime; this data type maps to a two-elements Elixir tuple ({{yy, mm, dd}, {hh, mm, ss}}). An %Ecto.DateTime{} struct is not a two-elements tuple, hence the compilation error.
You may want to set the type of your fields to Ecto.DateTime, it should all work seamlessly.
Here is the relevant documentation about primitive types and non-primitive types.
PS you may also want to have a look at Ecto.Schema.timestamps/1, which is macro that expands to basically what you wrote manually (it adds the created_at and updated_at fields and it let's you choose what type they should be, defaulting to Ecto.DateTime):
schema "users" do
field :name, :string
field :email, :string
timestamps
end
You could also consider having the default not be in the schema, but in the migration: "created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP"
I am attempting to order the results of a query by the value of a specific embedded document, but even with what seems to be a valid set of options and using the $elemMatch operator, my results are coming back in natural order.
My model is composed of Cards, which embeds_many :card_attributes, which in turn reference a specific CardAttributeField and contain an Integer value. I would like to be able to order a collection of Cards by that value.
I am able to isolate a collection of Cards which have a CardAttribute referencing a specific CardAttributeField like this:
cards = Card.where(:card_attributes.elem_match => {
:card_attribute_field_id => card_attribute_field.id
})
If I knew the order in which the card_attributes were set, I could use MongoDB array notation, like this:
cards.order_by(['card_attributes.0.value', :asc])
This does deliver my expected results in test scenarios, but it won't work in the real world.
After much messing around, I found a syntax which I thought would allow me to match a field without using array notation:
cards.asc(:'card_attributes.value'.elem_match => {
:card_attribute_field_id => card_attribute_field.id
})
This produced a set of options on the resulting Mongoid::Criteria which looked like:
{:sort=>{"{#<Origin::Key:0x2b897548 #expanded=nil, #operator=\"$elemMatch\", #name=:\"card_attributes.value\", #strategy=:__override__, #block=nil>=>{:card_attribute_field_id=>\"54c6c6fe2617f55611000068\"}}"=>1}}
However, the results here came back in the same order regardless or whether I called asc() or desc().
Is there any way to do what I'm after? Am I taking the wrong approach, or do I have a mistake in my implementation? Thanks.
Simplified, my model is:
class Card
include Mongoid::Document
# various other fields
has_many :card_attribute_fields
embeds_many :card_attributes do
def for_attribute_field card_attribute_field
where(:card_attribute_field_id => card_attribute_field.id)
end
end
end
class CardAttributeField
include Mongoid::Document
belongs_to :card
field :name, type: String
field :default_value, type: String
field :description, type: String
end
class CardAttribute
include Mongoid::Document
embedded_in :card
field :card_attribute_field_id, type: Moped::BSON::ObjectId
field :value, type: Integer
end