Rethinking relational many-to-many relationships for MongoDB - mongodb

I am just starting a new Rails 3 project using Mongoid ORM for MongoDB. There is just one thing I can not get my head around, and that is how to effectively have a many-to-many relationship. Now there is a good chance that I may be approaching this problem wrong, but as far as I know, there is at least two containers in my project that need a many-to-many relationship. I would prefer to treat both models as "first class" models and allocate each with its own container.
This is the simplest way I can think to structure my many-to-many relationship:
// Javascript pseudo modeling
// -------------------- Apps
{
app: {
_id: "app1",
name: "A",
event_ids: ["event1","event2"]
}
}
{
app: {
_id: "app2",
name: "B",
event_ids: ["event1"]
}
}
// -------------------- Events
{
event: {
_id: "event1",
name: "Event 1",
}
}
{
event: {
_id: "event2",
name: "Event 2",
}
}
As far as I can tell this is the minimum amount of information need to infer a many-to-many relationship. My assumption is that I might have to have a map reduce procedure to determine what apps belong to an event. I would also have to write post commit/save hooks on Event to update App.event_ids if an app is added to or removed from an event model.
Am I on the right track here? If someone has any Mongoid or Mongomapper code examples of a many-to-many relationship working, could you please share.

Your structure can work and you don't need a mapreduce function to determine what apps belong to an event. You can query the app collection on an eventid. You can index field collection.event_ids.
If you don't want to search apps on an eventid but on a event name, you will need to add that event name to the app collection (denormalization). That means that you also have to update the app collection when the name of an event changes. I don't know if that happens very often?
You often have to denormalize when you use MongoDB, so you don't store the minimal amount of information but you store some things "twice".

I was able to implement this design using Mongoid. I wrote extensive tests and I was able to get my solution working; however, I am not satisfied with my implementation. I believe that my implementation would be a difficult to maintain.
I'm posting my non-elegant solution here. Hopefully, this will help someone with the start of a better implementation.
class App
include Mongoid::Document
field :name
references_one :account
references_many :events, :stored_as => :array, :inverse_of => :apps
validates_presence_of :name
end
class Event
include Mongoid::Document
field :name, :type => String
references_one :account
validates_presence_of :name, :account
before_destroy :remove_app_associations
def apps
App.where(:event_ids => self.id).to_a
end
def apps= app_array
unless app_array.kind_of?(Array)
app_array = [app_array]
end
# disassociate existing apps that are not included in app_array
disassociate_apps App.where(:event_ids => self.id).excludes(:id => app_array.map(&:id)).to_a
# find existing app relationship ids
existing_relationship_ids = App.where(:event_ids => self.id, :only => [:id]).map(&:id)
# filter out existing relationship ids before making the new relationship to app
push_apps app_array.reject { |app| existing_relationship_ids.include?(app.id) }
end
def push_app app
unless app.event_ids.include?(self.id)
app.event_ids << self.id
app.save!
end
end
def disassociate_app app
if app.event_ids.include?(self.id)
app.event_ids -= [self.id]
app.save!
end
end
def push_apps app_array
app_array.each { |app| push_app(app) }
end
def disassociate_apps app_array
app_array.each { |app| disassociate_app(app) }
end
def remove_app_associations
disassociate_apps apps
end
end

Related

Getting ElasticSearch document fields inside of loaded records in searchkick

Is it possible to get ElasticSearch document fields inside of loaded AR records?
Here is a gist that illustrates what I mean: https://gist.github.com/allomov/39c30905e94c646fb11637b45f43445d
In this case I want to avoid additional computation of total_price after getting response from ES. The solution that I currently see is to include the relationship and run total_price computation for each record, which is not so optimal way to perform this operation, as I see it.
result = Product.search("test", includes: :product_components).response
products_with_total_prices = result.map do |product|
{
product: product
total_price: product.product_components.map(&:price).compact.sum
}
end
Could you please tell if it is possible to mix ES document fields into AR loaded record?
As far as I'm aware it isn't possible to get a response that merges the document fields into the loaded record.
Usually I prefer to completely rely on the data in the indexed document where possible (using load: false as a search option), and only load the AR record(s) as a second step if necessary. For example:
result = Product.search("test", load: false).response
# If you also need AR records, could do something like:
product_ids = result.map(&:id)
products_by_id = {}
Product.where(id: product_ids).find_each do |ar_product|
products_by_id[ar_product.id] = ar_product
end
merged_result = result.map do |es_product|
es_product[:ar_product] = products_by_id[es_product.id]}
end
Additionally, it may be helpful to retrieve the document stored in the ES index for a specific record, which I would normally do by defining the following method in your Product class:
def es_document
return nil unless doc = Product.search_index.retrieve(self).presence
Hashie::Mash.new doc
end
You can use select: true and the with_hit method to get the record and the search document together. For your example:
result = Product.search("test", select: true)
products_with_total_prices =
result.with_hit.map do |product, hit|
{
product: product,
total_price: hit["_source"]["total_price"]
}
end

Mongoid/MongoDB: Order a query by the value of an embedded document?

I am attempting to order the results of a query by the value of a specific embedded document, but even with what seems to be a valid set of options and using the $elemMatch operator, my results are coming back in natural order.
My model is composed of Cards, which embeds_many :card_attributes, which in turn reference a specific CardAttributeField and contain an Integer value. I would like to be able to order a collection of Cards by that value.
I am able to isolate a collection of Cards which have a CardAttribute referencing a specific CardAttributeField like this:
cards = Card.where(:card_attributes.elem_match => {
:card_attribute_field_id => card_attribute_field.id
})
If I knew the order in which the card_attributes were set, I could use MongoDB array notation, like this:
cards.order_by(['card_attributes.0.value', :asc])
This does deliver my expected results in test scenarios, but it won't work in the real world.
After much messing around, I found a syntax which I thought would allow me to match a field without using array notation:
cards.asc(:'card_attributes.value'.elem_match => {
:card_attribute_field_id => card_attribute_field.id
})
This produced a set of options on the resulting Mongoid::Criteria which looked like:
{:sort=>{"{#<Origin::Key:0x2b897548 #expanded=nil, #operator=\"$elemMatch\", #name=:\"card_attributes.value\", #strategy=:__override__, #block=nil>=>{:card_attribute_field_id=>\"54c6c6fe2617f55611000068\"}}"=>1}}
However, the results here came back in the same order regardless or whether I called asc() or desc().
Is there any way to do what I'm after? Am I taking the wrong approach, or do I have a mistake in my implementation? Thanks.
Simplified, my model is:
class Card
include Mongoid::Document
# various other fields
has_many :card_attribute_fields
embeds_many :card_attributes do
def for_attribute_field card_attribute_field
where(:card_attribute_field_id => card_attribute_field.id)
end
end
end
class CardAttributeField
include Mongoid::Document
belongs_to :card
field :name, type: String
field :default_value, type: String
field :description, type: String
end
class CardAttribute
include Mongoid::Document
embedded_in :card
field :card_attribute_field_id, type: Moped::BSON::ObjectId
field :value, type: Integer
end

Sails.js/Waterline cascading delete for a many-to-many association

As shown in that stackoverflow answer, having no support for cascading (cascading deletes in particular) in Waterline there is a workaround for one-to-many associations by using the afterDestroy (or afterUpdate for soft-delete) lifecycle callback and deleting the associated records with a second query. This is possible via ManyModel.destroy({ oneModel: _.pluck(destroyedOneModels, "id") }) from within afterDestroy.
How do we do that for a many-to-many relationship (having in mind that a junction table is used internally and we have to delete records from it)?
I did some tests using the Pet / User example from the documentation with sails 0.11.
Writting this lifecycle callback in the Pet model deletes all the users associated to a pet before deleting it.
// models/Pet.js
module.exports = {
attributes: {
name:'string',
color:'string',
owners: {
collection: 'user',
via: 'pets'
}
},
beforeDestroy: function(criteria, cb) {
// Destroy any user associated to a deleted pet
Pet.find(criteria).populate('owners').exec(function (err, pets){
if (err) return cb(err);
pets.forEach(function(recordToDestroy) {
User.destroy({id: _.pluck(recordToDestroy.owners, 'id')}).exec(function(err) {
console.log('The users associated to the pet ' + recordToDestroy.name + ' have been deleted');
});
});
cb();
})
}
};
I couldn't do it in the afterDestroy lifecycle callback because the many-to-many properties of the deleted records are missing there.
Waterline is deleting the records of the junction table automatically.
The problem with this feature is that it probably would delete too much things if some pets share some owners. Following the example of the documentation, if you delete the pet Rainbow Dash, you will delete the users Mike, Cody and Gabe, and the pets Pinkie Pie and Applejack would be orphans.
If you define a many-to-many relation like this one but you know that the pets cannot have any owner in common, then it works fine. Otherwise, you should add a test to check that you will not make another pet an orphan.

Can I join the column names with search_related in DBIx?

I have a DBIx Class schema where I have;
A Device that has many Interfaces
An Interface has many Rules Applied
Each Rule has many Rule Entries.
I want to search for all of the Rule Entries for a Particular device name and Rule Name.
I am still learning DBIx so I don’t know if this is even the most efficient way.
I am doing this like so;
my $rs = $self->search( { devicename => ‘DeviceA’ } )->search_related('interfaces')->search_related(’Rules’, { rulename => ‘RuleA’ } )->search_related(‘RuleEntries’, {},
{ columns => [qw/source destination port/], result_class => 'DBIx::Class::ResultClass::HashRefInflator'} );
What I am trying to do is get the ‘RuleName’ as a column of my result set.
at the moment I’m getting all of the Rule Entries for DeviceA with a RuleName on an interface called RuleA, The columns returned are
‘source destination port’.
I want this to look like
‘rulename source destination port’
As you are already restricting the rule name it doesn't make sense to query it from the database.
Besides that you should always search for objects of the type you want to get back, in your case that's rule entries:
my $rs = $schema->resultset('Rule_Entries')->search({
'rel_device.name' => 'DeviceA',
'rel_rule.name' => 'Rule',
},{
columns => [ 'rel_rule.name', 'me.source', 'me.destination', 'me.port' ],
join => { rel_rule => { rel_interface => 'rel_device' }},
});
It seems your doing something very similar what I do: storing firewall rules. You might want to have the rule directly related to the device and the interface being an optional attribute of the rule because some vendors don't have interface specific rules (Checkpoint).

Injecting relationships in DBIx::Class

I have a handful of DBIx::Class::Core objects that model various database tables.
For some of those models (those that have a 'queue' column), I have another class inject subs (basically, to 'move' the model object along it's queue states).
I'd like to also have that class inject has_many relationships ala
class($name)->has_many('queue_history','MySchema::Result::QueueHistory',
{ 'foreign.record_id'=>'self.id' },
{ where => { type => $name }} );
but I can't seem to get the relationships to register properly (keep getting "No Such Relationship" errors - however, when calling the relationship method on the sources provides back the relationship).
Any clues as to what's wrong?
After some digging around, the following works:
$class = $schema->class($name)->has_many('queue_history','MySchema::Result::QueueHistory',
{ 'foreign.record_id'=>'self.id' },
{ where => { type => $name }} );
$schema->unregister_source($name);
$schema->register_class($name,$class);
The key being the unregister/register methods in order to generate all the appropriate other methods that get added by having a new has_many relationship.