BSON::InvalidDocument: Cannot serialize an object into BSON - mongodb

I'm trying to follow along with http://mongotips.com/b/array-keys-allow-for-modeling-simplicity/
I have a Story document and a Rating document. The user will rate a story, so I wanted to create a many relationship to ratings by users as such:
class StoryRating
include MongoMapper::Document
# key <name>, <type>
key :user_id, ObjectId
key :rating, Integer
timestamps!
end
class Story
include MongoMapper::Document
# key <name>, <type>
timestamps!
key :title, String
key :ratings, Array, :index => true
many :story_ratings, :in => :ratings
end
Then
irb(main):006:0> s = Story.create
irb(main):008:0> s.ratings.push(Rating.new(user_id: '0923ksjdfkjas'))
irb(main):009:0> s.ratings.last.save
=> true
irb(main):010:0> s.save
BSON::InvalidDocument: Cannot serialize an object of class StoryRating into BSON.
from /usr/local/lib/ruby/gems/1.9.1/gems/bson-1.6.2/lib/bson/bson_c.rb:24:in `serialize' (...)
Why?

You should be using the association "story_rating" method for your push/append rather than the internal "rating" Array.push to get what you want to follow John Nunemaker's "Array Keys Allow For Modeling Simplicity" discussion. The difference is that with the association method, MongoMapper will insert the BSON::ObjectId reference into the array, with the latter you are pushing a Ruby StoryRating object into the Array, and the underlying driver driver cant serialize it.
Here's a test that works for me, that shows the difference. Hope that this helps.
Test
require 'test_helper'
class Object
def to_pretty_json
JSON.pretty_generate(JSON.parse(self.to_json))
end
end
class StoryTest < ActiveSupport::TestCase
def setup
User.delete_all
Story.delete_all
StoryRating.delete_all
#stories_coll = Mongo::Connection.new['free11513_mongomapper_bson_test']['stories']
end
test "Array Keys" do
user = User.create(:name => 'Gary')
story = Story.create(:title => 'A Tale of Two Cities')
rating = StoryRating.create(:user_id => user.id, :rating => 5)
assert_equal(1, StoryRating.count)
story.ratings.push(rating)
p story.ratings
assert_raise(BSON::InvalidDocument) { story.save }
story.ratings.pop
story.story_ratings.push(rating) # note story.story_ratings, NOT story.ratings
p story.ratings
assert_nothing_raised(BSON::InvalidDocument) { story.save }
assert_equal(1, Story.count)
puts Story.all(:ratings => rating.id).to_pretty_json
end
end
Result
Run options: --name=test_Array_Keys
# Running tests:
[#<StoryRating _id: BSON::ObjectId('4fa98c25e4d30b9765000003'), created_at: Tue, 08 May 2012 21:12:05 UTC +00:00, rating: 5, updated_at: Tue, 08 May 2012 21:12:05 UTC +00:00, user_id: BSON::ObjectId('4fa98c25e4d30b9765000001')>]
[BSON::ObjectId('4fa98c25e4d30b9765000003')]
[
{
"created_at": "2012-05-08T21:12:05Z",
"id": "4fa98c25e4d30b9765000002",
"ratings": [
"4fa98c25e4d30b9765000003"
],
"title": "A Tale of Two Cities",
"updated_at": "2012-05-08T21:12:05Z"
}
]
.
Finished tests in 0.023377s, 42.7771 tests/s, 171.1084 assertions/s.
1 tests, 4 assertions, 0 failures, 0 errors, 0 skips

Related

Ecto will not insert a :date field into the database - why?

I am facing an issue while learning Elixir & Ecto. The idea is to build a standard posts/comments page to understand how the basics work. I am at a point where I have schemas defined, a migration written and encounter an error when trying to insert data into the database (PostgreSQL) via the Repo. I have done a fair deal of web searching and documentation reading, which leads me to believe it's a scenario that should just work and I am making a stupid mistake somewhere, which I just can't see.
They are defined as follows:
lib/hello/schemas.ex
defmodule Hello.PostAuthor do
use Ecto.Schema
schema "post_authors" do
field :name, :string
end
end
defmodule Hello.CommentAuthor do
use Ecto.Schema
schema "comment_authors" do
field :name, :string
end
end
defmodule Hello.Comment do
use Ecto.Schema
schema "comments" do
has_one :author, Hello.CommentAuthor
field :date, :date
field :body, :string
end
end
defmodule Hello.Post do
use Ecto.Schema
schema "posts" do
has_one :author, Hello.PostAuthor
field :date, :date
field :body, :string
has_many :comments, Hello.Comment
end
end
as you can see, I have two fields with :date type - on post and comment schemas. The corresponding migration is as follows:
defmodule Hello.Repo.Migrations.CreatePosts do
use Ecto.Migration
def change do
create table(:post_authors) do
add :name, :string
end
create table(:comment_authors) do
add :name, :string
end
create table(:comments) do
add :author, references(:comment_authors)
add :date, :date
add :body, :string
end
create table(:posts) do
add :author, references(:post_authors), null: false
add :date, :date
add :body, :string
add :comments, references(:comments)
timestamps()
end
end
end
Now, when I start iex -S mix I can successfully create all structs:
iex(1)> post_author = %Hello.PostAuthor{name: "John"}
%Hello.PostAuthor{
__meta__: #Ecto.Schema.Metadata<:built, "post_authors">,
id: nil,
name: "John"
}
iex(2)> comment_author = %Hello.PostAuthor{name: "Adam"}
%Hello.PostAuthor{
__meta__: #Ecto.Schema.Metadata<:built, "post_authors">,
id: nil,
name: "Adam"
}
iex(3)> comment = %Hello.Comment{author: comment_author, date: ~D[2019-01-01], body: "this is a comment"}
%Hello.Comment{
__meta__: #Ecto.Schema.Metadata<:built, "comments">,
author: %Hello.PostAuthor{
__meta__: #Ecto.Schema.Metadata<:built, "post_authors">,
id: nil,
name: "Adam"
},
body: "this is a comment",
date: ~D[2019-01-01],
id: nil
}
iex(4)> post = %Hello.Post{author: post_author, date: ~D[2019-01-01], body: "this is a post", comments: [comment]}
%Hello.Post{
__meta__: #Ecto.Schema.Metadata<:built, "posts">,
author: %Hello.PostAuthor{
__meta__: #Ecto.Schema.Metadata<:built, "post_authors">,
id: nil,
name: "John"
},
body: "this is a post",
comments: [%Hello.Comment{
__meta__: #Ecto.Schema.Metadata<:built, "comments">,
author: %Hello.PostAuthor{
__meta__: #Ecto.Schema.Metadata<:built, "post_authors">,
id: nil,
name: "Adam"
},
body: "this is a comment",
date: ~D[2019-01-01],
id: nil
}],
date: ~D[2019-01-01],
id: nil
}
The problem arises when I call Hello.Repo.insert(post) (where post is the struct representing the Hello.Post schema). I receive what looks like serialization error:
iex(8)> Hello.Repo.insert(post) [debug] QUERY OK db=0.1ms
begin []
[debug] QUERY ERROR db=1.6ms
INSERT INTO "posts" ("body","date") VALUES ($1,$2) RETURNING "id" ["this is a post", ~D[2019-01-01]]
[debug] QUERY OK db=0.1ms
rollback []
** (DBConnection.EncodeError) Postgrex expected a binary, got ~D[2019-01-01]. Please make sure the value you are passing matches the definition in your table or in your query or convert the value accordingly.
(postgrex) lib/postgrex/type_module.ex:897: Postgrex.DefaultTypes.encode_params/3
(postgrex) lib/postgrex/query.ex:75: DBConnection.Query.Postgrex.Query.encode/3
(db_connection) lib/db_connection.ex:1148: DBConnection.encode/5
(db_connection) lib/db_connection.ex:1246: DBConnection.run_prepare_execute/5
(db_connection) lib/db_connection.ex:540: DBConnection.parsed_prepare_execute/5
(db_connection) lib/db_connection.ex:533: DBConnection.prepare_execute/4
(postgrex) lib/postgrex.ex:198: Postgrex.query/4
(ecto_sql) lib/ecto/adapters/sql.ex:666: Ecto.Adapters.SQL.struct/10
(ecto) lib/ecto/repo/schema.ex:651: Ecto.Repo.Schema.apply/4
(ecto) lib/ecto/repo/schema.ex:262: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
(ecto) lib/ecto/repo/schema.ex:916: anonymous fn/3 in Ecto.Repo.Schema.wrap_in_transaction/6
(ecto_sql) lib/ecto/adapters/sql.ex:898: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
(db_connection) lib/db_connection.ex:1415: DBConnection.run_transaction/4
This is where I am lost. Both the schema and the migration are expecting a :date . I believe that ~D[2019-01-01] is a date. PostgreSQL defines date as a 4 byte binary value. I am expecting Ecto.Adapters.Postgres to translate elixir date struct into the Postgres binary value. This is not happening. Why?
Struct itself is just raw data. You should go through Ecto.Changeset as shown in the documentation, specifically to all types to be cast to the respective DB types with Ecto.Changeset.cast/4.
The conversion will be done automagically, but you need to explicitly call cast/4 (hence the Changeset,) otherwise the adapter has no idea of how to convert your ecto types.

SugarCRM Rest API set_relationship between Contacts and Documents

I am trying to (link/set_relationship) between a document and a contact on SugarCRM. I am not sure how to construct the "name_value_list" specifically for this. At least that is what I believe to be wrong.
I have tried the following:
1.
'name_value_list': []
2.
'name_value_list' : [{
'name': "documents_contacts",
'value': 'Other',
}],
3.
'name_value_list': [{'table': "%s_%s" % (ModuleName, LinkedModuleName)},
{'fields': [
{"id": str(uuid.uuid1())},
{"date_modified": str(datetime.datetime.now())},
{"deleted": '0'},
{"document_id": RecordID},
{"contact_id": LinkedRecordID},
]
4.
'name_value_list':[{"%s_%s" % (ModuleName, LinkedModuleName): 'Other',
"id": str(uuid.uuid1()),
"date_modified": str(datetime.datetime.now()),
"deleted": '0',
"document_id": RecordID,
"contact_id": LinkedRecordID
}]
SugarCRM CE Version 6.5.20 (Build 1001)
SugarCRM v4_1 Rest API Documentation:
* Set a single relationship between two beans. The items are related by module name and id.
*
* #param String $session -- Session ID returned by a previous call to login.
* #param String $module_name -- name of the module that the primary record is from. This name should be the name the module was developed under (changing a tab name is studio does not affect the name that should be passed into this method)..
* #param String $module_id - The ID of the bean in the specified module_name
* #param String link_field_name -- name of the link field which relates to the other module for which the relationship needs to be generated.
* #param array related_ids -- array of related record ids for which relationships needs to be generated
* #param array $name_value_list -- The keys of the array are the SugarBean attributes, the values of the array are the values the attributes should have.
* #param integer $delete -- Optional, if the value 0 or nothing is passed then it will add the relationship for related_ids and if 1 is passed, it will delete this relationship for related_ids
* #return Array - created - integer - How many relationships has been created
* - failed - integer - How many relationsip creation failed
* - deleted - integer - How many relationships were deleted
* #exception 'SoapFault' -- The SOAP error, if any
*/
Method [ public method set_relationship ] {
- Parameters [7] {
Parameter #0 [ $session ]
Parameter #1 [ $module_name ]
Parameter #2 [ $module_id ]
Parameter #3 [ $link_field_name ]
Parameter #4 [ $related_ids ]
Parameter #5 [ $name_value_list ]
Parameter #6 [ $delete ]
}
}
Python 3.7
def SetRelationship(self, ModuleName, ModuleID, LinkFieldName, RelatedID):
method = 'set_relationship'
data = {
'session':self.SessionID,
'module_name':ModuleName,
'module_id':ModuleID,
'link_field_name':LinkFieldName,
'related_ids':[RelatedID, ]
}
response = json.loads(self.request(method, data))
SetRelationship('Documents', 'e9d22076-02fe-d95d-1abb-5d572e65dd46', 'Contacts', '2cdc28d8-763e-6232-2788-57f4e19a9ea0')
Result:
{'created': 0, 'failed': 1, 'deleted': 0}
Expected Result:
{'created': 1, 'failed': 0, 'deleted': 0}
You probably meant to call
SetRelationship('Documents', 'e9d22076-02fe-d95d-1abb-5d572e65dd46', 'contacts', '2cdc28d8-763e-6232-2788-57f4e19a9ea0')
Notice the lowercase contacts here, as the API expects the name of the link field in Documents, not the name of the module.
If that still doesn't fix the issue, check the sugarcrm.log and the php log for errors.

How use values mode in Orion?

Reading FIWARE-NGSI v2 Specification (http://telefonicaid.github.io/fiware-orion/api/v2/latest/)
In section Simplified Entity Representation
I couldn't test values mode as recomend. My test fail:
values mode. This mode represents the entity as an array of attribute
values. Information about id and type is left out. See example below.
The order of the attributes in the array is specified by the attrs URI
param (e.g. attrs=branch,colour,engine). If attrs is not used, the
order is arbitrary.
[ 'Ford', 'black', 78.3 ]
Where and how I referenced an entityID?
POST /v2/entities/Room1?options=values&attrs=branch,colour,engine
payload:
[ 'Ford', 'black', 78.3 ]
Answer:
{
"error": "MethodNotAllowed",
"description": "method not allowed"
}
POST /v2/entities?options=values
payload:
[ 'Ford', 'black', 78.3 ]
Answer:
{
"error": "ParseError",
"description": "Errors found in incoming JSON buffer"
}
Version:
GET /version
{
"orion": {
"version": "1.10.0-next",
"uptime": "0 d, 0 h, 1 m, 34 s",
"git_hash": "0f92803495a8b6c145547e19f35e8f633dec92e0",
"compile_time": "Fri Feb 2 09:45:41 UTC 2018",
"compiled_by": "root",
"compiled_in": "77ff7f334a88",
"release_date": "Fri Feb 2 09:45:41 UTC 2018",
"doc": "https://fiware-orion.readthedocs.org/en/master/"
}
}
"options=values" is a representation format for querying data not for posting new entity data for obvious reasons, when you are creating new entities you have to specify the entity id and the entity type and with the values representation format you can't ...

FactoryGirl gives wrong value on enum status field

I am trying to build a FactoryGirl factory for the Client.rb model:
Client.rb
enum status: [ :unregistered, :registered ]
has_many :quotation_requests
#Validations
validates :first_name,
presence: true,
length: {minimum: 2}
validates :last_name,
presence: true,
length: {minimum: 2}
validates :email, email: true
validates :status, presence: true
Factory:
FactoryGirl.define do
factory :client do
first_name "Peter"
last_name "Johnson"
sequence(:email) { |n| "peterjohnson#{n}#example.com" }
password "somepassword"
status "unregistered"
end
end
client_spec.rb
require 'rails_helper'
RSpec.describe Client, type: :model do
describe 'factory' do
it "has a valid factory" do
expect(FactoryGirl.build(:client).to be_valid
end
end
end
I get the following errorL
1) Client factory has a valid factory
Failure/Error: expect(FactoryGirl.build(:client, status: 'unregistered')).to be_valid
expected #<Client id: nil, email: "peterjohnson1#example.com", encrypted_password: "$2a$04$urndfdXNfKVqYB5t3kERZ.c.DUitIVXEZ6f19FNYZ2C...", first_name: "Peter", last_name: "Johnson", status: "0", reset_password_token: nil, reset_password_sent_at: nil, remember_created_at: nil, sign_in_count: 0, current_sign_in_at: nil, last_sign_in_at: nil, current_sign_in_ip: nil, last_sign_in_ip: nil, created_at: nil, updated_at: nil> to be valid, but got errors: Status can't be blank
The error is that Status can't be blank.
I don't understand how this is possible as the factory is clearly assigning a value to the status attribute.
How can I get this factory to build a valid client object?
Rails 4.2
Using factory_girl 4.7.0
Using factory_girl_rails 4.7.0
This error was caused by the data type I used for the status attribute. I chose string instead of integer.
I solved the problem by running a new migration to change the data type of the status to integer.
class ChangeColumnTypeClientStatus < ActiveRecord::Migration
def change
change_column :clients, :status, :integer, default: 0
end
end
Now it works perfectly.
I think that you forgot the
let(:client) { FactoryGirl.create(:client) }
on your client_spec.rb
Where're you creating the client object?
Other issue may be that you assign on Factory:
status "unregistered"
instead of:
status :unregistered
as a symbol or due to is an enum maybe you should make
status 0 # :unregistered

What does "fastmod" means in mongodb logs

I have thought fastmod specifies some operations like update-in-place.
In my app I'm doing update by _id using '$' modifiers, for example:
$colleciton->update(
array('_id' => $id),
array(
'$inc' => array('hits' => new MongoInt32(1)),
'$set' => array(
'times.gen' => gettimeofday(true),
'http.code' => new MongoInt32(200)
)
),
array('safe'=>false,'multiple'=>false,'upsert'=>false)
);
I've got such logs:
Wed Jul 25 11:08:36 [conn7002912] update mob.stat_pages query: { _id: BinData } update: { $inc: { hits: 1 }, $set: { times.gen: 1343203715.684896, http.code: 200 } } nscanned:1 nupdated:1 keyUpdates:0 locks(micros) w:342973 342ms
In logs as you can see I don't have any "fastmod" flags. There is no "moved" flag, because I set fields 'times.gen' and 'http.code' on insert, so padding factor is 1.0.
Am I doing something wrong, or I misunderstood meaning of fastmod?
You are correct that "fastmod" in the logs means an in-place update. Some possible reasons for the omission of logged fastmod/in-place operations:
You are actually setting or incrementing a field that doesn't exist, so it must be added, not an in place operation
The logs only show slow queries (default >100ms), so the in-place ones are probably happening too fast to be logged
You seem to be using 2.1 or 2.2 judging by the log - did the messages disappear if/when you switched to the new version?
In terms of looking into this further:
Have a look at the profiler, try with different settings, note: profiling adds load - so use carefully.
You can also try setting the slowms value lower, either on start up or:
> db.setProfilingLevel(0,20) // slow threshold=20ms