Jade Template Code
each member in team.members
if member == user._id
- flag=true;
Here, team is a document in the collection 'teams' in Mongo DB, and 'members' is a property having Arrays as value.
Here values of #{member} are 53fecc27bace957811633dcc and 5412a08fa51c105017b9f1fe respectively. Both are Object Id's.
Value of #{user.id} is 53fecc27bace957811633dcc
However, this comparison returns as false and flag is never set to true.
Why is it so?
EDITED FOR CLARITY:
The mongodb-native driver uses a custom ObjectID type. Even if the ObjectIDs are the same, member and user._id would still refer to two distinct objects, and therefore the comparison fails.
Here's a simplified example of what I'm talking about. In the example below I have a constructor function for a TestObject (simulating the mongo ObjectID). I instantiate two objects (i.e. x1 and x2) both with the same parameter. While x1 and x2 are instances of the same object and both were created with the same arguments, they are actually just two different objects with the same data/payload. Therefore, the first direct comparison fails, because they in fact are not the same. To take the simulation a bit further, I added a prototype method .equals() which ensures that both objects are instances of TestObject and the payloads are identical.
function TestObject(data) {
this.payload = data;
}
TestObject.prototype.equals = function(anotherObject) {
return (anotherObject instanceof TestObject) && (this.payload === anotherObject.payload);
};
var x1 = new TestObject(1), x2 = new TestObject(1);
console.log(x1 == x2); // <-- false
console.log(x1.equals(x2)); // <-- true
You can compare them by converting the ObjectIDs to strings.
each member in team.members
if member.toString() == user._id.toString()
- flag=true;
Another possible solution would be to use the ObjectId .equal() method:
each member in team.members
if member.equals(user._id)
- flag=true;
MongoDB objectIds are probably not simple strings. As per docs, they are BSON types.
It's possible that member and user._id are two different objects that is why a "reference equals" comparison is failing, even though their string representations are the same.
On a related note, it's a bad idea to use MongoDB generated object IDs in your application. A more robust design would be to have your own identifier for every record type.
Related
I have a custom rest action like this (in a class that extends yii/rest/ActiveController):
public function actionTest()
{
return ["9" => "Nine", "1" => "one"];
}
When calling the API, the array output is in reverse order, ie:
{
"1": "One"
"9": "Nine",
}
I would like to have it in the original (expected) order...
Seems like the array was sorted somewhere after the array was returned in the action, but I can't figure out where. This only happens when the array key is an integer, an array like this is sorted as expected:
["id-9" => "Nine", "id-1" => "one"]
Have tried using an ArrayDataProvider setting 'sort' = false, but that made no difference.
Since you're exporting it as json and looking at that, from this question - Keeping dictionary keys of JSON Object in order in Objective C, the answer left by Henning Makholm says that:
In JSON objects, by definition, the order of the key-value pairs is not meaningful. The specification allows a JSON producer to permute them any way it want, even at random -- and does not require a parser to preserve the ordering. RFC 4627 says:
An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.
So json has no ordering, since it uses a dictionary as its data structure and typically dictionaries will have no implicit or explicit ordering due to the way they hash the keys for quick access.
It may be that the actual underlying representation in your program is ordered, but the json output has no such guarantee.
One way to fix this would be to move to a different data structure where order is preserved.
I am trying to dereference a reference field on my Flask backend and return the complete object with that certain field dereferenced.
The field I am trying to dereference is defined like this:
vouches_received = db.ListField(db.ReferenceField('Vouch'))
The way I am trying to dereference it is like this:
unverified_vouches = []
for vouch in usr.vouches_received:
unverified_vouches.append(vouch.to_mongo())
usr.vouches_received = unverified_vouches
However, when I then do:
usr.to_json()
On the object, then I get a ValidationError like so:
ValidationError: u'{...}' is not a valid ObjectId, it must be a
12-byte input of type 'str' or a 24-character hex string
The 3 dots (...) is basically the document dereferenced, it has mostly Strings, a Date Field, and some other reference fields I do not wish to dereference.
I am aware this is a valid error, as it is expecting an ObjectID for the reference field, but then arises the question, how do I succeed at dereferencing that field and return the document.
Thanks
The ListField is expecting elements of ObjectId and because you've de-referenced them it throws that error. I'm not sure this is the most elegant way but could you convert the usr.to_json() to a dict and then replace the vouches_received list with a deferenced list afterwards - I can't test it but something like?
user_dict = json.loads(usr.to_json())
unverified_vouches = []
for vouch in usr.vouches_received:
user_dict['vouches_received'].append(vouch.to_mongo())
usr_json = json.dumps(user_dict)
A better solution may be to use an EmbededDocument.
Let's say I have a Postgres database (9.3) and there is a table called Resources. In the Resources table I have the fields id which is an int and data which is a JSON type.
Let's say I have the following records in said table.
1, {'firstname':'Dave', 'lastname':'Gallant'}
2, {'firstname':'John', 'lastname':'Doe'}
What I want to do is write a query that would return all the records in which the data column has a json element with the lastname equal to "Doe"
I tried to write something like this:
records = db_session.query(Resource).filter(Resources.data->>'lastname' == "Doe").all()
Pycharm however is giving me a compile error on the "->>"
Does anyone know how I would write the filter clause to do what I need?
Try using astext
records = db_session.query(Resource).filter(
Resources.data["lastname"].astext == "Doe"
).all()
Please note that the column MUST have a type of a JSONB. The regular JSON column will not work.
Also you could explicitly cast string to JSON (see Postgres JSON type doc).
from sqlalchemy.dialects.postgres import JSON
from sqlalchemy.sql.expression import cast
db_session.query(Resource).filter(
Resources.data["lastname"] == cast("Doe", JSON)
).all()
If you are using JSON type (not JSONB) the following worked for me:
Note the '"object"'
query = db.session.query(ProductSchema).filter(
cast(ProductSchema.ProductJSON["type"], db.String) != '"object"'
)
I have some GeoJSON in a JSON (not JSONB) type column and none of the existing solutions worked, but as it turns out, in version 1.3.11 some new data casters were added, so now you can:
records = db_session.query(Resource).filter(Resources.data["lastname"].as_string() == "Doe").all()
Reference: https://docs.sqlalchemy.org/en/14/core/type_basics.html#sqlalchemy.types.JSON
Casting JSON Elements to Other Types
Index operations, i.e. those invoked by calling upon the expression
using the Python bracket operator as in some_column['some key'],
return an expression object whose type defaults to JSON by default, so
that further JSON-oriented instructions may be called upon the result
type. However, it is likely more common that an index operation is
expected to return a specific scalar element, such as a string or
integer. In order to provide access to these elements in a
backend-agnostic way, a series of data casters are provided:
Comparator.as_string() - return the element as a string
Comparator.as_boolean() - return the element as a boolean
Comparator.as_float() - return the element as a float
Comparator.as_integer() - return the element as an integer
These data casters are implemented by supporting dialects in order to
assure that comparisons to the above types will work as expected, such
as:
# integer comparison
data_table.c.data["some_integer_key"].as_integer() == 5
# boolean comparison
data_table.c.data["some_boolean"].as_boolean() == True
According sqlalchemy.types.JSON, you can do it like this
from sqlalchemy import JSON
from sqlalchemy import cast
records = db_session.query(Resource).filter(Resources.data["lastname"] == cast("Doe", JSON)).all()
According to this, pre version 1.3.11, the most robust way should be like this, as it works for multiple database types, e.g. SQLite, MySQL, Postgres:
from sqlalchemy import cast, JSON, type_coerce, String
db_session.query(Resource).filter(
cast(Resources.data["lastname"], String) == type_coerce("Doe", JSON)
).all()
From version 1.3.11 onward, type-specific casters is the new and neater way to handle this:
db_session.query(Resource).filter(
Resources.data["lastname"].as_string() == "Doe"
).all()
I haven't tested this against the 1.4.16 release that came out a couple of weeks ago but there is nothing in the release notes about it.
The problem occurs with predicates where the value you are comparing is identical to the name of a property on any entity that breeze knows about. A simple test case is :
var query = breeze.EntityQuery.from('Items');
var pred = breeze.Predicate.create('name', breeze.FilterQueryOp.Contains, searchTerm);
query = query.where(pred);
Where searchTerm is equal to any string other than "name" this produces an oData query as below:
Items?$filter=(substringof(%27somevalue%27%2CName)%20eq%20true)
but if searchTerm = "name" then it produces the following query
Items?$filter=(substringof(Name%2CName)%20eq%20true)
Which istead of comparing the string 'name' against the property Name, it compares the property Name with itself.
I have not tested every operator but as far as I can tell it does not matter which you use you get the same behaviour.
You also get the same problem when querying navigation properties but it usually results in an invalid query. Below is a predicate for the same entity but against a navigation property tags that contains a collection of ItemTag entities that have a "Tag" property on them.
breeze.Predicate.create('tags', breeze.filterQueryOp.Any, 'tag', breeze.filterQueryOp.Contains, searchTerm)
It works fine for any searchTerm other than "tag" where it produces an oData request as below:
Items?$filter=Tags%2Fany(x1%3A%20substringof(%27somevalue%27%2Cx1%2FTag)%20eq%20true)
but if the searchTerm is "tag" then it requests:
Items?$filter=Tags%2Fany(x1%3A%20substringof(Tag%2Cx1%2FTag)%20eq%20true)
which produces an error of "Could not find a property named 'Tag' on type 'Item'" because the property Tag exists on the ItemTag entity.
In short breeze seems to infer that any search term that is identical to the name of a property it knows about, refers to that property rather than being a string literal value.
Has anyone else encountered this?
Is this a bug, or is there a way to explicitly tell breeze to interpret that value as a string literal and not a reference to a property?
I am not sure it is relevant as the server seems to be responding correctly to the requests and it is breeze that is creating incorrect requests but on the server side I am using Web API oData controllers with EF as ORM data layer.
Try
var pred = breeze.Predicate.create('name', breeze.FilterQueryOp.Contains,
{ value: searchTerm, isLiteral: true} );
This is described here ( under the explanation of the value parameter):
http://www.breezejs.com/sites/all/apidocs/classes/Predicate.html#method_create
if the value can be interpreted as a property expression it will be, otherwise it will be treated as a literal.
In most cases this works well, but you can also force the interpretation by making the value argument itself an object with a 'value' property and an 'isLiteral' property set to either true or false.
Breeze also tries to infer the dataType of any literal based on context, if this fails you can force this inference by making the value argument an object with a 'value' property and a 'dataType'property set
to one of the breeze.DataType enumeration instances.
The reason for this logic is to allow expressions where both sides of the expression are properties. For example to query for employees with the same first and last name you'd do this:
var q = EntityQuery.from("Employees")
.where("lastName", "==", "firstName");
whereas if you wanted employees with a lastName of 'firstName' you'd do this:
var q = EntityQuery.from("Employees")
.where("lastName", "startsWith", { value: "firstName", isLiteral: true })
I have the following Mongoengine document:
class MyEmbed(EmbeddedDocument):
embedField = StringField(primary_key=True)
varField = StringField()
class TestDoc(Document):
myField = StringField()
embed_list = ListField(EmbeddedDocumentField(MyEmbed))
So I keep a list of embedded documents, to which I wish to add new documents if they don't exist already. The problem is that when I use the atomic update operator add_to_set things don't turn out the way I want them to.
This is what I am trying to do:
embed1 = models.MyEmbed(embedField="F1")
parent = models.TestDoc(myField="ParentField")
embed_list = []
embed_list.append(embed1)
parent.embed_list = embed_list
parent.save()
embed2 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed2)
The problem is that after doing this, I have in the DB a list of embedded documents with 2 elements. And what I want is to decide upon one field (embedField in this case) whether 2 EmbeddedDocuments are equal or not, and not by taking into account all the properties. My questions are:
What are the default criteria according to which Mongoengine decides whether 2 EmbeddedDocuments are equal or not?
How can I redefine the function that makes Mongoengine decide when 2 EmbeddedDocuments are equal or not?
Thanks!
The actual checking is done inside MongoDB and not mongoengine.
The object sent to mongodb should be the same, but this is where it gets tricky as with BSON order is important and in python with dictionaries its not. When converting to send to mongodb mongoengine just passes a dictionary. This is a bug - so I've added #296 and will fix for 0.8
See https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/document.py#L51 and https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/base/document.py#L52:
def __eq__(self, other):
if isinstance(other, self.__class__):
return self._data == other._data
return False
It compare dicts of Embedded documents data. So you can override this method.
If you look at Document update that calls QuerySet update (find add_to_set and addToSet) you can find that mongoengine doesnt't check exists document in list and just call mongo $addToSet operation: https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/queryset/transform.py#L156.
In your code you have document MyEmbed(embedField="F1") and try add another document MyEmbed(embedField="F1", varField="varField") so logic right: it add new document. If you try next code:
embed1 = models.MyEmbed(embedField="F1")
parent = models.TestDoc(myField="ParentField")
embed_list = []
embed_list.append(embed1)
parent.embed_list = embed_list
parent.save()
embed2 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed2)
embed3 = models.MyEmbed(embedField="F1")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed3)
embed4 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed4)
you can find that parent contains only embed1 and embed2.
So, to resolve you problem you can override __eq__ method and check document in list, but you must find another solution for update document list, because it have direct call of mongo method.