I am trying to dereference a reference field on my Flask backend and return the complete object with that certain field dereferenced.
The field I am trying to dereference is defined like this:
vouches_received = db.ListField(db.ReferenceField('Vouch'))
The way I am trying to dereference it is like this:
unverified_vouches = []
for vouch in usr.vouches_received:
unverified_vouches.append(vouch.to_mongo())
usr.vouches_received = unverified_vouches
However, when I then do:
usr.to_json()
On the object, then I get a ValidationError like so:
ValidationError: u'{...}' is not a valid ObjectId, it must be a
12-byte input of type 'str' or a 24-character hex string
The 3 dots (...) is basically the document dereferenced, it has mostly Strings, a Date Field, and some other reference fields I do not wish to dereference.
I am aware this is a valid error, as it is expecting an ObjectID for the reference field, but then arises the question, how do I succeed at dereferencing that field and return the document.
Thanks
The ListField is expecting elements of ObjectId and because you've de-referenced them it throws that error. I'm not sure this is the most elegant way but could you convert the usr.to_json() to a dict and then replace the vouches_received list with a deferenced list afterwards - I can't test it but something like?
user_dict = json.loads(usr.to_json())
unverified_vouches = []
for vouch in usr.vouches_received:
user_dict['vouches_received'].append(vouch.to_mongo())
usr_json = json.dumps(user_dict)
A better solution may be to use an EmbededDocument.
Related
const tours = await Tour.find()
when i use console.log(typeof tours) it shows object in console log.
but when i use console.log(tours) it shows an array of objects.
so i'm bit confused about what it actually returns back?
In JavaScript the value of typeof when used on an array is "object". The correct way to check if a variable is an array is Array.isArray() - it will return true or false depending if the argument passed is an array.
Let's say I have a Postgres database (9.3) and there is a table called Resources. In the Resources table I have the fields id which is an int and data which is a JSON type.
Let's say I have the following records in said table.
1, {'firstname':'Dave', 'lastname':'Gallant'}
2, {'firstname':'John', 'lastname':'Doe'}
What I want to do is write a query that would return all the records in which the data column has a json element with the lastname equal to "Doe"
I tried to write something like this:
records = db_session.query(Resource).filter(Resources.data->>'lastname' == "Doe").all()
Pycharm however is giving me a compile error on the "->>"
Does anyone know how I would write the filter clause to do what I need?
Try using astext
records = db_session.query(Resource).filter(
Resources.data["lastname"].astext == "Doe"
).all()
Please note that the column MUST have a type of a JSONB. The regular JSON column will not work.
Also you could explicitly cast string to JSON (see Postgres JSON type doc).
from sqlalchemy.dialects.postgres import JSON
from sqlalchemy.sql.expression import cast
db_session.query(Resource).filter(
Resources.data["lastname"] == cast("Doe", JSON)
).all()
If you are using JSON type (not JSONB) the following worked for me:
Note the '"object"'
query = db.session.query(ProductSchema).filter(
cast(ProductSchema.ProductJSON["type"], db.String) != '"object"'
)
I have some GeoJSON in a JSON (not JSONB) type column and none of the existing solutions worked, but as it turns out, in version 1.3.11 some new data casters were added, so now you can:
records = db_session.query(Resource).filter(Resources.data["lastname"].as_string() == "Doe").all()
Reference: https://docs.sqlalchemy.org/en/14/core/type_basics.html#sqlalchemy.types.JSON
Casting JSON Elements to Other Types
Index operations, i.e. those invoked by calling upon the expression
using the Python bracket operator as in some_column['some key'],
return an expression object whose type defaults to JSON by default, so
that further JSON-oriented instructions may be called upon the result
type. However, it is likely more common that an index operation is
expected to return a specific scalar element, such as a string or
integer. In order to provide access to these elements in a
backend-agnostic way, a series of data casters are provided:
Comparator.as_string() - return the element as a string
Comparator.as_boolean() - return the element as a boolean
Comparator.as_float() - return the element as a float
Comparator.as_integer() - return the element as an integer
These data casters are implemented by supporting dialects in order to
assure that comparisons to the above types will work as expected, such
as:
# integer comparison
data_table.c.data["some_integer_key"].as_integer() == 5
# boolean comparison
data_table.c.data["some_boolean"].as_boolean() == True
According sqlalchemy.types.JSON, you can do it like this
from sqlalchemy import JSON
from sqlalchemy import cast
records = db_session.query(Resource).filter(Resources.data["lastname"] == cast("Doe", JSON)).all()
According to this, pre version 1.3.11, the most robust way should be like this, as it works for multiple database types, e.g. SQLite, MySQL, Postgres:
from sqlalchemy import cast, JSON, type_coerce, String
db_session.query(Resource).filter(
cast(Resources.data["lastname"], String) == type_coerce("Doe", JSON)
).all()
From version 1.3.11 onward, type-specific casters is the new and neater way to handle this:
db_session.query(Resource).filter(
Resources.data["lastname"].as_string() == "Doe"
).all()
Jade Template Code
each member in team.members
if member == user._id
- flag=true;
Here, team is a document in the collection 'teams' in Mongo DB, and 'members' is a property having Arrays as value.
Here values of #{member} are 53fecc27bace957811633dcc and 5412a08fa51c105017b9f1fe respectively. Both are Object Id's.
Value of #{user.id} is 53fecc27bace957811633dcc
However, this comparison returns as false and flag is never set to true.
Why is it so?
EDITED FOR CLARITY:
The mongodb-native driver uses a custom ObjectID type. Even if the ObjectIDs are the same, member and user._id would still refer to two distinct objects, and therefore the comparison fails.
Here's a simplified example of what I'm talking about. In the example below I have a constructor function for a TestObject (simulating the mongo ObjectID). I instantiate two objects (i.e. x1 and x2) both with the same parameter. While x1 and x2 are instances of the same object and both were created with the same arguments, they are actually just two different objects with the same data/payload. Therefore, the first direct comparison fails, because they in fact are not the same. To take the simulation a bit further, I added a prototype method .equals() which ensures that both objects are instances of TestObject and the payloads are identical.
function TestObject(data) {
this.payload = data;
}
TestObject.prototype.equals = function(anotherObject) {
return (anotherObject instanceof TestObject) && (this.payload === anotherObject.payload);
};
var x1 = new TestObject(1), x2 = new TestObject(1);
console.log(x1 == x2); // <-- false
console.log(x1.equals(x2)); // <-- true
You can compare them by converting the ObjectIDs to strings.
each member in team.members
if member.toString() == user._id.toString()
- flag=true;
Another possible solution would be to use the ObjectId .equal() method:
each member in team.members
if member.equals(user._id)
- flag=true;
MongoDB objectIds are probably not simple strings. As per docs, they are BSON types.
It's possible that member and user._id are two different objects that is why a "reference equals" comparison is failing, even though their string representations are the same.
On a related note, it's a bad idea to use MongoDB generated object IDs in your application. A more robust design would be to have your own identifier for every record type.
I haven't tested this against the 1.4.16 release that came out a couple of weeks ago but there is nothing in the release notes about it.
The problem occurs with predicates where the value you are comparing is identical to the name of a property on any entity that breeze knows about. A simple test case is :
var query = breeze.EntityQuery.from('Items');
var pred = breeze.Predicate.create('name', breeze.FilterQueryOp.Contains, searchTerm);
query = query.where(pred);
Where searchTerm is equal to any string other than "name" this produces an oData query as below:
Items?$filter=(substringof(%27somevalue%27%2CName)%20eq%20true)
but if searchTerm = "name" then it produces the following query
Items?$filter=(substringof(Name%2CName)%20eq%20true)
Which istead of comparing the string 'name' against the property Name, it compares the property Name with itself.
I have not tested every operator but as far as I can tell it does not matter which you use you get the same behaviour.
You also get the same problem when querying navigation properties but it usually results in an invalid query. Below is a predicate for the same entity but against a navigation property tags that contains a collection of ItemTag entities that have a "Tag" property on them.
breeze.Predicate.create('tags', breeze.filterQueryOp.Any, 'tag', breeze.filterQueryOp.Contains, searchTerm)
It works fine for any searchTerm other than "tag" where it produces an oData request as below:
Items?$filter=Tags%2Fany(x1%3A%20substringof(%27somevalue%27%2Cx1%2FTag)%20eq%20true)
but if the searchTerm is "tag" then it requests:
Items?$filter=Tags%2Fany(x1%3A%20substringof(Tag%2Cx1%2FTag)%20eq%20true)
which produces an error of "Could not find a property named 'Tag' on type 'Item'" because the property Tag exists on the ItemTag entity.
In short breeze seems to infer that any search term that is identical to the name of a property it knows about, refers to that property rather than being a string literal value.
Has anyone else encountered this?
Is this a bug, or is there a way to explicitly tell breeze to interpret that value as a string literal and not a reference to a property?
I am not sure it is relevant as the server seems to be responding correctly to the requests and it is breeze that is creating incorrect requests but on the server side I am using Web API oData controllers with EF as ORM data layer.
Try
var pred = breeze.Predicate.create('name', breeze.FilterQueryOp.Contains,
{ value: searchTerm, isLiteral: true} );
This is described here ( under the explanation of the value parameter):
http://www.breezejs.com/sites/all/apidocs/classes/Predicate.html#method_create
if the value can be interpreted as a property expression it will be, otherwise it will be treated as a literal.
In most cases this works well, but you can also force the interpretation by making the value argument itself an object with a 'value' property and an 'isLiteral' property set to either true or false.
Breeze also tries to infer the dataType of any literal based on context, if this fails you can force this inference by making the value argument an object with a 'value' property and a 'dataType'property set
to one of the breeze.DataType enumeration instances.
The reason for this logic is to allow expressions where both sides of the expression are properties. For example to query for employees with the same first and last name you'd do this:
var q = EntityQuery.from("Employees")
.where("lastName", "==", "firstName");
whereas if you wanted employees with a lastName of 'firstName' you'd do this:
var q = EntityQuery.from("Employees")
.where("lastName", "startsWith", { value: "firstName", isLiteral: true })
I know another users had asked this question. I test and it doesn't work. I find this problem when use the $coll->remove({"_id" => "xxxx"}) not hehaviour as expected.
Following is the summary of my test:
print Dumper $db->posts->find_one({"_id" => "4d92740b2239007c16130000"});
$VAR1= undef;
print Dumper $db->posts->find_one({"_id" => $conn->oid("4d92740b2239007c16130000")});
print out a document with _id => "4d9274032e62007c16110000"
Does the autogenerated _id object can be used to manage documents?
http://www.mongodb.org/display/DOCS/Removing
db.things.remove({_id: myobject._id});
The _id is not a string. It's a 12 byte binary value stored as a BSON object:
http://www.mongodb.org/display/DOCS/Object+IDs
You can remove by the _id but you can't use the 24 digit hex string representation. This is exactly the same behavior as find_one() which you have already discovered.
$coll->remove({"_id" => $myObject->{_id}});
There is a clear distinction between an objectid and its string representation. Why should the first find_one() with the string representation return a result if you are actually using an ObjectId as _id here? So the behavior is completely correct - independent of the driver used. If you introduce your string object ids for whatever reason then you will be able to search by string. As long as the driver injects decicated objectid (which aren't strings) you will have to search by them and not by their string representation.