I know another users had asked this question. I test and it doesn't work. I find this problem when use the $coll->remove({"_id" => "xxxx"}) not hehaviour as expected.
Following is the summary of my test:
print Dumper $db->posts->find_one({"_id" => "4d92740b2239007c16130000"});
$VAR1= undef;
print Dumper $db->posts->find_one({"_id" => $conn->oid("4d92740b2239007c16130000")});
print out a document with _id => "4d9274032e62007c16110000"
Does the autogenerated _id object can be used to manage documents?
http://www.mongodb.org/display/DOCS/Removing
db.things.remove({_id: myobject._id});
The _id is not a string. It's a 12 byte binary value stored as a BSON object:
http://www.mongodb.org/display/DOCS/Object+IDs
You can remove by the _id but you can't use the 24 digit hex string representation. This is exactly the same behavior as find_one() which you have already discovered.
$coll->remove({"_id" => $myObject->{_id}});
There is a clear distinction between an objectid and its string representation. Why should the first find_one() with the string representation return a result if you are actually using an ObjectId as _id here? So the behavior is completely correct - independent of the driver used. If you introduce your string object ids for whatever reason then you will be able to search by string. As long as the driver injects decicated objectid (which aren't strings) you will have to search by them and not by their string representation.
Related
I am trying to call the hint method on a MongoDB::Cursor object. However, it throwing an exception when it's trying to execute the query. See the code sample below:
sub some_method_which_returns_cursor {
my $cursor = $collection->find($filter);
if ($hint) {
$cursor->hint({‘some_index’ => 1}); #failing here.
}
if ($sort) {
$cursor->sort($sort);
}
return $cursor;
}
Any thoughts as to what's going on and how I can fix this?
Harish asked me via email and I'll repeat my answer here for posterity:
The hint method takes a string when given an index name, or an array reference when given keys/order pairs:
$cursor->hint("some_index"); # by name
$cursor->hint([field1 => 1, field2 => -1]); # by keys
It also takes a hash reference, but don't use that because modern Perls randomize key order when serializing, so your hint may not match an index.
I have a custom rest action like this (in a class that extends yii/rest/ActiveController):
public function actionTest()
{
return ["9" => "Nine", "1" => "one"];
}
When calling the API, the array output is in reverse order, ie:
{
"1": "One"
"9": "Nine",
}
I would like to have it in the original (expected) order...
Seems like the array was sorted somewhere after the array was returned in the action, but I can't figure out where. This only happens when the array key is an integer, an array like this is sorted as expected:
["id-9" => "Nine", "id-1" => "one"]
Have tried using an ArrayDataProvider setting 'sort' = false, but that made no difference.
Since you're exporting it as json and looking at that, from this question - Keeping dictionary keys of JSON Object in order in Objective C, the answer left by Henning Makholm says that:
In JSON objects, by definition, the order of the key-value pairs is not meaningful. The specification allows a JSON producer to permute them any way it want, even at random -- and does not require a parser to preserve the ordering. RFC 4627 says:
An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.
So json has no ordering, since it uses a dictionary as its data structure and typically dictionaries will have no implicit or explicit ordering due to the way they hash the keys for quick access.
It may be that the actual underlying representation in your program is ordered, but the json output has no such guarantee.
One way to fix this would be to move to a different data structure where order is preserved.
I am trying to dereference a reference field on my Flask backend and return the complete object with that certain field dereferenced.
The field I am trying to dereference is defined like this:
vouches_received = db.ListField(db.ReferenceField('Vouch'))
The way I am trying to dereference it is like this:
unverified_vouches = []
for vouch in usr.vouches_received:
unverified_vouches.append(vouch.to_mongo())
usr.vouches_received = unverified_vouches
However, when I then do:
usr.to_json()
On the object, then I get a ValidationError like so:
ValidationError: u'{...}' is not a valid ObjectId, it must be a
12-byte input of type 'str' or a 24-character hex string
The 3 dots (...) is basically the document dereferenced, it has mostly Strings, a Date Field, and some other reference fields I do not wish to dereference.
I am aware this is a valid error, as it is expecting an ObjectID for the reference field, but then arises the question, how do I succeed at dereferencing that field and return the document.
Thanks
The ListField is expecting elements of ObjectId and because you've de-referenced them it throws that error. I'm not sure this is the most elegant way but could you convert the usr.to_json() to a dict and then replace the vouches_received list with a deferenced list afterwards - I can't test it but something like?
user_dict = json.loads(usr.to_json())
unverified_vouches = []
for vouch in usr.vouches_received:
user_dict['vouches_received'].append(vouch.to_mongo())
usr_json = json.dumps(user_dict)
A better solution may be to use an EmbededDocument.
I am querying a collection that includes an integer value among it's values, and loading resulting documents into this struct:
type Subscription struct {
Id bson.ObjectId "_id,omitempty"
Listen string
Job string
TimeoutSeconds int
Data string
}
var subscription Subscription
subscriptions := subscriptionsCol.Find(bson.M{"listen": "example_channel"}).Iter()
for subscriptions.Next(&subscription) {
log("Pending job: %s?%s (timeout: %d)\n",
subscription.Job,
subscription.Data,
subscription.TimeoutSeconds)
}
This is what phpMoAdmin shows me:
[_id] => MongoId Object (
[$id] => 502ed8d84eaead30a1351ea7
)
[job] => partus_test_job_a
[TimeoutSeconds] => 30
[listen] => partus.test
[data] => a=1&b=9
It puzzles me that subscription.TimeoutSeconds contains always 0, when I'm positive I have 30 in the document I inserted in the collection.
All other values are retrieved OK.
What's wrong with the int type?
Have you tried setting the "key" value for that field?
Unmarshal
The lowercased field name is used as the key for each exported field,
but this behavior may be changed using the respective field tag.
type Subscription struct {
Id bson.ObjectId "_id,omitempty"
Listen string
Job string
TimeoutSeconds int "TimeoutSeconds"
Data string
}
The other fields are working fine because their lowercase value matches your Mongo fields in the collection, whereas TimeoutSeconds is using the TitleCase. What is happening is the int field is being left at its zero value, since the Unmarshal can't map a field to it.
When UnMarshalling data, there are multiple keys that are supported.
Below are some examples:
type T struct {
A bool
B int "myb"
C string "myc,omitempty"
D string `bson:",omitempty" json:"jsonkey"`
E int64 ",minsize"
F int64 "myf,omitempty,minsize"
}
The general spec for 1 key-value pair during marshal is :
"[<key>][,<flag1>[,<flag2>]]"
`(...) bson:"[<key>][,<flag1>[,<flag2>]]" (...)`
GO provides support for particular keywords like bson (for mongo keys) and json for setting the json key in a resposne.
Check the Marshal GO Reference for more information.
Similarly there are some frameworks which provide further options to define the keys befor parsing. For example, in sql jinzhu github library gives support for setting default values, column ids to map, etc.
Anyone can use this feature and provide customized support.
How to use dot in field name ?
I see error in example:
db.test2.insert({ "a.a" : "b" })
can't have . in field names [a.a]
You can replace dot symbols of your field name to Unicode equivalent of \uff0E
db.test.insert({"field\uff0ename": "test"})
db.test.find({"field\uff0ename": "test"}).forEach(printjson)
{ "_id" : ObjectId("5193c053e1cc0fd8a5ea413d"), "field．name" : "test" }
See more:
http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping
http://docs.mongodb.org/manual/core/document/#dot-notation
Actualy you may use dots in queries. See: http://www.mongodb.org/display/DOCS/Dot+Notation+%28Reaching+into+Objects%29
Because of this special dot symbol mean you cannot use it in field names. Like you cannot use dot symbol in identifiers in most of programming languages.
You may write query db.test2.find({ "a.a" : "b" }) but if you want to be able to write such a query you need to insert your object like so: db.test2.insert({"a": {"a": "b"}}). This will create document with the field named "a" with the value of embeded document containing the field named "a" (again) with the value "b".
You can also write a SONManipulator using the pymongo library that transforms the data going to and back out of mongodb. There are downsides; there is a performance hit (impact depends on your use case) and you have to transform your keys when you do searches using find.
Here's code with an example of how to use it in the comment for the KeyTransform class:
from pymongo.son_manipulator import SONManipulator
class KeyTransform(SONManipulator):
"""Transforms keys going to database and restores them coming out.
This allows keys with dots in them to be used (but does break searching on
them unless the find command also uses the transform).
Example & test:
# To allow `.` (dots) in keys
import pymongo
client = pymongo.MongoClient("mongodb://localhost")
db = client['delete_me']
db.add_son_manipulator(KeyTransform(".", "_dot_"))
db['mycol'].remove()
db['mycol'].update({'_id': 1}, {'127.0.0.1': 'localhost'}, upsert=True,
manipulate=True)
print db['mycol'].find().next()
print db['mycol'].find({'127_dot_0_dot_0_dot_1': 'localhost'}).next()
Note: transformation could be easily extended to be more complex.
"""
def __init__(self, replace, replacement):
self.replace = replace
self.replacement = replacement
def transform_key(self, key):
"""Transform key for saving to database."""
return key.replace(self.replace, self.replacement)
def revert_key(self, key):
"""Restore transformed key returning from database."""
return key.replace(self.replacement, self.replace)
def transform_incoming(self, son, collection):
"""Recursively replace all keys that need transforming."""
for (key, value) in son.items():
if self.replace in key:
if isinstance(value, dict):
son[self.transform_key(key)] = self.transform_incoming(
son.pop(key), collection)
else:
son[self.transform_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_incoming(value, collection)
return son
def transform_outgoing(self, son, collection):
"""Recursively restore all transformed keys."""
for (key, value) in son.items():
if self.replacement in key:
if isinstance(value, dict):
son[self.revert_key(key)] = self.transform_outgoing(
son.pop(key), collection)
else:
son[self.revert_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_outgoing(value, collection)
return son
def remove_dots(data):
for key in data.keys():
if type(data[key]) is dict: data[key] = remove_dots(data[key])
if '.' in key:
data[key.replace('.', '\uff0E')] = data[key]
del data[key]
return data
this recursive method replaces all dot characters from keys of a dict with \uff0E
as suggested by Fisk
I replaced the key value using myString.replace(".","\u2024") before inserting it into the JsonObject.
Initially I used a simple recursion to replace all "." characters with its unicode equivalent but figured it out that even the dots in the values was getting replaced. So I thought that we should replace the dots only from keys and made the changes accordingly in case "if isinstance(input, dict)".
I thought it should be a sufficient condition to do the magic but I forgot that dict value can also be a dict or a list and then I finally added that check that if value of a dict was not string then, go inside recursively and was finally able to come up with this solution which eventually did the trick.
def remove_dots(data):
if isinstance(data, dict):
return {remove_dots(key): value if isinstance(value, str) else remove_dots(value) for key,value in data.iteritems()}
elif isinstance(data, list):
return [remove_dots(element) for element in data]
elif isinstance(data, str):
return data.replace('.','\u002e')
else:
return data
I've only really come across this problem when trying to serialize Dictionaries and such where the offending dot can appear as a key name.
Edited to show the references.
The quick and dirty C# approach:
using MongoDB.Bson;
using Newtonsoft.Json.Linq;
using System.Text.RegularExpressions;
public static T Sanitize<T>(T obj)
{
var str = JObject.FromObject(obj).ToJson();
var parsed = Regex.Replace(str, #"\.(?=[^""]*"":)", "_"); //i.e. replace dot with underscore when found as a json property name { "property.name": "don't.care.what.the.value.is" }
return JObject.Parse(parsed).ToObject<T>();
}