I have an application that uses MongoDB (on AWS DocumentDB) to stores documents with a large string in one of its fields which we call field X.
Few notes to start:
I'm using pymongo so the method names you might see here are taken from there
As of the nature of field X it is not being indexed
On field X we use MongoDB find method using a query with regex condition limiting it by both maxTimeMS and limit to a small amount of results.
When we get the results we iterate the cursor to fetch all the results to a list (inline loop).
Most of the times the query works properly but I'm starting to get more and more of the following error:
pymongo.errors.OperationFailure: Cannot update value (error code 14)
This is being thrown after the query return a cursor and we iterating the results and occurs after trying to _refresh the cursor connection by calling the next method and being thrown by _check_command_response at its last line meaning this is a default exception(?).
The query:
collection.find(condition).max_time_ms(MAX_QUERY_TIME_MS).sort(sort_order) \
.limit(RESULT_LIMIT)
results = [document for document in cursor] # <--- here we get the error
Stack trace:
pymongo/helpers.py in _check_command_response at line 155
pymongo/cursor.py in __send_message at line 982
pymongo/cursor.py in _refresh at line 1104
pymongo/cursor.py in next at line 1189
common/my_code.py in <listcomp> at line xxx
I'm trying to understand the origin of the exception to handle it correctly or use a different approach for handling the cursor.
what is being updated at the refresh method of the cursor that might
throw the above exception?
Thanks in advance.
Related
I have been trying to do a LIKE comparison in postgres but repeatedly receive an error telling me that the column "%#firstname%" doesn't exist.
I should clarify, this query is executed in a function, "#firstname" is the parameter passed into the function.
The relevant section of the query is below:
WHERE u."firstname" LIKE "%#firstname%"
I do not want an exact comparison which is why I am trying to add the %% to the query. It works just fine without them for exact queries. Whenever, I add the % then it assumes that they are part of the variable name and subsequently can't find it.
I have also tried the following:
'%"#firstname"%' which results in an empty array being returned even though it should have matched
"%'#firstname'%" which results in error: column "%'#filter'%" does not exist
%"#firstname"% which results in error: column "%'#filter'%" does not exist
If "#firstname" is a parameter you need something like:
WHERE u.firstname LIKE concat('%', "#firstname", '%');
There is a collection in my MongoDB that im not able to drop.
The name of the collection ends with a ` (backquote) character and I got error when executing the drop command.
db.MyCollection`.drop()
Error is
2017-12-07T07:42:35.711+0000 SyntaxError: Unexpected token ILLEGAL
Is there any other way of removing or is it here forever?
Just use getCollection function.
Example:
db.getCollection("MyCollection`").drop();
I'm checking with MongoDB application. There is application installed on my system, when I enter single quote(') in the input box it pop up the following error:
A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.
/usr/lib/cgi-bin/mongo/2.2.3/dbparse.py in ()
41 print "</th>"
42 if where:
=> 43 for record in collection.find(where):
44 print "<tr>"
45 print "<td align=\"center\">"+record["test"]+"</td>"
record undefined, collection = Collection(Database(MongoClient('localhost', 27017), u'test_d'), u'london_garages'), collection.find = <bound method Collection.find of Collection(Data...', 27017), u'test_d'), u'l_g')>, where = {'$where': "this.test== ''--'"}
What is the meaning of the error? If you have another pointer to check the security of this application please let me know.
If you look at the error and the following part:
where = {'$where': "this.test== ''--'"}
I assume the single quote goes to the where clause (some sort of search), so your code is probably something like the following:
where = {'$where': "this.test== '[YOUR TEXT BOX INPUT]--'"}
A single quote terminated your where clause prematurely. This is a good demonstration for a NoSQL injection.
Background:
I've got a python script using pymongo that pulls some XML data, parses it into an array of dictionaries called 'all_orders'. I then try to insert it into the collection "orders" and I invariably get this exception. I am reasonably certain that my array of dictionaries is correct because when the list is small it tends to work (I think). I've also found that 8 out of the ~1300 documents I tried to insert into the collection worked.
Question:
Do you know what causes this AutoReconnect(str(e)) exception? Do you know how to work around or avoid this issue?
Error Trace:
File "mongovol.py", line 152, in get_reports
orders.insert(all_orders)
File "/Users/ashutosh/hrksandbox/lumoback-garden2/venv/lib/python2.7/site-packages/pymongo/collection.py", line 359, in insert
continue_on_error, self.__uuid_subtype), safe)
File "/Users/ashutosh/hrksandbox/lumoback-garden2/venv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 853, in _send_message
raise AutoReconnect(str(e))
MongoCursorException' with message '$ operator made object too large
I was trying to update some objects with an update command like
$collection->update({"_id" => array('$in' => array(2,3,4,5)) } ....)
and got this exception. Any hints about what it means ? So that I can proceed to debug.
Based on the source code, I believe it means that the update made the object larger than the document size allowed: https://github.com/mongodb/mongo/blob/master/db/update.cpp
Max document size was made 8Mb (from 4Mb) on 1.7.2: https://jira.mongodb.org/browse/SERVER-1918
Then made 16Mb on 1.7.4: https://jira.mongodb.org/browse/SERVER-431