I am working on several modules of my python library, doing most of the testing through IPython.
Anytime I attempt to reload (deep_reload) a module that uses sqlalchemy, the reload throws an Exception and fails to reload the module (I must start a new kernel and re-import). More specifically, any module that imports my sqlalchemy declarative models.
The traceback is quite long, so I will summarize for now, and provide more if requested.
Before the Exception, StdOUT reads (this is only the bottom portion):
Reloading pysqlite2
Reloading sqlalchemy.dialects.sqlite.sqlite3
Reloading sqlite3
Reloading sqlite3.dbapi2
Reloading sqlite3.datetime
Reloading sqlite3.time
Reloading sqlite3._sqlite3
Reloading _sqlite3
Then the exception is thrown, and here is the most recent entry of the traceback:
\pydir\lib\site-packages\sqlalchemy-0.7.9-py2.7.egg\sqlalchemy\util\_collections.pyc in __setitem__(self, object, value)
697 if oid not in self._weakrefs:
698 self._weakrefs[oid] = self._ref(object)
--> 699 weakref.WeakKeyDictionary.__setitem__(self, object, value)
700
701 def __delitem__(self, object):
TypeError: unbound method __setitem__() must be called with WeakKeyDictionary instance as first argument (got WeakIdentityMapping instance instead)
Edit: Problem Update
When calling reload, I started adding exclusions based on the module on the top of the traceback stack ('_sqlite3' in the example above).
What I found was that I kept getting the same Error, only it would come from a new module ('pysqlite2', 'sqlite3'). And in each case, there was always SqlAlchemy higher up on the traceback, more specifically, 'sqlalchemy.orm.mapper'.
When defining my model orm classes, I was defining my orm classes and adding them to the session directly from the module on import. The solution was to wrap these few lines in a function that should not be executed on import. Seems to fix.
I think this was just a case of some lazy coding finally catching up with me.
You might add sqlite3 (or pysqlite2) to deep reload's exclusions:
dreload(mymod, exclude=['sys', 'os.path', '__builtin__', '__main__', 'sqlite3'])
etc.
Related
I have been getting very frustrated with Dart as runtime exceptions seem to fail silently. Program execution will continue after some type of failure and leave me totally stumped about what is not working and why. For example, in using aqueduct for dart server (this is not an issue unique to aqueduct, but dart in general) I have this code:
jsonData.forEach((jsonObject) async {
ManagedObject record = Document();
record.read(jsonObject);
print(Util.jsonFormatToString(record.asMap()));
....more code
}
In this case, the application fails silently on record.read() falls out of the forEach loop and then continues code execution. During debugging, the application returns completely strange results and after debugging, presumably there is a problem with the jsonObject attempting to be read into the managed object, but Dart gives no notice that there is a problem or what that problem might be.
This is one example of many I have been running into.
Why does Dart fail so silently or am I missing some setting somewhere that is hiding critical info? I am using IntelliJ IDE.
Probably a long shot but I'm wondering if anyone has seen an error like this before, as I can not reproduce it outside of a production environment. Essentially the situation is as follows:
I have a module called My::Budget::Module (renamed for simplicity) which is responsible for updating the "budget" for a given object in the application
The My::Budget::Module uses a Moo object that I built called My::Bulk::Update::Module which does the following:
build up an array of database rows that need to be updated
build a MySQL update query string / statement which will update all rows at once
actually update all rows at once
The My::Bulk::Update::Module will then perform the update and mark the rows that have been updated as "stale" so that they will not be cached
The error always seems to occur somewhere after adding a row to be updated but before the code which actually applies the update returns.
If you look at the stack trace that I have included below you can see that the error takes the form
Attempt to bless into a reference at...
and the point at which this occurs is in the constructor of Moo/Object.pm which is Version 2.003002 of Moo from cpan(see here).
Attempt to bless into a reference at /path/to/module/from/cpan/Moo/Object.pm line 25 at /path/to/module/from/cpan/Moo/Object.pm line 25.
Moo::Object::new(My::Bulk::Update::Module=HASH(0xf784b50)) called at (eval 1808) line 28
MongoDB::Collection::new(My::Bulk::Update::Module=HASH(0xf784b50)) called at /path/to/my/bulk/update/module line XXXX
My::Bulk::Update::Module::apply_bulk_update(My::Bulk::Update::Module=HASH(0xf784b50)) called at /path/to/my/budget/module line XXXX
My::Budget::Module::update_budget(My::Budget::Module=HASH(0xf699a38)) called at /path/to/my/budget/module line XXXX
Moving backwards through the stack trace leads to MongoDB::Collection & this is where things start to get very weird.
MongoDB::Collection is also a cpan module but the module which appears at this point varies and I can't see a pattern here except that it is always a Moo object. Moreover, I'm unsure why this module is being instantiated as there is no call to MongoDB::Collection::new at the line mentioned.
In addition, from the stack trace it looks like MongoDB::Collection and Moo::Object are instantiated with the first argument being My::Bulk::Update::Module=HASH(0xf784b50). Given the application logic I do not believe MongoDB::Collection should be instantiated here nor should My::Bulk::Update::Module be passed to MongoDB::Collection at all.
Other than the fact that it is a Moo object, My::Bulk::Update::Module does not extend any other module and is designed to be a stand alone "utility" module. It is only used at one place in the entire application.
Has anyone seen something similar before?
EDIT: Adding some more code - apply_bulk_update doesn't do much at all. There is no call to MongoDB::Collection here and MongoDB::Collection just "happens" to be the moudule included in the stack trace in this particular example. This is not always MongoDB::Collection - I've also seen MongoDB::Timestamp, MongoDB::Cursor, Search::Elasticsearch::Serializer::JSON, Search::Elasticsearch::Logger::LogAny etc etc
sub apply_bulk_update
{
my $self = shift;
my ($db) = #_; # wrapper around DBI module
my $query = $self->_generate_query(); # string UPDATE table SET...
my $params = $self->_params; # arrayref
return undef unless $params && scalar #$params;
$db->do($query, undef, #$params);
}
The code sometimes dies as soon as apply_bulk_update is called, sometimes on the call to _generate_query and sometimes after the query executes on the last line...
Just in case anyone was interested...
After a chunk of further debugging the error was traced to the exact point where My::Bulk::Update::Module::apply_bulk_update or My::Bulk::Update::Module::_generate_query was called but logging code inside these subroutines determined that they were not being executed as expected.
To determine what was going on B::Deparse was used to rebuild the source code for the body of these subroutines (or at least the source code located at the memory address to which these subs were pointing)
After using this library e.g.
B::Deparse->new->coderef2text(\&My::Bulk::Update::_generate_query)
it became obvious that the error occurred when My::Bulk::Update::_generate_query was pointing at a memory location which contained something entirely different (i.e. MongoDB::Collection::new etc).
This issue appears to have been solved upstream by the following commit in the Sub::Defer module (which is a dependency for Moo).
https://github.com/moose/Sub-Quote/commit/4a38f034366e79b76d29fec903d8e8d02ee01896
If you read the summary of the commit you can see the change that was made:
Prevent defer_info and undefer_sub from operating on expired subs. Validate that the arguments to defer_info and undefer_sub refer to
actual live subs. Use the weak refs we are storing to the deferred and
undeferred subs to make sure the original subs are still alive, and we
aren't returning data related to a reused memory address. Also make sure we don't expire data related to unnamed subs. Since the
user can capture the undeferred sub via undefer_sub, we can't track the
expiry without using a fieldhash. For now, avoid introducing that
complexity, since the amount we leak should not be that great.
Upgrading the version of Sub::Defer appears to have solved the issue.
I have a django application using a C++ library (imported via swig).
The C++ library launches own thread which calls callbacks in Python code.
I cannot setup a breakpoint in python code, neither in PyDev nor PyCharm.
Tried also 'gevent compatibility' option too with no luck.
I verified the callbacks are properly called as logging.info dumps what expected. Breakpoints set in other threads work fine. So it seems that python debuggers cannot manage breakpoints in python code called by threads created in non-python code.
Does anyone know a workaround? Maybe there is some 'magic' thread initialization sequence I could use?
You have to setup the debugger machinery for it to work on non-python threads (this is done automatically when a Python thread is created, but when you create a thread for which Python doesn't have any creation hook, you have to do it yourself) -- note that for some frameworks -- such as QThread/Gevent -- things are monkey patched so that we know about the initialization and start the debugger, but for other frameworks you have to do it yourself.
To do that, after starting the thread you have to call:
import pydevd
pydevd.settrace(suspend=False, trace_only_current_thread=True)
Note that if you had put suspend=True, it'd simulate a manual breakpoint and would stop at that point of the code.
This is a follow-up to #fabio-zadrozny answer.
Here is a mixin I've created that my class (which gets callbacks from a C-thread) inherits from.
class TracingMixing(object):
"""The callbacks in the FUSE Filesystem are C threads and breakpoints don't work normally.
This mixin adds callbacks to every function call so that we can breakpoint them."""
def __call__(self, op, path, *args):
pydevd.settrace(suspend=False, trace_only_current_thread=True, patch_multiprocessing=True)
return getattr(self, op)(path, *args)
In GWT application I have RPC interface. Some methods works fine (i.e. RemoeServiceServlet configured fine), but when I try to invoke another method, it always fails with onFailure() method. Ajax call also don't occur (I can see it using FireBug, also on server side method invocation don't occur), but another methods of this service performs Ajax calls as well.
When I try to log error using e.getMessage() I get "undefined" message. Also I tried to wrap RPC calling code using try-catch - no error message.
Can this issue be related with GWT-RPC Serialization?
EDIT: Opera Dragonfly showed error on following method inside generated JavaScript (compiled with PRETTY mode):
function $check(this$static, typeSignature){
if (isNull($get_3(this$static.methodMapNative, typeSignature))) {
Unhandled Object: undefined
throw new SerializationException_1(typeSignature);
}
}
with error message
Unhandled Object: undefined
I would guess that you have a Serialization issue, remember that Java Serialization is not the same as GWT Serialization.
There is often no meaningful error message on Serialization errors when using RPC.
must have 0-ary constructor
final fields are inherently transient (ie. do NOT use final fields in classes intended to be serialized)
collections (ex List and Set) must be annotated with #gwt.typeArgs. #gwt.typeArgs is a JavaDoc annotation, thus it must be wrapped in a JavaDoc comment
ex.: /** #gwt.typeArgs */
For more details see:
GWT Serialization
Another thing to try:
When running GWT from the eclipse-plugin, a folder in the eclipse project is created (I belive its called gwt-unitCache). Sometimes my own GWT projects get ill and output strange exceptions, I can solve this by deleting the folder and run the project again.
I'm coding a web crawler and I've been using WWW::Mechanize::Firefox to navigate some pages (for the others I use WWW::Mechanize) which keep loading content after the page loaded and I've never had an issue with that.
Yesterday I added DBI and DBD::mysql to the script, adding queries to export data to a database (this works perfectly), but suddenly MozRepl started giving this error:
(in cleanup) Can't call method "execute" on an undefined value at /Library/Perl/5.10.0/MozRepl.pm line 372 during global destruction.
(in cleanup) Can't call method "execute" on an undefined value at /Library/Perl/5.10.0/MozRepl.pm line 372 during global destruction.
and terminating the script after 1 cycle (it should run until it gets to the end of a specific text file, which it doesn't).
I haven't touched anything from this part of the script (don't need to use the db with those pages), at least willingly. I checked with a file compare app and couldn't solve anything.
Posting the code could be tricky, it's pretty long and I have no idea where the problem may lie.
EDIT
Sometimes it also gives this error instead of the previous one:
(in cleanup) Can't call method "cmd" on an undefined value at /Library/Perl/5.10.0/MozRepl/Client.pm line 186 during global destruction.
This has nothing to do with DBI or DBD::mysql.The messages are nothing to worry about, but I admit they are unsightly.
The messages come as remaining Perl/Javascript objects get destroyed in an unordered way during Perl Global Destruction. If you want to avoid them, destroy your $mech object before
quitting your application.
undef $mech;
# end of program
If the $mech object is released before the program gets shut down, the Perl/Javascript bridge can also shut down in an orderly fashion.
Also note that the preferred forum for questions about WWW::Mechanize::Firefox is http://perlmonks.org :)