I'm using celery with a function which writes data into a database table.
This table doesn't have a related model inside models.py because I create it dynamically using django-mutant.
When I run my task, it correctly writes on my dynamic table, but at the very end of the task I get the following error:
[2013-07-10 09:10:45,707: CRITICAL/MainProcess] Task topology_py.Functions.Functions.objectAggregate[aff70510-1e93-4610-b08c-c3675c92afe9] INTERNAL ERROR: PicklingError("Can't pickle <class 'mutant.apps.tss.models.1_tmp'>: import of module mutant.apps.tss.models failed",)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/task/trace.py", line 261, in trace_task
for callback in task_request.callbacks or []]
File "/usr/lib/python2.7/site-packages/celery/canvas.py", line 197, in apply_async
return self._apply_async(args, kwargs, **options)
File "/usr/lib/python2.7/site-packages/celery/app/task.py", line 472, in apply_async
**options)
File "/usr/lib/python2.7/site-packages/celery/app/amqp.py", line 249, in publish_task
**kwargs
File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 157, in publish
compression, headers)
File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 233, in _prepare
body) = encode(body, serializer=serializer)
File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 170, in encode
payload = encoder(data)
File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 356, in dumps
return dumper(obj, protocol=pickle_protocol)
PicklingError: Can't pickle <class 'mutant.apps.tss.models.1_tmp'>: import of module mutant.apps.tss.models failed
The model that celery is searching for, 1_tmp is not stored in my app, named tss, but inside mutant app tables.
My problem is that if I chain this function as a subtask to another subtask, celery terminates with this error at the end of the first one!
Is there a way to tell celery where to find this model or anyway to skip this error and go further to next subtasks?
Thanks in advance.
EDIT: at the moment I can run even 5 functions in a chain using 5 immutable subtasks. They work correctly and fill correctly the dynamic table, but the first one, and only the first one, continue producing the error described...Can you give me and idea of why only the first one give me the error since that they access the dynamic model in the same identical way?
I resolved: it was an issue inside my first function: I was returning an object of the dynamic model at the end of its execution and that made a celery/django-mutant mess.
This object was completely useless and so deleting its return solved the problem.
Thank you everybody.
Related
I'm using the GUI for QGIS, not writing my own Python script. I'm using the QNEAT3 plugin to create isochrones using the Iso-Area as Polygons (from Point) algorithm. I'm getting the following error when I run the algorithm (screenshot of my parameters is attached):
Traceback (most recent call last):
File "C:\Users/emmaj/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QNEAT3\algs\IsoAreaAsPolygonsFromPoint.py", line 249, in processAlgorithm
polygon_featurelist = net.calcIsoPolygons(max_dist, interval, output_path)
File "C:\Users/emmaj/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\QNEAT3\Qneat3Framework.py", line 470, in calcIsoPolygons
band_in = ds_in.GetRasterBand(1)
AttributeError: 'NoneType' object has no attribute 'GetRasterBand'
The parameters attached are what I'm using right now, but I will need to change the size of Iso-area eventually to create different isochrones. However, I've replicated this error with different size values.
How can I fix this error?
I have successfully created a classifier using 20 classes, with 200 images in each class. When I try to add additional classes to this classifier, the API call fails:
Traceback (most recent call last):
File "create_classifier.py", line 34, in <module>
print visual_recognition.update_classifier('flowers_1337689264', **myhash)
File "/usr/local/lib/python2.7/dist-packages/watson_developer_cloud/visual_recognition_v3.py", line 96, in update_classifier
params=params, accept_json=True)
File "/usr/local/lib/python2.7/dist-packages/watson_developer_cloud/watson_developer_cloud_service.py", line 268, in request
raise WatsonException(error_message)
watson_developer_cloud.watson_developer_cloud_service.WatsonException: Unknown error
Here is my code:
import json
from watson_developer_cloud import VisualRecognitionV3
from glob import glob
visual_recognition = VisualRecognitionV3('2016-05-20', api_key='xxxx')
myhash = {}
for f in sorted(glob('*.zip'))[20:30]:
clas = f.replace('.zip','');
clas = clas.replace(' ', '-')
clas += '_positive_examples'
myhash[clas] = open(f, 'rb')
print myhash
#print visual_recognition.create_classifier('test', **myhash)
print visual_recognition.update_classifier('test_1337689264', **myhash)
Not sure if this is your problem, but there is an issue for retraining - If your training .zip file included files that are named like images (for example, *.jpg or *.png), but are not actually valid image files, the system will think it has incomplete data when trying to marshal all the vectors for retraining. This can occur frequently when using .zip files from MacOS, which might include a "hidden" __MACOSX folder with files named like this.
A solution for this issue of non-image files causing retraining to fail was deployed on Aug 10, 2016. Classifiers created after that date should not be hindered in retraining from non-image files presence in the training examples. It's not retroactive, though, so the workaround for classifiers trained before that date is to start the training process over with the same input zip files to create a new one, and then delete the old classifier.
I'm working with Python using the standard xmlrpclib library communicating between a JavaServer (that host xmlrpc server) and my python client. I have a problem and I hope that you should solve my issue.
I want to send a DOM instance through xmlrpc protocol. I know that the instance are impossible to manage by xmlrpclib library in Python but this is what I need. I'm trying to understand why is not possible to marshal the class Nodelist and I recognize that the target function is dumps(self, values) in class Marshaller because she couldn't find this type of Python Object.
For Instance, the Fault code is this:
Traceback (most recent call last):
File "testRestSwi.py", line 31, in <module>
conn.xxx.xxx(dom3,"xxx","xxx")
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1572, in __request
allow_none=self.__allow_none)
File "/usr/lib/python2.7/xmlrpclib.py", line 1085, in dumps
data = m.dumps(params)
File "/usr/lib/python2.7/xmlrpclib.py", line 632, in dumps
dump(v, write)
File "/usr/lib/python2.7/xmlrpclib.py", line 654, in __dump
f(self, value, write)
File "/usr/lib/python2.7/xmlrpclib.py", line 756, in dump_instance
self.dump_struct(value.__dict__, write)
File "/usr/lib/python2.7/xmlrpclib.py", line 735, in dump_struct
dump(v, write)
File "/usr/lib/python2.7/xmlrpclib.py", line 646, in __dump
raise TypeError, "cannot marshal %s objects" % type(value)
TypeError: cannot marshal <class 'xml.dom.minicompat.NodeList'> objects
Now, I would want to solve this problem and for sure there are many solutions. But I don't know how to develop or implement something within the xmlrpclib in order to avoid the "marshalling" problem. Keep in mind that the file object must be a DOM, is aim, unchangeable.
For Instance, I'm developing the following:
import xml.dom.minidom as parser
import xmlrpclib
dom3 = parser.parseString("xxx")
conn = xmlrpclib.ServerProxy('xxx',allow_none=True,verbose=True)
conn.xxx.xxx(dom3,"xxx","xxx") #<--- The problem Here.
Might you help me?
I'm a having a problem with the python-docx library. I'm following this page's example however it doesn't seem to work. Can anyone provide some assistance?
>>> all_runs[0]
<docx.text.Run object at 0x7f49b899c5d0>
>>> play=all_runs[0]
>>> play.add_break(WD_BREAK.PAGE)
>>> play.breaks
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Run' object has no attribute 'breaks'
That page is part of the design documentation, not as-built documentation (notice the title 'Candidate' protocol). There is no breaks property implemented on Run as yet.
If you can say a bit about what you're trying to accomplish I may be able to help.
I am trying to create a graph of the connections between the users in my database using nodebox(ubuntu 12.04, python 2.7, django 1.3), but when I enter the following instructions, I get the error message underneath it:
"""
g.draw(weighted=False, directed=False, highlight=[], traffic=None)
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2721, in run_code
exec code_obj in self.user_global_ns, self.user_ns
File "<ipython-input-1-0a219395b690>", line 1, in <module>
g.draw(weighted=False, directed=False, highlight=[], traffic=None)
File "/home/raymond/Documents/exchange/cet/cet/graph/__init__.py", line 453, in draw
self.update()
File "/path/to/myproject//graph/__init__.py", line 416, in update
self.x = _ctx.WIDTH - max.x*self.d - min_.x*self.d
AttributeError: 'NoneType' object has no attribute 'WIDTH'
"""
How can I get nodebox to draw my graph? should I add a try/catch or if statement to graph/init.py in order to prevent object of type none from being written to? I know displaying more code would help solve the problem faster; I am running into formatting problem, but I will add code to the question as soon as I can.
Nodebox has a graph web page which is concise, informative and seemed what I need. As the file to download is on the same page it seemed ideal, BUT it fails to mention it is Nodebox 1 which is for Mac ...only!
Then a further rummage finds NodeBox OpenGL which has a new methodology and is cross platform, BUT it only lists compatibility with python 2.5-2.6 (so with 2.7 should be worth a try?) BUT installing the graphics util Pyglet failed the first time ....so more investigation required.
Sadly Nodebox 3 seems all different and haven't yet seen a comparable graph command
Looking back on the outcome
After achieving an install, I prototyped network viewer with a mouse selectable attribute viewer, low frame rate and a constant data set. A flavour of the interactiveness is demonstrated in this site
http://www.visualthesaurus.com/app/view?word=link
(try a click and drag on the central word)
But problems I needed to solve were how to
show a live data set consistently ie dealing with orphans,
transitions as branches you may be viewing may disappear from the
data
partitioning the viewing of large amounts of data ie view 2 to 20 nodes from several hundred items
displaying data without overlapping/obscuring or going off the display area
displaying differences when not on the current level
portability
...
In my case a periodically updated, structured html table with colours and zoom levels was the best solution