Currently have made a flask application which crashes when I do a postgres database restart, because the cursor which was opened is stale ...
How do I handle this situation. Currently connecting the flask app to postgres via psycopg2....
I am not a database expert...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/var/www/flaskapps/capp/override.py", line 15, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1689, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1356, in full_dispatch_request
rv = self.preprocess_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1539, in preprocess_request
rv = func()
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 321, in _load_user
self.reload_user()
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 350, in reload_user
user = self.user_callback(user_id)
File "/var/www/flaskapps/capp/login_setup.py", line 163, in load_user
cursor.execute(qstr)
File "/usr/share/pyshared/psycopg2/extras.py", line 123, in execute
return _cursor.execute(self, query, vars)
InterfaceError: cursor already closed
This is one of the many cases where your code needs to detect a transient failure and re-try the transaction, re-opening the connection if necessary.
Other cases include deadlocks and serialization failures.
The sqlstate on the exception will let you determine which error cases to retry and how. See the PostgreSQL documentation on error codes for guidance on the meaning of the sqlstate codes.
Sometimes your database interface will through a typed exception that tells you enough just by its data type, too. This doesn't look like one of those cases.
Related
we are trying to use SQLFluff in our project to avoid sql parser errors before deployment.
in our case we have subdirectories which contains sql files.
During Development we are running command sqlfluff lint command in root directory, we found that lint command is working for one level subdirectory sql path like below,
sqlfluff lint demo/complexquery.sql --dialect snowflake
But when we try for 2 level subdirectory sql file path, lint command is not working and giving error as below. could you please let me know whether iam missing syntax.
sqlfluff lint SQLScript/demo/complexquery.sql --dialect snowflake
Traceback (most recent call last):
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 395, in loads
value, vtype = decoder.load_value(multilinestr)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 849, in load_value
raise ValueError("Found tokens after a closed " +
ValueError: Found tokens after a closed string. Invalid TOML.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\ar\AppData\Roaming\Python\Python310\Scripts\sqlfluff.exe_main.py", line 7, in
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 549, in lint
config = get_config(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\cli\commands.py", line 361, in get_config
return FluffConfig.from_root(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 613, in from_root
c = loader.load_config_up_to_path(
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in load_config_up_to_path
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 414, in
[self.load_config_at_path(p) for p in config_paths]
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 338, in load_config_at_path
configs = self.load_config_file(p, fname, configs=configs)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 305, in load_config_file
elems = self._get_config_elems_from_toml(file_path)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\sqlfluff\core\config.py", line 191, in _get_config_elems_from_toml
config = toml.load(fpath)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 134, in load
return loads(ffile.read(), _dict, decoder)
File "C:\Users\ar\AppData\Roaming\Python\Python310\site-packages\toml\decoder.py", line 397, in loads
raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Found tokens after a closed string. Invalid TOML. (line 54 column 1 char 4192)
So, I started with this article https://towardsdatascience.com/predict-age-and-gender-using-convolutional-neural-network-and-opencv-fd90390e3ce6 for age and gender detection and, I am facing a trivial problem. I am not able to run caffe on pictures downloaded from google. Actually, it's running only on the pictures that I take from my phone or webcam. Is there any specific reason or am I doing something incorrectly? Also I wrapping all of this with flask.
for example:- when I feed this image that i took from google search https://www.hanselman.com/blog/content/binary/WindowsLiveWriter/DIYMakingaWideAngleWebcam_1478B/2010-02-16%2023-01-29.283_2.jpg
I get this as my logs:-
127.0.0.1 - - [12/Mar/2020 11:51:57] "?[1m?[35mPOST /predicWithImage HTTP/1.1?[0m" 500 -
Traceback (most recent call last):
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 2463, in __call__
return self.wsgi_app(environ, start_response)
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 2449, in wsgi_app
response = self.handle_exception(e)
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask_cors\extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 1866, in handle_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\_compat.py", line 39, in reraise
raise value
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 1952, in full_dispatch_request
return self.finalize_request(rv)
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 1967, in finalize_request
response = self.make_response(rv)
File "C:\Users\abc\AppData\Local\Continuum\anaconda3\lib\site-
packages\flask\app.py", line 2097, in make_response
"The view function did not return a valid response. The"
TypeError: The view function did not return a valid response. The
function either returned None or ended without a return statement.
vs Logs If I feed the picture taken from my webcam/phone.
Found 1 faces
printing the blob
Gender: Male
Age Range: (15, 20)
127.0.0.1 - - [12/Mar/2020 11:56:07] "?[37mPOST /predicWithImage HTTP/1.1?[0m" 200 -
As you can see i am getting 200 for pictures from webcam vs 500 for google pictures. It's not an issue with flask wrapper, rather I tested the code directly with a picture on my disk into cv.imread(), the Caffe model is not picking it up.
I suspect the code below ran out connection capacity, etc. Is there any interface I can send batch requests with? Or sleep a few ms?
def archive_pending_blobs(bucket: Bucket, blobs: typing.List[Blob], pending_prefix: str,
loaded_prefix: str) -> None:
"""Archive pending blobs to loaded prefix."""
try:
for b in blobs:
bucket.copy_blob(b, bucket, b.name.replace(pending_prefix, loaded_prefix))
bucket.delete_blobs(blobs)
except Exception as e:
print('gcs achieving error for path: {} err: {}'.format(pending_prefix, e))
raise e
Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request six.raise_from(e, None) File "", line 2, in raise_from File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request httplib_response = conn.getresponse()
File "/opt/python3.7/lib/python3.7/http/client.py", line 1321, in getresponse response.begin() File "/opt/python3.7/lib/python3.7/http/client.py", line 296, in begin version, status, reason = self._read_status() File "/opt/python3.7/lib/python3.7/http/client.py", line 257, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/python3.7/lib/python3.7/socket.py", line 589, in readinto return self._sock.recv_into(b) File "/opt/python3.7/lib/python3.7/ssl.py", line 1049, in recv_into return self.read(nbytes, buffer) File "/opt/python3.7/lib/python3.7/ssl.py", line 908, in read return self._sslobj.read(len, buffer) ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/requests/adapters.py", line 445, in send timeout=timeout File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen _stacktrace=sys.exc_info()[2]) File "/env/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 367, in increment raise six.reraise(type(error), error, _stacktrace) File "/env/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise raise value.with_traceback(tb) File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request six.raise_from(e, None) File "", line 2, in raise_from File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request httplib_response = conn.getresponse() File "/opt/python3.7/lib/python3.7/http/client.py", line 1321, in getresponse response.begin() File "/opt/python3.7/lib/python3.7/http/client.py", line 296, in begin version, status, reason = self._read_status() File "/opt/python3.7/lib/python3.7/http/client.py", line 257, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/python3.7/lib/python3.7/socket.py", line 589, in readinto return self._sock.recv_into(b) File "/opt/python3.7/lib/python3.7/ssl.py", line 1049, in recv_into return self.read(nbytes, buffer) File "/opt/python3.7/lib/python3.7/ssl.py", line 908, in read return self._sslobj.read(len, buffer) urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/user_code/main.py", line 230, in bq_merge archive_pending_blobs(bucket, blobs[min_idx:max_idx], pending_prefix, loaded_prefix) File "/user_code/main.py", line 44, in archive_pending_blobs raise e File "/user_code/main.py", line 40, in archive_pending_blobs bucket.copy_blob(b, bucket, b.name.replace(pending_prefix, loaded_prefix)) File "/env/local/lib/python3.7/site-packages/google/cloud/storage/bucket.py", line 711, in copy_blob _target_object=new_blob, File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 290, in api_request headers=headers, target_object=_target_object) File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 183, in _make_request return self._do_request(method, url, headers, data, target_object) File "/env/local/lib/python3.7/site-packages/google/cloud/_http.py", line 212, in _do_request url=url, method=method, headers=headers, data=data) File "/env/local/lib/python3.7/site-packages/google/auth/transport/requests.py", line 201, in request method, url, data=data, headers=request_headers, **kwargs) File "/env/local/lib/python3.7/site-packages/requests/sessions.py", line 512, in request resp = self.send(prep, **send_kwargs) File "/env/local/lib/python3.7/site-packages/requests/sessions.py", line 622, in send r = adapter.send(request, **kwargs) File "/env/local/lib/python3.7/site-packages/requests/adapters.py", line 495, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions_v1beta2/worker.py", li
Per this StackOverflow answer about Connection reset by peer, seems like this is a fatal error were the remote server sends a RST packet to inmediatly drop the connection.
This other SO answer tackles how to solve it. The solution given is using time.sleep but, as we discussed on the comments, didn't work in your case. That's why I'm suggesting a different approach by using truncated exponential backoff:
Truncated exponential backoff is a standard error handling strategy for network applications in which a client periodically retries a failed request with increasing delays between requests.
[...]
Accessing Cloud Storage through a client library. Note that some client libraries, such as the Cloud Storage Client Library for Node.js, have built-in exponential backoff.
There's no built-in exponential backoff for Python, but there's an example of how handle retries in Python with this method.
i have installed openerp. i have created postgres database. While i open ¨localhost:8060¨ in browser, display the followin error:
Client Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/http.py", line 204, in dispatch
response["result"] = method(self, **self.params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 761, in get_list
monodb = db_monodb(req)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 129, in db_monodb
return db_redirect(req, True)[0]
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 109, in db_redirect
dbs = db_list(req, True)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/controllers/main.py", line 90, in db_list
dbs = proxy.list(force)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 30, in proxy_method
result = self.session.send(self.service_name, method, *args)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 103, in send
raise xmlrpclib.Fault(openerp.tools.ustr(e), formatted_info)
Server Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/web/session.py", line 89, in send
return openerp.netsvc.dispatch_rpc(service_name, method, args)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/netsvc.py", line 292, in dispatch_rpc
result = ExportService.getService(service_name).dispatch(method, params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/service/web_services.py", line 122, in dispatch
return fn(*params)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/service/web_services.py", line 359, in exp_list
cr = db.cursor()
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 484, in cursor
return Cursor(self._pool, self.dbname, serialized=serialized)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 182, in __init__
self._cnx = pool.borrow(dsn(dbname))
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 377, in _locked
return fun(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/sql_db.py", line 440, in borrow
result = psycopg2.connect(dsn=dsn, connection_factory=PsycoConnection)
File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
connection_factory=connection_factory, async=async)
OperationalError: FATAL: role "bala" does not exist
otherwise i started like following ¨openerp-server start¨, error following like
Traceback (most recent call last):
File "/usr/local/bin/openerp-server", line 5, in <module>
pkg_resources.run_script('openerp==7.0-20140110-002122', 'openerp-server')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/EGG-INFO/scripts/openerp-server", line 5, in <module>
openerp.cli.main()
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/cli/__init__.py", line 51, in main
__import__(m)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/modules/module.py", line 133, in load_module
mod = imp.load_module('openerp.addons.' + module_part, f, path, descr)
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/l10n_si/__init__.py", line 22, in <module>
import account_wizard
File "/usr/local/lib/python2.7/dist-packages/openerp-7.0_20140110_002122-py2.7.egg/openerp/addons/l10n_si/account_wizard.py", line 22, in <module>
import tools
ImportError: No module named tools
Anyone help for error correction. i cannot clear this error even two days i spend for this.
OperationalError: FATAL: role "bala" does not exist
A user that you have told OpenERP to use does not exist in the database. You probably need to create it. I expect you missed one or more installation steps.
Look for a CREATE USER, CREATE ROLE, or createuser command in the instructions.
GRANT commands or a database ownership assignment may also be nececssary depending on what the user is supposed to be able to do.
(It might also be assumed that you'll know to create the user to connect to the DB as).
My IPython parallel program crashed at yesterday midnight. Below is the error message. I do not know what does this message exactly mean? How can I trace the exception? What does the message "BadStatusLine" mean?
File "/usr/local/lib/python2.7/dist-packages/ipython-0.12-py2.7.egg/IPython/parallel/client/view.py", line 597, in execute
return self._really_apply(util._execute, args=(code,), block=block, targets=targets)
File "<string>", line 2, in _really_apply
File "/usr/local/lib/python2.7/dist-packages/ipython-0.12-py2.7.egg/IPython/parallel/client/view.py", line 59, in sync_results
ret = f(self, *args, **kwargs)
File "<string>", line 2, in _really_apply
File "/usr/local/lib/python2.7/dist-packages/ipython-0.12-py2.7.egg/IPython/parallel/client/view.py", line 48, in save_ids
ret = f(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ipython-0.12-py2.7.egg/IPython/parallel/client/view.py", line 535, in _really_apply
return ar.get()
File "/usr/local/lib/python2.7/dist-packages/ipython-0.12-py2.7.egg/IPython/parallel/client/asyncresult.py", line 104, in get
raise self._exception
IPython.parallel.error.CompositeError: one or more exceptions from call to method: _execute
[11:apply]: BadStatusLine: ''
[12:apply]: BadStatusLine: ''
[13:apply]: BadStatusLine: ''
BadStatusLine is an error from httplib. IPython is not raising this error, your code on the engines is. IPython propagates your errors up from the engines to your client. What does your code do? If it is fetching things from the network, some of those requests are failing with an error code httplib does not understand.