DataError: (DataError) invalid input syntax for integer: "None" - postgresql

Background
We're using Flask to develop a fairly simple application to "scratch an itch" we realized we had while working on a previous project. Session management is handled through the Flask-Login extension, coupled with the Github-Flask extension which we use for user authentication. The app uses Flask-SQLAlchemy and Psycopg2 (2.6.3) to connect to a PostgreSQL 9.1 database where user data is stored.
Issue
When the app is run locally (or deployed to a remote server) for testing we're seeing a strange condition during the first login attempt. The first time I log in I successfully get through the Github authentication sets, but immediately see this error.
DataError: (DataError) invalid input syntax for integer: "None"
LINE 3: WHERE "user".id = 'None'
^
'SELECT "user".id AS user_id, "user".date_created AS user_date_created,
"user".date_modified AS user_date_modified, "user".nickname AS user_nickname,
"user".email AS user_email, "user".about_me AS user_about_me,
"user".github_access_token AS user_github_access_token, "user".github_id AS
user_github_id \nFROM "user" \nWHERE "user".id = %(param_1)s' {'param_1': u'None'}
I can close the browser tab and then revisit the application URL and see the same message. However, if I close the browser window, completely clear my browser's cache, and then attempt to log back in, it works as expected and I'm able to use the application. From that point on I can log out, switch browsers, clear browser cache again, etc., and there don't appear to be any problems -- it's only that first login until browser cache is cleared that this happens.
My testing shows that this happens on any user and two different users can be in two different states at the same time (one has cleared cache and now isn't having problems, the other who hasn't taken those steps is still stuck in the error state.
Originally we used a local SQLite database instead of Postgres. The issue does not exist there. It only has occurred since the switch to Postgres.
I'm at a loss for what steps I can take to remedy this situation. So far the questions I know to ask haven't been able to lead to the right answers when I search online.
For reference, I'll include the full stack trace as well.
Full Stack Trace
Traceback (most recent call last):
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask_debugtoolbar/__init__.py", line 124, in dispatch_request
return view_func(**req.view_args)
File "/Users/dev/PRODUCT/app/modules/mod_profile/controllers.py", line 26, in profile
title='PRODUCT')
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/templating.py", line 126, in render_template
ctx.app.update_template_context(context)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask/app.py", line 716, in update_template_context
context.update(func())
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask_login.py", line 825, in _user_context_processor
return dict(current_user=_get_user())
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask_login.py", line 794, in _get_user
current_app.login_manager._load_user()
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask_login.py", line 363, in _load_user
return self.reload_user()
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/flask_login.py", line 325, in reload_user
user = self.user_callback(user_id)
File "/Users/dev/PRODUCT/app/modules/mod_auth/controllers.py", line 31, in load_user
return User.query.get(id)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 840, in get
return loading.load_on_ident(self, key)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", line 231, in load_on_ident
return q.one()
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2395, in one
ret = list(self)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2438, in __iter__
return self._execute_and_instances(context)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2453, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute
return meth(self, multiparams, params)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
compiled_sql, distilled_params
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
context)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1159, in _handle_dbapi_exception
exc_info
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
context)
File "/Users/dev/PRODUCT/venv/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
cursor.execute(statement, parameters)
DataError: (DataError) invalid input syntax for integer: "None"
LINE 3: WHERE "user".id = 'None'
^
'SELECT "user".id AS user_id, "user".date_created AS user_date_created, "user".date_modified AS user_date_modified, "user".nickname AS user_nickname, "user".email AS user_email, "user".about_me AS user_about_me, "user".github_access_token AS user_github_access_token, "user".github_id AS user_github_id \nFROM "user" \nWHERE "user".id = %(param_1)s' {'param_1': u'None'}

Resolved!
This was a race condition. During the login process our application redirects an authenticated user to their profile page. This was getting triggered correctly, but for new users it was getting called before the user was actually committed to the database.
Visually it happened at the same time, so you could confirm that the callback was supplied with the "right" information, but it wasn't until I deconstructed and then reconstructed the login steps that I realized that it was hitting the route just before the record was actually committed to the database. Since the function that added the record wasn't interrupted by the redirect it did what it was supposed to do, but just a hair late.
Perhaps someone else will find this extract of my code helpful when troubleshooting a similar issue:
## from mod_auth/controllers.py
#mod_auth.before_app_request
def before_request():
g.user = current_user
#mod_auth.route('/login', methods=['GET'])
def login():
if g.user.is_authenticated():
redirect(url_for('mod_home.index'))
return github.authorize()
#login_manager.user_loader
def load_user(id):
return User.query.get(id)
#github.access_token_getter
def token_getter():
user = g.user
if user is not None:
return user.github_access_token
#mod_auth.route('/github')
#github.authorized_handler
def authorized(oauth_token):
next_url = request.args.get('next') or url_for('mod_home.index')
if oauth_token is None:
flash("Authorization failed.")
return redirect(next_url)
user = User.query.filter_by(github_access_token=oauth_token).first()
if user is None:
user = User(github_access_token=oauth_token)
db.session.add(user)
db.session.commit() ## <-- adding a commit here fixed the issue
user.github_access_token = oauth_token
login_user(user)
## login_user called, but because the oauth_token exist and had been
## added to the session, the user was flagged as "is_authenticated"
## and the redirect was triggered
ghinfo = github.get('user')
if 'login' in ghinfo:
user.nickname = ghinfo['login'].lower()
else:
user.nickname = 'nameless'
db.session.commit() ## this was the only commit, originally
return redirect(url_for('mod_profile.profile', username=user.nickname))

Related

Odoo 16: Playing with translations and Boom ! No longer can access my custom model data in my server

After successfully adding a pot file to my new i18n folder in my local machine, as well as setting "translate=True" in a couple of fields in my carddecks module, and verifying that in localhost I could acess my model data, I decided to update my server.
But when I try to access my model data I get the following error:
LINE 1: ..."write_date", COALESCE("carddecks_card"."cardText"->>'pt_PT'...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
Anyone might know what may be causing this?
source code for the module can be found at https://github.com/diogocsc/carddecks
Full error:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/odoo/http.py", line 1579, in _serve_db
return service_model.retrying(self._serve_ir_http, self.env)
File "/usr/lib/python3/dist-packages/odoo/service/model.py", line 134, in retrying
result = func()
File "/usr/lib/python3/dist-packages/odoo/http.py", line 1608, in _serve_ir_http
response = self.dispatcher.dispatch(rule.endpoint, args)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 1805, in dispatch
result = self.request.registry['ir.http']._dispatch(endpoint)
File "/usr/lib/python3/dist-packages/odoo/addons/website/models/ir_http.py", line 235, in _dispatch
response = super()._dispatch(endpoint)
File "/usr/lib/python3/dist-packages/odoo/addons/base/models/ir_http.py", line 144, in _dispatch
result = endpoint(**request.params)
File "/usr/lib/python3/dist-packages/odoo/http.py", line 698, in route_wrapper
result = endpoint(self, *args, **params_ok)
File "/usr/lib/python3/dist-packages/odoo/addons/web/controllers/dataset.py", line 42, in call_kw
return self._call_kw(model, method, args, kwargs)
File "/usr/lib/python3/dist-packages/odoo/addons/web/controllers/dataset.py", line 33, in _call_kw
return call_kw(request.env[model], method, args, kwargs)
File "/usr/lib/python3/dist-packages/odoo/api.py", line 457, in call_kw
result = _call_kw_model(method, model, args, kwargs)
File "/usr/lib/python3/dist-packages/odoo/api.py", line 430, in _call_kw_model
result = method(recs, *args, **kwargs)
File "/usr/lib/python3/dist-packages/odoo/addons/web/models/models.py", line 62, in web_search_read
records = self.search_read(domain, fields, offset=offset, limit=limit, order=order)
File "/usr/lib/python3/dist-packages/odoo/models.py", line 4968, in search_read
result = records.read(fields, **read_kwargs)
File "/usr/lib/python3/dist-packages/odoo/models.py", line 2992, in read
self._read(stored_fields)
File "/usr/lib/python3/dist-packages/odoo/models.py", line 3235, in _read
cr.execute(query_str, params + [sub_ids])
File "/usr/lib/python3/dist-packages/odoo/sql_db.py", line 315, in execute
res = self._obj.execute(query, params)
psycopg2.errors.UndefinedFunction: operator does not exist: character varying ->> unknown
LINE 1: ..."write_date", COALESCE("carddecks_card"."cardText"->>'pt_PT'...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above server error caused the following client error:
OwlError: The following error occurred in onWillStart: "Odoo Server Error"
at wrapError (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1445:77)
at onWillStart (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1451:117)
at useModel (https://www.relationalgames.com/web/assets/505-11285c6/web.assets_backend.min.js:4709:1)
at ListController.setup (https://www.relationalgames.com/web/assets/505-11285c6/web.assets_backend.min.js:4430:645)
at new ComponentNode (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1407:136)
at https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1929:6
at View.slot1 (eval at compile (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1892:370), <anonymous>:15:36)
at callSlot (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1508:25)
at WithSearch.template (eval at compile (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1892:370), <anonymous>:8:12)
at Fiber._render (https://www.relationalgames.com/web/assets/504-41b52e3/web.assets_common.min.js:1336:96)
Caused by: RPC_ERROR: Odoo Server Error
at makeErrorFromResponse (https://www.relationalgames.com/web/assets/505-11285c6/web.assets_backend.min.js:967:163)
at XMLHttpRequest.<anonymous> (https://www.relationalgames.com/web/assets/505-11285c6/web.assets_backend.min.js:974:13)
Found the answer for this. In my localhost I was updating my module. In my server i was just running docker-compose up. Not updating the module in the db.
The solution was going to settings -> apps search by cardecks and upgrade it.

Postgres + Airflow db: permission denied for schema public

I'm trying to setup Postgres as backend database for my Airflow installation.
I followed what the documentation suggests. Particularly, I created a specific user for Airflow in Postgres, a specific new table, and granted all privileges to the user. I executed the commands, similar as suggested:
CREATE DATABASE airflow_db;
CREATE USER airflow_user WITH PASSWORD 'airflow_pass';
GRANT ALL PRIVILEGES ON DATABASE airflow_db TO airflow_user;
Also, I edited the sql_alchemy_conn string in airflow.cfg.
Now, when I run airflow db init, I always get the same error:
DB: postgresql+psycopg2://airflow_user:***#localhost:5434/airflow_db
[2022-11-10 15:19:38,319] {migration.py:204} INFO - Context impl PostgresqlImpl.
[2022-11-10 15:19:38,319] {migration.py:207} INFO - Will assume transactional DDL.
Traceback (most recent call last):
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.InsufficientPrivilege: permission denied for schema public
LINE 2: CREATE TABLE dag_code (
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/phil/airflowv/bin/airflow", line 8, in <module>
sys.exit(main())
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/__main__.py", line 39, in main
args.func(args)
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 52, in command
return func(*args, **kwargs)
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/cli/commands/db_command.py", line 37, in initdb
db.initdb()
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/utils/session.py", line 75, in wrapper
return func(*args, session=session, **kwargs)
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/utils/db.py", line 683, in initdb
_create_db_from_orm(session=session)
File "/Users/phil/airflowv/lib/python3.10/site-packages/airflow/utils/db.py", line 666, in _create_db_from_orm
Base.metadata.create_all(settings.engine)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 4785, in create_all
bind._run_ddl_visitor(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3110, in _run_ddl_visitor
conn._run_ddl_visitor(visitorcallable, element, **kwargs)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2113, in _run_ddl_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
return meth(obj, **kw)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 846, in visit_metadata
self.traverse_single(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
return meth(obj, **kw)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 890, in visit_table
self.connection.execute(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 77, in _execute_on_connection
return connection._execute_ddl(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
ret = self._execute_context(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
self._handle_dbapi_exception(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2026, in _handle_dbapi_exception
util.raise_(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/Users/phil/airflowv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InsufficientPrivilege) permission denied for schema public
LINE 2: CREATE TABLE dag_code (
^
[SQL:
CREATE TABLE dag_code (
fileloc_hash BIGINT NOT NULL,
fileloc VARCHAR(2000) NOT NULL,
last_updated TIMESTAMP WITH TIME ZONE NOT NULL,
source_code TEXT NOT NULL,
CONSTRAINT dag_code_pkey PRIMARY KEY (fileloc_hash)
)
]
(Background on this error at: https://sqlalche.me/e/14/f405)
The SQLAlchemy doc didn't help. I tried to resolve this, but no success. Again, I granted ALL privileges to the airflow user on the new airflow database.
After trying out several different things, I found a simple fix:
ALTER DATABASE airflow_db OWNER TO airflow_user;
sets the owner of the database from the standard/admin user (e.g. postgres) to airflow_user.
Now I run airflow db init again, and the error doesn't come up, instead I get Initialization done.

Unable to set credentials with Google texttoaudio API

I was able to set to set the credentials for Google's translation but with texttospeech I'm having a lot of trouble. First, I couldn't get the credentials as a json file, but got it as just a string. I haven't been able to find anyone else whose credentials are strings every one else has a json file. I converted the string to a json file but I don't think that is helping because it seems that they json object has to be a dictionary. In any case when I try this:
from google.oauth2 import service_account
key1 = 'key.json'
credentials = service_account.Credentials.from_service_account_file(key1)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-13-c7d030662a35>", line 1, in <module>
credentials = service_account.Credentials.from_service_account_file(key1)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/oauth2/service_account.py", line 209, in from_service_account_file
filename, require=['client_email', 'token_uri'])
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_service_account_info.py", line 73, in from_filename
return data, from_dict(data, require=require)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_service_account_info.py", line 46, in from_dict
missing = keys_needed.difference(six.iterkeys(data))
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/six.py", line 575, in iterkeys
return iter(d.keys(**kw))
AttributeError: 'str' object has no attribute 'keys'
When I try this code the following happens:
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'key.json'
client = texttospeech.TextToSpeechClient()
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-14-d30e5cd41087>", line 2, in <module>
client = texttospeech.TextToSpeechClient()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/texttospeech_v1/gapic/text_to_speech_client.py", line 159, in __init__
address=api_endpoint, channel=channel, credentials=credentials
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/texttospeech_v1/gapic/transports/text_to_speech_grpc_transport.py", line 61, in __init__
channel = self.create_channel(address=address, credentials=credentials)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/texttospeech_v1/gapic/transports/text_to_speech_grpc_transport.py", line 91, in create_channel
address, credentials=credentials, scopes=cls._OAUTH_SCOPES, **kwargs
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 177, in create_channel
credentials, _ = google.auth.default(scopes=scopes)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_default.py", line 305, in default
credentials, project_id = checker()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_default.py", line 165, in _get_explicit_environ_credentials
os.environ[environment_vars.CREDENTIALS])
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/auth/_default.py", line 102, in _load_credentials_from_file
credential_type = info.get('type')
AttributeError: 'str' object has no attribute 'get'
I think this is because my json object is not a dict but a string. But the key that Google gave me was a string and not a json file, so I really don't know what to do here. Plus their documentation is too hard to understand.
When you say you have a string, are you referring to the Key ID? You will still need the json file associated with that key.
To create a new json file, go to Google Cloud Console -> IAM & Admin -> Service Accounts. Select one of your service accounts and click "Create Key" which will download the key as a json file. It can only be downloaded once.

flask form post : cannot submit Chinese charactor, only English words works

I cannot submit Chinese words in the form.
English word is OK.
How can I set the utf-8 ?
In the html or app.py? or html code?
Here is my page: http://shiqiu.pw/testpage
PS: I can submit Chinese word in my mac, no problem. But when I deploy my code to my server, Chinese word can not be submitted, it shows Internal Server Error
PPS:I send my post words to mysql, and then show mysql data in new page, perhaps there is something need to set with my mysql?
#myapp.py code part
with connection.cursor() as cursor:
# Create a new recrod
word = request.form.get('word')
print('word')
print(word)
meaning = request.form.get('meaning')
sql = 'INSERT INTO sqdict (word, meaning) VALUES (%s, %s)'
cursor.execute(sql, (word, meaning)) # execute
# connection is not autocommit by default. So you must commit to save your changes.
connection.commit()
with connection.cursor() as cursor:
sql = 'SELECT * from sqdict'
cursor.execute(sql)
# sqlresult = cursor.fetchone() # only show the first row
sqlresult = cursor.fetchall() # all rows
print('sqlresult')
print(sqlresult)
allwords = sqlresult
ERROR LOG BELOW
word
工作
ERROR:flask.app:Exception on /test [POST]
Traceback (most recent call last):
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 1993, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/werkzeug/wrappers/base_response.py", line 269, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/werkzeug/test.py", line 1119, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'InternalError' object is not callable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 2311, in wsgi_app
response = self.full_dispatch_request()
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 1835, in full_dispatch_request
return self.finalize_request(rv)
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 1850, in finalize_request
response = self.make_response(rv)
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 2001, in make_response
reraise(TypeError, new_error, sys.exc_info()[2])
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
raise value.with_traceback(tb)
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/flask/app.py", line 1993, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/werkzeug/wrappers/base_response.py", line 269, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/srv/data/web/vhosts/default/local/lib/python3.7/site-packages/werkzeug/test.py", line 1119, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'InternalError' object is not callable
The view function did not return a valid response. The return type must be a string, tuple, Response instance, or WSGI callable, but it was a InternalError.
My input Chinese charactor is showed as '工作'
I don't know which is the Internalerror object.
Full code is at github,code maybe not the same, but not big difference.
I have tried request.form.getunicode('word').
Also, I tried app.config['JSON_AS_ASCII' = False).
Or, set <meta charset="UTF-8"> in the html code.
BUT, no one works.
I hope the form can support Chinese words as value.

Celery backend cleanup failing with SQLAlchemy & MySQL

I'm facing following exception when celery is trying to cleanup back-end.
Most probably, this is happening due to MySQL disconnect issue and can be solved by using pool_recycle parameter and retrying the task later. But this is out of my hand - I guess celery needs to provide support for this?
Now my question is, what is backend cleanup task and how such a failed task may affect our system?
Log:
[2014-04-08 04:00:00,017: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2014-04-08 04:00:00,020: INFO/MainProcess] Received task: celery.backend_cleanup[b70acd50-e72d-43b1-a702-0bfa8e7e83a6] expires:[2014-04-08 16:00:00.018317+01:00]
[2014-04-08 04:00:00,036: ERROR/MainProcess] Task celery.backend_cleanup[b70acd50-e72d-43b1-a702-0bfa8e7e83a6] raised unexpected: OperationalError('(OperationalError) MySQL Connection not available.',)
Traceback (most recent call last):
File "/webapps/phoenix/lib/python3.3/site-packages/celery/app/trace.py", line 238, in trace_task
R = retval = fun(*args, **kwargs)
File "/webapps/phoenix/lib/python3.3/site-packages/celery/app/trace.py", line 416, in __protected_call__
return self.run(*args, **kwargs)
File "/webapps/phoenix/lib/python3.3/site-packages/celery/app/builtins.py", line 56, in backend_cleanup
app.backend.cleanup()
File "/webapps/phoenix/lib/python3.3/site-packages/celery/backends/database/__init__.py", line 180, in cleanup
Task.date_done < (now - expires)).delete()
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/orm/query.py", line 2626, in delete
delete_op.exec_()
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/orm/persistence.py", line 866, in exec_
self._do_exec()
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/orm/persistence.py", line 991, in _do_exec
params=self.query._params)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/orm/session.py", line 978, in execute
clause, params or {})
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/base.py", line 664, in execute
return meth(self, multiparams, params)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/sql/elements.py", line 282, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/base.py", line 828, in _execute_context
None, None)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/base.py", line 1023, in _handle_dbapi_exception
exc_info
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/util/compat.py", line 174, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/util/compat.py", line 167, in reraise
raise value.with_traceback(tb)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/base.py", line 824, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 507, in _init_compiled
self.cursor = self.create_cursor()
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 671, in create_cursor
return self._dbapi_connection.cursor()
File "/webapps/phoenix/lib/python3.3/site-packages/sqlalchemy/pool.py", line 548, in cursor
return self.connection.cursor(*args, **kwargs)
File "/webapps/phoenix/lib/python3.3/site-packages/mysql/connector/connection.py", line 1231, in cursor
raise errors.OperationalError("MySQL Connection not available.")
sqlalchemy.exc.OperationalError: (OperationalError) MySQL Connection not available. 'DELETE FROM celery_taskmeta WHERE celery_taskmeta.date_done < %(date_done_1)s' [{}]
PS I've checked this SO question but seems like it's due to a different exception: Celery log shows cleanup failed
The closest Celery has to retrying is short lived sessions.
The task is cleaning out un-read task results. If it's failing, you may see those results start to build up, but should be OK otherwise.
You're right that there's very little documentation about it!