psycopg2: "UnboundLocalError","evalue":"local variable 'connection' referenced before assignment" - postgresql
I have a psycopg2 connection which I am using to connect to postgresql from pyspark. Here is my code -
host = 'IP Address'
port = 'Port'
user = 'postgres'
db = 'postgres'
password = 'password'
def move_records(main_table,stg_table):
try:
connection = psycopg2.connect(host=host,
database=db,
user=user,
password=password,
#driver = driver,
port = port)
cursor = connection.cursor()
move_query = "INSERT INTO " +main_table+ " select * from "+stg_table+" where country ='USA'"
cursor.execute(move_query)
connection.commit()
logger.debug("Record moved successfully")
except (Exception, psycopg2.DatabaseError) as error :
logger.error("%s Error in transction Reverting all other operations of a transaction ", error)
global flag
flag = False
connection.rollback()
finally:
if(connection):
cursor.close()
connection.close()
logger.debug("PostgreSQL connection is closed")
move_records(table_1,table_2)
But I keep getting error below error on line if(connection):
"UnboundLocalError","evalue":"local variable 'connection' referenced before assignment"
Can not figure out what is the issue. Need help.
I am no expert in Python but I have worked on similar thing, connecting from Python to Postgres in in AWS Lambda using psycopg2.
I believe the error lies somewhere in scope of variable. You need to declare all variables(host, port, user, db, password) inside function once again as global or nonlocal and then try to run function.
For your reference, check out this link:-
Related
How can I connect to postgres data base by input parameters?
I'm trying to connect to a postgres database by using psycopg2. I ask for the user and user password via input. But I think, this way doesnt let me connect to the database. Is there another way to place the values?
I try to connect like this:
connection = psycopg2.connect("host=localhost dbname=dbname user=%s password=%s", (username, password))
This is the error I get:
connection = _connect(dsn, connection_factory=connection_factory, **kwasync)
TypeError: 'tuple' object is not callable
There are a couple of ways to build the connection string from inputs, first:
# From input
user_name = 'some_name'
pwd = 'some_pwd'
connection = psycopg2.connect(host=localhost, dbname=dbname, user=user_name, password=pwd)
Second from here Make dsn:
from psycopg2.extensions import make_dsn
dsn = make_dsn('host=localhost dbname=dbname', user=user_name, password=pwd)
connection = psycopg2.connect(dsn)
UPDATE, forgot the most obvious way:
dsn = 'host=localhost dbname=dbname user=' + user_name + ' password=' + pwd
connection = psycopg2.connect(dsn)
sqlalchemy connect to databasename with whitespace
I'm trying to connect to a postgres-DB, which unfortunately has a name with a whitespace in it:
%load_ext sql
from sqlalchemy import create_engine
%sql postgresql://postgres:dbpass#localhost/Test DB
(psycopg2.OperationalError) FATAL: database "Test DB" does not exist
I've tried to follow some tipps on the internet and used:
import urllib.parse
urllib.parse.quote_plus("Test DB")
which simply results in a string "Test+DB" (this does not work).
How can I adress the database, without changing its name?
Best regards!
I was able to solve it by using sqlalchemy's create_engine(), therefore being able to simply save the string (with its whitespace) as a variable (eg database_name):
import sqlalchemy as db
database_name = 'Test DB'
engine = db.create_engine('postgresql://' + 'user_name' + ':' + 'password' + '#localhost/' + database_name)
connection = engine.connect()
s = 'SELECT id FROM user'
df = pd.read_sql_query(s, engine)
df.head()
Hope it helps, best regards.
Password change plugin in poste.io fails
I am experiment with poste.io mail server. It uses rouncube as its web interface. I tried to enable the password plugin.
I see below error whenever I try to set the password:
[21-Mar-2017 13:00:31 +0100]: DB Error: [1] no such function: update_passwd (SQL Query: SELECT update_passwd('$1$LXeDlIT0$NGunS8gcCOSrKK2ZJ6RIW/', 'naidu#example.com')) in /opt/www/webmail/program/lib/Roundcube/rcube_db.php on line 539 (POST /webmail/?_task=settings&_action=plugin.password-save)
The internet is full of using mysql as the database. I think I have to update the password change query in
/opt/www/webmail/plugins/password/config.inc.php
from
$sql = 'SELECT update_passwd(%c, %u)';
to
$sql = 'UPDATE mailaccount SET password=%c WHERE nname=%u LIMIT 1';
The UPDATE statement above is valid for mysql. What is the equivalent for sqlite3 database?
I have a setup consist of postfix with sqlite and my sql query is as below: UPDATE mailbox SET password=%c WHERE username=%u LIMIT 1 My sqlite config as below: $config['password_db_dsn'] = 'sqlite:////var/vmail/postfixadmin.db?mode=0646'; $config['password_query'] = 'UPDATE mailbox SET password=%c WHERE username=%u LIMIT 1'; Add this for debugging: $config['debug_level'] = 4; $config['sql_debug'] = true; $config['imap_debug'] = true; $config['ldap_debug'] = true; $config['smtp_debug'] = true; Hope this helps.
How can I check the connection of Mongoid
Does Mongoid has any method like ActiveRecord::Base.connected?? I want to check if the connection that's accessible.
We wanted to implement a health check for our running Mongoid client that tells us whether the established connection is still alive. This is what we came up with: Mongoid.default_client.database_names.present? Basically it takes your current client and tries to query the databases on its connected server. If this server is down, you will run into a timeout, which you can catch.
My solution:
def check_mongoid_connection
mongoid_config = File.read("#{Rails.root}/config/mongoid.yml")
config = YAML.load(mongoid_config)[Rails.env].symbolize_keys
host, db_name, user_name, password = config[:host], config[:database], config[:username], config[:password]
port = config[:port] || Mongo::Connection::DEFAULT_PORT
db_connection = Mongo::Connection.new(host, port).db(db_name)
db_connection.authenticate(user_name, password) unless (user_name.nil? || password.nil?)
db_connection.collection_names
return { status: :ok }
rescue Exception => e
return { status: :error, data: { message: e.to_s } }
end
snrlx's answer is great.
I use following in my puma config file, FYI:
before_fork do
begin
# load configuration
Mongoid.load!(File.expand_path('../../mongoid.yml', __dir__), :development)
fail('Default client db check failed, is db connective?') unless Mongoid.default_client.database_names.present?
rescue => exception
# raise runtime error
fail("connect to database failed: #{exception.message}")
end
end
One thing to remind is the default server_selection_timeout is 30 seconds, which is too long for db status check at least in development, you can modify this in your mongoid.yml.
Erlang and PostgreSQL
I try to execute simple PostgreSQL query with erlang and epgsql.
I do:
{ok, C} = pgsql:connect("localhost", "shk", "qwerty", [{database, "mydb"}]).
>> {ok,<0.34.0>}
Then:
{ok, Cols, Rows} = pgsql:squery(C, "select * from users").
But i got error:
=ERROR REPORT==== 27-Apr-2012::17:58:23 ===
** State machine <0.34.0> terminating
** Last message in was {'EXIT',<0.32.0>,
{{badmatch,
{error,
{error,'ð\236ð¨ð\230ð\221ð\232ð\220',<<"42P01">>,
<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>,
[{position,<<"15">>}]}}},
[{erl_eval,expr,3}]}}
** When State == ready
** Data == {state,undefined,<0.35.0>,5000,
[{<<"application_name">>,<<>>},
{<<"client_encoding">>,<<"UTF8">>},
{<<"DateStyle">>,<<"ISO, DMY">>},
{<<"integer_datetimes">>,<<"on">>},
{<<"IntervalStyle">>,<<"postgres">>},
{<<"is_superuser">>,<<"off">>},
{<<"server_encoding">>,<<"UTF8">>},
{<<"server_version">>,<<"9.0.7">>},
{<<"session_authorization">>,<<"shk">>},
{<<"standard_conforming_strings">>,<<"off">>},
{<<"TimeZone">>,<<"posixrules">>}],
undefined,undefined,undefined,
{30932,488494147},
{statement,undefined,undefined,undefined},
73}
** Reason for termination =
** {{badmatch,{error,{error,'ð\236ð¨ð\230ð\221ð\232ð\220',<<"42P01">>,
<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>,
[{position,<<"15">>}]}}},
[{erl_eval,expr,3}]}
What's wrong i do? How can i fix it?
Thank you.
The error string seems to be in russian if I recognize the characters.
To view the response you can use the following command:
io:format("~ts",[<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>]).
отношение "mydb" не существует
A quick google translate makes me think the database mydb does not exist or you do not have permissions to use it.
try simply doing
Response = pgsql:squery(C, "select * from mydb"),
io:format("~p~n",[Response]).
And see what he is giving back from the server, maybe you have typo or table don't exists etc.
also check this out http://www.erlangatwork.com/2009/01/erlang-and-postgresql.html
From epgsql docs:
Errors
Errors originating from the PostgreSQL backend are returned as {error, #error{}},
see pgsql.hrl for the record definition. epgsql functions may also return
{error, What} where What is one of the following:
{unsupported_auth_method, Method} - required auth method is unsupported
timeout - request timed out
closed - connection was closed
sync_required - error occured and pgsql:sync must be called
Try to include pgsql.hrl, capture the error and print the error message, that should point you to the right direction.