Looking at the output of select * from pg_stat_activity;, I see a column called application_name, described here.
I see psql sets this value correctly (to psql...), but my application code (psycopg2/SQLAlchemy) leaves it blank.
I'd like to set this to something useful, like web.1, web.2, etc, so I could later on correlate what I see in pg_stat_activity with what I see in my application logs.
I couldn't find how to set this field using SQLAlchemy (and if push comes to shove - even with raw sql; I'm using PostgresSQL 9.1.7 on Heroku, if that matters).
Am I missing something obvious?
the answer to this is a combination of:
http://initd.org/psycopg/docs/module.html#psycopg2.connect
Any other connection parameter supported by the client library/server can be passed either in the connection string or as keywords. The PostgreSQL documentation contains the complete list of the supported parameters. Also note that the same parameters can be passed to the client library using environment variables.
where the variable we need is:
http://www.postgresql.org/docs/current/static/runtime-config-logging.html#GUC-APPLICATION-NAME
The application_name can be any string of less than NAMEDATALEN characters (64 characters in a standard build). It is typically set by an application upon connection to the server. The name will be displayed in the pg_stat_activity view and included in CSV log entries. It can also be included in regular log entries via the log_line_prefix parameter. Only printable ASCII characters may be used in the application_name value. Other characters will be replaced with question marks (?).
combined with :
http://docs.sqlalchemy.org/en/rel_0_8/core/engines.html#custom-dbapi-args
String-based arguments can be passed directly from the URL string as query arguments: (example...) create_engine() also takes an argument connect_args which is an additional dictionary that will be passed to connect(). This can be used when arguments of a type other than string are required, and SQLAlchemy’s database connector has no type conversion logic present for that parameter
from that we get:
e = create_engine("postgresql://scott:tiger#localhost/test?application_name=myapp")
or:
e = create_engine("postgresql://scott:tiger#localhost/test",
connect_args={"application_name":"myapp"})
If you're using asyncpg driver, you should use
conn = await asyncpg.connect(server_settings={'application_name': 'foo'})
src - https://github.com/MagicStack/asyncpg/issues/204#issuecomment-333917251
Related
In a parameterized query issued from c# code to PostgreSQL 10.14 via dotConnect 7.7.832 .NET connector, I select either a parameter value or the local timestamp, if the parameter is NULL:
using (var cmd = new PgSqlCommand("select COALESCE(#eventTime, LOCALTIMESTAMP)", connection)
When executed, this statement throws the error in subject. If I comment out the corresponding parameter
cmd.Parameters.Add("#eventTime", PgSqlType.TimeStamp).Value = DateTime.Now;
and hardcode
using (var cmd = new PgSqlCommand("select COALESCE('11/6/2020 2:36:58 PM', LOCALTIMESTAMP)", connection)
or if I cast the parameter
using (var cmd = new PgSqlCommand("select COALESCE(cast(#eventTime as timestamp without time zone), LOCALTIMESTAMP)", connection)
then it works. Can anyone explain what # operator in the error is referring to and why the error?
In the case that doesn't work, your .Net connection library seems to be passing an SQL command containing a literal # to the database, rather than substituting it. The database assumes you are trying to use # as a user defined operator, as it doesn't know what else it could possibly be. But no such operator has been defined.
Why is it doing that? I have no idea. That is a question about your .Net connection library, not about PostgreSQL itself, so you might want to add tag.
The error message you get from the database should include the text of the query it received (as opposed to the text you think it was sent) and it is often useful to see that in situations like this. If that text is not present in the client's error message (some connection libraries do not faithfully pass this info along) you should be able to pull it directly from the PostgreSQL server's log file.
For performance reasons, I'm trying to limit the round trips I make to the database. Currently I connect and the issue a series of statements like SET TIMEZONE='America/New_York' to postgres. Many databases allow session variables and similar settings to be passed to the database either as part of the jdbc url i.e. jdbc:postgresql://localhost:5435/test?timezone=America/New_York or by adding the it to the property list like this...
String url = "jdbc:postgresql://localhost/test:5435";
Properties props = new Properties();
props.setProperty("user","fred");
props.setProperty("password","secret");
props.setProperty("timezone","America/New_York");
Connection conn = DriverManager.getConnection(url, props);
I can't find definitive information if this is supported by postgres and tests have failed.
The documentation says:
Connection Parameters
[...]
options = String
Specify 'options' connection initialization parameter.
The value of this property may contain spaces or other special characters, and it should be properly encoded if provided in the connection URL. Spaces are considered to separate command-line arguments, unless escaped with a backslash (\); \\ represents a literal backslash.
So you should use
props.setProperty("options", "-c timezone=America/New_York");
I have a Python script that runs a pgSQL file through SQLAlchemy's connection.execute function. Here's the block of code in Python:
results = pg_conn.execute(sql_cmd, beg_date = datetime.date(2015,4,1), end_date = datetime.date(2015,4,30))
And here's one of the areas where the variable gets inputted in my SQL:
WHERE
( dv.date >= %(beg_date)s AND
dv.date <= %(end_date)s)
When I run this, I get a cryptic python error:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) argument formats can't be mixed
…followed by a huge dump of the offending SQL query. I've run this exact code with the same variable convention before. Why isn't it working this time?
I encountered a similar issue as Nikhil. I have a query with LIKE clauses which worked until I modified it to include a bind variable, at which point I received the following error:
DatabaseError: Execution failed on sql '...': argument formats can't be mixed
The solution is not to give up on the LIKE clause. That would be pretty crazy if psycopg2 simply didn't permit LIKE clauses. Rather, we can escape the literal % with %%. For example, the following query:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%';
would need to be modified to:
SELECT *
FROM people
WHERE start_date > %(beg_date)s
AND name LIKE 'John%%';
More details in the pscopg2 docs: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
As it turned out, I had used a SQL LIKE operator in the new SQL query, and the % operand was messing with Python's escaping capability. For instance:
dv.device LIKE 'iPhone%' or
dv.device LIKE '%Phone'
Another answer offered a way to un-escape and re-escape, which I felt would add unnecessary complexity to otherwise simple code. Instead, I used pgSQL's ability to handle regex to modify the SQL query itself. This changed the above portion of the query to:
dv.device ~ E'iPhone.*' or
dv.device ~ E'.*Phone$'
So for others: you may need to change your LIKE operators to regex '~' to get it to work. Just remember that it'll be WAY slower for large queries. (More info here.)
For me it's turn out I have % in sql comment
/* Any future change in the testing size will not require
a change here... even if we do a 100% test
*/
This works fine:
/* Any future change in the testing size will not require
a change here... even if we do a 100pct test
*/
I have an MS SQL Server 2008 Database, from which I am fetching data using perl DBD::Sybase module. But there are some special characters in the DB, like the Copyright symbol, Trademark symbol etc., which are not getting imported properly. Perl seems to change all of these special characters to a Question mark character. Is there a way to fix this?
I have tried specifying charset=utf8 in the connection string. The doc mentions a syb_enable_utf8 (bool) setting, but whenever I try that, I get an error:
Can't locate object method "syb_enable_utf8" via package "DBI::db"
One solution I found was this:
use Encode qw(encode_utf8);
Then, wherever you are writing data to a file or anywhere else, use Encode::encode_utf8($data);
where $data is the column/value which you have fetched from MSSQL.
I don't use DBD::Sybase but a) I use a lot of other DBDs and b) I am currently collecting information about unicode support in DBDs. According to the pod you need at least OpenClient 15.x when using syb_enable_utf8. Are you using 15.x or later? Perhaps syb_enable_utf8 is not defined if your client is less than 15.x or perhaps you have too old a version of DBD::Sybase. Unfortunately I cannot see from the Changes file when syb_enable_utf8 was added.
However, when you say "can't locate method" I think that is a clue as syb_enable_utf8 is not a method, it is an attribute (it is under Sybase Specific Attributes) in the pod. So you need to add it to your connect call or set it via a connection handle like this:
my $h = DBI->connect("dbi:Sybase:something","user","password", {syb_enable_utf8 => 1});
or
$h->{syb_enable_utf8} = 1;
You should also read the bits in the pod describing what happens when syb_enable_utf8 is set as it appears from the documents it only applies to UNIVARCHAR, UNICHAR, and UNITEXT columns.
Lastly, you need to ensure you insert the data correctly in the first place. I'd guess if it is not inserted from Perl with syb_enable_utf8 and charset=utf8 and your data is not proper unicode characters in Perl before you insert you'll get garbage back.
The comment Raze2dust made had nothing to do with your issue but is worth heeding if you are going to write the data retrieved from your database elsewhere. Just remember to decode any data input to your script and encode any data output.
I m using a update Query.
i.e:-
UPDATE tbl_ecpuser
SET ecpuser_fullname = 'Operator',
ecpuser_password = encrypt(E'Op1111/1\1/1\1' , 'ENCRYPE_KEY', 'ENCRYPE_ALGORITHM'),
where ecpuser_key = '0949600348'
Query is Executing Successfully.
But when I m trying to retrive the value for the Column ecpuser_password, it
returns with some extra character (i.e-00)
The Query for the Retrive the Password is:-
SELECT
decrypt(ecpuser_password,'ENCRYPE_KEY','ENCRYPE_ALGORITHM') AS PASSWORD
FROM tbl_ecpuser
WHERE
ecpuser_key = '0949600348'
This query returens
"Op1111/1\001/1\001"
but it should return "Op1111/1\1/1\1" and I need this.
So can any body help me about this.
Thanks.
One place where PostgreSQL was not conforming to the SQL standard was the treatment of a backslash in string literals.
Since 8.2 a configuration property standard_conforming_strings is available that configures PostgreSQL to comply with the standard here.
If you set that to "on" '\1' is treated correctly as a string with two characters (one backslash and the character 1).
However if that is turned on, the prefix E enables escape sequences again.
So (if I understand your problem correctly) you should set standard_conforming_strings = on and specify the string literal without the leading E.
Seems like E'\1' is treated as chr(1) and returned accordingly.
You probably want: E'\\1'.