I was trying to trace the slow queries. I'm new to Pg9.6.
I could not find the /pg_log/ folder in the new version. It was available in /data/pg_log/ in older versions(I was using 9.2)..
If this is a repeating question, please tag.
connect to your postgres and run:
t=# show log_directory;
log_directory
---------------
pg_log
(1 row)
t=# show logging_collector ;
logging_collector
-------------------
on
(1 row)
https://www.postgresql.org/docs/9.6/static/runtime-config-logging.html
log_directory (string)
When logging_collector is enabled, this parameter determines the
directory in which log files will be created. It can be specified as
an absolute path, or relative to the cluster data directory. This
parameter can only be set in the postgresql.conf file or on the server
command line. The default is pg_log.
You could also want to check all not default values with
select name,setting from pg_settings where source <>'default' and name like 'log%';
Related
db2 how to configure external tables using extbl_location, extbl_strict_io. Could you please give insert example for system table how to set up this parameters. I need to create external table and upload data to external table.
I need to know how to configure parameters extbl_location, extbl_strict_io.
I created table like this.
CREATE EXTERNAL TABLE textteacher(ID int, Name char(50), email varchar(255)) USING ( DATAOBJECT 'teacher.csv' FORMAT TEXT CCSID 1208 DELIMITER '|' REMOTESOURCE 'LOCAL' SOCKETBUFSIZE 30000 LOGDIR '/tmp/logs' );
and tried to upload data to it.
insert into textteacher (ID,Name,email) select id,name,email from teacher;
and get exception [428IB][-20569] The external table operation failed due to a problem with the corresponding data file or diagnostic files. File name: "teacher.csv". Reason code: "1".. SQLCODE=-20569, SQLSTATE=428IB, DRIVER=4.26.14
If I correct understand documentation parameter extbl_location should pointed directory where data will save. I suppose full directory will showed like
$extbl_location+'/'+teacher.csv
I found some documentation about error
https://www.ibm.com/support/pages/how-resolve-sql20569n-error-external-table-operation
I tried to run command in docker command line.
/opt/ibm/db2/V11.5/bin/db2 get db cfg | grep -i external
but does not information about external any tables.
CREATE EXTERNAL TABLE statement:
file-name
...
When both the REMOTESOURCE option is set to LOCAL (this is its default value) and the extbl_strict_io configuration parameter is set
to NO, the path to the external table file is an absolute path and
must be one of the paths specified by the extbl_location configuration
parameter. Otherwise, the path to the external table file is relative
to the path that is specified by the extbl_location configuration
parameter followed by the authorization ID of the table definer. For
example, if extbl_location is set to /home/xyz and the authorization
ID of the table definer is user1, the path to the external table file
is relative to /home/xyz/user1/.
So, If you use relative path to a file as teacher.csv, you must set extbl_strict_io to YES.
For an unload operation, the following conditions apply:
If the file exists, it is overwritten.
Required permissions:
If the external table is a named external table, the owner must have read and write permission for the directory of this file.
If the external table is transient, the authorization ID of the statement must have read and write permission for the directory of this file.
Moreover you must create a sub-directory equal to your username (in lowercase) which is owner of this table in the directory specified in extbl_location and ensure, that this user (not the instance owner) has rw permission to this sub-directory.
Update:
To setup presuming, that user1 runs this INSERT statement.
sudo mkdir -p /home/xyz/user1
# user1 must have an ability to cd to this directory
sudo chown user1:$(id -gn user1) /home/xyz/user1
db2 connect to mydb
db2 update db cfg using extbl_location /home/xyz extbl_strict_io YES
I am trying to encrypt a column in my prostrgres DB. The column name is "test" of type "bytea".
My enity code is below,
#ColumnTransformer(read = "pgp_sym_decrypt(" + " test, "
+ " current_setting('encrypt.key')"
+ ")", write = "pgp_sym_encrypt( " + " ?, "
+ " current_setting('encrypt.key')" + ") ")
#Column(columnDefinition = "bytea")
private String test;
postgresql.conf configuration file:
encrypt.key = 'Wow! So much security.
Placed the postgresql.conf configuration file in src/main/resources of spring boot appln. But the encryption.key value is not being picked up. And is there a way to pass the key using application.properties?
postgresql.conf is the configuration file for the Postgres server. It's stored inside the data directory (aka "cluster") on the server.
You can't put it on the client side (where your application runs). It has no meaning there.
To change values in there, you need to edit the file (on the server) or use ALTER SYSTEM.
If you want to change a configuration setting for the current session, use SET or set_config()
The latter two are probably the ones you are looking for to initialize the custom property for your encryption/decryption functions.
The way to use encrypt.key, not only for current session, it's store it in postgresql.conf.
The correct place is at the end of this file, in the "Customized Options" section:
#------------------------------------------------------------------------------
# CUSTOMIZED OPTIONS
#------------------------------------------------------------------------------
# Add settings for extensions here
encrypt.key=123456
Reload the configuration of the database server:
systemctl reload postgresql.service
To testing if it's working correctly. Open a pgsql session and type:
mydb=# show encrypt.key;
encrypt.key
-------------
123456
(1 row)
Example of encrypt:
mydb=# select pgp_sym_encrypt('Hola mundo',current_setting('encrypt.key'));
pgp_sym_encrypt
------------------------------------------------------------------------------------------------------------------------------------------------------------
\xc30d04070302255230e388dfe25e7dd23b01c5b8e62d148088a3417d3c27ed2cc11655d863b271672b9f076fffb82f1a7f074f2ecbe973df04642cd7a4f76ca5cff4a13b9a71e7cc6e693827
(1 row)
Example of decrypt:
mydb=# select pgp_sym_decrypt('\xc30d04070302255230e388dfe25e7dd23b01c5b8e62d148088a3417d3c27ed2cc11655d863b271672b9f076fffb82f1a7f074f2ecbe973df04642cd7a4f76ca5cff4a13b9a71e7cc6e693827',current_setting('encrypt.key'));
pgp_sym_decrypt
-----------------
Hola mundo
(1 row)
What is the reason for this query give no results:
SELECT name FROM users WHERE LOWER(name) LIKE LOWER('%после%');
When this works fine:
SELECT name FROM users WHERE LOWER(name) LIKE LOWER('%После%');
Name is 'Последователь'. If i use name like 'Post', then search works fine.
Version: PostgreSQL 11.2 (Ubuntu 11.2-100) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0, 64-bit
Server and db encoding is UTF8. Client encoding is UNICODE.
The lower function works according to the database collation, so the result will vary depending on how your database is defined.
It will work well in this case:
test=# CREATE DATABASE rus TEMPLATE template0
ENCODING UTF8 LC_COLLATE "ru_RU.utf8" LC_CTYPE "ru_RU.utf8";
CREATE DATABASE
test=# \c rus
You are now connected to database "rus" as user "postgres".
rus=# SELECT LOWER('%после%') = LOWER('%После%');
?column?
----------
t
(1 row)
But it won't work with the C collation, because that does not know how to properly lower case cyrillic characters:
rus=# \c test
You are now connected to database "test" as user "postgres".
test=# DROP DATABASE rus;
DROP DATABASE
test=# CREATE DATABASE rus TEMPLATE template0
ENCODING UTF8 LC_COLLATE "C" LC_CTYPE "C";
CREATE DATABASE
test=# \c rus
You are now connected to database "rus" as user "postgres".
rus=# SELECT LOWER('%после%') = LOWER('%После%');
?column?
----------
f
(1 row)
So if you want your query to work well, make sure that you are using a collation that knows how to convert the characters to upper and lower case.
Functions LOWER and UPPER do not work correct if database has wrong locale.
Step-1. View locale of databases in Postgres:
\l
Step-2. View available locales in terminal:
locale -a
or by your language, for example ru:
locale -a | grep ru
Step-3. Use locale name from Step-2(for example ru_RU.utf8) in updating database YOUR_DATABASE_NAME in Postgres:
update pg_database set datcollate='ru_RU.utf8', datctype='ru_RU.utf8' where datname='YOUR_DATABASE_NAME';
or you can create new database with locale:
CREATE DATABASE NEW_DATABASE_NAME TEMPLATE template0 ENCODING UTF8 LC_COLLATE "ru_RU.UTF-8" LC_CTYPE "ru_RU.UTF-8";
PS My macOS has name ru_RU.UTF-8, Ubuntu has name ru_RU.utf8
Look Closer. LIKE statement is sometimes case sensitive. Depens on the program you use
The first statement is :
SELECT name FROM users WHERE LOWER(name) LIKE LOWER('%после%');
while the second is :
SELECT name FROM users WHERE LOWER(name) LIKE LOWER('%После%');
What you seek is
П оследователь
note that in the first statement you wrote with (п) while the second one is (П)
In the PostgreSQL shell I can type:
SHOW log_directory
and I can see where the log file is saved.
The question is - is there a SELECT statement which will give me the same information?
TIA!
Yes, these two are equivalent:
knayak=# show log_directory;
log_directory
---------------
log
(1 row)
knayak=# select setting FROM pg_settings where name = 'log_directory';
setting
---------
log
(1 row)
I've postgresql-9.4 up and running, and I've enabled pg_stat_statements module lately by the help of official documentation.
But I'm getting following error upon usage:
postgres=# SELECT * FROM pg_stat_statements;
ERROR: relation "pg_stat_statements" does not exist
LINE 1: SELECT * FROM pg_stat_statements;
postgres=# SELECT pg_stat_statements_reset();
ERROR: function pg_stat_statements_reset() does not exist
LINE 1: SELECT pg_stat_statements_reset();
I'm logged in to psql with the postgres user.
I've also checked the available extension lists:
postgres=# SELECT * FROM pg_available_extensions WHERE name = 'pg_stat_statements'
;
name | default_version | installed_version | comment
--------------------+-----------------+-------------------+-----------------------------------------------------------
pg_stat_statements | 1.2 | | track execution statistics of all SQL statements executed
(1 row)
And here's the results of the extension versions query:
postgres=# SELECT * FROM pg_available_extension_versions WHERE name = 'pg_stat_statements';
name | version | installed | superuser | relocatable | schema | requires | comment
--------------------+---------+-----------+-----------+-------------+--------+----------+-----------------------------------------------------------
pg_stat_statements | 1.2 | f | t | t | | | track execution statistics of all SQL statements executed
(1 row)
Any help will be appreciated.
Extension isn't installed:
SELECT *
FROM pg_available_extensions
WHERE
name = 'pg_stat_statements' and
installed_version is not null;
If the table is empty, create the extension:
CREATE EXTENSION pg_stat_statements;
I've faced with this issue at configuring Percona Monitoring and Management (PMM) because by some strange reason PMM connecting to database with name postgres, so pg_stat_statements extension have to be created in this database:
yourdb# \c postgres
postgres# CREATE EXTENSION pg_stat_statements SCHEMA public;
Follow below steps:
Create the extension
CREATE EXTENSION pg_stat_statements;
Change in config
alter system set shared_preload_libraries='pg_stat_statements';
Restart
$ systemctl restart postgresql
Verify changes applied or not.
select * from pg_file_Settings where name='shared_preload_libraries';
The applied attribute must be 'true'.
I Had the same issue when deploying the environment using liquibase for the first time.
I understand that my reply maybe is not related with your problem but was the first google result so I think that other guys like me can arrive here with my the same Liquibase Issue.
These are PosGreSQL metadata tables that are retrieved by liquibase when you generate your first xml file.
In my case it only was useless autogenerated code, so I solved it deleteing these lines:
<changeSet author="martinlarizzate (generated)" id="1588181532394-7">
<createView fullDefinition="false" viewName="pg_stat_statements"> SELECT pg_stat_statements.userid,
pg_stat_statements.dbid,
pg_stat_statements.queryid,
pg_stat_statements.query,
pg_stat_statements.calls,
pg_stat_statements.total_time,
pg_stat_statements.min_time,
pg_stat_statements.max_time,
pg_stat_statements.mean_time,
pg_stat_statements.stddev_time,
pg_stat_statements.rows,
pg_stat_statements.shared_blks_hit,
pg_stat_statements.shared_blks_read,
pg_stat_statements.shared_blks_dirtied,
pg_stat_statements.shared_blks_written,
pg_stat_statements.local_blks_hit,
pg_stat_statements.local_blks_read,
pg_stat_statements.local_blks_dirtied,
pg_stat_statements.local_blks_written,
pg_stat_statements.temp_blks_read,
pg_stat_statements.temp_blks_written,
pg_stat_statements.blk_read_time,
pg_stat_statements.blk_write_time
FROM pg_stat_statements(true) pg_stat_statements(userid, dbid, queryid, query, calls, total_time, min_time, max_time, mean_time, stddev_time, rows, shared_blks_hit, shared_blks_read, shared_blks_dirtied, shared_blks_written, local_blks_hit, local_blks_read, local_blks_dirtied, local_blks_written, temp_blks_read, temp_blks_written, blk_read_time, blk_write_time);</createView>
</changeSet>