PostgreSQL "initdb" (Database Initialization) on Linux - postgresql

I'm working on creating a Database Cluster (single database) in PostgreSQL 9.x working on a Linux system (CentOS - RedHat - Fedora). I've installed the correct PostgreSQL packages (server & client) however, I'm unable to create a database and get some type of initializing dependencies error: Bus Error / Exit Code 135. I've changed my user to "postgres" with "su postgres" and then tried to initialize the database with "initdb" (this may be the problem)
Installed: postgresql-libs-9.2.13-1.el7_1.x86_64
Installed: postgresql-9.2.13-1.el7_1.x86_64
Installed: postgresql-server-9.2.13-1.el7_1.x86_64
$ initdb -D /usr/local/pgsql/data
http://www.postgresql.org/docs/9.2/interactive/creating-cluster.html
Error:
$ initdb -D /usr/local/pgsql/data
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
creating directory /usr/local/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /usr/local/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... sh: line 1: 12616 Bus error (core dumped) "/usr/bin/postgres" --single -F -O -c search_path=pg_catalog -c exit_on_error=true template1 > /dev/null
child process exited with exit code 135
Any ideas?

After installing PostgreSQL (server and client tools) one needs to run the following commands as ROOT ("su"). The key step is to start "service postgresql initdb" and let it initialize your PostgreSQL database.
If you have any errors you need to remove the empty install "data" directories and read all log files carefully.
# service postgresql initdb
# systemctl enable postgresql
# systemctl start postgresql
After doing the above verify that postgres is in /var/lib/pgsql and a running process with "ps -ef | grep postgres" (its on port 5432)
If you run into any other problems you may need to create or modify a postgres user/password or clean a postgres data directory out.

If you installed from packages, you should use the package's provided methods for creating the DB. For the PDGD RPMs (from http://yum.postgresql.org/) that's documented in the README.rpm-dist:
/usr/pgsql-9.4/bin/postgresql94-setup initdb
However, the error you're getting really shouldn't happen. It suggests a hardware incompatibility or a low level issue like an incompatible C library. Perhaps you force-installed RPMs from a different OS or version?
Update:
Seems very likely to be a C library incompatibility. Perhaps an issue between RHEL and CentOS? Or version related? That's a fault in the dynamic linker. dl-lookup.c will be glibc/elf/dl-lookup.c and it seems to be crashing during symbol lookup. So there's something really wonky here, like a corrupt symbol hash table in the binary or an incompatibility between the binary and the dynamic linker used. Or a memory fault, disk fault, CPU cache issue, or other hardware error.
If rebooting makes it go away I'd be very suspicious of the hardware. If it doesn't, you might have something really wonky on the system like some 3rd party unpackaged installer overwriting the original C library / dynamic linker, that sort of weirdness.

Related

Getting error "could not access file "pg_stat_statements": No such file or directory"

I have the package postgresql11-contrib installed.
$ yum list installed | grep contrib
postgresql11-contrib.x86_64 11.11-1PGDG.rhel7 #pgdg11
Here is the version of postgres.
psql (PostgreSQL) 11.6
I have the below entries in postgresql.conf which is causing the error.
shared_preload_libraries = 'pg_stat_statements' # (change requires restart)
pg_stat_statements.max = 10000
pg_stat_statements.track = all
Upon some online searching I found that, I need to run "Run CREATE EXTENSION pg_stat_statements in the database(s) of my choice". But to do that I first commented above mentioned 3 lines from conf file because my psql server was failing to start with error could not access file "pg_stat_statements": No such file or directory".
https://www.oreilly.com/library/view/mastering-postgresql-11/9781789537819/a6a44124-558b-42f9-a0f3-eb52ea2799d4.xhtml
https://www.postgresql.org/docs/11/pgstatstatements.html
Now when I execute the command CREATE EXTENSION pg_stat_statements; I see error "could not open extension control file "/usr/postgresql/share/extension/pg_stat_statements.control": No such file or directory" and upon looking into the mentioned directory, these is no file pg_stat_statements.control
What am I missing here? Please help.
You seem to have installed two versions of Postgres on your Redhat/CentOS machine.
You probably have installed postgresql11 from the PGDG repository (yum.postgresql.org) as well as the default postgresql provided by CentOS 7. If you do rpm -aq | grep postgres I'll bet you'll see a v. 11 and a v. 9.2.
Just yum erase postgresql, which will delete the v. 9.2 instance that's in the CentOS repository. However, before you do that you'll need to dump the database(s) and stop Postgres. After you delete v. 9.2, you'll need to do a fresh initdb before starting Postgres, and re-load the data you had dumped.

Trying to dump a PostgreSQL-10 DB running in a CentOS 7 machine and restore it into a Windows 10 machine

I am trying to execute a backup of my PostgreSQL-10 database running on a CentOS 7 machine and then to restore it in a development machine running Windows 10, but I am getting errors during the restore process:
pg_restore: [custom archiver] WARNING: ftell mismatch with expected position -- ftell used
I have made sure that the commands' parameters passed in both dump and restore are the same:
pg_dump --format=c --compress=9 --encoding=UTF-8 -n public --verbose --username=postgres databaseName -W -f /usr/local/production-dump.backup
However it does not work at all. Even though the schema is restored, the data is not, because right before the restore process is going to start restoring data, it gives a "pipe has ended" error and does not proceed with the full restore process. I am using the "custom" format because the plain SQL or tar formats generate huge backup files.
What am I doing wrong? Is there any parameter that I need to pass to the dump or restore commands?
The likely explanation is that the file was modified during file transfer. Could you calculate a checksum of the file before and after transfer and verify that it is the same?
If the file did not change, then you have probably found a PostgreSQL bug. If you have a dump that you can share and that exhibits the problem, please report this problem to PostgreSQL.

installing postgres via puppet with customer data_directory

I'm trying to install postgres 9.1 on an ubuntu 12.04 machine using puppet v3.4.3 and puppetlabs/postgresql module v3.3.0. I want the data_directory to point to a large disk I've mounted.
If I change the datadir property of postgresql::globals it doesn't seem to do anything. The postgres.conf file still has data_directory pointing to /var/lib/postgresql/9.1/main
Then I tried also using postgresql::server::config_entry to change the data_directory param in postgres.config but that gives the following error:
Debug: Executing 'service postgresql reload'
Notice: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]/returns: * Reloading PostgreSQL 9.1 database server
Notice: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]/returns: * pg_ctl: PID file "/data/PGDATA/postmaster.pid" does not exist
Notice: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]/returns: Is server running?
Notice: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]/returns: ...fail!
Error: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: Failed to call refresh: service postgresql reload returned 1 instead of one of [0]
Error: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: service postgresql reload returned 1 instead of one of [0]
I believe this fails because postgres was started before configuring the data_directory, and this parameter is part of the startup of the process so when you change the value it can no longer find the process to stop and start it. If I kill postgres and try to start it I get an error saying /data/PGDATA is not a valid directory, as the database was not created in the data_directory location I specified.
Finally I tried creating the user, group and file path, and then mounting the disk to /var/lib/postgresql/9.1/main all in puppet before puppet installs postgres, but I get this error:
Notice: /Stage[main]/Mm_postgres::Server/Postgresql::Server::Config_entry[checkpoint_segments]/Postgresql_conf[checkpoint_segments]/ensure: created
Error: Puppet::Util::FileType::FileTypeFlat could not write /etc/postgresql/9.1/main/postgresql.conf: No such file or directory - /etc/postgresql/9.1/main/postgresql.conf
Error: /Stage[main]/Mm_postgres::Server/Postgresql::Server::Config_entry[checkpoint_segments]/Postgresql_conf[checkpoint_segments]: Could not evaluate: Puppet::Util::FileType::FileTypeFlat could not write /etc/postgresql/9.1/main/postgresql.conf: No such file or directory - /etc/postgresql/9.1/main/postgresql.conf
I believe everything was done in the correct order, and permissions were fine, but I'll keep looking into this. Does anyone know if it is possible to install postgres via puppet on ubuntu and change the data_directory, and if so, how?
Thanks
Problem is that postgresql module from puppetlabs doesn't quite work with that kind of customization. I think there are even some bug reports out there. This is what I did, and it works without a problem:
file { '/data/postgresql':
ensure => directory,
}
file { '/var/lib/postgresql':
ensure => link,
target => '/data/postgresql',
before => Class['::postgresql::server::install'],
}
class { '::postgresql::globals':
}->class { '::postgresql::server':
}
Hope it helps!
PS. I used puppetlabs-postgresql version 3.3.3

PostgreSQL with Homebrew on Mac

First I use homebrew and install postgresql and I get this success message:
==> Summary 🍺 /usr/local/Cellar/postgresql/9.2.2: 2819 files, 39M, built in 68 seconds
Then I need to run this command:
initdb `brew --prefix`/var/postgres -E utf8
But this is the message I get:
What should I do with this?
Also running on OSX 10.8.2
The files belonging to this database system will be owned by user
"JonyIve". This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default text search configuration will be set to "english".
initdb: directory "/usr/local/var/postgres" exists but is not empty If
you want to create a new database system, either remove or empty the
directory "/usr/local/var/postgres" or run initdb with an argument
other than "/usr/local/var/postgres".
Assuming this is a new installation and not a re-installation I think solving your problem is as simple as:
initdb `brew --prefix`/var/postgres/data -E utf8
Typically the data directory is called "data' and is underneath the postgresql home directory. this allows for the possibility of sharing log file access and the like.
If you know that you can do this without breaking anything, another option is always just to remove the directory as per the message
remove or empty the directory "/usr/local/var/postgres"
Via
rm -rf /usr/local/var/postgres

Use pg_restore to restore from a newer version of PostgreSQL

I have a (production) DB server running PostgreSQL v9.0 and a development machine running PostgreSQL v8.4. I would like to take a dump of the production DB and use it on the development machine. I cannot upgrade the postgres on the dev machine.
On the production machine, I run:
pg_dump -f nvdls.db -F p -U nvdladmin nvdlstats
On the development machine, I run:
pg_restore -d nvdlstats -U nvdladmin nvdls.db
And I got this error:
pg_restore: [archiver] unsupported version (1.12) in file header
This occurs regardless of whether I choose the custom, tar, or plain_text format when dumping.
I found one discussion online which suggests that I should use a newer version of pg_restore on the dev machine. I tried this by simply copying the 9.0 binary to the dev machine, but this fails (not unexpectedly) due to linking problems.
I thought that the point of using a plain_text dump was that it would be raw, portable SQL. Apparently not.
How can I get the 9.0 DB into my 8.4 install?
pg_restore is only for restoring dumps taken in the "custom" format.
If you do a "plain text" dump you have to use psql to run the generated SQL script:
psql -f nvdls.db dbname username
Using pg_dump/pg_restore to move from 9.0 to 8.4 is not supported - only moving forward is supported.
However, you can usually get the data across (in a data-only dump), and in some cases you can get the schema - but that's mostly luck, it depends on which features you're using.
You should normally use the target version of pg_dump and pg_restore - meaning in this case you should use the binaries from 8.4. But you should use the same version of pg_dump and pg_restore. Both tools will work fine across the network, so there should be no need to copy the binaries around.
And as a_horse_with_no_name says, you may be better off using pg_dump in plaintext mode - that will allow you to hand-edit the dump if necessary. In particular, you can make one schema only dump (with -s) and one data only dump - only the schema dump is likely to require any editing.
If the 9.0 database contains any bytea columns, then bigger problems await.
These columns will be exported by pg_dump using the "hex" representation and appear in your dump file like:
SELECT pg_catalog.lowrite(0, '\x0a2')
Any version of the postgres backend below 9.0 can't grok the hex representation of bytea, and I can't find an option to tell pg_dump on the 9.0 side to not use it. Setting the default "bytea_output" setting to ESCAPE for either the database or the whole server is seemingly ignored by pg_dump.
I suppose it would be possible to post-process the dump file and actually change every hex-encoded bytea value to an escaped one, but the risk of untraceably corrupting the kind of things normally stored in a bytea (images, PDFs etc) does not excite me.
I solved this by upgrading postgresql from 8.X to 9.2.4. If you're using brew on Mac OS-X, use -
brew upgrade postgresql
Once this is done, just make sure your new postgres installation is at the top of your path. It'll look something like (depending on the version installation path) -
export PATH=/usr/local/Cellar/postgresql/9.2.4/bin:$PATH
I had same issue. I used pgdump and psql for export/import DB.
1.Set PGPASSWORD
export PGPASSWORD='h0ld1tn0w';
2.Export DB with pg_dump
pg_dump -h <<host>> -U <<username>> <<dbname>> > /opt/db.out
/opt/db.out is dump path. You can specify of your own.
3.Then set again PGPASSWORD of you another host. If host is same or password is same then this is not required.
4.Import db at your another host
psql -h <<host>> -U <<username>> -d <<dbname>> -f /opt/db.out
If username is different then find and replace with your local username in db.out file. And make sure on username is replaced and not data.