I'm encountering an error when trying to do migrations bootstrap for kong, can anyone please help?
serene#ubuntu:~$ sudo kong migrations bootstrap -c /etc/kong/kong.conf --v
2020/10/19 02:21:04 [verbose] Kong: 2.1.4
2020/10/19 02:21:04 [verbose] reading config file at /etc/kong/kong.conf
2020/10/19 02:21:04 [verbose] prefix in use: /usr/local/kong
Error:
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:95: [PostgreSQL error] failed to retrieve PostgreSQL server_version_num: Transport endpoint is not connected
stack traceback:
[C]: in function 'assert'
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:95: in function 'cmd_exec'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
[C]: in function 'xpcall'
/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
/usr/local/bin/kong:9: in function 'file_gen'
init_worker_by_lua:50: in function <init_worker_by_lua:48>
[C]: in function 'xpcall'
init_worker_by_lua:57: in function <init_worker_by_lua:55>
I am using WSL, Ubuntu 18.04 LTS, I've installed kong v2.1.4, postgresql 12 on it
serene#ubuntu:~$ kong version
2.1.4
serene#ubuntu:~$ dpkg -l | grep postgres
ii pgdg-keyring 2018.2 all keyring for apt.postgresql.org
ii postgresql-12 12.4-1.pgdg18.04+1 amd64 object-relational SQL database, version 12 server
ii postgresql-client-12 12.4-1.pgdg18.04+1 amd64 front-end programs for PostgreSQL 12
ii postgresql-client-common 220.pgdg18.04+1 all manager for multiple PostgreSQL client versions
ii postgresql-common 220.pgdg18.04+1 all PostgreSQL database-cluster manager
I have started the postgresql service.
I have configured kong.conf as follows
database = postgres
pg_host = 127.0.0.1
pg_port = 5433
pg_timeout = 5000
pg_user = kong
pg_password = kong
pg_database = kong
pg_ssl = off
pg_ssl_verify = off
pg_max_concurrent_queries = 0
pg_semaphore_timeout = 60000
I have also configured a user in postgres called kong with password 'kong' as owner of database called 'kong.
serene#ubuntu:~$ sudo -u postgres psql
psql (12.4 (Ubuntu 12.4-1.pgdg18.04+1))
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+---------+-----------------------
kong | kong | UTF8 | C.UTF-8 | C.UTF-8 | =Tc/kong +
| | | | | kong=CTc/kong
postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
postgres=#
Initially I was using WSL Ubuntu 20.04, Kong 2.1.4, Postgresql 13. Same error.
Did a fresh install of those. Same error.
Deleted Ubuntu 20.04 and installed Ubuntu 18.04 and reinstall Kong 2.1.4 and Postgresql 13. Same error.
Heard someone had troubles with Postgresql 13 when installing Kong, so I went and did a fresh install for Postgresql 12, still same error.
Is there anyone who knows what went wrong and how I can rectify this?
Related
I'm trying to upgrade a postgres server from 10 12, and am having problems with the encodings. I'm following what I believe to be established recipes.
Behold:
postgres#serverbot:~$ psql -l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-------------+----------+-----------+---------+-------+-----------------------
postgres | postgres | SQL_ASCII | C | C |
template0 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | SQL_ASCII | C | C | postgres=CTc/postgres+
| | | | | =c/postgres
thingsboard | postgres | SQL_ASCII | C | C | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | nagios=c/postgres
(4 rows)
Notice the encodings on the 10 database. Time to create the database for 12.
postgres#serverbot:~$ sudo service postgresql stop
postgres#serverbot:~$ /usr/lib/postgresql/12/bin/initdb -E SQL_ASCII --locale=C -D /var/lib/postgresql/12/data
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /var/lib/postgresql/12/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/New_York
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/lib/postgresql/12/bin/pg_ctl -D /var/lib/postgresql/12/data -l logfile start
Excellent! Let's fire up the new server.
postgres#serverbot:~$ /usr/lib/postgresql/12/bin/pg_ctl -D /var/lib/postgresql/12/data -l logfile start
waiting for server to start.... done
server started
And verify the encodings...
postgres#serverbot:~$ psql -l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+-----------+---------+-------+-----------------------
postgres | postgres | SQL_ASCII | C | C |
template0 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
Everything matches... time to upgrade!
postgres#serverbot:~$ /usr/lib/postgresql/12/bin/pg_upgrade \
--old-datadir=/var/lib/postgresql/10/main \
--new-datadir=/var/lib/postgresql/12/main \
--old-bindir=/usr/lib/postgresql/10/bin \
--new-bindir=/usr/lib/postgresql/12/bin \
--old-options '-c config_file=/etc/postgresql/10/main/postgresql.conf' \
--new-options '-c config_file=/etc/postgresql/12/main/postgresql.conf' \
--link --check
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
Checking database user is the install user ok
Checking database connection settings ok
Checking for prepared transactions ok
Checking for reg* data types in user tables ok
Checking for contrib/isn with bigint-passing mismatch ok
Checking for tables WITH OIDS ok
Checking for invalid "sql_identifier" user columns ok
encodings for database "postgres" do not match: old "SQL_ASCII", new "UTF8"
Failure, exiting
postgres#serverbot:~$
Doh!
What is wrong here? I assert that the encodings do match, and I'm stuck.
Can anyone offer any advice?
What I see is:
postgres#serverbot:~$ /usr/lib/postgresql/12/bin/pg_ctl -D /var/lib/postgresql/12/data -l logfile start
waiting for server to start.... done
server started
and then:
postgres#serverbot:~$ /usr/lib/postgresql/12/bin/pg_upgrade \
--old-datadir=/var/lib/postgresql/10/main \
--new-datadir=/var/lib/postgresql/12/main \
[...]
Note the $PGDATA directories. The database cluster you did the initdb for is different from the one you are doing the pg_upgrade to.
UPDATE. Since you seem to be using a Debian based OS and it's Postgres packaging, it might work better to stick with the packaging tools:
sudo pg_createcluster --locale=C 12 ascii
Creating new PostgreSQL cluster 12/ascii ...
/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/ascii --auth-local peer --auth-host md5 --locale C
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/12/ascii ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/Los_Angeles
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctlcluster 12 ascii start
Ver Cluster Port Status Owner Data directory Log file
12 ascii 5434 down postgres /var/lib/postgresql/12/ascii /var/log/postgresql/postgresql-12-ascii.log
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+-----------+---------+-------+-----------------------
postgres | postgres | SQL_ASCII | C | C |
template0 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
Also I would use pg_upgradecluster, see here for more information. This keeps everything in the same system.
I updated my docker file to version 3 but now the inicial schemas are not being created. i already tried with a different volume
I run it with : docker-compose -f docker-compose.yml up
version: '3'
services:
db-service:
image: postgres:11.2
volumes:
- ./dbscripts/:/docker-entrypoint-initdb.d/
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=sample
ports:
- 5432:5432
Check these 2 things:
The init scripts are triggered only on the first deploy. On the subsequent docker-compose ups you probably see as well on the logs the following message:
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
To force the re-initialization, you can wipe the data volume(This will delete the entries, not only the schema).
To find the path, simply docker inspect <postgres-container-id> and look for the HostConfig.Binds
The script executes till the first type "longvarbinary" does not exist error. You will need to fix this.
db-service_1 | CREATE TABLE
db-service_1 | 2020-07-26 19:29:34.121 UTC [68] ERROR: type "longvarbinary" does not exist at character 68
I opened a console in the postgres container and your sample database is there, as well as a table that is created before the .sql error occurs in the script:
docker exec -ti 22b0bb87561f sh
# psql -U postgres
psql (11.2 (Debian 11.2-1.pgdg90+1))
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
sample | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
postgres=# \c sample
You are now connected to database "sample" as user "postgres".
sample=# \dt
List of relations
Schema | Name | Type | Owner
--------+----------------------+-------+----------
public | oauth_client_details | table | postgres
(1 row)
I ran into the same issue, and I figured it was because I forgot to clear the volume when I take down docker compose with command docker-compose down.
The solution that helps is when you take down docker compose, you specify that all volumes must be removed docker-compose down --volumes. Also, make sure to check and delete any related volumes using docker volume ls.
WARNING: Any subsequent command docker-compose down should not have the flag --volume or else your persistent data will be wiped out. This may or may not be desirable depending on your use case. The idea is to ONLY run the init script on the very first time you run docker-compose up. Any subsequent command up after that will skip initialization because data is already populated, and you don't want to override it.
Hope it helps!
I am trying to setup GitLab using separate docker containers for both GitLab and PostgreSQL. I am using RancherOS v1.0.3 with Kubernetes enabled. For now, all i want is to have a single node having both the containers. Later, i will look into configuring them on separate nodes.
[rancher#rancher-agent-2 ~]$ cat postgresql.sh
docker run --name=postgresql -d \
--env 'DB_NAME=gitlabhq_production' \
--env 'DB_USER=gitlab' --env 'DB_PASS=password' \
--env 'DB_EXTENSION=pg_trgm' \
--volume /opt/postgresql:/var/lib/postgresql \
postgres:9.6.3
[rancher#rancher-agent-2 ~]$ cat gitlab.sh
docker run --name=gitlab -d --link postgresql:postgresql \
-v /opt/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:9.3.9-ce.0
Now when i run the PostgreSQL container and try logging in, i get errors.
[rancher#rancher-agent-2 ~]$ docker exec -it postgresql bash
root#a6cef780c594:/# psql -d gitlabhq_production -U gitlab
psql: FATAL: role "gitlab" does not exist
Seems the db (gitlabhq_production) has not been created.
root#a6cef780c594:/# psql -U postgres
psql (9.6.3)
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
Why is it not creating the db despite passing the parameters?
UPDATE:
PostgreSQL container log:
[rancher#rancher-agent-2 ~]$ docker logs postgresql
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
...
...
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
ALTER ROLE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
...
...
You are using the wrong environment vars names:
DB_NAME should be POSTGRES_DB
DB_USER should be POSTGRES_USER
DB_PASS should be POSTGRES_PASSWORD
Check https://hub.docker.com/_/postgres/
I'm trying to build Vagrant box provisioned with Puppet.
I use Ubuntu 12.04 LTS box from http://files.vagrantup.com/precise64.box.
I also use http://forge.puppetlabs.com/puppetlabs/postgresql module to install PostgreSQL with the following manifest:
class db {
class { 'postgresql': version => '9.1' }
class { 'postgresql::server': }
}
class { 'db': }
It installs correctly, but databases are created with SQL_ASCII encoding:
$ psql -l -U postgres
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+-----------+---------+-------+-----------------------
postgres | postgres | SQL_ASCII | C | C |
template0 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | SQL_ASCII | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
After googling I've thought it might be a locale problem, but it looks correct to me:
$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
Do you know what should I do to be able to install PostgreSQL with UTF8 encoding default? It'd be also nice to be able to use Puppet for this.
Apparently, it is Puppet bug http://projects.puppetlabs.com/issues/4695. Added the following code from issue comments as a workaround:
# workaround for http://projects.puppetlabs.com/issues/4695
# when PostgreSQL is installed with SQL_ASCII encoding instead of UTF8
exec { 'utf8 postgres':
command => 'pg_dropcluster --stop 9.1 main ; pg_createcluster --start --locale en_US.UTF-8 9.1 main',
unless => 'sudo -u postgres psql -t -c "\l" | grep template1 | grep -q UTF',
require => Class['postgresql::server'],
path => ['/bin', '/sbin', '/usr/bin', '/usr/sbin'],
}
I have a postgresql (9.1.4) server running on a remote machine (Ubuntu 12.04), and I'm having trouble accessing it on my local machine. Specifically, on my local machine, I can access the remote server using only the username "postgres" and the database name "postgres", even though I have other user names and databases. First some of the background information. While in the interactive environment invoked with "psql" on the remote machine, I created another user, called "mxtxdb", and set the password for that user. I also created a database called "mxtxdb". To demonstrate their existence, I logged onto the remote machine and ran:
sudo su postgres
Password:
postgres#myhost:~$ psql
psql (9.1.4)
Type "help" for help.
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------+-----------
mxtxdb | | {}
postgres | Superuser, Create role, Create DB, Replication | {}
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
mxtxdb | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | mxtxdb=CTc/postgres
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
Furthermore, while still on the remote machine, I can access the database from the "mxtxdb" postgresql account:
postgres#myhost:~$ psql -h localhost -U mxtxdb -d mxtxdb
Password for user mxtxdb:
psql (9.1.4)
Type "help" for help.
mxtxdb=>
But when I try to do the same on the local machine, I get this:
psql -h <server's address> -U postgres -d "postgres" -p 5432
Password for user postgres:
psql (9.1.4)
Type "help" for help.
postgres=#
and
psql -h <server's address> -U "postgres" -d "mxtxdb" -p 5432
Password for user postgres:
psql: FATAL: database "mxtxdb" does not exist
and
psql -h <server's address> -U "mxtxdb" -d "mxtxdb" -p 5432
Password for user mxtxdb:
psql: FATAL: role "mxtxdb" does not exist
The last two lines of my pg_hba.conf file are:
host all mxtxdb 0.0.0.0/0 trust
host all postgres 0.0.0.0/0 trust
and the listen_addresses is set to '*' (and uncommented) in the postgresql.conf file.
Any idea why I cannot access other accounts or databases remotely, but I can when ssh'd into the remote machine?
First of all, thanks to Craig Richard for your suggestions, and in fact you were both correct. Here's what happened: I installed PostgreSQL using sudo apt-get install postgresql-9.1, and then I followed the instructions for configuring the server at http://www.postgresql.org/docs/9.1/interactive/runtime.html . Unfortunately, what I did not notice was that the apt-get install process creates a postgresql database cluster by default, so when I followed the instructions to create a new one, I then had two clusters, one at /var/lib/postgresql/9.1/main and one at /usr/local/pgsql/data, and I was interacting with different clusters when I logged in locally vs remotely. When I removed the second cluster, the issue was resolved and I was able to connect remotely.
My takeaways from this process: when installing postgresql with apt-get, and then following the instructions in the postgresql manual, be aware that the install process will create a postgres operating system user by default, and will also create a database cluster by default, so those steps in the manual do not need to be performed. If anyone experiences an error like:
"FATAL: role/database '<name>' does not exist"
when you are sure you've created that user or database, check to ensure that you're operating with the correct database cluster.
Thanks again!