I want to establish a gitlab test stage that uses a gitlab service postgres database. The problem is, that everytime I try to access the database via a script call in the pipeline I get the following error:
psql: error: could not connect to server: Connection refused
Is the server running on host "postgres" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
yaml looks the following
image: some_image:latest
stages:
- test
tests:
image: node:latest
stage: test
services:
- postgres:latest
before_script:
- apt-get update && apt-get install -y postgresql-client libpq-dev
# access database script from another repo here through git clone
- psql -U postgres -h postgres < ./create-database.sql
script:
- npm install
- npm run tests
only:
- master
Am I missing something - is the database maybe not created and I am calling to soon?
You're missing the configuration properties of the DB. They can not be configured on the UI - or they can but won't be passed to the actual DB service.
So please add the following properties to the script:
variables:
POSTGRES_DB: <db name>
POSTGRES_USER: <db user>
POSTGRES_PASSWORD: <some PW>
POSTGRES_HOST_AUTH_METHOD: trust
Documentation link: https://docs.gitlab.com/ee/ci/services/postgres.html
I have a PostgreSQL server running on a Linux Red Hat VM (installed from rh-postgresql96) and am trying to create a role in the database through the psql shell using the following Ansible task.
- name: Setup database
become_user: postgres
become: yes
block:
- name: Enter psql shell
shell: |
scl enable rh-postgresql96 bash
- name: Create user1 role
shell: |
psql -c "CREATE ROLE user1 WITH LOGIN PASSWORD 'pass!123' VALID UNTIL 'infinity';"
But when I do this, it just hangs on the psql shell task...
Any ideas?
Please note, I do not want to use the postgresql ansible plugin to achieve this.
It may probably be waiting for some sort of input such as password. While executing mysql queries with ansible, I used the following format:
- name: Set max queries for hour for specified user
expect:
command: mysql -u root -p -e "ALTER USER '{{username}}'#'{{userhost}}' WITH MAX_QUERIES_PER_HOUR {{MAX_QUERIES_PER_HOUR}};"
echo: yes
responses:
'password': "{{mysql_root_password}}"
This query is irrelevant to the topic, but I used expect module to give the password input.
Usually I do :
sudo -su postgres
psql
\password
\q
to change postgres password. Now I want to automate this step using ansible.
Ansible postgresql_user module will help to set/reset the database user password. Here is the sample
- name: pd reset database user
become: yes
become_method: sudo
become_user: postgres
postgresql_user:
db: test
name: test
password: ""
For Reference, here is ansible link: https://docs.ansible.com/ansible/latest/collections/community/postgresql/postgresql_user_module.html#ansible-collections-community-postgresql-postgresql-user-module
I am running PostgreSQL 9.3 on FreeBSD. FreeBSD uses pgsql as the default system user for PostgreSQL. My /usr/local/pgsql/data/pg_hba.conf looks like this:
# TYPE DATABASE USER ADDRESS METHOD
local all pgsql peer
local all all md5
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
With this configuration I can connect to the database as pgsql without a password.
$ su pgsql
$ psql template1
template1=# \l
List of databases
...
That works as intended.
On a remote machine, I have an Ansible task to create a database on the FreeBSD server.
- name: Create the postgresql database
postgresql_db: name=mydatabase login_user=pgsql
Executing this task fails with the error Peer authentication failed for user "pgsql".
PLAY [web] ********************************************************************
GATHERING FACTS ***************************************************************
ok: [host.example.org]
TASK: [database | Create the postgresql database] *****************************
failed: [host.example.org] => {"failed": true}
msg: unable to connect to database: FATAL: Peer authentication failed for user "pgsql"
FATAL: all hosts have already failed -- aborting
Why does this fail when peer authentication for the user pgsql is clearly working?
This worked for me:
- name: Create postgres database
become: true
become_user: postgres
postgresql_db:
name: <database-name>
In your specific case the user might be pgsql, but I think usually the user is postgres.
Or with slightly different syntax (from Ansible 1.9) and for user creation (might be helpful for someone)
- name: Create postgres user
postgresql_user: name={{ pg_user }} password={{ pg_password }}
become: true
become_user: postgres
For those running into "Failed to set permissions on the temporary files Ansible needs to create..." in order to switch to the postgres user with become_user you can leverage pipelining on Ubuntu hosts.
Create a ansible.cfg in your playbook directory and add the following lines:
[ssh_connection]
pipelining=True
Update: according to #lolcode Ansible 2.9.0 has updated to ansible_pipelining
[ssh_connection]
ansible_pipelining = true
Update 4/30/2020: for those who still have issues, try installing acl which will cause Ansible to use this acl filesystem to mount module that need to be accessible by the 2nd user instead of making them readable by everyone. Thanks #Andreas Florath
- name: install setfacl support
become: yes
apt: pkg=acl
I had the same problem. In my case I overlooked that I configured my Ansible-playbook to run as another Linux user than the one with peer access (pgsql in your case). Solution is either run the Ansible play as pgsql:
- name: Create the postgresql database
remote_user: pgsql
postgresql_db: name=mydatabase login_user=pgsql
...
Or run it as root, and su to pgsql for the command:
- name: Create the postgresql database
remote_user: root
become: yes
become_user: pgsql
postgresql_db: name=mydatabase login_user=pgsql
...
... depending on your access rights via ssh.
This is using Ansible 2.0.
Another workaround is to connect via host (localhost) rather than the default local peer authentication method:
- name: Create the postgresql database
postgresql_db:
name: mydatabase
login_user: postgres
login_host: 127.0.0.1
Depending on the settings in pg_hba.conf, you may also need to provide login_password. You can circumvent this by setting
host all postgres 127.0.0.1/32 md5
to
host all postgres 127.0.0.1/32 trust
I notice your Postgres version is really out of date (9.3). I had this issue recently when working on an Ubuntu 14 server with Postgres 9.3.
I tried a dozen different things, and finally what worked was installing the acl package via apt. Ansible uses it for navigating some of it's permissions issues. The package is installed by default on newer distros, hence why I've only seen this problem crop up on an old server.
Thanks to this threat I made a variant of mdh's post. When I set up a database I generate a password for the postgres user and I store it in a file under the root directory.
I thought why not store it also (or instead) in a .pgpass file for root. So I created a template like this (only last line is important):
#### password file for posgres connection ###
#### *:*:*:*
#### works like
#### * : * : * : *
#### <ip addr> : <port nr> : <db name> : <password>
127.0.0.1:*:*:postgres:{{ new_postgres_pass }}
Store the .pgpass file in the home directory of root. Now you can use the module as root without switching user of having to change the pg_hba.conf:
- name: Ensure postgresql mydatabase
postgresql_db:
name: mydatabase
login_user: postgres
login_host: 127.0.0.1
If you don't have sudo (debian, etc) but have access to root
- name: Create database
remote_user: root
become: yes
become_method: su
become_user: postgres
postgresql_db:
name: my_db
This problem due to postgres authentication methods configured at /etc/postgresql/10/main/pg_hba.conf , the configuration location may vary depending on the version installed, /etc/postgresql//main/pg_hba.conf
sample ansible recipe
---
- name : "Dump PostgreSQL database"
gather_facts: true
hosts: posgre_host
tasks:
- name: Dump existing PostgreSQL database
community.postgresql.postgresql_db:
name: demo
state: dump
target: /tmp/backup/backup.sql.gz
become: true
From
local all postgres peer
To
local all postgres trust
Here trust authentication method anyone connect to the server is authorized to access the database, since ansible is working based on ssh, it is safe to change.
So if I understood well you are on a remote machine, and maybe you should change /usr/local/pgsql/data/pg_hba.conf to allow remote connections 'host all all 0.0.0.0/0 md5' or another specific network address.
i am on ubuntu 12.04 server and i am trying to install postgresql. As of now, i have successfully installed it but unable to configure it. I need to create a role to move ahead and i ran this command in terminal :
root#hostname: createuser -s -r postgres
and it said :
createuser: could not connect to database postgres: FATAL: role "root" does not exist
Fine, so i did :
su - postgres
and then tried again
postgres#hostname: createuser -s -r postgres
and i got the error
createuser: could not connect to database postgres: FATAL: role "postgres" does not exist
and i get the same error when i do
psql -d dbname
Its like a loop, i am unable to create a role postgres because a role postgres does not already exist.
How do i fix this ?
The postgres version seems to be 9.1.x and the ubuntu version is 12.10
Turns out i had installed postgres-xc and postgresql on my machine. I had to knock off postgres-xc completely. And it was a little difficult to do that because, there was always an error --purge remove postgres-xc and the uninstallation could not continue.
There seems to be some kind of a packaging bug. (details on launchpad).
Eventually, i ended up doing this to make it work.
After that i uninstalled postgresql and installed it back to make it work.
Read postgresql tutorial it doesn't matter if it's Ubuntu or other Linux.
EDIT
before creating role or anything else on fresh install you need to create database cluster: have you created it?
initdb -D /usr/local/pgsql/data
You need to be logged as user postgres on linux machine. Here is more info.