New to Ansible, Running version 2.1.0. I've written an Ansible playbook that runs a PostgreSQL query against a group of hosts. When I specify the SQL DB password in the shell command it works, but I'm looking to run the playbook against a group of hosts and need a better way to input the passwords as they are all unique. Could anyone suggest a better way to do this?
---
- hosts: Test_Hosts
sudo: yes
sudo_user: root
gather_facts: yes
tasks:
- name: Login to DB and run command
shell: export PGPASSWORD='Password'; psql -U 'user' -d 'db' -c 'select * FROM table';
register: select_all_from_table
- name: Display table contents
debug: msg="{{ select_all_from_table.stdout }}"
I saw another thread on the topic but was not sure how to implement the suggestion: Run a postgresql command with ansible playbook. Postgresql requires password
Ansible allows you to set environment variables for a task using the environment parameter to any task.
So in your case you could just do this:
- name: Login to DB and run command
shell: psql -U 'user' -d 'db' -c 'select * FROM table';
register: select_all_from_table
environment:
PGPASSWORD: '{{ pgpassword }}'
And then set the pgpassword variable at the group or host level.
I just ran into this issue today and this is what worked for me. On Linux you can package all the credentials into a ~/.pgpass hidden file.
Just create it locally (in this case in ./files/pgpass) and then use ansible to copy it onto the host before you run the psql commands.
- name: set passwd file for PSQL
copy:
src: files/pgpass
dest: ~/.pgpass
mode: 0600 ### important: will not work with wrong permissions
- name: PSQL command
shell: "psql -U 'user' -d 'db' -c 'select * FROM table'"
register: select_all_from_table
The file contents must be in the following format:
hostname:port:database:username:password
However, you can use wildcards, so mine looks like this, for example:
*:*:db1:user1:passwd1
*:*:db2:user2:passwd2
See documentation for more details:
https://www.postgresql.org/docs/9.1/static/libpq-pgpass.html
Related
I have a simple dockerfile:
FROM postgres:latest
ENV POSTGRES_PASSWORD password
ENV POSTGRES_USER postgres
ENV POSTGRES_DB evesde
COPY init.sh /docker-entrypoint-initdb.d/
and my init file is chmod' to 777:
#!/bin/bash
psql -U "postgres" -d "evesde" -e "create role yaml with login encrypted password 'password';"
when running a container it will say:
psql: warning: extra command-line argument "create role yaml with
login encrypted password 'password';" ignored
Im not sure why this is happening, and when doing an interactive terminal, this command seemingly worked. I dont see any additional information and wasnt sure what was going wrong.
The postgres docker page is: https://hub.docker.com/_/postgres
When looking at it deeper, I was noticing that running the command itself fails in an interactive Terminal with the same error, but the command runs when I am in postgres: psql -U "postgres" -d "evesde" and run the command, it works.
I think it may be related to passing the command in through the exec command is where it fails. likely related to '.
You want -c instead of -e.
-e turns on "echo queries"
-c runs the command and exits
Have you considered putting just the create role command in a file called create_role.sql and copying that into /docker-entrypoint-initdb.d/?
Based on testing, it looks like an equivalent but simpler solution is to put the SQL command as one line in a file, 00_roles.sql, and copy that into the container instead of the init.sh script.
I'm using Postgres within a docker-compose environment to host databases for multiple containers. I basically want to add a database per application directly from docker-compose without the need of manually creating the databases and users. For this, I'm using the init script feature of the Postgres docker image and copy the following bash script by mounting a volume:
db:
image: postgres:9.6
container_name: db
restart: always
volumes:
- postgres-data:/var/lib/postgresql/data
- /opt/docker/pgsql-entrypoint:/docker-entrypoint-initdb.d
environment:
- POSTGRES_PASSWORD={{ vault_pgsql_root_password }}
- POSTGRES_MULTIPLE_DATABASES=confluence-{{ confluence_pgsql_password }},keycloak-{{ keycloak_pgsql_password }},gitlab-{{ gitlab_pgsql_password }},jira-{{ jira_pgsql_password }}
Basically the POSTGRES_MULTIPLE_DATABASESenvironment variable contains all the databases and users that should be created. The password is as follows:
#!/bin/bash
set -e
set -u
function create_user_and_database() {
local database=$1
local password=$2
echo " Creating user and database '$database'"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER $database WITH PASSWORD '$password';
CREATE DATABASE $database;
GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}
if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
for entry in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
db=$(echo $entry | cut -f1 -d-)
pw=$(echo $entry | cut -f2 -d-)
create_user_and_database $db $pw
done
echo "Multiple databases created"
fi
My problem is: at a certain point (now ;) ) I may want to add an additional service. Just adding an additional pair to the environment variable does not work, as the Postgres image is skipping the init step if data already exists. Is there a way to still achieve this behaviour?
Edit: I should have specified that i want to do it automatically from the compose file, by just changing the environment variable. It’s clear that it can be done manually of course.
You can always connect to the server with a database client and do it manually. If you don't want to expose the database port to the host then you can run the postgres client from the terminal in the postgres container. To open the terminal in the container:
> docker exec -it <container_name> /bin/sh
Then change users to postgres and start the client
# su postgres
postgres#1778e9755f65:/$ psql
Once inside just create a database:https://www.postgresql.org/docs/9.0/sql-createdatabase.html
I ended up not to use docker-compose but using sensible to deploy the database container directly and also make sure the appropriate users, databases and permissions are present. I could not find a meaningful way to do that during startup and environment variables.
I've added three (3) environment variables which I'm trying to used to setup the db correctly.
These are the variables:
DB_NAME
DB_USER
DB_PW
In the before_script section of .travis.yml I would run something like this:
before_script:
- psql -c "\set pw `echo "$db_pw"`"
- psql -c "\set user `echo "$db_user"`"
- psql -c "\set name `echo "$db_name"`"
- psql -c "create database :'name'"
- psql -c "CREATE :user WITH PASSWORD ':pw';"
However, that doesn't work since the result of \set is lost after each command.
How can I setup the db based on what on my environment variables?
I’d like to use Ansible to manage the configuration of a our Hadoop cluster (running Red Hat).
I have sudo access and can manually ssh into the nodes to execute commands. However, I’m experiencing problems when I try to run Ansible modules to perform the same tasks. Although I have sudo access, I can’t become root. When I try to execute Ansible scripts that require elevated privileges, I get an error like this:
Sorry, user awoolford is not allowed to execute '/bin/bash -c echo
BECOME-SUCCESS- […] /usr/bin/python
/tmp/ansible-tmp-1446662360.01-231435525506280/copy' as awoolford on
[some_hadoop_node].
Looking through the documentation, I thought that the become_allow_same_user property might resolve this, and so I added the following to ansible.cfg:
[privilege_escalation]
become_allow_same_user=yes
Unfortunately, it didn't work.
This post suggests that I need permissions to sudo /bin/sh (or some other shell). Unfortunately, that's not possible for security reasons. Here's a snippet from /etc/sudoers:
root ALL=(ALL) ALL
awoolford ALL=(ALL) ALL, !SU, !SHELLS, !RESTRICT
Can Ansible work in an environment like this? If so, what am I doing wrong?
Well, you simply cannot execute /bin/sh or /bin/bash as your /etc/sudoers shows. What you could do is change ansible's default shell to something else (variable executable in ansible.conf).
Since your sudo policy allows everything by default (does not seem like really secure to me), and I suppose ansible expects an sh-compatible shell, as a really dirty hack you could copy /bin/bash to some other path/name and set the executable variable accordingly (not tested).
In the playbook (some.yml) file, set
runthisplaybook.yml
---
- hosts: label_which_will_work_on_some_servers
sudo: yes
roles:
- some_role_i_want_to_run
Next, in the role//tasks/main.yml for the action which you have to run as sudo.. use something like become_user (where common_user is a variable defined in some role's defaults\main.yml file as common_user: "this_user_can_sudo":
- name: Run chkconfig on init script
command: "sudo -u root /sbin/chkconfig --add tomcat"
# Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
# OR Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "sudo -u firstuser sudo -u seconduser chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
# OR Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
PS: While running ansible-playbook,
ansible-playbook runthisplaybook.yml --sudo-user=this_user_can_sudo -i hosts.yml -u user_which_will_connect_from_source_machine --private-key ${DEPLOYER_KEY_FILE} --extra-vars "target_svr_type=${server_type} deploy_environment=${DEPLOY_ENVIRONMENT} ansible_user=${ANSIBLE_USER}"
After a research over the subject, as of Ansible 2.8 it doesn't seem you have a way to run commands as a different user using become without root permissions.
There's another way to achieve what you were asking without being so, how to put it, 'hacky'.
You can use the shell module with sudo su - <user> -c "COMMAND" to execute a command as a different user, without the need for root access to the original user.
For example,
1 ---
2 - hosts: target_host
3
4 tasks:
5 - shell: 'sudo su EXEC_USER -c "whoami"'
6 register: x
7
8 - debug:
9 msg: "{{ x.stdout_lines }}" # This returns EXEC_USER
However, if your play is complex, you would need to break it down and wrap only the commands that are required to be executed as different user.
This isn't best practice (using sudo + shell instead of become), however that's a solution, and in my opinion a better one than creating dummy shell on every node you manage.
I think now sudo: yes is depricated and replace with become: yes
---
- hosts: servers_on_which_you_want_to_run
become: yes
roles:
- some_role
The smiplist solution is just create a ansible.cfg in your playbook directory with the following content, if it doesn't accept root user:
[defaults]
sudo_user = UsernameToWhichYouWantToUse
Hope, this will solve your problem.
I am looking for a way to run a Postgres script using Ansible. While I found a reasonably good example Here, I need to:
Run the script as user postgres
I don't necessarily need to keep a copy of the script on the server so if I need to have a copy, it will only be for temp use.
Can anyone tell me if this is possible and if so an example of running it. Here is what I tried so far using Ansible and it just hung at these points:
- name: Testing DB to make sure it is available
command: psql -U bob image
register: b
- debug: b
- name: Verifying Tables exist in Image
shell: \d image
register: c
- debug: c
- name: Exiting Image DB
shell: \q
register: d
- debug: d
- name: Going to Agent DB
command: psql -U bob agent
register: e
- debug: e
This always hangs at the first part of it when logging into the image DB.
Why it doesn't work
This:
- name: Testing DB to make sure it is available
command: psql -U bob image
register: b
- debug: b
- name: Verifying Tables exist in Image
shell: \d image
register: c
- debug: c
doesn't do what you think it does.
The first command runs psql -U bob image. This starts a psql session. psql waits for input from stdin. Ansible will never send any, it is simply waiting for the command you specified to exit, so it can check the exit code.
So Ansible waits for psql to exit, and psql waits for Ansible to send some input.
Each task in Ansible is independent. The shell or command modules do not change the shell that subsequent commands run in. You simply can't do this the way you expect.
Even if psql exited after the first task (or went to the background), you'd just get an error from the second task like:
bash: d: command not found
So the way you're trying to do this just isn't going to work.
How to do it
You need to run each task as a separate psql command, with a command string:
- name: Testing DB to make sure it is available
command: psql -U bob image -c 'SELECT 1;'
- name: Verifying Tables exist in Image
command: psql -U bob image -c '\d image'
... or with standard input, except that Ansible doesn't seem to support supplying a variable as stdin to a command.
... or with a (possibly templated) SQL script:
- name: Template sql script
template: src="my.sql.j2" dest="{{sometemplocation}}/my.sql"
- name: Execute sql script
shell: "psql {{sometemplocation}}/my.sql"
- name: Delete sql script
file: path="{{sometemplocation}}/my.sql" state=absent
Alternately you can use Ansible's built-in support for querying PostgreSQL to do it, but in that case you cannot use the psql client's backslash commands like \d, you'd have to use only SQL. Query information_schema for table info, etc.
Here's how some of my code looks
Here's an example from an automation module I wrote that does a lot with PostgreSQL.
Really, I should just suck it up and write a psql Ansible task that runs commands via psql, rather than using shell, which is awful and clumsy. For now, though, it works. I use connection strings that're assigned from variables or generated using set_fact to reduce the mess a bit and make connections more flexible.
- name: Wait for the target node to be ready to be joined
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_join_target_dsn}}' -qAtw 'SELECT bdr.bdr_node_join_wait_for_ready();'"
- name: Template pre-BDR-join SQL script
template: src="{{bdr_pre_join_sql_template}}" dest="{{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql"
- name: Execute pre-BDR-join SQL script
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -f {{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql"
- name: Delete pre-BDR-join SQL script
file: path="{{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql" state=absent
- name: bdr_group_join
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -c \"SELECT bdr.bdr_group_join(local_node_name := '{{inventory_hostname}}', node_external_dsn := '{{bdr_node_dsn}}', join_using_dsn := '{{bdr_join_target_dsn}}');\""
- name: Template post-BDR-join SQL script
template: src="{{bdr_post_join_sql_template}}" dest="{{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql"
- name: Execute post-BDR-join SQL script
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -f {{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql"
- name: Delete post-BDR-join SQL script
file: path="{{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql" state=absent
The answer that Craig gives is good, but fails to solve the problem of running the commands as a specific user. That can be done with my additions to his code:
- name: Testing DB to make sure it is available
become: true
become_user: postgres
command: psql -U bob image -c 'SELECT 1;'
- name: Verifying Tables exist in Image
become: true
become_user: postgres
command: psql -U bob image -c '\d image'
Note the "become" and "become_user" parameters. These will tell Ansible to change to the correct user before running the commands.
IMPORTANT: Ansible Version 1.9 and earlier use sudo: yes and sudo_user: postgres instead of become: true and become_user: postgres
Building on the excellent responses above, you can also specify environment variables in your Ansible task as shown below. Note that this assumes you have set up a .pgpass file with the password for the target db.
- name: Execute some sql via psql
command: psql -f /path/to/your/sql
environment:
PGUSER: "{{ db_user }}"
PGDATABASE: "{{ db_name }}"
PGHOST: "{{ db_host }}"
PGPASS: "{{ pgpass_filepath }}"