I’d like to use Ansible to manage the configuration of a our Hadoop cluster (running Red Hat).
I have sudo access and can manually ssh into the nodes to execute commands. However, I’m experiencing problems when I try to run Ansible modules to perform the same tasks. Although I have sudo access, I can’t become root. When I try to execute Ansible scripts that require elevated privileges, I get an error like this:
Sorry, user awoolford is not allowed to execute '/bin/bash -c echo
BECOME-SUCCESS- […] /usr/bin/python
/tmp/ansible-tmp-1446662360.01-231435525506280/copy' as awoolford on
[some_hadoop_node].
Looking through the documentation, I thought that the become_allow_same_user property might resolve this, and so I added the following to ansible.cfg:
[privilege_escalation]
become_allow_same_user=yes
Unfortunately, it didn't work.
This post suggests that I need permissions to sudo /bin/sh (or some other shell). Unfortunately, that's not possible for security reasons. Here's a snippet from /etc/sudoers:
root ALL=(ALL) ALL
awoolford ALL=(ALL) ALL, !SU, !SHELLS, !RESTRICT
Can Ansible work in an environment like this? If so, what am I doing wrong?
Well, you simply cannot execute /bin/sh or /bin/bash as your /etc/sudoers shows. What you could do is change ansible's default shell to something else (variable executable in ansible.conf).
Since your sudo policy allows everything by default (does not seem like really secure to me), and I suppose ansible expects an sh-compatible shell, as a really dirty hack you could copy /bin/bash to some other path/name and set the executable variable accordingly (not tested).
In the playbook (some.yml) file, set
runthisplaybook.yml
---
- hosts: label_which_will_work_on_some_servers
sudo: yes
roles:
- some_role_i_want_to_run
Next, in the role//tasks/main.yml for the action which you have to run as sudo.. use something like become_user (where common_user is a variable defined in some role's defaults\main.yml file as common_user: "this_user_can_sudo":
- name: Run chkconfig on init script
command: "sudo -u root /sbin/chkconfig --add tomcat"
# Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
# OR Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "sudo -u firstuser sudo -u seconduser chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
# OR Set execute permission on run_jmeter_test.sh
- name: Set execute permission on run_jmeter_test.sh
command: "chmod -R 755 {{ jmeter_perf_tests_results }}"
become_user: "{{ common_user }}"
PS: While running ansible-playbook,
ansible-playbook runthisplaybook.yml --sudo-user=this_user_can_sudo -i hosts.yml -u user_which_will_connect_from_source_machine --private-key ${DEPLOYER_KEY_FILE} --extra-vars "target_svr_type=${server_type} deploy_environment=${DEPLOY_ENVIRONMENT} ansible_user=${ANSIBLE_USER}"
After a research over the subject, as of Ansible 2.8 it doesn't seem you have a way to run commands as a different user using become without root permissions.
There's another way to achieve what you were asking without being so, how to put it, 'hacky'.
You can use the shell module with sudo su - <user> -c "COMMAND" to execute a command as a different user, without the need for root access to the original user.
For example,
1 ---
2 - hosts: target_host
3
4 tasks:
5 - shell: 'sudo su EXEC_USER -c "whoami"'
6 register: x
7
8 - debug:
9 msg: "{{ x.stdout_lines }}" # This returns EXEC_USER
However, if your play is complex, you would need to break it down and wrap only the commands that are required to be executed as different user.
This isn't best practice (using sudo + shell instead of become), however that's a solution, and in my opinion a better one than creating dummy shell on every node you manage.
I think now sudo: yes is depricated and replace with become: yes
---
- hosts: servers_on_which_you_want_to_run
become: yes
roles:
- some_role
The smiplist solution is just create a ansible.cfg in your playbook directory with the following content, if it doesn't accept root user:
[defaults]
sudo_user = UsernameToWhichYouWantToUse
Hope, this will solve your problem.
Related
I have a Python docker container that needs to wait until another container (postgres server) finishes setup. I tried the standard wait-for-it.sh but several commands weren't included. I tried a basic sleep (again in an sh file) but now it's reporting exec: 300: not found when trying to finally execute the command I'm waiting on.
How do I get around this (preferably without changing the image, or having to extend an image.)
I know I could also just run a Python script, but ideally I'd like to use wait-for-it.sh to wait for the server to turn up rather than just sleep.
Dockerfile (for stuffer):
FROM python:2.7.13
ADD ./stuff/bin /usr/local/bin/
ADD ./stuff /usr/local/stuff
WORKDIR /usr/local/bin
COPY requirements.txt /opt/updater/requirements.txt
COPY internal_requirements.txt /opt/stuff/internal_requirements.txt
RUN pip install -r /opt/stuff/requirements.txt
RUN pip install -r /opt/stuff/other_requirements.txt
docker-compose.yml:
version: '3'
services:
local_db:
build: ./local_db
ports:
- "localhost:5432:5432"
stuffer:
build: ./
depends_on:
- local_db
command: ["./wait-for-postgres.sh", "-t", "300", "localhost:5432", "--", "python", "./stuffing.py", "--file", "./afile"]
Script I want to use (but can't because no psql or exec):
#!/bin/bash
# wait-for-postgres.sh
set -e
host="$1"
shift
cmd="$#"
until psql -h "$host" -U "postgres" -c '\l'; do >&2 echo "Postgres is unavailable - sleeping"
sleep 1
done
>&2 echo "Postgres is up - executing command"
exec $cmd
Sergey's comment. I had wrong argument order. This issue had nothing to do with docker and everything to do with my inability to read.
I made an example so you can see it working:
https://github.com/nitzap/wait-for-postgres
On the other hand also you can have errors inside the execution of the script to validate that the service is working. You should not refer as localhost .... because that is within the contexts of containers, if you want to point to another container has to be through the name of the service.
New to Ansible, Running version 2.1.0. I've written an Ansible playbook that runs a PostgreSQL query against a group of hosts. When I specify the SQL DB password in the shell command it works, but I'm looking to run the playbook against a group of hosts and need a better way to input the passwords as they are all unique. Could anyone suggest a better way to do this?
---
- hosts: Test_Hosts
sudo: yes
sudo_user: root
gather_facts: yes
tasks:
- name: Login to DB and run command
shell: export PGPASSWORD='Password'; psql -U 'user' -d 'db' -c 'select * FROM table';
register: select_all_from_table
- name: Display table contents
debug: msg="{{ select_all_from_table.stdout }}"
I saw another thread on the topic but was not sure how to implement the suggestion: Run a postgresql command with ansible playbook. Postgresql requires password
Ansible allows you to set environment variables for a task using the environment parameter to any task.
So in your case you could just do this:
- name: Login to DB and run command
shell: psql -U 'user' -d 'db' -c 'select * FROM table';
register: select_all_from_table
environment:
PGPASSWORD: '{{ pgpassword }}'
And then set the pgpassword variable at the group or host level.
I just ran into this issue today and this is what worked for me. On Linux you can package all the credentials into a ~/.pgpass hidden file.
Just create it locally (in this case in ./files/pgpass) and then use ansible to copy it onto the host before you run the psql commands.
- name: set passwd file for PSQL
copy:
src: files/pgpass
dest: ~/.pgpass
mode: 0600 ### important: will not work with wrong permissions
- name: PSQL command
shell: "psql -U 'user' -d 'db' -c 'select * FROM table'"
register: select_all_from_table
The file contents must be in the following format:
hostname:port:database:username:password
However, you can use wildcards, so mine looks like this, for example:
*:*:db1:user1:passwd1
*:*:db2:user2:passwd2
See documentation for more details:
https://www.postgresql.org/docs/9.1/static/libpq-pgpass.html
I've really been struggling over the past few days trying to setup some docker containers and shell scripts to create an environment for my application to run in.
The tall and short is that I have a web server which requires a database to operate. My aim is to have end users unzip the content onto their docker machine, run a build script (which just builds the relevant docker images), then run a OneTime.sh script (which creates the volumes and databases necessary), during this script, they are prompted for what user name and password they would like for the super user of the database.
The problem I'm having is getting those values to the docker image. Here is my script:
# Create the volumes for the data backend database.
docker volume create --name psql-data-etc
docker volume create --name psql-data-log
docker volume create --name psql-data-lib
# Create data store database
echo -e "\n${TITLE}[Data Store Database]${NC}"
docker run -v psql-data-etc:/etc/postgresql -v psql-data-log:/var/log/postgresql -v psql-data-lib:/var/lib/postgresql -p 9001:5432 -P --name psql-data-onetime postgres-setup
# Close containers
docker stop psql-data-onetime
docker rm psql-data-onetime
docker stop psql-transactions-onetime
docker rm psql-transactions-onetime
And here is the docker file:
FROM ubuntu
#Required environment variables: USERNAME, PASSWORD, DBNAME
# Add the PostgreSQL PGP key to verify their Debian packages.
# It should be the same key as https://www.postgresql.org/media/keys/ACCC4CF8.asc
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
# Add PostgreSQL's repository. It contains the most recent stable release
# of PostgreSQL, ``9.3``.
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" > /etc/apt/sources.list.d/pgdg.list
# Install ``python-software-properties``, ``software-properties-common`` and PostgreSQL 9.3
# There are some warnings (in red) that show up during the build. You can hide
# them by prefixing each apt-get statement with DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python-software-properties software-properties-common postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
# Note: The official Debian and Ubuntu images automatically ``apt-get clean``
# after each ``apt-get``
# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed``
USER postgres
# Complete configuration
USER root
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.3/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.3/main/postgresql.conf
# Expose the PostgreSQL port
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
# Run setup script
ADD Setup.sh /
CMD ["sh", "Setup.sh"]
The script 'Setup.sh' is the following:
echo -n " User name: "
read user
echo -n " Password: "
read password
echo -n " Database Name: "
read dbname
/etc/init.d/postgresql start
/usr/lib/postgresql/9.3/bin/psql --command "CREATE USER $user WITH SUPERUSER PASSWORD '$password';"
/usr/lib/postgresql/9.3/bin/createdb -O $user $dbname
exit
Why doesn't this work? (I don't get prompted to enter the text, and it throws an error that the parameters are bad). What is the proper way to do something like this? It feels like it's probably a pretty common problem to solve, but I cannot for the life of me find any non convoluted examples of this behaviour.
The main purpose of this is to make life easier for the end user, so if I could just prompt them for the user name, password, and dbname, (plus calling the correct scripts), that would be ideal.
EDIT:
After running the log file looks like this:
User name:
Password:
Database Name:
Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..]
EDIT 2:
After updating to CMD ["sh", "-x", "Setup.sh"]
I get:
echo -n User name:
+read user
:bad variable nameuser
echo -n Password:
+read password
:bad variable namepassword
echo -n Database Name:
+read dbname
:bad variable dbname
I am looking for a way to run a Postgres script using Ansible. While I found a reasonably good example Here, I need to:
Run the script as user postgres
I don't necessarily need to keep a copy of the script on the server so if I need to have a copy, it will only be for temp use.
Can anyone tell me if this is possible and if so an example of running it. Here is what I tried so far using Ansible and it just hung at these points:
- name: Testing DB to make sure it is available
command: psql -U bob image
register: b
- debug: b
- name: Verifying Tables exist in Image
shell: \d image
register: c
- debug: c
- name: Exiting Image DB
shell: \q
register: d
- debug: d
- name: Going to Agent DB
command: psql -U bob agent
register: e
- debug: e
This always hangs at the first part of it when logging into the image DB.
Why it doesn't work
This:
- name: Testing DB to make sure it is available
command: psql -U bob image
register: b
- debug: b
- name: Verifying Tables exist in Image
shell: \d image
register: c
- debug: c
doesn't do what you think it does.
The first command runs psql -U bob image. This starts a psql session. psql waits for input from stdin. Ansible will never send any, it is simply waiting for the command you specified to exit, so it can check the exit code.
So Ansible waits for psql to exit, and psql waits for Ansible to send some input.
Each task in Ansible is independent. The shell or command modules do not change the shell that subsequent commands run in. You simply can't do this the way you expect.
Even if psql exited after the first task (or went to the background), you'd just get an error from the second task like:
bash: d: command not found
So the way you're trying to do this just isn't going to work.
How to do it
You need to run each task as a separate psql command, with a command string:
- name: Testing DB to make sure it is available
command: psql -U bob image -c 'SELECT 1;'
- name: Verifying Tables exist in Image
command: psql -U bob image -c '\d image'
... or with standard input, except that Ansible doesn't seem to support supplying a variable as stdin to a command.
... or with a (possibly templated) SQL script:
- name: Template sql script
template: src="my.sql.j2" dest="{{sometemplocation}}/my.sql"
- name: Execute sql script
shell: "psql {{sometemplocation}}/my.sql"
- name: Delete sql script
file: path="{{sometemplocation}}/my.sql" state=absent
Alternately you can use Ansible's built-in support for querying PostgreSQL to do it, but in that case you cannot use the psql client's backslash commands like \d, you'd have to use only SQL. Query information_schema for table info, etc.
Here's how some of my code looks
Here's an example from an automation module I wrote that does a lot with PostgreSQL.
Really, I should just suck it up and write a psql Ansible task that runs commands via psql, rather than using shell, which is awful and clumsy. For now, though, it works. I use connection strings that're assigned from variables or generated using set_fact to reduce the mess a bit and make connections more flexible.
- name: Wait for the target node to be ready to be joined
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_join_target_dsn}}' -qAtw 'SELECT bdr.bdr_node_join_wait_for_ready();'"
- name: Template pre-BDR-join SQL script
template: src="{{bdr_pre_join_sql_template}}" dest="{{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql"
- name: Execute pre-BDR-join SQL script
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -f {{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql"
- name: Delete pre-BDR-join SQL script
file: path="{{postgres_install_dir}}/bdr_pre_join_{{inventory_hostname}}.sql" state=absent
- name: bdr_group_join
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -c \"SELECT bdr.bdr_group_join(local_node_name := '{{inventory_hostname}}', node_external_dsn := '{{bdr_node_dsn}}', join_using_dsn := '{{bdr_join_target_dsn}}');\""
- name: Template post-BDR-join SQL script
template: src="{{bdr_post_join_sql_template}}" dest="{{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql"
- name: Execute post-BDR-join SQL script
shell: "{{postgres_install_dir}}/bin/psql '{{bdr_node_dsn}}' -qAtw -f {{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql"
- name: Delete post-BDR-join SQL script
file: path="{{postgres_install_dir}}/bdr_post_join_{{inventory_hostname}}.sql" state=absent
The answer that Craig gives is good, but fails to solve the problem of running the commands as a specific user. That can be done with my additions to his code:
- name: Testing DB to make sure it is available
become: true
become_user: postgres
command: psql -U bob image -c 'SELECT 1;'
- name: Verifying Tables exist in Image
become: true
become_user: postgres
command: psql -U bob image -c '\d image'
Note the "become" and "become_user" parameters. These will tell Ansible to change to the correct user before running the commands.
IMPORTANT: Ansible Version 1.9 and earlier use sudo: yes and sudo_user: postgres instead of become: true and become_user: postgres
Building on the excellent responses above, you can also specify environment variables in your Ansible task as shown below. Note that this assumes you have set up a .pgpass file with the password for the target db.
- name: Execute some sql via psql
command: psql -f /path/to/your/sql
environment:
PGUSER: "{{ db_user }}"
PGDATABASE: "{{ db_name }}"
PGHOST: "{{ db_host }}"
PGPASS: "{{ pgpass_filepath }}"
Testing capistrano with a simple recipe.
$ cap deploy:setup
* executing `deploy:setup'
* executing "sudo -p 'sudo password: ' mkdir -p /u/apps/ [..]
[..]
deploy:setup works as exprected.
However
$ cap deploy:check
* executing `deploy:check'
* executing "test -d /u/apps/[..]
[..]
When running deploy:check I get the following error:
The following dependencies failed. Please check them and try again:
--> You do not have permissions to write to `/u/apps/
[..]
--> `/u/apps/app/shared is not writable [..]
It seems that capistrano is not using sudo while in deploy:check mode.
I don't get it!
While in deploy:setup the whole directory structure was created by capistrano without any issue?
Why capistrano doesn't use sudo as in deploy:check?
I also ran into this issue and it turned out capistrano was creating all of the folders under the <user> group except for the shared folder. SSH onto your server and do a long listing ls -l. If you see - root - root - for the shared folder, you'll just need to update the permissions on the folder:
sudo chown <user> shared
sudo chgrp <user> shared
I ran into the same issue: the trick is to explicitly configure Capistrano not to use sudo.
You can turn that off in your deploy.rb file with:
set :use_sudo, false
If you need to use sudo, how about using the sudo DSL Action Invocation in your commands:
run "#{sudo} apachectl restart"