Ansible postgresql_db task fails after a very long pause - postgresql

The following ansible task (in a vagrant VM) fails :
- name: ensure database is created
postgresql_db: name={{dbname}}
sudo_user: postgres
the task pauses for a few minutes before failing
the vagrant VM is a centos6.5.1,
the tasks output is :
TASK: [postgresql | ensure database is created] *******************************
fatal: [192.168.78.6] => failed to parse:
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo via ansible, key=glxzviadepqkwddapvjheeuillbdakly] password:
FATAL: all hosts have already failed -- aborting
I have verified that postgres is prooperly installed
by doing vagrant ssh and connecting vial psql.
I also validated that I can do a "sudo su postgres" within the VM ...
======== update
It looks like the problem is the sudo_user: postgres, because removing the
above postgres tasks and replacing with this one causes the same problem :
- name: say hello from postgress
command: echo "hello"
sudo_user: postgres
the output is exactly the same as above, so it's really a problem of
ansible doing a sudo_user on centos6.5
one interesting observation, although I can do "sudo su postgres" from
inside the vm
when I call "psql" (as the postgres user) I get the message :
could not change directory to "/home/vagrant": Permission denied
but the psql shell still starts successfully
======== conclusion
Problem was fixed by changing to a stock centos box,
lesson learned : when using ansible/vagrant, only use stock OS images...

I am using the wait for host:
- local_action: wait_for port=22 host="{{PosgresHost}}" search_regex=OpenSSH delay=1 timeout=60
ignore_errors: yes
PS:
I think you should use gather_facts: False and do setup after ssh is up.
Example main.yml:
---
- name: Setup
hosts: all
#connection: local
user: root
gather_facts: False
roles:
- main
Example roles/main/tasks/main.yml
- debug: msg="System {{ inventory_hostname }} "
- local_action: wait_for port=22 host="{{ inventory_hostname}}" search_regex=OpenSSH delay=1 timeout=60
ignore_errors: yes
- action: setup
ansible-playbook -i 127.0.0.1, main.yml
PLAY [Setup] ******************************************************************
TASK: [main | debug msg="System {{ inventory_hostname }} "] *******************
ok: [127.0.0.1] => {
"msg": "System 127.0.0.1 "
}
TASK: [main | wait_for port=22 host="{{ inventory_hostname}}" search_regex=OpenSSH delay=1 timeout=60] ***
ok: [127.0.0.1 -> 127.0.0.1]
PLAY RECAP ********************************************************************
127.0.0.1 : ok=2 changed=0 unreachable=0 failed=0

Related

Running command during docker compose or docker build failed

I am trying to build mongo inside docker and I want to push database, collection and document inside the collection I tried with docker build and below my Dockerfile
FROM mongo
RUN mongosh mongodb://127.0.0.1:27017/demeter --eval 'db.createCollection("Users")'
RUN mongosh mongodb://127.0.0.1:27017/demeter --eval 'var document = {"_id": "61912ebb4b6d7dcc7e689914","name": "Test Account","email":"test#test.net", "role": "admin", "company_domain": "test.net","type": "regular","status": "active","createdBy": "61901a01097cb16e554f5a19","twoFactorAuth": false, "password": "$2a$10$MPDjDZIboLlD8xpc/RfOouAAAmBLwEEp2ESykk/2rLcqcDJJEbEVS"}; db.Users.insert(document);'
EXPOSE 27017
and using Docker Compose
version: '3.9'
services:
web:
build:
context: ./server
dockerfile: Dockerfile
ports:
- "8080:8080"
demeter_db:
image: "mongo"
volumes:
- ./mongodata:/data/db
ports:
- "27017:27017"
command: mongosh mongodb://127.0.0.1:27017/demeter --eval 'db.createCollection("Users")'
demeter_redis:
image: "redis"
I want to add the below records because the Web Server is using them in backend. if there is a better way of doing it I would be thankful.
What I get is the below error
demeter_db_1 | Current Mongosh Log ID: 61dc697509ee790cc89fc7aa
demeter_db_1 | Connecting to: mongodb://127.0.0.1:27017/demeter?directConnection=true&serverSelectionTimeoutMS=2000
demeter_db_1 | MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017
Knowing when I connect to interactive shell inside mongo container and add them manually things works fine.
root#8b20d117586d:/# mongosh 127.0.0.1:27017/demeter --eval 'db.createCollection("Users")'
Current Mongosh Log ID: 61dc64ee8a2945352c13c177
Connecting to: mongodb://127.0.0.1:27017/demeter?directConnection=true&serverSelectionTimeoutMS=2000
Using MongoDB: 5.0.5
Using Mongosh: 1.1.7
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
------
The server generated these startup warnings when booting:
2022-01-10T16:52:14.717+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-01-10T16:52:15.514+00:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
------
{ ok: 1 }
root#8b20d117586d:/# exit
exit
Cheers

Error creating a connection to Postgresql while preparing konga database

After installing Konga, we are trying to prepare Konga database on the already running Postgresql database. by using suggested command i.e.
node ./bin/konga.js prepare --adapter postgres --uri postgresql://localhost:5432/konga
But we are facing the error as below:
Error creating a connection to Postgresql using the following settings:
postgresql://localhost:5432/konga?host=localhost&port=5432&schema=true&ssl=false&adapter=sails-postgresql&user=postgres&password=XXXX&database=konga_database&identity=postgres
* * *
Complete error details:
error: password authentication failed for user "root"
error: A hook (`orm`) failed to load!
error: Failed to prepare database: error: password authentication failed for user "root"
We even created the schema konga_database manually and have tried several variations for prepare command but no fate
node ./bin/konga.js prepare --adapter postgres --uri postgresql://kong:XXXX#localhost:5432/konga_database
node ./bin/konga.js prepare --adapter postgres --uri postgresql://kong#localhost:5432/konga
node ./bin/konga.js prepare --adapter postgres --uri postgresql://kong#localhost:5432/konga_database
Below is config/connections.js
postgres: {
adapter: 'sails-postgresql',
url: process.env.DB_URI,
host: process.env.DB_HOST || 'localhost',
user: process.env.DB_USER || 'postgres',
password: process.env.DB_PASSWORD || 'XXXX',
port: process.env.DB_PORT || 5432,
database: process.env.DB_DATABASE ||'konga_database',
// schema: process.env.DB_PG_SCHEMA ||'public',
// poolSize: process.env.DB_POOLSIZE || 10,
ssl: process.env.DB_SSL ? true : false // If set, assume it's true
},
Below is .env file configuration
PORT=1337
NODE_ENV=production
KONGA_HOOK_TIMEOUT=120000
DB_ADAPTER=postgres
DB_URI=postgresql://localhost:5432/konga
DB_HOST=localhost
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=XXXX
KONGA_LOG_LEVEL=info
TOKEN_SECRET=
kong and postgresql are already running on the AWS linux AMI 2 server on there respective ports i.e. 8443 & 5432
Please help us to prepare DB and start konga service. Also. let us know in case you need more info.
Node v: v12.19.0
NPM v: 6.14.8
Regards
Nitin G
Maybe I overlooked it, but what version of PostreSQL are you using?
Konga is not able to support postgresql 12:
https://github.com/pantsel/konga/issues/487
Have you tried like this?
.env
DB_URI=postgresql://userdb:passworddb#localhost:5432/kongadb
I tried on Postgresql 9.6
https://www.rosehosting.com/blog/how-to-install-postgresql-9-6-on-ubuntu-20-04/

Graylog2 mongo profiler plugin won't connect to mongo instance, error: "Exception opening socket" (everything dockerized)

I am trying to play with graylog's mongo profiler plugin using docker to run everything. But I can't get any profiling logs into graylog.
When I start the mongo input from the graylog UI it eventually times out with an error:
Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:37017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}].
This is my setup based on following the graylog dockerhub installation and the mongo profiler plugin guide and modifying bits:
(1) I bring up graylog, mongo and elastic using a docker-compose file:
version: '2'
services:
some-mongo:
image: "mongo:3"
some-elasticsearch:
image: "elasticsearch:2"
command: "elasticsearch -Des.cluster.name='graylog'"
graylog:
image: graylog2/server:2.2.1-1
environment:
GRAYLOG_PASSWORD_SECRET: somepasswordpepper
GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
GRAYLOG_WEB_ENDPOINT_URI: http://127.0.0.1:9000/api
links:
- some-mongo:mongo
- some-elasticsearch:elasticsearch
ports:
- "9000:9000"
- "514:514/udp"
- "12202:12202"
- "37017:37017"
That has worked fine so far and I've been able to send in syslog udp messages and gelf http messages.
(2) I created a separate mongo docker container with ports mapped because I worry that if I use 27017, that graylog might look in its own internal mongodb container:
docker run -d -p 37017:27017 mongo:2.4
I start a mongo session and enable profiling for a "graylog" database:
$ mongo --port 37017
> use graylog
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "ok" : 1 }
> db.foo.insert({_id:1})
// Check that profiling data is being written to system.profile:
> db.system.profile.find().limit(1).sort( { ts : -1 } ).pretty()
{
"op" : "query",
"ns" : "graylog.foo",
"query" : {
},
"ntoreturn" : 0,
"ntoskip" : 0,
....
"allUsers" : [ ],
"user" : ""
}
So it seems like the mongod instance is running and profiling is working.
(3) I download the plugin jar and docker cp it into the plugins dir inside the graylog docker container. Something like:
docker cp graylog-plugin-mongodb-profiler-2.0.1.jar e89a2decda37:/usr/share/graylog/plugin
Then restart graylog.
I can see that the file is there:
$ docker exec -it e89a2decda37 /bin/sh
# ls /usr/share/graylog/plugin
graylog-plugin-anonymous-usage-statistics-2.2.1.jar graylog-plugin-map-widget-2.2.1.jar
graylog-plugin-beats-2.2.1.jar graylog-plugin-mongodb-profiler-2.0.1.jar
graylog-plugin-collector-2.2.1.jar graylog-plugin-pipeline-processor-2.2.1.jar
graylog-plugin-enterprise-integration-2.2.1.jar
So that part seemed to work fine and I can see an entry "Mongo profiler input" in the list of input types in the graylog UI.
(4) I create a "Mongo profiler input" input with:
hostname: localhost
port: 37017
database: graylog
(5) After I click save, the input tries to start then eventually fails like above. Restarting graylog or trying to restart the input results in the same failures.
I have tried step (2) with different versions of mongo in case there was some driver incompatibility but they all fail with same error. I've tried docker images:
mongo:3
mongo:2.6
mongo:2.4
Thanks in advance!
Like Thilo suggested above, the hostname for the graylog input shouldn't be "localhost" as that will point graylog to the docker container hosting it.
So I found the ip of the host machine using:
docker exec -it [CONTAINER ID] /bin/sh
/sbin/ip route|awk '/default/ { print $3 }'
and rewired the input and Bob's your Uncle!

Setting up travis ci with phoenix + heroku

I'm running into issues with automating deployments with Travis CI to Heroku for my Phoenix app. Here's Travis CI build error:
(Mix) The database for AgilePulse.Repo couldn't be created: tcp connect: connection refused - :econnrefused
Here's my .travis.yml config:
language: elixir
elixir:
- 1.3.2
otp_release:
- 19.0
sudo: false
addons:
postgresql: '9.5'
notifications:
email: false
env:
- MIX_ENV=test
before_script:
- cp config/travis_ci_test.exs config/test.secret.exs
- mix do ecto.create, ecto.migrate
Here's my travis_ci_test.exs:
use Mix.Config
# Configure your database
config :agile_pulse, AgilePulse.Repo,
adapter: Ecto.Adapters.Postgres,
username: "postgres",
password: "",
database: "travis_ci_test",
hostname: "localhost",
pool: Ecto.Adapters.SQL.Sandbox
Any pointers would be greatly appreciated!
Additional info:
GitHub repo: https://github.com/cscairns/agile-pulse-api
On a second look: judging by the travis log you posted, looks like you're bootstrapping an Ubuntu 12.04 Precise for your build; I suspect that Postgres 9.5 is not available on precise:
https://docs.travis-ci.com/user/database-setup/#Using-a-different-PostgreSQL-Version
Could you try switching to Postgres 9.4 and see if that works?

Chef server 12 reconfiguration is not working on Ubuntu 14.04

I'm trying to install a Chef server on an Ubuntu 14.04 box. I've downloaded the .deb file from the site and installed with sudo dpkg -i chef-server-core_12.0.8-1_amd64.deb but when I do sudo chef-server-ctl reconfigure all goes well until it reaches the postgresql part:
Running handlers:
[2015-05-03T23:16:07-04:00] ERROR: Running exception handlers
Running handlers complete
[2015-05-03T23:16:07-04:00] ERROR: Exception handlers complete
[2015-05-03T23:16:07-04:00] FATAL: Stacktrace dumped to /opt/opscode/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 44 resources updated in 198.107797872 seconds
[2015-05-03T23:16:08-04:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: private-chef_pg_database[opscode-pgsql] (private-chef::postgresql line 127) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[create_database_opscode-pgsql] (/opt/opscode/embedded/cookbooks/cache/cookbooks/private-chef/providers/pg_database.rb line 13) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of createdb --template template0 --encoding UTF-8 opscode-pgsql ----
STDOUT:
STDERR: createdb: could not connect to database template1: FATAL: role "opscode-pgsql" does not exist
---- End output of createdb --template template0 --encoding UTF-8 opscode-pgsql ----
Ran createdb --template template0 --encoding UTF-8 opscode-pgsql returned 1
Am I missing any step? The installation instruction does not say anything about any other middle task to perform.
Thank you very much for any help you can give me.
Saw exact same error installing chef-server-core-12.0.8-1.el6.x86_64.rpm on RHEL6.6. Had gone through every pre-req step listed at https://docs.chef.io/install_server_pre.html ... finally chalked it up to having multiple versions of Postgres on the system.
Attempted the same install and reconfigure command (chef-server-ctl reconfigure) on a system without Postgres installed and had success.
HTH.
Cannot reproduce your problem. The installation of chef server 12.0.8-1 worked fine on my kitchen/vagrant setup
Example
├── Berksfile
├── .kitchen.yml
├── roles
   └── chef-server.json
Berksfile
source "https://supermarket.chef.io"
cookbook "chef-server"
kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-14.04
driver:
network:
- ["private_network", {ip: "192.168.38.34"}]
customize:
memory: 4096
suites:
- name: default
run_list:
- role[chef-server]
attributes:
roles/chef-server.json
{
"name": "chef-server",
"description": "Chef server",
"run_list": [
"recipe[chef-server]"
]
}