Create multiple Postgres instances on same machine - postgresql

To test streaming replication, I would like to create a second Postgres instance on the same machine. The idea is that if it can be done on the test server, then it should be trivial to set it up on the two production servers.
The instances should use different configuration files and different data directories. I tried following the instructions here http://ubuntuforums.org/showthread.php?t=1431697 but I haven't figured out how to get Postgres to use a different configuration file. If I copy the init script, the scripts are just aliases to the same Postgres instance.
I'm using Postgres 9.3 and the Postgres help pages say to specify the configuration file on the postgres command line. I'm not really sure what this means. Am I supposed to install some client for this to work? Thanks.

I assume you can work your way out on using postgresql utilities.
Create the clusters
$ initdb -D /path/to/datadb1
$ initdb -D /path/to/datadb2
Run the instances
$ pg_ctl -D /path/to/datadb1 -o "-p 5433" -l /path/to/logdb1 start
$ pg_ctl -D /path/to/datadb2 -o "-p 5434" -l /path/to/logdb2 start
Test streaming
Now you have two instances running on ports 5433 and 5434. Configuration files for them are in data dirs specified by initdb. Tweak them for streaming replication.
Your default installation remains untouched in port 5432.

On Debian based distros you could use pg_createcluster instead of initdb:
$ pg_createcluster -u [user] -g [group] -d /path/to/data -l /path/to/log -p 5433
Also pg_ctlcluster is an alternative to pg_ctl.

Steps to create New Server Instance on PostgreSQL 9.5
On command prompt run:
initdb -D Instance_Directory_path -U username -W
(prompts for password)
Once the new Instance Directory is created. Run command prompt as Administrator
pg_ctl register -N service_name -D Instance_Directory_path -o "-p port_no"
After the service is registered, start server
pg_ctl start -D Instance_Directory_path -o "-p port_no"

To complete other answers, on CentOS 6 AND 7.
After running something like
$ initdb -D /path/to/newdb
You'll have to change at least port configuration option and, probably, listen_addresses in config file postgresql.conf.
Instead of starting inmediatly this new instance, which has been explained in previous answers, maybe you want new instance to run automatically on system start (in case of shutdown, e.g.). To do this, as CentOS doesn't have pg_ctl register option (only for Windows) you'll have to create a new service file and register it in order systemctl or service can start it up automatically.
Centos 6
Follow next commands to get service's init file:
[root#machine ~]# service postgresql-9.6 edit
Usage: /etc/init.d/postgresql-9.6 {start|stop|status|restart|upgrade|condrestart|try-restart|reload|force-reload|initdb|promote}
[root#machine ~]# cd /etc/init.d # Now we know where service file is
[root#machine init.d]# cp -p postgresql-9.6 postgresql-9.6_5433
[root#machine init.d]# vi postgresql-9.6_5433
Now you can change PGDATA directory with the one where new instance resides. If you're using Postgresql version previous to 9.4 (which you shouldn't by the time of this answer) you'll have to change PGPORT too with the value where new instance is listening to.
The name of the new service is up to you. I usually take original service name and add port number at the end.
Now you only have to register new service:
[root#machine init.d]# chkconfig postgresql-9.6_5433 on # service registered!
[root#machine init.d]# service postgresql-9.6_5433 start
Iniciando servicios postgresql-9.6_5433: [ OK ]
[root#machine init.d]# service postgresql-9.6_5433 status
Se está ejecutando postgresql-9.6_5433 (pid 120993)...
Centos 7
In CentOS 7 instead of service to control services running on the machine you have systemctl and commands and paths change a bit. But the process is the same: create new service file, edit with the new location/port, register and start:
[root#localhost ~]# locate postgresql.service
/etc/systemd/system/multi-user.target.wants/postgresql.service
/usr/lib/systemd/system/postgresql.service
[root#localhost ~]# cd /usr/lib/systemd/system
[root#localhost ~]# cp -p postgresql.service postgresql_5433.service
[root#localhost ~]# vi postgresql_5433.service
# Change PGDATA and maybe PGPORT if PG version <9.4
[root#localhost ~]# systemctl enable postgresql_5433.service
[root#localhost ~]# systemctl start postgresql_5433.service
[root#localhost ~]# systemctl list-unit-files | grep postgres
postgresql.service enabled
postgresql_5433.service enabled

Related

Unable to install postgresql ubuntu with error initdb

I am using Ubuntu server. I am trying to install postgresql on it. when I try to start postgres using
sudo service postgresql start
it result me into
*No PostgreSQL clusters exist; see "man pg_createcluster"
and when I try to create cluster using command pg_createcluster --start 9.6 main (*9.6 is my postgres version)
The result is as follows
Creating new PostgreSQL cluster 9.6/main ...
/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/9.6/main --
auth-local peer --auth-host md5
initdb: could not look up effective user ID 108: Permission denied
Error: initdb failed
I am not able to figure out what is the issue please help.
It looks like your /etc/passwd is not readable. I'm not sure which distro would require it, but it is a problem for initdb.
I believe fast solution would be running as root chmod 644 /etc/passwd
And follow with
/usr/lib/postgresql/9.6/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start
command

Postgres in Docker persistent data

I'm running postgres inside a docker container to limit the amount of system resources it has access to. I'm having some trouble understanding how to make the data persistent. I've read the following articles:
https://www.andreagrandi.it/2015/02/21/how-to-create-a-docker-image-for-postgresql-and-persist-data/
http://container42.com/2013/12/16/persistent-volumes-with-docker-container-as-volume-pattern/
Which suggest using a data only container, and then having my postgres container link to it. What I'm failing to understand is; what's the advantage to this? As far as I can tell, if for some reason the docker-machine shut down (for example; moving it to a different physical machine), the data only container stops running, and all of it's contents are lost? I've tried creating a volume in the postgres container, but it doesn't actually seem to save anything to the disk.
Here's my docker file. What am I doing wrong?
FROM ubuntu
MAINTAINER Andrew Broadbent <andrew.broadbent#manchester.ac.uk>
# Add the PostgreSQL PGP key to verify their Debian packages.
# It should be the same key as https://www.postgresql.org/media/keys/ACCC4CF8.asc
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
# Add PostgreSQL's repository. It contains the most recent stable release
# of PostgreSQL, ``9.3``.
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" > /etc/apt/sources.list.d/pgdg.list
# Install ``python-software-properties``, ``software-properties-common`` and PostgreSQL 9.3
# There are some warnings (in red) that show up during the build. You can hide
# them by prefixing each apt-get statement with DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python-software-properties software-properties-common postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
# Note: The official Debian and Ubuntu images automatically ``apt-get clean``
# after each ``apt-get``
# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed``
USER postgres
# Create a PostgreSQL role named ``docker`` with ``docker`` as the password and
# then create a database `docker` owned by the ``docker`` role.
# Note: here we use ``&&\`` to run commands one after the other - the ``\``
# allows the RUN command to span multiple lines.
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" &&\
createdb -O docker docker
# Complete configuration
USER root
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.3/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.3/main/postgresql.conf
# Expose the PostgreSQL port
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
# Set the default command to run when starting the container
USER postgres
CMD ["/usr/lib/postgresql/9.3/bin/postgres", "-D", "/var/lib/postgresql/9.3/main", "-c", "config_file=/etc/postgresql/9.3/main/postgresql.conf"]
This one answers your question about data container:
docker mounting volumes on host
Regarding to your dockerfile, I would suggest you either:
1) use data container pattern
2) mount the volume to host machine by specifying: docker run -v [host-path]:[container-path] ..., so that data will be kept at one place in your host and will not be lost after the container is removed.
Ref: https://docs.docker.com/engine/tutorials/dockervolumes/#/mount-a-host-directory-as-a-data-volume

memcached doesn't start on vagrant up with CentOS 6.5

I'm trying to provision a development box with Vagrant and a CentOS 6.5 base box. I want memcached to automatically start at system boot/vagrant up.
I have tried adding memcached -d -l localhost -p11211 to /etc/rc.d/rc.local and this does not work.
I have also tried adding to /etc/init/vagrant-mounted.conf
start on vagrant-mounted
memcached -d -l localhost -p11211
[EDIT]
I've updated /etc/rc.d/rc.local to now use the following
chkconfig memcached on
service memcached start
I'm not seeing anything in the /var/log/boot.log. It looks like rc.local is not being run at all. It has ugo+x permissions; so the file is definitely executable, but it doesn't appear to run at all.
Does memcached -d -l localhost -p11211 exit immediately or spawn a process?
If it keeps running, try: nohup memcached -d -l localhost -p11211 &
Also, try putting it in /etc/rc.local as
memcached -d -l localhost -p11211 >/var/log/memcached.log 2>&1
That will give you a log file with possible errors.
Lastly, does your install of memcached not have an init.d file in /etc/init.d ?
if it does, simply do chkconfig servicename on && service servicename start

Set Docker_Opts in centos

I need to set docker to listen to tcp://0.0.0.0/4243 on my host machine running amazon linux (centos). All the documentation I have seen has told me to run the following command
echo DOCKER_OPTS="-H=tcp://127.0.0.1:4243" >> /etc/default/docker
Which will write the correct docker_opts to /etc/default/docker. I've done this, but when I restart docker it does not listen to 127.0.0.1
I can make docker run correctly by typing
sudo /usr/bin/docker -H tcp://0.0.0.0:4243 -d &
That works, but I want the default option to be listening on tcp://0.0.0.0:4243 without having to specify it every time.
It seems that docker is completely ignoring my /etc/default/docker file so the settings are being ignored. I also tried writing the file to /etc/default/docker.io and /etc/default/docker-io (didn't really expect much to happen)
I need to be able to start docker with just
service docker start
or it will cause issues in my current deployment playbook.
Any thoughts on what I can do to set DOCKER_OPTS and not have to do it every time I restart docker?
In RHEL7, instead of modifying your docker.service unit file, you can also just edit your /etc/sysconfig/docker file:
# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
OPTIONS=--selinux-enabled -H unix:///var/run/docker.sock -H tcp://0.0.0.0:4243
and then restart your docker service.
To me, this is more reliable than modifying the service script.
For CentOS 7 (RHEL 7):
Find the systemd docker.service unit file. Mine is located at: /usr/lib/systemd/system/docker.service
In this file, edit the line in the [Service] section beginning with ExecStart=. Add the "-H tcp://0.0.0.0:4243" into the line. (notice there's no "=" between the -H and the IP address as in your example DOCKER_OPTS line above.)
On my system, the entire contents of docker.service then looks like:
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target docker.socket
Requires=docker.socket
[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=/usr/bin/docker -d -H tcp://127.0.0.1:4243 -H fd:// $OPTIONS
LimitNOFILE=1048576
LimitNPROC=1048576
[Install]
Also=docker.socket
(I only need Docker to listen on the loopback, instead of all IP addresses.)
After making this edit to the systemd unit file and restarting the Docker service via systemctl restart docker, I see the following process:
root 8574 0.0 0.2 321708 10564 ? Ssl 00:42 0:00 /usr/bin/docker -d -H tcp://127.0.0.1:4243 -H fd:// --selinux-enabled
As you can see, it does now listen on the configured TCP address, and will persist over reboots and service stop/starts.
I cannot believe how many answers there are for this. So here is another one for:
CentOS 7.3
Docker Version = 17.03.1-ce, API Version = 1.27
This answer is built upon an unbelievable playing around combination of this answer and this one and this one.
sudo vim /usr/lib/systemd/system/docker.service
insert " -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
sudo systemctl daemon-reload //refresh your file changes above
sudo systemctl restart docker
netstat -l | grep 4243 //verify port is open
connect to your docker host from somewhere, like Jenkins Docker Plugin, i.e. tcp://[server_ip]:4243
Editing /etc/docker/daemon.json seems to be the new, supported way.
With Docker 1.7.1 on CentOS 7 neither modifying /usr/lib/systemd/system/docker.service or /etc/sysconfig/docker worked for me. It seems that in systemd sets up the socket, so in order to change the group you have to edit SocketGroup in /usr/lib/systemd/system/docker.socket
[Unit]
Description=Docker Socket for the API
PartOf=docker.service
[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=jenkins
[Install]
WantedBy=sockets.target
I Think on CentOS, you can try setting the options as below in the file /etc/sysconfig/docker
other_args="-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock"
Then restart the docker and try checking if the port is opening using
netstat -plt | grep 4243
This should list if listening
I needed to change the default bridge interface docker0 to use my own bridge interface br0 and putting the following content in that file solved my issue:
CentOS 7.2 and docker 1.10.3
/usr/lib/systemd/system/docker.service.d/docker.conf
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon --bridge=br0 -H fd://
and of course the following need to be performed after:
sudo systemctl daemon-reload
sudo systemctl restart docker
ip link del docker0
It worked for me when I followed how its shown in the posts above with drop-in replacement files in: /etc/systemd/system/docker.service.d
I am working on centos 7.
I just want to add insecure-registry in docker config file then I changed "DOCKER_OPTS=--insecure-registry=...." in /etc/sysconfig/docker while it did not work.
While I saw a INSECURE_REGISTRY in the config so I changed this variable and it WORKS!
So I guess DOCKER_OPTS does not work here!
But it worked on my unbuntu 14!
It is really frustrating when using docker!
Based on https://docs.docker.com/engine/admin/configuring/
sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/docker.conf
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// -D -H tcp://127.0.0.1:4243
sudo systemctl daemon-reload
sudo systemctl restart docker
1、edit /usr/lib/systemd/system/docker.service to add two param in the service section:
# vim /usr/lib/systemd/system/docker.service
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock
2、reload the configuration,and then restart docker。
# systemctl daemon-reload
# systemctl restart docker
3、to check for success, see if the return the following response。
# ps -ef|grep docker
root 26208 1 0 23:51 ? 00:00:00 /usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock
reference from Expose the Docker Remote API on Centos 7?
I believe things have changed now, this answer by Brian Ogden had worked for me earlier but didn't work on my environment today, probably with the updated versions of the docker, kernel, and OS.
CentOS 7.4.1708 (on AWS)
Docker 17.03.2-ce
API 1.27
This is what worked after few hit and trials. I could not find it documented anywhere.
In file /etc/systemd/system/docker.service.d/execstart.conf, replace the last ExecStart (there are two) with below
ExecStart=/usr/bin/dockerd --graph=/var/lib/docker --storage-driver=overlay -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
Now, my files looks like this
# cat /etc/systemd/system/docker.service.d/execstart.conf
[Service]
Restart=always
StartLimitInterval=0
RestartSec=15
ExecStartPre=-/sbin/ip link del docker0
ExecStart=
ExecStart=/usr/bin/dockerd --graph=/var/lib/docker --storage-driver=overlay -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
#
Once, the above file is changed just the run the below command to activate the changes.
# systemctl daemon-reload && systemctl stop docker && rm -f /var/run/docker.sock && systemctl start docker
To verify if everything is working fine, you can run any (or all) of below commands
# systemctl status docker.service | grep tcp
├─21812 /usr/bin/dockerd --graph=/var/lib/docker --storage-driver=overlay -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
#
# netstat -an | grep 4243
tcp6 0 0 :::4243 :::* LISTEN
#
# ps aux | grep [4]243
root 21812 1.0 0.8 1017512 67876 ? Ssl 15:11 0:06 /usr/bin/dockerd --graph=/var/lib/docker --storage-driver=overlay -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
#
# docker -H :4243 info

I have installed PostgreSQL via MacPorts, but cannot access it

As I said in title, I've installed PostgreSQL usind MacPorts, but cannot access it.
The installation process was
$ sudo port install postgresql83-server
$ sudo mkdir -p /opt/local/var/db/postgresql83/webcraft
$ sudo chown postgres:postgres /opt/local/var/db/postgresql83/webcraft
$ sudo su postgres -c '/opt/local/lib/postgresql83/bin/initdb -D /opt/local/var/db/postgresql83/webcraft'
$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.postgresql83-server.plist
My PATH is
/opt/local/lib/postgresql83/bin:/opt/local/lib/mysql5/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
I try to connect the server using psql client
$ psql
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Here is some info
$ ps ax | grep postgres | grep -v grep
52 ?? Ss 0:00.00 /opt/local/bin/daemondo --label=postgresql83-server --start-cmd /opt/local/etc/LaunchDaemons/org.macports.postgresql83-server/postgresql83-server.wrapper start ; --stop-cmd /opt/local/etc/LaunchDaemons/org.macports.postgresql83-server/postgresql83-server.wrapper stop ; --restart-cmd /opt/local/etc/LaunchDaemons/org.macports.postgresql83-server/postgresql83-server.wrapper restart ; --pid=none
Did you try running:
which psql
I imagine psql is still referencing /usr/bin/psql, and the macports version of psql is suffixed with the version number, in your case psql83. You can alias psql to psql83 as a simple workaround. Better would be to change the default:
sudo port select --set postgresql postgresql83
That will do the proper routing.
There is a very easy solution to this, but it's not well documented in my opinion:
MacPorts encourages installing their *_select ports to manage potentially multiple versions of software (say you want Postgres93 and Postgres94 at the same time). It's a great feature, but it adds an extra step that is for some reason rarely mentioned in the docs:
$ sudo port install postgresql94-server
Many failed attempts at starting the server later..
$ sudo port install postgresql_select
$ sudo port select postgresql
Available versions for postgresql:
none (active)
postgresql94
Well that can't be good!
$ sudo port select postgresql postgresql94
$ sudo port load postgresql94-server
You're kidding me. Now it's running?
Simply installing Postgres doesn't fully setup symlinks to make it easily runnable. Installing postrgresql_select gives MacPorts the information it needs to do that via port select. Once you've selected the active version of your choice, starting the Posgres server via luanchctl is as easy as port load postgresqlXX-server.
I know this is a very late answer and doesn't answer your full question, but launchctl will show different results depending on if you are superuser or not.
Try doing:
sudo launchctl list | grep postgres
I had exactly the same problem on my MacBook Pro. I could resolve the problem after I rode this blog post here and all the comments:
http://benscheirman.com/2010/06/installing-postgresql-for-rails-on-mac-os-x
The Problem is that postgres is not really running. I recognized this after I did a port scan to my own machine and realized that nothing is running on Port 5432.
I created a small script "start_pg_server.sh":
#!/bin/sh
sudo su postgres -c 'pg_ctl start -D /opt/local/var/db/postgresql83/defaultdb/'
after executing this script the server was running and I could connect me with pgAdmin. I was also able to run my ruby stuff with rake db:create and rake db:migrate.
After I restored using Timemachine I had the same problem.
The reason was that the permissions were mangled and postgres could not write the pid file.
Running this solved it for me:
sudo chown -R postgres:postgres /opt/local/var/db/postgresql91/
sudo port unload postgresql91-server
sudo port load postgresql91-server
Did you by any chance create your postgres user with a shell of /usr/bin/false? If so, the startup script won't work because it uses su which passes commands you send it through the shell.
If you did set it to /usr/bin/false, try changing it to /bin/bash and that might fix things.