I just deployed a docker with Postgres on it on AWS EKS.
Below is the description details.
How do i access or test if postgres is working. I tried accessing both IP with post within VPC from worker node.
psql -h #IP -U #defaultuser -p 55432
Below is the deployment.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:10.4
imagePullPolicy: "IfNotPresent"
ports:
- containerPort: 55432
# envFrom:
# - configMapRef:
# name: postgres-config
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: postgredb
volumes:
- name: postgredb
persistentVolumeClaim:
claimName: efs
Surprisingly I am able to connect to psql but on 5432. :( Not sure what I am doing wrong. I passed containerPort as 55432
In short, you need to run the following command to expose your database on 55432 port.
kubectl expose deployment postgres --port=55432 --target-port=5432 --name internal-postgresql-svc
From now on, you can connect to it via port 55432 from inside your cluster by using the service name as a hostname, or via its ClusterIP address:
kubectl get internal-postgresql-svc
What you did in your deployment manifest file, you just attached additional information about the network connections a container uses, between misleadingly, because your container exposes 5432 port only (you can verify it by your self here). You should use a Kubernetes Service - abstraction which enables access to your PODs, and does the necessary port mapping behind the scene.
Please check also different port Types, if you want to expose your postgresql database outside of the Kubernetes cluster.
To test if progress is running fine inside POD`s container:
kubectl run postgresql-postgresql-client --rm --tty -i --restart='Never' --namespace default --image bitnami/postgresql --env="PGPASSWORD=<HERE_YOUR_PASSWORD>" --command -- psql --host <HERE_HOSTNAME=SVC_OR_IP> -U <HERE_USERNAME>
Related
I'm just adding the containers part of the spec. Everything is otherwise set up and working fine and values are hardcoded here. This is a simple Postgres pod that is part of a single replica deployment with its own PVC to persist state. But the problem is having nothing to do with my pod/deployment setup.
containers:
- name: postgres-container
image: postgres
imagePullPolicy: Always
volumeMounts:
- name: postgres-internal-volume
mountPath: /var/lib/postgresql/data
subPath: postgres
envFrom:
- configMapRef:
name: postgres-internal-cnf
ports:
- containerPort: 5432
command: ['psql']
args: [-U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'dominion'" | grep -q 1 || psql -h localhost -p 5432 -U postgres -c "CREATE DATABASE dominion"]
This command will create a database if it does not already exist. If I create the deployment and exec into the pod and run this command everything works fine. If I however run it here the pod fails to spin up and I get this error:
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I was under the impression that this error comes from the default connection values being incorrect, but here I am hardcoding the localhost and the port number.
With your pod spec, you've replaced the default command -- which starts up the postgres server -- with your own command, so the server never starts. The proper way to perform initialization tasks with the official Postgres image is in the documentation.
You want to move your initialization commands into a ConfigMap, and then mount the scripts into /docker-entrypoint-initdb.d as described in those docs.
The docs have more details, but here's a short example. We want to run
CREATE DATABASE dominion when the postgres server starts (and only
if it is starting with an empty data directory). We can define a
simple SQL script in a ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-init-scripts
data:
create-dominion-db.sql: |
CREATE DATABASE dominion
And then mount that script into the appropriate location in the pod
spec:
volumes:
- name: postgres-init-scripts
configMap:
name: postgres-init-scripts
containers:
- name: postgres-container
image: postgres
imagePullPolicy: Always
volumeMounts:
- name: postgres-internal-volume
mountPath: /var/lib/postgresql/data
subPath: postgres
- name: postgres-init-scripts
mountPath:
/docker-entrypoint-initdb.d/create-dominion-db.sql
subPath: create-dominion-db.sql
envFrom:
- configMapRef:
name: postgres-internal-cnf
ports:
- containerPort: 5432
I have setup a Postgres pod on my Kubernetes cluster, and I am trying to troubleshoot it a bit.
I would like to use the official Postgres image and deploy it to my Kubernetes cluster using kubectl. Given that my Postgres server connection details are:
host: mypostgres
port: 5432
username: postgres
password: 12345
And given that I think the command will be something like:
kubectl run -i --tty --rm debug --image=postgres --restart=Never -- sh
What do I need to do so that I can deploy this image to my cluster, connect to my Postgres server and start running SQL command against it (for troubleshooting purposes)?
If your primarily interested in troubleshooting, then you're probably looking for the kubectl port-forward command, which will expose a container port on your local host. First, you'll need to deploy the Postgres pod; you haven't shown what your pod manifest looks like, so I'm going to assume a Deployment like this:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: postgres
name: postgres
namespace: sandbox
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- env:
- name: POSTGRES_PASSWORD
value: secret
- name: POSTGRES_USER
value: example
- name: POSTGRES_DB
value: example
image: docker.io/postgres:13
name: postgres
ports:
- containerPort: 5432
name: postgres
protocol: TCP
volumeMounts:
- mountPath: /var/lib/postgresql
name: postgres-data
strategy: Recreate
volumes:
- emptyDir: {}
name: postgres-data
Once this is running, you can access postgres with the port-forward
command like this:
kubectl -n sandbox port-forward deploy/postgres 5432:5432
This should result in:
Forwarding from 127.0.0.1:5432 -> 5432
Forwarding from [::1]:5432 -> 5432
And now we can connect to Postgres using psql and run queries
against it:
$ psql -h localhost -U example example
psql (13.4)
Type "help" for help.
example=#
kubectl port-forward is only useful as a troubleshooting mechanism. If
you were trying to access your postgres pod from another pod, you
would create a Service and then use the service name as the hostname
for your client connections.
Update
If your goal is to deploy a client container so that you can log
into it and run psql, the easiest solution is just to kubectl rsh
into the postgres container itself. Assuming you were using the
Deployment shown earlier in this question, you could run:
kubectl rsh deploy/postgres
This would get you a shell prompt inside the postgres container. You
can run psql and not have to worry about authentication:
$ kubectl rsh deploy/postgres
$ psql -U example example
psql (13.4 (Debian 13.4-1.pgdg100+1))
Type "help" for help.
example=#
If you want to start up a separate container, you can use the kubectl debug command:
kubectl debug deploy/postgres
This gets you a root prompt in a debug pod. If you know the ip address
of the postgres pod, you can connect to it using psql. To get
the address of the pod, run this on your local host:
$ kubectl get pod/postgres-6df4c549f-p2892 -o jsonpath='{.status.podIP}'
10.130.0.11
And then inside the debug container:
root#postgres-debug:/# psql -h 10.130.0.11 -U example example
In this case you would have to provide an appropriate password,
because you are accessing postgres from "another machine", rather than
running directly inside the postgres pod.
Note that in the above answer I've used the shortcut
deploy/<deployment_name, which avoids having to know the name of the
pod created by the Deployment. You can replace that with
pod/<podname> in all cases.
I have created a mongodb pod in my Minikube local cluster with the following configuration, now I would like to migrate the data of my existing mongodb database(run in AWS EC2 instance) to this database, how can I accomplish this?
apiVersion: v1
kind: Pod
metadata:
name: mongodb
labels:
app: mongodb
spec:
volumes:
- name: mongo-vol
persistentVolumeClaim:
claimName: mongo-pvc
containers:
- image: mongo
name: container1
command:
- "mongod"
- "--bind_ip"
- "0.0.0.0"
ports:
- containerPort: 27017
volumeMounts:
- name: mongo-vol
mountPath: /data/db
Can't added a comment because I do not have more than 50 reputation but here is the thing:
You created a Pod using MongoDB and I see the volumeMount set to /data/db. What is this path? Is this volume a hostPath, NFS, external storage?
The migration itself could be accomplished in two ways:
by running a mongodump on your EC2 instance:
mongodump -host hostname --port 27017 --out /tmp/mongodb-dump
Then a mongorestore on your /data/db path.
You can manually log in on the pod
#copy your dump from host to pod using
kubectl cp /tmp/foo_dir <some-pod>:/tmp/bar_dir
#log on the pod
kubectl exec mongodb -it /bin/bash
#run restore
mongorestore /tmp/mongodb-dump
or
You can copy all files/the entire /data from your EC2 instance and dropped on the volumeMount your pod is mounting /data/db.
I am trying to set a postgres parameter (shared_buffers) into my postgres database pod. I am trying to set an init container to set the db variable, but it is not working because the init container runs as the root user.
What is the best way to edit the db variable on the pods? I do not have the ability to make the change within the image, because the variable needs to be different for different instances. If it helps, the command I need to run is a "postgres -c" command.
"root" execution of the PostgreSQL server is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromise. See the documentation for
more information on how to properly start the server.
You didn't share your Pod/Deployment definition, but I believe you want to set shared_buffers from the command line of the actual container (not the init container) in your Pod definition. Something like this if you are using a deployment:
apiVersion: v1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:12.2
imagePullPolicy: "IfNotPresent"
command: ["postgres"] # <-- add this
args: ["-D", "-c", "shared_buffers=128MB"] # <-- add this
ports:
- containerPort: 5432
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: postgredb
- name: postgresql-config-volume # <-- use if you are using a ConfigMap (see below)
mountPath: /var/lib/postgres/data/postgresql.conf
volumes:
- name: postgredb
persistentVolumeClaim:
claimName: postgres-pv-claim # <-- note: you need to have this already predefined
- name: postgresql-config-volume # <-- use if you are using a ConfigMap (see below)
configMap:
name: postgresql-config
Notice that if you are using a ConfigMap you can also do this (note that you may want to add more configuration options besides shared_buffers):
apiVersion: v1
kind: ConfigMap
metadata:
name: postgresql-config
data:
postgresql.conf: |
shared_buffers=256MB
In my case, the #Rico answer didn't help me out of the box because I don't use postgres with a persistent storage mount, which means there is no /var/lib/postgresql/data folder and pre-existed database (so both proposed options have failed in my case).
To successfully apply postgres settings, I used only args (without command section).
In that case, k8s will pass these args to the default entrypoint defined in the docker image (docs), and as for postgres entrypoint, it is made so that any options passed to the docker command will be passed along to the postgres server daemon (look section Database Configuration at: https://hub.docker.com/_/postgres)
apiVersion: v1
kind: Pod
metadata:
name: postgres
spec:
containers:
- image: postgres:9.6.8
name: postgres
args: ["-c", "shared_buffers=256MB", "-c", "max_connections=207"]
To check that the settings applied:
$ kubectl exec -it postgres -- bash
root#postgres:/# su postgres
$ psql -c 'show max_connections;'
max_connections
-----------------
207
(1 row)
What is the best practice for backing up a Postgres database running on Google Cloud Container Engine?
My thought is working towards storing the backups in Google Cloud Storage, but I am unsure of how to connect the Disk/Pod to a Storage Bucket.
I am running Postgres in a Kubernetes cluster using the following configuration:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: postgres-deployment
spec:
replicas: 1
template:
metadata:
labels:
app: postgres
spec:
containers:
- image: postgres:9.6.2-alpine
imagePullPolicy: IfNotPresent
env:
- name: PGDATA
value: /var/lib/postgresql/data
- name: POSTGRES_DB
value: my-database-name
- name: POSTGRES_PASSWORD
value: my-password
- name: POSTGRES_USER
value: my-database-user
name: postgres-container
ports:
- containerPort: 5432
volumeMounts:
- mountPath: /var/lib/postgresql
name: my-postgres-volume
volumes:
- gcePersistentDisk:
fsType: ext4
pdName: my-postgres-disk
name: my-postgres-volume
I have attempted to create a Job to run a backup:
apiVersion: batch/v1
kind: Job
metadata:
name: postgres-dump-job
spec:
template:
metadata:
labels:
app: postgres-dump
spec:
containers:
- command:
- pg_dump
- my-database-name
# `env` value matches `env` from previous configuration.
image: postgres:9.6.2-alpine
imagePullPolicy: IfNotPresent
name: my-postgres-dump-container
volumeMounts:
- mountPath: /var/lib/postgresql
name: my-postgres-volume
readOnly: true
restartPolicy: Never
volumes:
- gcePersistentDisk:
fsType: ext4
pdName: my-postgres-disk
name: my-postgres-volume
(As far as I understand) this should run the pg_dump command and output the backup data to stdout (which should appear in the kubectl logs).
As an aside, when I inspect the Pods (with kubectl get pods), it shows the Pod never gets out of the "Pending" state, which I gather is due to there not being enough resources to start the Job.
Is it correct to run this process as a Job?
How do I connect the Job to Google Cloud Storage?
Or should I be doing something completely different?
I'm guessing it would be unwise to run pg_dump in the database Container (with kubectl exec) due to a performance hit, but maybe this is ok in a dev/staging server?
As #Marco Lamina said you can run pg_dump on postgres pod like
DUMP
// pod-name name of the postgres pod
// postgres-user database user that is able to access the database
// database-name name of the database
kubectl exec [pod-name] -- bash -c "pg_dump -U [postgres-user] [database-name]" > database.sql
RESTORE
// pod-name name of the postgres pod
// postgres-user database user that is able to access the database
// database-name name of the database
cat database.sql | kubectl exec -i [pod-name] -- psql -U [postgres-user] -d [database-name]
You can have a job pod that does run this command and exports this to a file storage system such as AWS s3.
I think running pg_dump as a job is a good idea, but connecting directly to your DB's persistent disk is not. Try having pg_dump connect to your DB over the network! You could then have a second disk onto which your pg_dump command dumps the backups. To be on the safe side, you can create regular snapshots of this second disk.
The reason for the Jobs POD to stay in Pending state is that it forever tries to attach/mount the GCE persistent disk and fails to do so because it is already attached/mounted to another POD.
Attaching a persistent disk to multiple PODs is only supported if all of them attach/mount the volume in ReadOnly mode. This is of course no viable solution for you.
I never worked with GCE, but it should be possible to easily create a snapshot from the PD from within GCE. This would not give a very clean backup, more like something in the state of "crashed in the middle, but recoverable", but this is probably acceptable for you.
Running pg_dump inside the database POD is a viable solution, with a few drawbacks as you already noticed, especially performance. You'd also have to move out the resulting backup from the POD afterwards, e.g. by using kubectl cp and another exec to cleanup the backup in the POD.
You can use Minio Client
First of all use simple dockerfile to make docker image contains postgres along with minio client (let name this image postgres_backup):
FROM postgres
RUN apt-get update && apt-get install -y wget
RUN wget https://dl.min.io/client/mc/release/linux-amd64/mc
RUN chmod +x mc
RUN ./mc alias set gcs https://storage.googleapis.com BKIKJAA5BMMU2RHO6IBB V8f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12
Now you can use postgres_backup image in your CronJob (I assumed you made backups bucket in your Google storage):
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: backup-job
spec:
# Backup the database every day at 2AM
schedule: "0 2 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: postgres-backup
image: postgres_backup
env:
- name: POSTGRES_HOST_AUTH_METHOD
value: trust
command: ["/bin/sh"]
args: ["-c", 'pg_dump -Fc -U [Your Postgres Username] -W [Your Postgres Password] -h [Your Postgres Host] [Your Postgres Database] | ./mc pipe gcs/backups/$(date -Iseconds).dump']
restartPolicy: Never
A lot of tutorials use kubectl cp or transfer the file inside the pod, but you can also pipe the pg_dump container output directly to another process.
kubectl run --env=PGPASSWORD=$PASSWORD --image=bitnami/postgresql postgresql -it --rm -- \
bash -c "pg_dump -U $USER -h $HOST -d $DATABASE" |\
gzip > backup.sql.gz
The easiest way to dump without storing any additional copies on your pod:
kubectl -n [namespace] exec -it [pod name] -- bash -c "export PGPASSWORD='[db password]'; pg_dump -U [db user] [db name]" > [database].sql