I'm trying to query a CSV into a DB, but always returns the error (No such file or directory).
The file should be hosted on the /tmp folder on the container?
Database: postgres:13.2-alpine (Docker container)
Backend: node + slonik (on top of pg)
docker-compose.yml
version: '3.8'
services:
db:
image: postgres:13.2-alpine
ports:
- 5432:5432
environment:
POSTGRES_USER: <>
POSTGRES_PASSWORD: <>
POSTGRES_DB: <>
db_admin:
image: adminer
ports:
- 8080:8080
depends_on:
- db
Desired query:
\COPY rams(ram) FROM '/Users/…/rams.csv' WITH (FORMAT CSV) HEADER;
Error:
node:59682) UnhandledPromiseRejectionWarning: error: could not open file "/Users/…/rams.csv" for reading: No such file or directory
Best,
There are several options to make connection between your local machine and a docker container, running the database. As already mentioned in my comment, looking up the section on volumes in docker documentation provides an overview. But this does not really fit, when you have an already running database container. This is what I do in such a case:
docker cp my_products.csv <database docker image>:/tmp
How do we get the "database docker image"? Simply by doing a docker ps (name is in the last column).
REFERENCE/CREDITS:
https://dev.to/adron/getting-copy-for-bulk-csv-working-on-a-container-running-postgresql-3kk9
Related
I have a Postgres database running on my host. The datafiles for the database is stored at /usr/local/var/postgresql#13.
To get the full system running easily I'd like to have a Docker with a Postgres service running for other Docker apps to connect to. I would however like to have the Docker Postgres service to use the existing datafiles on the host ...
How do I set up the volume correctly to point to the hosts database files?
Do have to have a user/password when running the Docker against existing datafiles?
I have the following but can get the volume to work ...
version: "3.9"
services:
web:
build: .
ports:
- 8081:3011
depends_on:
- db
environment:
- PGHOST=db
- PGDATABASE=loggingtestdb
- PGUSER=postgres
- PGPASSWORD=postgres
db:
image: postgres
ports:
- 5432:5432
volumes:
- /usr/local/var/postgresql#13 <--- Need help here.
How do I map the container pg datafile location to the hosts pg datafile location? 🙏
Update 1
This is the datafile folder for the db on the host
After comments I updated the volumes to below
volumes:
- /usr/local/var/postgresql#13:/var/lib/postgresql/data
But when running docker compose I only get
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /usr/local/var/postgresql#13
Update 2
/use/local works fine. But as soon as I add the /var folder to the path Docker for some reason can’t find it … What am I missing here?
I am running postgres in a docker container using docker-compose and it spins up with no issue and I am able to connect to the database. But now I want to go into the container and execute the postgis shp2pgsql to load a shape file but the command seems to be nonexistent. Below is my code:
docker-compose.yaml
version: '3'
services:
db:
container_name: pg_container
image: postgis/postgis
restart: always
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: root
POSTGRES_DB: test_db
volumes:
- ./data:/var/lib/postgresql/
- ./postgres_init:/postgres_init
ports:
- 5433:5433
networks:
- ch_ntw
networks:
ch_ntw:
driver: bridge
ipam:
config:
- subnet: 10.222.1.0/24
Getting into the container:
docker exec -it pg_container bash
Connecting to the db without issue using psql:
psql --host=pg_container --dbname=test_db --username=root
But then if I try to invoke shp2pgsql from bash I get the following:
shp2pgsql -s 2263:4326 postgres_init/nyct2010_15b/nyct2010.shp | psql -d test_db
bash: shp2pgsql: command not found
I would think since this is a postgis container that the function should be accessible no?
shp2pgsql is a client package. The postgis/postgis image is the PostGIS server components only. If you want to use shp2pgsql or other client tools, install them locally on your host, or in another container.
I would like to access my Postgres database (docker container) from PhpStorm.
docker-compose.yml
# Run docker-compose build
# Run docker-compose up
# Live long and prosper
version: '3.1'
services:
apache:
build: .docker/apache
container_name: sf-apache
ports:
- 82:80
volumes:
- .docker/config/vhosts:/etc/apache2/sites-enabled
- ${SYMFONY_APP}:/home/wwwroot/sf3
depends_on:
- php
postgres:
container_name: postgres
restart: always
image: 'postgres:12.6'
ports:
- "5432:5432"
environment:
- "POSTGRES_USER=${PGSQL_ADMIN_USER}"
- "POSTGRES_PASSWORD=${PGSQL_ADMIN_PASSWORD}"
volumes:
- ./API/var/postgres:/var/lib/postgresql/data
- .docker/postgresql/init-database.sh:/docker-entrypoint-initdb.d/init-database.sh
My PhpStorm config :
I can access to my database via docker exec -it postgres bash
If php storm is on the same host then you need to use localhost. If both phpstorm and pg is part of the same compose file , then you would use the service name since both would be in the same virtual network
I found the solution.
I have a Postgres local and a Postgres with docker. My Postgres local get the upper hand on my docker Postgres. I have killed the service and put the container service name on it.
Works perfectly.
Thanks for you help,
I have docker-compose.yml as:
version: '2.1'
services:
postgres:
image: postgres:latest
container_name: postgres
environment:
- "POSTGRES_USER=docker"
- "POSTGRES_PASSWORD=docker"
ports:
- "5432:5432"
volumes:
- "./sql:/docker-entrypoint-initdb.d"
In folder sql, I have a .sql file containing:
DROP DATABASE IF EXISTS mytestdb;
CREATE DATABASE mytestdb;
\c mytestdb;
docker-compose gets postgres container up and running. But when I connect to the server via pgadmin, I see no database with name 'mytestdb' created. Instead I see one with name 'docker'.
What am I missing here?
I am using docker compose to combine 2 images (tomcat with my app and database - postgres).
My compose file looks like this :
version: '3'
services:
tomcat:
build: ./tomcat-img
ports:
- "8080:8080"
depends_on:
- "db"
db:
build: ./db-img
volumes:
- db-data:/var/lib/postgres/data
ports:
- "5433:5432"
volumes:
db-data:
and here is dockerfile for database image:
FROM postgres:9.5-alpine
ENV POSTGRES_DB mydb
ENV POSTGRES_USER xxxx
ENV POSTGRES_PASSWORD xxxx
COPY init-db.sql /docker-entrypoint-initdb.d/
EXPOSE 5432
CMD ["postgres"]
Next I started my containers with docker-compose cli docker-compose -f docker-compose.yml up
and run psql tool with:
docker exec -it container_id psql -d xxxx -U xxxx
and insert new record. After that I check if there really is:
select * from my_table;
After that I tried stopped docker compose and remove containers with:
docker-compose -f docker-compose.yml down
and start it again
docker-compose -f docker-compose.yml up
when I run again psql tool of db container and select data in my_table, there is no previous inserted record ... Can you help me to fix it please? I need init my db with init-db.sql just once and next using that persist storage. Thanks for answers.
In my dockerized Postgresql with a data volume I am binding to /var/lib/postgresql and not to /var/lib/postgres/data. Try changing your compose file to
volumes:
- db-data:/var/lib/postgresql