How to insert data using docker compose - postgresql

Question

Overview: I would like to insert data in file.json in db within docker-compose using CURL.
My Try: I have tried to mount file.json inside the FROST container, then run the command CURL within docker-compose.
curl -X POST -H "Content-Type: application/json" -d file.json http://localhost:8080/FROST-Server/v1.1/Things
as the result the docker-compose yaml is so
version: '2'
services:
web:
image: fraunhoferiosb/frost-server:latest
environment:
- serviceRootUrl=http://localhost:8080/FROST-Server
- http_cors_enable=true
- http_cors_allowed.origins=*
- persistence_db_driver=org.postgresql.Driver
- persistence_db_url=jdbc:postgresql://database:5432/sensorthings
- persistence_db_username=sensorthings
- persistence_db_password=ChangeMe
- persistence_autoUpdateDatabase=true
ports:
- 8080:8080
- 1883:1883
volumes:
- ./file.json:file.json
depends_on:
- database
command:
- curl
- -X
- POST
- --data-binary
- ./file.json
- -H
- "Content-Type: application/json" # <-- YAML string quoting
- -H
- "Accept: application/json"
- "http://localhost:8080/FROST-Server/v1.1/Things"
database:
image: postgis/postgis:14-3.2-alpine
environment:
- POSTGRES_DB=sensorthings
- POSTGRES_USER=sensorthings
- POSTGRES_PASSWORD=ChangeMe
volumes:
- postgis_volume:/var/lib/postgresql/data
volumes:
postgis_volume:
After running docker-compuse up, i got this error
Creating network "frost_default" with the default driver
Creating frost_database_1 ... done
Creating frost_web_1 ... done
Attaching to frost_database_1, frost_web_1
database_1 |
database_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1 |
database_1 | 2022-06-13 11:24:06.420 UTC [1] LOG: starting PostgreSQL 14.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
database_1 | 2022-06-13 11:24:06.421 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
database_1 | 2022-06-13 11:24:06.421 UTC [1] LOG: listening on IPv6 address "::", port 5432
database_1 | 2022-06-13 11:24:06.458 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1 | 2022-06-13 11:24:06.486 UTC [22] LOG: database system was shut down at 2022-06-13 09:58:59 UTC
database_1 | 2022-06-13 11:24:06.498 UTC [1] LOG: database system is ready to accept connections
web_1 | % Total % Received % Xferd Average Speed Time Time Time Current
web_1 | Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
web_1 | curl: (7) Failed to connect to localhost port 8080: Connection refused
frost_web_1 exited with code 7
note: docker-compose.yml and file.json are in same path.
file.json
{
"name": "Car",
"description": "Smart connected car equipped with sensors.",
"properties": {
"vin": "5YJ3EDEB5KF223462"
},
"Locations": [
{
"name": "Parking lot",
"description": "The parking lot of the fictive company.",
"encodingType": "application/vnd.geo+json",
"location": {
"type": "Point",
"coordinates": [8.419432640075684, 49.01395040719586]
}
}
],
"Datastreams": [
{
"name": "Oil pump",
"description": "Measuring the motor oil pressure.",
"observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
"unitOfMeasurement": {
"name": "Bar",
"symbol": "bar",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#Bar"
},
"Sensor": {
"name": "OIL_PRES_SENSOR2",
"description": "Oil pressure sensor",
"encodingType": "application/pdf",
"metadata": "..."
},
"ObservedProperty": {
"name": "Pressure",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#Pressure",
"description": "The oil pressure."
},
"Observations": [
{
"phenomenonTime": "2022-01-10T10:00:00Z",
"result": 2.1
},
{
"phenomenonTime": "2022-01-10T10:01:10Z",
"result": 2.3
},
{
"phenomenonTime": "2022-01-10T10:02:20Z",
"result": 2.7
},
{
"phenomenonTime": "2022-01-10T10:03:30Z",
"result": 2.9
},
{
"phenomenonTime": "2022-01-10T10:04:40Z",
"result": 4.1
},
{
"phenomenonTime": "2022-01-10T10:05:50Z",
"result": 3.7
}
]
},
{
"name": "Motor Temperature",
"description": "The temperature of the motor.",
"observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
"unitOfMeasurement": {
"name": "Centigrade",
"symbol": "C",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#DegreeCentigrade"
},
"Sensor": {
"name": "DHT22/Temperature",
"description": "Temperature sensor of a DHT22",
"encodingType": "application/pdf",
"metadata": "https://www.sparkfun.com/datasheets/Sensors/Temperature/DHT22.pdf"
},
"ObservedProperty": {
"name": "Temperature",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#ThermodynamicTemperature",
"description": "The temperature."
},
"Observations": [
{
"phenomenonTime": "2019-03-14T10:00:00Z",
"result": 21.0
},
{
"phenomenonTime": "2019-03-14T10:01:00Z",
"result": 23.1
},
{
"phenomenonTime": "2019-03-14T10:02:00Z",
"result": 40.5
},
{
"phenomenonTime": "2019-03-14T10:03:00Z",
"result": 47.1
},
{
"phenomenonTime": "2019-03-14T10:04:00Z",
"result": 32.2
},
{
"phenomenonTime": "2019-03-14T10:05:00Z",
"result": 30.3
}
]
},
{
"name": "Access",
"description": "The access state (e.g., open, closed) of the vehicle.",
"observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
"unitOfMeasurement": {
"name": "Centimeter",
"symbol": "cm",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#Centimeter"
},
"Sensor": {
"name": "Distance Sensor",
"description": "Measure the distance.",
"encodingType": "application/pdf",
"metadata": "..."
},
"ObservedProperty": {
"name": "Length",
"definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#Length",
"description": "The length of the measured distance."
},
"Observations": [
{
"phenomenonTime": "2019-03-14T10:00:00Z",
"result": 0.0
},
{
"phenomenonTime": "2019-03-14T10:01:00Z",
"result": 0.0
},
{
"phenomenonTime": "2019-03-14T10:02:00Z",
"result": 21.4
},
{
"phenomenonTime": "2019-03-14T10:03:00Z",
"result": 21.7
},
{
"phenomenonTime": "2019-03-14T10:04:00Z",
"result": 20.9
},
{
"phenomenonTime": "2019-03-14T10:05:00Z",
"result": 0.0
}
]
}
]
}
any other solution is welcome too:)

Answer 1

I think you're executing your "curl" command before your "web" container starts. If I were you, I will run the "web" on "0.0.0.0" instead of "localhost", and use separate container to sleep for some time, then execute the curl command (don't forget about the container entrypopoint and override it if required).
I did run your docker-compose and execute the curl manually, but I got 400:
curl -vvv -X POST --data-binary ./file.json -H "Content-Type: application/json" -H "Accept: application/json" "http://localhost:8080/FROST-Server/v1.1/Things"
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /FROST-Server/v1.1/Things HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Content-Type: application/json
> Accept: application/json
> Content-Length: 11
>
* upload completely sent off: 11 out of 11 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 400
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Location
< Content-Length: 224
< Date: Tue, 14 Jun 2022 12:43:06 GMT
< Connection: close
<
* Closing connection 0
{"code":400,"type":"error","message":"Unexpected character ('.' (code 46)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: (BufferedReader); line: 1, column: 2]"}
Since I'm not familiar with this service, I don't know how to fix this, but you can test bellow docker-compose if you could fix this:
version: '2'
services:
curl:
image: alpine/curl:latest
entrypoint: /bin/sh
command:
- -c
- sleep 10
- curl
- -vvv
- -X
- POST
- --data-binary
- /file.json
- -H
- "Content-Type: application/json" # <-- YAML string quoting
- -H
- "Accept: application/json"
- "http://web:8080/FROST-Server/v1.1/Things"
volumes:
- ./file.json:/file.json
depends_on:
- web
web:
image: fraunhoferiosb/frost-server:latest
environment:
- serviceRootUrl=http://0.0.0.0:8080/FROST-Server
- http_cors_enable=true
- http_cors_allowed.origins=*
- persistence_db_driver=org.postgresql.Driver
- persistence_db_url=jdbc:postgresql://database:5432/sensorthings
- persistence_db_username=sensorthings
- persistence_db_password=ChangeMe
- persistence_autoUpdateDatabase=true
ports:
- 8080:8080
- 1883:1883
# volumes:
# - ./file.json:/usr/local/tomcat/file.json
depends_on:
- database
# command:
# - curl
# - -X
# - POST
# - --data-binary
# - ./file.json
# - -H
# - "Content-Type: application/json" # <-- YAML string quoting
# - -H
# - "Accept: application/json"
# - "http://localhost:8080/FROST-Server/v1.1/Things"
database:
image: postgis/postgis:14-3.2-alpine
environment:
- POSTGRES_DB=sensorthings
- POSTGRES_USER=sensorthings
- POSTGRES_PASSWORD=ChangeMe
volumes:
- postgis_volume:/var/lib/postgresql/data
volumes:
postgis_volume:
Edit 1: volume was missing from "curl" container

Answer 2

The End Solution of the problem. I would like to thank #RoohAllahGodazgar for helping me out.
Explanation: we need to have another image as curl image which we can run curl request on it after the FROST server is run. on curl service, we should mount first the dummy data inside the container and after that run sleep 20s so that we make sure that FROST server is run, then run crul method. after that the container will be exited.
version: '2'
services:
curl:
image: alpine/curl:latest
command: sh -c "sleep 20 && curl -X POST -d #/file.json http://web:8080/FROST-Server/v1.1/Things"
volumes:
- ./file.json:/file.json
depends_on:
- web
web:
image: fraunhoferiosb/frost-server:latest
environment:
- serviceRootUrl=http://0.0.0.0:8080/FROST-Server
- http_cors_enable=true
- http_cors_allowed.origins=*
- persistence_db_driver=org.postgresql.Driver
- persistence_db_url=jdbc:postgresql://database:5432/sensorthings
- persistence_db_username=sensorthings
- persistence_db_password=ChangeMe
- persistence_autoUpdateDatabase=true
ports:
- 8080:8080
- 1883:1883
depends_on:
- database
database:
image: postgis/postgis:14-3.2-alpine
environment:
- POSTGRES_DB=sensorthings
- POSTGRES_USER=sensorthings
- POSTGRES_PASSWORD=ChangeMe
volumes:
- postgis_volume:/var/lib/postgresql/data
volumes:
postgis_volume: