Invalid response error with socket time out while building Dockerfile with docker-compose.yml - docker-compose

I'm now trying to make a monorepo project.
My directory structure looks like this:
word_mono
|- apps
|- shadowing_list
|- backend
|- (Nest project here, made up with nest command)
|- Dockerfile
|- frontend
|- shadowing_trend
|- shared
|- database
|- Dockerfile
|- schema.prisma
|- .env
|- docker-compose.yml
On current stage, I'm now implementing shadowing_list's backend with postgresql, prisma, and nest js with docker.
So my docker-compose.yml file is like this
version: "3.8"
services:
shadowing-list-backend:
container_name: shadowing-list-backend
restart: on-failure:10
ports:
- "8000:8000"
build:
context: ./
dockerfile: ./shadowing_list/backend/Dockerfile
volumes:
- /usr/src/app/node_modules
- ./shadowing_list/backend/:/usr/src/app
env_file:
- .env
depends_on:
- prisma
- db
prisma:
image: prismagraphql/prisma:1.34
container_name: prisma
restart: always
ports:
- "4466:4466"
environment:
PRISMA_CONFIG: |
port: 4466
databases:
default:
connector: postgres
host: db
port: 5432
user: ${POSTGRES_USER}
password: ${POSTGRES_PASSWORD}
database: ${POSTGRES_DB}
migrations: true
env_file:
- .env
links:
- db
db:
build:
context: ./shared/database
container_name: postgres
env_file:
- .env
ports:
- "5432:5432"
volumes:
- ./shared/database/db-data:/var/lib/postgresql/data
volumes:
db-data:
the Dockerfile in shadowing_list/backend:
# Build context /apps/
FROM node:16
WORKDIR /usr/src/app
# Install prisma
RUN npm install -g prisma
# Copy package.json
COPY shadowing_list/backend/package.json ./
RUN npm install
# Cache all after package installed
COPY shadowing_list/backend ./
# Use schema.prisma file in shared folder
COPY shared/database/schema.prisma .
RUN prisma generate
EXPOSE 8000
CMD ["npm", "start"]
the package.json file of shadowing_list/backend:
{
"name": "backend",
"version": "0.0.1",
"description": "",
"author": "",
"private": true,
"license": "UNLICENSED",
"scripts": {
"prebuild": "rimraf dist",
"build": "nest build",
"format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
"start": "nest start",
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"test": "jest",
"test:watch": "jest --watch",
"test:cov": "jest --coverage",
"test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
"test:e2e": "jest --config ./test/jest-e2e.json"
},
"dependencies": {
"#nestjs/common": "^9.0.0",
"#nestjs/core": "^9.0.0",
"#nestjs/graphql": "^10.1.7",
"#nestjs/platform-express": "^9.0.0",
"#prisma/client": "^4.9.0",
"dotenv": "^16.0.3",
"reflect-metadata": "^0.1.13",
"rimraf": "^3.0.2",
"rxjs": "^7.2.0"
},
"devDependencies": {
"#nestjs/cli": "^9.0.0",
"#nestjs/schematics": "^9.0.0",
"#nestjs/testing": "^9.0.0",
"#types/express": "^4.17.13",
"#types/jest": "28.1.8",
"#types/node": "^16.0.0",
"#types/supertest": "^2.0.11",
"#typescript-eslint/eslint-plugin": "^5.0.0",
"#typescript-eslint/parser": "^5.0.0",
"eslint": "^8.0.1",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-prettier": "^4.0.0",
"jest": "28.1.3",
"prettier": "^2.3.2",
"source-map-support": "^0.5.20",
"supertest": "^6.1.3",
"ts-jest": "28.0.8",
"ts-loader": "^9.2.3",
"ts-node": "^10.0.0",
"tsconfig-paths": "4.1.0",
"typescript": "^4.7.4"
},
"jest": {
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".*\\.spec\\.ts$",
"transform": {
"^.+\\.(t|j)s$": "ts-jest"
},
"collectCoverageFrom": [
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
}
}
The problem is there is an error while I'm trying to build my backend application with
docker-compose up
command.
The error is like this:
> [apps-shadowing-list-backend 5/8] RUN npm install:
#0 836.2 npm ERR! code FETCH_ERROR
#0 836.2 npm ERR! errno FETCH_ERROR
#0 836.2 npm ERR! invalid json response body at https://registry.npmjs.org/ts-jest reason: Invalid response body while trying to fetch https://registry.npmjs.org/ts-jest: Socket timeout
#0 836.2
#0 836.2 npm ERR! A complete log of this run can be found in:
#0 836.2 npm ERR! /root/.npm/_logs/2023-01-23T04_19_52_861Z-debug-0.log
------
failed to solve: executor failed running [/bin/sh -c npm install]: exit code: 1
And also there were some trouble in volumes of backend application. The mounting is not working properly.
I expected this backend application can communicate with prisma service by using graphql in nest js. And also changes have to be applied while it's running on docker container.

Related

Container on same network not communicating with each other

I have a mongodb container which i name e-learning
and i have a docker image which should connect to the mongodb container to update my database but it's not working i get this error:
Unknown, Last error: connection() error occurred during connection handshake: dial tcp 127.0.0.1:27017: connect: connection refused }
here's my docker build file
# syntax=docker/dockerfile:1
FROM golang:1.18
WORKDIR /go/src/github.com/worker
COPY go.mod go.sum main.go ./
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM jrottenberg/ffmpeg:4-alpine
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
ENV LD_LIBRARY_PATH=/usr/local/lib
COPY --from=jrottenberg/ffmpeg / /
COPY app.env /root
COPY --from=0 /go/src/github.com/worker/app .
CMD ["./app"]
my docker compose file
version: "3.9"
services:
worker:
image: worker
environment:
- MONGO_URI="mongodb://localhost:27017/"
- MONGO_DATABASE=e-learning
- RABBITMQ_URI=amqp://user:password#rabbitmq:5672/
- RABBITMQ_QUEUE=upload
networks:
- app_network
external_links:
- e-learning
- rabbitmq
volumes:
- worker:/go/src/github.com/worker:rw
networks:
app_network:
external: true
volumes:
worker:
my docker inspect network
[
{
"Name": "app_network",
"Id": "f688edf02a194fd3b8a2a66076f834a23fa26cead20e163cde71ef32fc1ab598",
"Created": "2022-06-27T12:18:00.283531947+03:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.20.0.0/16",
"Gateway": "172.20.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"2907482267e1f6e42544e5e8d852c0aac109ec523c6461e003572963e299e9b0": {
"Name": "rabbitmq",
"EndpointID": "4b46e091e4d5a79782185dce12cb2b3d79131b92d2179ea294a639fe82a1e79a",
"MacAddress": "02:42:ac:14:00:03",
"IPv4Address": "172.20.0.3/16",
"IPv6Address": ""
},
"8afd004a981715b8088af53658812215357e156ede03905fe8fdbe4170e8b13f": {
"Name": "e-learning",
"EndpointID": "1c12d592a0ef6866d92e9989f2e5bc3d143602fc1e7ad3d980efffcb87b7e076",
"MacAddress": "02:42:ac:14:00:02",
"IPv4Address": "172.20.0.2/16",
"IPv6Address": ""
},
"ad026f7e10c9c1c41071929239363031ff72ad1b9c6765ef5c977da76f24ea31": {
"Name": "video-transformation-worker-1",
"EndpointID": "ce3547014a6856725b6e815181a2c3383d307ae7cf7132e125c58423f335b73f",
"MacAddress": "02:42:ac:14:00:04",
"IPv4Address": "172.20.0.4/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]
Change MONGO_URI="mongodb://localhost:27017/" to MONGO_URI="mongodb://e-learning:27017/" (working on the assumption that e-learning is the mongo container).
Within a container attached to a bridge network (the default) localhost (127.0.0.1) is the container itself. So your app container is trying to access the database at port 27017 on itself (not on the host or on the db container). The easiest solution is to use the automatic DNS resolution between containers that docker provides.
I added extra hosts and changed my mongo uri to host.docker.internal
and it solved my problems
version: "3.9"
services:
worker:
image: worker
environment:
- MONGO_URI="mongodb://host.docker.internal:27017/"
- MONGO_DATABASE=e-learning
- RABBITMQ_URI=amqp://user:password#rabbitmq:5672/
- RABBITMQ_QUEUE=upload
networks:
- app_network
external_links:
- e-learning
- rabbitmq
volumes:
- worker:/go/src/github.com/worker:rw
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
app_network:
external: true
volumes:
worker:

Unable to connect to Mongodb container from flask container

#Docker compose file
version: "3.4" # optional since v1.27.0
services:
flaskblog:
build:
context: flaskblog
dockerfile: Dockerfile
image: flaskblog
restart: unless-stopped
environment:
APP_ENV: "prod"
APP_DEBUG: "False"
APP_PORT: 5000
MONGODB_DATABASE: flaskdb
MONGODB_USERNAME: flaskuser
MONGODB_PASSWORD:
MONGODB_HOSTNAME: mongodbuser
volumes:
- appdata:/var/www
depends_on:
- mongodb
networks:
- frontend
- backend
mongodb:
image: mongo:4.2
#container_name: mongodb
restart: unless-stopped
command: mongod --auth
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD:
MONGO_INITDB_DATABASE: flaskdb
MONGODB_DATA_DIR: /data/db2
MONDODB_LOG_DIR: /dev/null
volumes:
- mongodbdata:/data/db2
#- ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro
networks:
- backend
networks:
backend:
driver: bridge
volumes:
mongodbdata:
driver: local
appdata:
driver: local
#Flask container file
FROM python:3.8-slim-buster
#python:3.6-stretch
LABEL MAINTAINER="Shekhar Banerjee"
ENV GROUP_ID=1000 \
USER_ID=1000 \
SECRET_KEY="4" \
EMAIL_USER="dankml.com" \
EMAIL_PASS="nzw"
RUN mkdir /home/logix3
WORKDIR /home/logix3
ADD . /home/logix3/
RUN pip install -r requirements.txt
RUN pip install gunicorn
RUN groupadd --gid $GROUP_ID www
RUN useradd --create-home -u $USER_ID --shell /bin/sh --gid www www
USER www
EXPOSE 5000/tcp
#CMD ["python","run.py"]
CMD [ "gunicorn", "-w", "4", "--bind", "127.0.0.1:5000", "run:app"]
These are my docker-compose file and Dockerfile for an application . The container for MongoDB and Flask are working fine individually, but when I try to run them together, I do not get any response on the localhost, There are no error messages in the logs or the containers
Curl in the system gives this :
$ curl -i http://127.0.0.1:5000
curl: (7) Failed to connect to 127.0.0.1 port 5000: Connection refused
Can anyone suggest how do I debug this ??
**Only backend network is functional
[
{
"Name": "mlspace_backend",
"Id": "e7e37cad0058330c99c55ffea2b98281e0c6526763e34550db24431b30030b77",
"Created": "2022-05-15T22:52:25.0281001+05:30",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.20.0.0/16",
"Gateway": "172.20.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"12c50f5f70d18b4c7ffc076177b59ff063a8ff81c4926c8cae0bf9e74dc2fc83": {
"Name": "mlspace_mongodb_1",
"EndpointID": "8278f672d9211aec9b539e14ae4eeea5e8f7aaef95448f44aab7ec1a8c61bb0b",
"MacAddress": "02:42:ac:14:00:02",
"IPv4Address": "172.20.0.2/16",
"IPv6Address": ""
},
"75cde7a34d6b3c4517c536a06349579c6c39090a93a017b2c280d987701ed0cf": {
"Name": "mlspace_flaskblog_1",
"EndpointID": "20489de8841d937f01768170a89f74c37ed049d241b096c8de8424c51e58704c",
"MacAddress": "02:42:ac:14:00:03",
"IPv4Address": "172.20.0.3/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "backend",
"com.docker.compose.project": "mlspace",
"com.docker.compose.version": "1.23.2"
}
}
]
I checked the logs of the containers. no errors found, flask engine seem to be running but the site wont run , even curl won't given any output

docker compose with network, connect ECONNREFUSED

Hello I have the following docker setup
I have a postgres image and my container node alpine
my docker file:
#building code FROM node:lts-alpine
RUN mkdir -p /home/node/api/node_modules && chown -R node:node /home/node/api
WORKDIR /home/node/api
COPY ormconfig.json .env package.json yarn.* ./
USER node
RUN yarn
COPY --chown=node:node . .
EXPOSE 4000
CMD ["yarn", "dev"]
and my docker compose file:
version: '3.7'
services:
ci-api:
build: .
container_name: ci-api
volumes:
- .:/home/node/api
- /home/node/api/node_modules
ports:
- '${SERVER_PORT}:${SERVER_PORT}'
depends_on:
- ci-postgres
networks:
- ci-network
ci-postgres:
image: postgres:12
container_name: ci-postgres
environment:
- ALLOW_EMPTY_PASSWORD=no
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASS}
- POSTGRES_DB=${DB_NAME}
volumes:
- ci-postgres-data:/data
networks:
- ci-network
volumes:
ci-postgres-data:
networks:
ci-network:
driver: bridge
my env file:
SERVER_PORT=4000
DB_HOST=ci-postgres
DB_PORT=5432
DB_USER=spirit
DB_PASS=api
DB_NAME=db
my ormconfig:
{
"type": "postgres",
"host": "${DB_HOST}",
"port": 5432,
"username": "spirit",
"password": "emasa",
"database": "emasa_base",
"synchronize": true,
"logging": false,
"entities": ["dist/src/entity/**/*.js"],
"migrations": ["dist/src/migration/**/*.js"],
"subscribers": ["dist/src/subscriber/**/*.js"],
"cli": {
"entitiesDir": "dist/src/entity",
"migrationsDir": "dist/src/migration",
"subscribersDir": "dist/src/subscriber"
}
}
my package.json:
{
"name": "training",
"version": "1.0.0",
"main": "index.js",
"license": "MIT",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "nodemon src/index.ts --exec ts-node",
"build": "tsc -b",
"lint": "eslint src/**/*.ts",
"format": "eslint src/**/*.ts --fix"
},
"devDependencies": {
"#types/express": "^4.17.6",
"#types/node": "^14.0.6",
"#typescript-eslint/eslint-plugin": "^3.0.2",
"#typescript-eslint/parser": "^3.0.2",
"eslint": "^7.1.0",
"eslint-config-node": "^4.1.0",
"eslint-config-prettier": "^6.11.0",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-prettier": "^3.1.3",
"prettier": "^2.0.5",
"ts-node": "^8.10.2",
"typescript": "^3.9.3"
},
"dependencies": {
"apollo-server": "^2.14.1",
"apollo-server-express": "^2.14.1",
"express": "^4.17.1",
"graphql": "^15.0.0",
"nodemon": "^2.0.4",
"reflect-metadata": "^0.1.13",
"type-graphql": "^1.0.0-rc.2",
"typeorm": "^0.2.25"
}
}
and i get this errors:
error An unexpected error occurred:
"https://registry.yarnpkg.com/express/-/express-4.17.1.tgz: connect
ECONNREFUSED 104.16.26.35:443".
ERROR: Service 'ci-api' failed to build: The command '/bin/sh -c yarn'
returned a non-zero code: 1

Proxy configuration for ui5-server

I created project with ui5-cli tool. I add external js libraries to my project. ui5 serve command start ui5-server and it is maping ui5 library folder as /resource. I need to map resource/libs folder to another location in my project.
How can I configure ui5-server proxy setting for mapping/routing?
My package.json file:
{
"name": "openui5-sample-app",
"version": "0.2.0",
"description": " ",
"private": true,
"scripts": {
"start": "ui5 serve",
"lint": "eslint webapp",
"karma": "karma start",
"karma-ci": "rimraf coverage && karma start karma-ci.conf.js",
"watch": "npm run karma",
"test": "npm run lint && npm run karma-ci",
"build": "ui5 build -a --clean-dest",
"build-self-contained": "ui5 build self-contained -a --clean-dest",
"serve-dist": "ws --compress -d dist"
},
"dependencies": {
"#openui5/sap.m": "^1.70.0",
"#openui5/sap.ui.core": "^1.70.0",
"#openui5/themelib_sap_fiori_3": "^1.70.0",
"js-cookie": "^2.2.1"
},
"devDependencies": {
"#ui5/cli": "^1.9.0",
"eslint": "^5.16.0",
"karma": "^4.3.0",
"karma-chrome-launcher": "^3.1.0",
"karma-coverage": "^2.0.1",
"karma-ui5": "^1.1.0",
"local-web-server": "^3.0.7",
"rimraf": "^3.0.0"
}
}
My component.js file:
jQuery.sap.registerModulePath('libs.js-cookie', '../resources/libs/js.cookie-2.2.1.min');
sap.ui.define([
...,
"libs/js-cookie"
], function(UIComponent, Device, models) {
...
});
My yaml file:
specVersion: '1.0'
metadata:
name: openui5-sample-app
type: application
resources:
configuration:
propertiesFileSourceEncoding: "UTF-8"
You must update your cli to the newest version at the time or writing it is "#ui5/cli": "^1.11.1". I personally can recomend you ncu to check regulare for updates.
This config needs a CDN bootraping in your index.html.
ui5-tooling has it`s own dependencies section, add there your packages
{
"name": "xyz",
"version": "1.0.0",
"description": "xyz",
"dependencies": {
"#sap/approuter": "6.5.1"
},
"devDependencies": {
"#ui5/cli": "^1.11.1",
"ui5-middleware-simpleproxy": "^0.1.3",
"ui5-tooling-livereload": "^0.1.4",
"js-cookie": "^2.2.1"
},
"sapui5-runtime": {
"version": "1.66.1"
},
"scripts": {
},
"ui5": {
"dependencies": [
"ui5-tooling-livereload",
"ui5-middleware-simpleproxy",
"js-cookie"
]
}
}
To see the great picture following the entire yaml. But you need only the things after "--- # Everything"
specVersion: '1.0'
metadata:
name: my.ui5.id
type: application
server:
customMiddleware:
- name: ui5-tooling-livereload
afterMiddleware: compression
configuration:
debug: true
ext: "xml,json,properties"
port: 35729
path:
- "webapp"
- name: ui5-middleware-simpleproxy
mountPath: /my/local/backend/service/odata/v2
afterMiddleware: compression
configuration:
baseUri: "http://localhost:8081/my/local/backend/service/odata/v2"
--- # Everything below this line could also be put into the ui5.yaml of a standalone extension module
specVersion: "1.0"
kind: extension
type: project-shim
metadata:
name: my.application.thirdparty
shims:
configurations:
js-cookie: # name as defined in package.json
specVersion: "1.0"
type: module # Use module type
metadata:
name: js-cookie
resources:
configuration:
paths:
/resource/libs: "/src/" #here is mapped: node_modules/js-cookie/src

Heroku can't find ExpressJS/Knex migrations

I have an Express.js api that works locally, using Postgresql via Knex/SQL queries.
I am trying to deploy to Heroku, however, when I attempt to migrate the DB I get the error that Heroku can't find the migration files.
Specifically, when I run the migration it throws:
Using environment: production Error: ENOENT: no such file or
directory, scandir '/app/migrations'
My migrations are located at my_app/db/migrations. The app folder mentioned in the error does not exist. Trying to figure out how to point to the correct folder.
Here is my package.json:
{
"name": "writerboard-express-api",
"version": "1.0.0",
"description": "",
"main": "server.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"serve": "cross-env NODE_ENV=development nodemon server.js"
},
"keywords": [],
"author": "dariusgoore <dariusgoore#gmail.com> (dariusgoore.com)",
"license": "MIT",
"dependencies": {
"bcryptjs": "^2.4.3",
"body-parser": "^1.19.0",
"cors": "^2.8.5",
"dotenv": "^8.1.0",
"express": "^4.17.1",
"express-jwt": "^5.3.1",
"jsonwebtoken": "^8.5.1",
"knex": "^0.19.3",
"objection": "^1.6.9",
"objection-password": "^2.0.0",
"passport": "^0.4.0",
"passport-local": "^1.0.0",
"pg": "^7.12.1"
},
"devDependencies": {
"cross-env": "^5.2.1",
"nodemon": "^1.19.1"
}
}
My knexfile (myapp/knexfile.js):
const dotenv = require('dotenv');
dotenv.config({ path: "../.env" });
module.exports = {
development: {
client: 'postgresql',
connection: {
database: 'writerboard_dev', // update with env var
user: 'dariusgoore' // update with env var
},
pool: {
min: 2,
max: 10
},
migrations: {
tableName: 'knex_migrations',
directory: './db/migrations'
},
seeds: {
directory: './db/seeds'
}
},
production: {
client: 'postgresql',
connection: process.env.DATABASE_URL,
pool: {
min: 2,
max: 10
},
migrations: {
tableName: 'knex_migrations'
},
ssl: true
}
};
here is the server log from heroku:
2019-09-16T14:25:00.251226+00:00 heroku[web.1]: State changed from down to starting
2019-09-16T14:25:00.063523+00:00 app[api]: Release v8 created by user dariusgoore#gmail.com
2019-09-16T14:25:00.063523+00:00 app[api]: Deploy b697b243 by user dariusgoore#gmail.com
2019-09-16T14:25:00.000000+00:00 app[api]: Build succeeded
2019-09-16T14:25:03.700742+00:00 heroku[web.1]: Starting process with command `node server.js`
2019-09-16T14:25:08.369304+00:00 heroku[web.1]: State changed from starting to up
2019-09-16T14:25:07.885086+00:00 app[web.1]: Running on localhost:58146
FYI, the answer was in the knexfile.js. Needed to set the migrations directory for production.