I'm working on a small test project and everything was working fine until I added date-logger "service" which uses mongoose.
I get the following error upon running docker-compose up --build:
Error: Cannot find module 'mongodb/node_modules/bson'
Require stack:
- /app/node_modules/mongoose/lib/drivers/node-mongodb-native/objectid.js
...
Everything else gets started and runs correctly, except for this service which immediately fails as soon as require('mongoose') is reached.
Here are some of the important files:
docker-compose.yml:
version: '3'
services:
postgres:
image: 'postgres:latest'
redis:
image: 'redis:latest'
api:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=${REDIS_HOST}
- REDIS_PORT=${REDIS_PORT}
- PG_USER=${PG_USER}
- PG_HOST=${PG_HOST}
- PG_DB=${PG_DB}
- PG_PASSWORD=${PG_PASSWORD}
- PG_PORT=${PG_PORT}
date-logger:
build:
dockerfile: Dockerfile.dev
context: ./date-logger
volumes:
- /app/node_modules
- ./date-logger:/app
environment:
- EXPRESS_PORT=${EXPRESS_PORT}
depends_on:
- "mongo"
client:
build:
dockerfile: Dockerfile.dev
context: ./client
volumes:
- /app/node_modules
- ./client:/app
worker:
build:
dockerfile: Dockerfile.dev
context: ./worker
volumes:
- /app/node_modules
- ./worker:/app
environment:
- REDIS_HOST=${REDIS_HOST}
- REDIS_PORT=${REDIS_PORT}
nginx:
restart: always
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- '3050:80'
mongo:
image: mongo
restart: always
container_name: mongo
Dockerfile.dev on date-logger
FROM node:alpine
WORKDIR '/app'
COPY ./package.json .
RUN npm install
COPY . .
CMD ["npm", "run", "dev"]
package.json on date-logger:
{
"dependencies": {
"express": "~4.17.0",
"mongoose": "~5.5.11",
"cors": "2.8.4",
"body-parser": "~1.19.0"
},
"devDependencies": {
"nodemon": "1.19.0"
},
"scripts": {
"start": "node index.js",
"dev": "nodemon"
}
}
index.js on date-logger:
let express = require('express');
let bodyParser = require('body-parser');
// This console.log works
console.log('date-logger hello ---');
// ****!!!! IT FAILS IMMEDIATELY ON THIS REQUIRE !!!****
let mongoose = require('mongoose');
let app = express();
app.use(
bodyParser.urlencoded({
extended: true
})
);
app.use(bodyParser.json());
mongoose.connect('mongodb://mongo/date-logger');
const ItemSchema = new mongoose.Schema({
value: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
});
Item = mongoose.model('item', ItemSchema);
var port = process.env.EXPRESS_PORT || 8080;
app.get('/', (req, res) => res.send('Hello World with Express'));
app.get('/date', (req, res) => res.send('data'));
app.get('/date/all', (req, res) => {
Item.find({}).exec().then(results => res.send(results))
});
app.post('/date/add', (req, res) => {
const newItem = new Item({
value: req.body.value
});
newItem.save().then(item => res.send('success'));
});
app.listen(port, function() {
console.log('Running RestHub on port ' + port);
});
I'm new to devops world so all/any help is super appreciated! Thanks!
I think your problem is related to mongoose and not docker,
check this issue
This is very old but I came across a similar problem recently (but using only docker, not docker-compose).
I solved the problem by removing package-lock.json from my .dockerignore file (it should have never been there at all).
And of course make sure you can run npm install in your local machine, try removing package-lock and node_modulues and running npm install again if you can't. And update any modules you can.
Related
im using sequlize with umzug - migrations work locally, when i create a job for it, it cannot find the neccessery modules.
I got a mirgrator.js file.
const { migrator } = require('./iumzug.js');
migrator.runAsCLI()
And an iumzug.ts file as well, which configured like this.
const { Sequelize } = require('sequelize');
const { envVar } = require('./src/utilities/env-var')
const { Umzug, SequelizeStorage } = require("umzug")
const sequelize = new Sequelize({
database: envVar.DB_DATABASE,
host: envVar.DB_HOST,
port: 5432,
schema: ["TEST"].includes(envVar.NODE_ENV) ? 'test' : 'public',
username: envVar.DB_USERNAME,
password: envVar.DB_PASSWORD,
dialect: 'postgres',
ssl: true,
dialectOptions: {
ssl: {
require: true,
},},});
const migrator = new Umzug({
migrations: {
glob: ["./src/database/*.ts", { cwd: __dirname }],
resolve: ({ name, path, context }) => {
// eslint-disable-next-line #typescript-eslint/no-var-requires
const migration = require(path);
return {
// adjust the parameters Umzug will
// pass to migration methods when called
name,
up: async () => migration.up(context, Sequelize),
down: async () => migration.down(context, Sequelize)
};
}
},
context: sequelize.getQueryInterface(),
storage: new SequelizeStorage({
sequelize,
modelName: "migration_meta"
}),
logger: console
});
module.exports = { migrator }
i created a migration job on my github yml file as followes:
migrations:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout#v3
- name: migrations step
run: |
node migrator.js up
when i run github action - i get this error
looking for alternatives / directions to fix it.
Let me know if i need to add anymore code / pictures of the process.
Everytime i docker-compose up. The app runs fine on 4040. But the api call give me errors when checked in console of web-browser. ERROR - GET http://backend:8888/api/screen net::ERR_NAME_NOT_RESOLVED
I am trying to get my backend-container ip-address in place of the api call 'backend':8888. But it takes the same name as i have given in the url below
void fetchData() async {
try {
setState(() {
isLoading = true;
});
var host = 'backend';
print(host);
var response = await dio.get(
"http://backend:8888/api/screen",
);
if (response.statusCode == 200) {
setState(() {
lineData = jsonDecode(response.data);
isLoading = false;
});
}
} on DioError catch (err) {
print(err);
isLoading = false;
} catch (_) {
setState(() {
isLoading = false;
});
}
}
My docker-compose-file
version: '3'
services:
playground-web-frontend:
build:
context: ./flutterapp
dockerfile: frontend.dockerfile
ports:
- 4040:4040
backend:
image: backend
ports:
- 8888:8888
I tried giving container-name,service-name. I exec into my flutter app container using bash and then curl onto the api using backend-container ip address. It works and gives me correct data
Error - ERROR - GET http://backend:8888/api/screen net::ERR_NAME_NOT_RESOLVED
By adding static ip address in docker-compose.yml file. Please handle cors server-side in flutter web apps.
version: '3'
networks:
fullstack:
ipam:
config:
- subnet: 172.20.0.0/24
services:
playground-web-frontend:
build:
context: ./flutterapp
dockerfile: frontend.dockerfile
networks:
fullstack:
ipv4_address: 172.20.0.6
ports:
- 4040:4040
backend:
image: backend
ports:
- 8888:8888
networks:
fullstack:
ipv4_address: 172.20.0.7
I have an error that's only happening in my Github Actions workflow (when I run my Jest script locally, it's fine). I've only found this SO answer and this one but error still persists. Any thoughts on what to check next?
Here's the error:
> jest server/test/test --config=server/test/jest.config.js
FAIL server/test/test.js
● Test suite failed to run
ReferenceError: TextEncoder is not defined
at Object.<anonymous> (../../node_modules/mongodb-connection-string-url/node_modules/whatwg-url/dist/encoding.js:2:21)
at Object.<anonymous> (../../node_modules/mongodb-connection-string-url/node_modules/whatwg-url/dist/url-state-machine.js:5:34)
Here's my Jest config and script:
jest.config.js
module.exports = {
preset: '#shelf/jest-mongodb'
};
test.js
const dotenv = require('dotenv');
const path = require('path');
process.env = dotenv.config({path: path.resolve(__dirname, '.env')}).parsed;
const request = require('supertest');
const {app, server} = require('../server');
const { MongoClient } = require('mongodb');
const { TextEncoder } = require('util');
global.TextEncoder = TextEncoder;
describe('GET /', () => {
let mongoClient;
beforeAll(async () => {
app.locals.mongoClient = mongoClient = await MongoClient.connect(process.env.MONGODB_URI, { useNewUrlParser: true, useUnifiedTopology: true });;
});
afterAll(async () => {
await mongoClient.close();
server.close();
});
it('responds with path of /', (done) => {
request(app).get('/').expect(JSON.stringify({path: '/'}), done);
});
});
My Github Actions workflow is erroring at the npm run test-server step:
name: cicd
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [ 10.x, 12.x, 14.x, 15.x ]
fail-fast: false
steps:
- uses: actions/checkout#v2
- name: Create environment variables
run: |
touch server/test/.env
echo MONGODB_URI=${{ secrets.QA_MONGODB_URI }} >> server/test/.env
- name: Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm run build --if-present
- run: npm run test-server
deploy:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: akhileshns/heroku-deploy#v3.12.12
with:
heroku_api_key: ${{ secrets.HEROKU_API_KEY }}
heroku_app_name: "my-app-name"
heroku_email: "my-email"
I had the same problem and a just inserted the two lines at the top
global.TextEncoder = require("util").TextEncoder;
global.TextDecoder = require("util").TextDecoder;
reference: ReferenceError: TextEncoder is not defined with mongodb nodes
global.TextEncoder = require("util").TextEncoder;
global.TextDecoder = require("util").TextDecoder;
import { MongoMemoryServer } from 'mongodb-memory-server';
import mongoose from 'mongoose';
let mongod: MongoMemoryServer;
beforeAll(async () => {
const mongod = await MongoMemoryServer.create();
const mongoUri = mongod.getUri();
await mongoose.connect(mongoUri);
});
beforeEach(async () => {
const collections = await mongoose.connection.db.collections();
for (let collection of collections) {
await collection.deleteMany({});
}
});
afterAll(async () => {
await mongod.stop(true);
await mongoose.connection.close();
});
I've tried to follow this tutorial https://www.youtube.com/watch?v=VF38U2qd27Q, but to no avail.
I realised that the syntax in the video already obsolete for example connectWithUri to become connect.
but when I tried to connect to mongo using deno_mongo with the latest docs, it still not working.
import { MongoClient } from "https://deno.land/x/mongo#v0.20.1/mod.ts";
const dbString = `mongodb://${mongoUser}:${mongoPass}#${mongoHost}:${mongoPort}`;
const client = new MongoClient();
client.connect(dbString);
const db = client.database(mongoDB)
this.users = db.collection<UserSchema>("users");
Then I found another library denodb but again can't connect to mongodb:
import { Database } from 'https://deno.land/x/denodb/mod.ts';
const dbString = `mongodb://${mongoUser}:${mongoPass}#${mongoHost}:${mongoPort}`;
this.db = new Database('mongo', {
uri: dbString,
database: mongoDB
});
the error message:
error: Uncaught AssertionError
deno | throw new AssertionError(msg);
deno | ^
deno | at assert (asserts.ts:152:11)
deno | at MongoClient.database (client.ts:48:5)
deno | at new connectDB (connectDB.ts:35:23)
which part is wrong?
Looking at the deno_mongo README on GitHub.
For a local database you should use
//Connecting to a Local Database
await client.connect("mongodb://localhost:27017");
And if you are connecting to Mongo Atlas Database (and probably any other remote database) you should use:
//Connecting to a Mongo Atlas Database
await client.connect({
db: "<db_name>",
tls: true,
servers: [
{
host: "<db_cluster_url>",
port: 27017,
},
],
credential: {
username: "<username>",
password: "<password>",
db: "<db_name>",
mechanism: "SCRAM-SHA-1",
},
});
FYI
If you are using Mongo Atlas make sure to split up the connection string you get from 'Connection Wizard' in the Mongo Atlas dashboard over 3 (or how many replicas you have) entries in the server array. Like this:
servers: [
{
host: this.dbUrl1, // e.g. <name-of-cluster>-00-00.fbnrc.mongodb.net
port: 27017,
},
{
host: this.dbUrl2, // e.g. <name-of-cluster>-00-01.fbnrc.mongodb.net
port: 27017,
},
{
host: this.dbUrl3, // e.g. <name-of-cluster>-00-02.fbnrc.mongodb.net
port: 27017,
}
]
You get the connection string by:
Click 'Clusters' under Data storage (left side of the screen)
Click 'Connect' button
Click 'Connect to Application'
Select Driver: 'Node.js' and Version: '2.2.12 or later'
Connection string is displayed below. The servers are listed in a comma separated manner like this:
...<name-of-cluster>-00-00.fbnrc.mongodb.net:27017,<name-of-cluster>-00-01.fbnrc.mongodb.net:27017,<name-of-cluster>-00-02.fbnrc.mongodb.net:27017...
FYI-2
Make sure the master replica is listed first in the server array. Because if you want to do insertions into the database the master replica should be targeted. For me this was the 2nd mongo url, therefore the following server array worked for me:
servers: [
{
host: this.dbUrl2,
port: 27017,
},
{
host: this.dbUrl1,
port: 27017,
},
{
host: this.dbUrl3,
port: 27017,
}
]
the below code is working for me.
import { DataTypes, Database, Model } from 'https://deno.land/x/denodb/mod.ts';
const db = new Database('mongo', {
host: 'mongodb://localhost:27017',
username: '',
password: '',
database: 'DBMYAPP',
});
console.log(db)
I also faced the same issue while updating deno_mongo to the latest version. Use await to resolve client.connect method
Try this:
import { MongoClient } from "https://deno.land/x/mongo#v0.20.1/mod.ts";
const dbString = `mongodb://${mongoUser}:${mongoPass}#${mongoHost}:${mongoPort}`;
const client = new MongoClient();
await client.connect(dbString);
const db = client.database(mongoDB)
this.users = db.collection<UserSchema>("users");
I had the same problem on windows 10, so try this
on your local mongodb:
await client.connect("mongodb://127.0.0.1:27017");
I'm having issues setting up an express server instance on serverless with nextJs. I keep getting a Cannot read property 'getCompilationErrors' of undefined when running the server function. It seems to be an issue with app.render.
When running debug it seems to be coming from within nextJs
Server.js
const express = require('express');
const path = require('path');
const dev = process.env.NODE_ENV !== 'production';
const next = require('next');
const pathMatch = require('path-match');
const app = next({ dev });
const handle = app.getRequestHandler();
const { parse } = require('url');
const server = express();
const route = pathMatch();
server.use('/_next', express.static(path.join(__dirname, '.next')));
server.get('/', (req, res) => app.render(req, res, '/'));
server.get('*', (req, res) => handle(req, res));
module.exports = server;
index.js
const sls = require('serverless-http')
const binaryMimeTypes = require('./binaryMimeTypes')
const server = require('./server')
module.exports.server = sls(server, {
binary: binaryMimeTypes
});
Serverless.yml
service: ssr-react-next
provider:
name: aws
runtime: nodejs8.10
stage: ${self:custom.secrets.NODE_ENV}
region: us-east-1
environment:
NODE_ENV: ${self:custom.secrets.NODE_ENV}
functions:
server:
handler: index.server
events:
- http: ANY /
- http: ANY /{proxy+}
plugins:
- serverless-apigw-binary
- serverless-domain-manager
custom:
secrets: ${file(secrets.json)}
apigwBinary:
types:
- '*/*'
customDomain:
domainName: ${self:custom.secrets.DOMAIN}
basePath: ''
stage: ${self:custom.secrets.NODE_ENV}
createRoute53Record: true
# endpointType: 'regional'
# if the ACM certificate is created in a region except for `'us-east-1'` you need `endpointType: 'regional'`
Figured a way around this, just needed to prepare the app with async
server.use(async(req, res, next) => {
await app.prepare();
next();
})
Did you add this in the server.js so it looks like this?
server.js after server.use('/_next', express.static(path.join(__dirname, '.next')))
server.use(async(req, res, next) => {
await app.prepare();
next();
})