No migrations are pending using Typeorm CLI and NestJS - postgresql

The issue I'm facing is with running migrations using #nestjs/typeorm ^9.0.1 and typeorm ^0.3.12. Despite being able to build the project and create migrations, when I try to run them, typeorm can't find the migration files (No migrations are pending).
I have noticed that the version 0.3.x of typeorm requires Datasource object and that in the version 9.x of #nestjs/typeorm, TypeOrmModuleOptions does not have the cli : {migrationsDir : string} attribute.
Here's a link to the project that reproduces the problem, and an example directory structure :
src\configs\database\dataSource.config.ts
src\configs\database\postgres.config.ts
src\configs\database\migrations\1676252827402-Chat.ts
postgres.config.ts
import { DataSourceOptions } from "typeorm";
import { join } from "path";
import { config } from "dotenv";
import CustomNamingStrategy from "./customNamingStrategy";
import { registerAs } from "#nestjs/config";
import { TypeOrmModuleOptions } from "#nestjs/typeorm";
config();
export const databaseConfig: DataSourceOptions = {
name: "default",
type: "postgres",
url: process.env.DATABASE_URL,
ssl:
process.env.DATABASE_ENABLE_SSL === "true"
? {
rejectUnauthorized: false,
}
: false,
logging: process.env.DATABASE_ENABLE_LOGGING === "true",
entities: [join(__dirname, "../../models/*/", "*.entity.{ts,js}")],
migrations: [join(__dirname, "migrations/*.{ts,js}")],
synchronize: false,
migrationsRun: false,
namingStrategy: new CustomNamingStrategy(),
};
export default registerAs(
"database",
() =>
({
...databaseConfig,
keepConnectionAlive: true,
} as TypeOrmModuleOptions)
);
dataSource.config.ts
import { DataSource } from "typeorm";
import { databaseConfig } from "./postgres.config";
export const AppDataSource = new DataSource(databaseConfig);
package.json
"typeorm": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli"
"migration:create": "yarn run typeorm migration:create"
"migration:run": "yarn run typeorm migration:run -d src/configs/database/dataSource.config.ts"
Although I can use the create command successfully, when I use the run command, it only displays the message "No migrations are pending". I attempted to resolve the issue by changing migrations path to the DataSourceOptions.migrations list, but it did not work. I also tried replacing the cli parameter datasource with the dist datasource, as well as adjusting the migration paths, but these changes did not fix the issue.

I'm facing a similar issue using the same version of typeorm (^0.3.12), so there might be a bug in this version.
It seems that the migrations are not found anymore if the DataSourceOptions.migrations property is set with the path to the migrations folder.
The ^0.3.12 version seems to be working fine only if the migration classes are directly referenced in DataSourceOptions.migrations, similar to the following:
import { Chat1676252827402 } from './migrations/1676252827402-Chat.ts';
export const databaseConfig: DataSourceOptions = {
...
migrations: [Chat1676252827402],
...
};

Related

MongoRuntimeError Unable to parse with URL

I am trying to connect to a mongo db using the nodejs mongo driver and I'm doing this in a cypress project. I get the error in the title. Below is the simplified version of my code.
import {MongoClient} from 'mongodb';
export class SomeRepository {
static insertSomething(): void {
// Error in the line below: MongoRuntimeError Unable to parse localhost:27017 with URL
const client = new MongoClient('mongodb://localhost:27017');
}
}
Mongodb is running because I can connect from the terminal. Also tried replacing localhost with 127.0.0.1 and adding the authSource parameter to the connection string.
The reason I'm mentioning cypress is because in a simple node project that only connects to mongodb everything works as expected. Package.json below
{
"name": "e2e",
"version": "1.0.0",
"description": "",
"main": "index.js",
"dependencies": {
"cypress": "10.8.0",
"cypress-wait-until": "1.7.2",
"headers-utils": "3.0.2",
"mongodb": "4.10.0",
"otplib": "12.0.1",
"pg": "8.7.3",
"pg-native": "3.0.1",
"typescript": "4.9.3"
}
}
The error is in the way you are passing the url, it is necessary that you follow a pattern, in mongodb to connect you need to have this pattern that I will pass below:
Format:
mongodb://<user>:<password>#<host>
Format with filled values:
mongodb://root:mypassword#localhost:27017/
The reason it’s not working is because you’re calling a NodeJS library in a cypress test. Cypress tests run inside a browser and cannot run nodejs libraries.
If you wanted to execute nodejs code in cypress you must create a cypress task https://docs.cypress.io/api/commands/task#Syntax
// cypress.config.js
import { SomeRepository } from ‘./file/somewhere’
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
on(‘task’, {
insertSomething() {
return SomeRepository.insertSomething();
}
}
}
}
})
// to call in a cypress test
it(‘test’, function () {
cy.task(‘insertSomething’).then(value => /* do something */);
}
});

I want to insert with mikro-orm, but it dont find my table :c (TableNotFoundException)

So
Console:
yarn dev
yarn run v1.22.10
$ nodemon dist/index.js
[nodemon] 2.0.7
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node dist/index.js`
[discovery] ORM entity discovery started, using ReflectMetadataProvider
[discovery] - processing entity Post
[discovery] - entity discovery finished, found 1 entities, took 21 ms
[info] MikroORM successfully connected to database postgres on postgresql://postgres:*****#127.0.0.1:5432
[query] begin
[query] insert into "post" ("created_at", "title", "updated_at") values ('2021-04-05T21:04:23.126Z', 'my first post', '2021-04-05T21:04:23.126Z') returning "_id" [took 12 ms]
[query] rollback
TableNotFoundException: insert into "post" ("created_at", "title", "updated_at") values ('2021-04-05T21:04:23.126Z', 'my first post', '2021-04-05T21:04:23.126Z') returning "_id" - relation "post" does not exist
at PostgreSqlExceptionConverter.convertException (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\postgresql\PostgreSqlExceptionConverter.js:36:24)
at PostgreSqlDriver.convertException (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\drivers\DatabaseDriver.js:194:54)
at P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\drivers\DatabaseDriver.js:198:24
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async PostgreSqlDriver.nativeInsert (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\knex\AbstractSqlDriver.js:150:21)
at async ChangeSetPersister.persistNewEntity (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\unit-of-work\ChangeSetPersister.js:55:21)
at async ChangeSetPersister.executeInserts (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\unit-of-work\ChangeSetPersister.js:24:13)
at async UnitOfWork.commitCreateChangeSets (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\unit-of-work\UnitOfWork.js:496:9)
at async UnitOfWork.persistToDatabase (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\unit-of-work\UnitOfWork.js:458:13)
at async PostgreSqlConnection.transactional (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\knex\AbstractSqlConnection.js:53:25)
at async UnitOfWork.commit (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\unit-of-work\UnitOfWork.js:183:17)
at async SqlEntityManager.flush (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\EntityManager.js:486:9)
at async SqlEntityManager.persistAndFlush (P:\.Projektek\lireddit-server\node_modules\#mikro-orm\core\EntityManager.js:438:9)
previous error: insert into "post" ("created_at", "title", "updated_at") values ('2021-04-05T21:04:23.126Z', 'my
first post', '2021-04-05T21:04:23.126Z') returning "_id" - relation "post" does not exist
at Parser.parseErrorMessage (P:\.Projektek\lireddit-server\node_modules\pg-protocol\dist\parser.js:278:15)
at Parser.handlePacket (P:\.Projektek\lireddit-server\node_modules\pg-protocol\dist\parser.js:126:29)
at Parser.parse (P:\.Projektek\lireddit-server\node_modules\pg-protocol\dist\parser.js:39:38)
at Socket.<anonymous> (P:\.Projektek\lireddit-server\node_modules\pg-protocol\dist\index.js:10:42)
at Socket.emit (events.js:315:20)
at Socket.EventEmitter.emit (domain.js:467:12)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
length: 166,
severity: 'ERROR',
code: '42P01',
detail: undefined,
hint: undefined,
position: '13',
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'd:\\pginstaller_13.auto\\postgres.windows-x64\\src\\backend\\parser\\parse_relation.c',
line: '1376',
routine: 'parserOpenTable'
}
Index.ts:
import { MikroORM } from "#mikro-orm/core";
import { __prod__ } from "./constants";
import { Post } from "./entities/Post";
import mikroConfig from "./mikro-orm.config";
const main = async () => {
const orm = await MikroORM.init(mikroConfig);
await orm.getMigrator().up;
const post = orm.em.create(Post, { title: "my first post" });
await orm.em.persistAndFlush(post);
};
main().catch((err) => {
console.error(err);
});
Post.ts:
import { Entity, PrimaryKey, Property } from "#mikro-orm/core";
#Entity()
export class Post {
#PrimaryKey()
_id!: number;
#Property({ type: "date" })
createdAt = new Date();
#Property({ type: "date", onUpdate: () => new Date() })
updatedAt = new Date();
#Property({ type: "text" })
title!: string;
}
mikro-orm.config.ts:
import { __prod__ } from "./constants";
import { Post } from "./entities/Post";
import { MikroORM } from "#mikro-orm/core";
import path from "path";
export default {
migrations: {
path: path.join(__dirname, "./migrations"),
pattern: /^[\w-]+\d+\.[tj]s$/,
},
entities: [Post],
dbName: "postgres",
debug: !__prod__,
type: "postgresql",
password: "hellothere",
} as Parameters<typeof MikroORM.init>[0];
And the migration I created with npx mikro-orm migration:create:
import { Migration } from '#mikro-orm/migrations';
export class Migration20210405205411 extends Migration {
async up(): Promise<void> {
this.addSql('create table "post" ("_id" serial primary key, "created_at" timestamptz(0) not null, "updated_at" timestamptz(0) not null, "title" text not null);');
}
}
After that im compiling it to js btw, but I guess the problem will be somewhere at my code or idk plz help me, I can give you more info just plz help, I've been trying to fix this bug for 5 hours :/
Btw Im doin Ben Awad's 14 hour fullstack tutorial if its matter.
The TableNotFoundException happens when you try to add data before initializing the table's schema (or structure).
Passing the --initial as in Mosh's Answer did not work for me, possibly because I am passing a username and password in ./mikro-orm.config.ts.
I used Mikro-ORM's SchemaGenerator to initialize the table as seen here in the official docs.
Add the following lines before adding data to post in your main function in index.ts:
const generator = orm.getSchemaGenerator();
await generator.updateSchema();
The main function in index.ts should now look like this:
const main = async () => {
const orm = await MikroORM.init(mikroConfig);
await orm.getMigrator().up;
const generator = orm.getSchemaGenerator();
await generator.updateSchema();
const post = orm.em.create(Post, { title: "my first post" });
await orm.em.persistAndFlush(post);
};
updateSchema creates a table or updates it based on .entities/Post.ts. This could cause issues when the Post file is updated, I haven't run in to any while following Ben's tutorial. Although, I'd still recommend creating ./create-schema.ts and running it when needed as shown in the official docs.
I have had the same issue. This is what I did:
I deleted the migrations folder as well as the dist folder
I ran npx mikro-orm migration:create --initial
After that, I restarted yarn watch and yarn dev and it worked for me.
Notice the --initial flag. I would recommend to check the official documentation. The migrations table is used to keep track of already executed migrations. When you only run npx mikro-orm migration:create, the table will not be created and therefore MikroORM is unable to check if the migration for the Post entity has already been performed (which includes creating the respective table on the database).
Ben does not use the --initial flag in his tutorial, he might have already ran it prior to the tutorial.
I had a similar problem myself (Also doing Ben Awad's tutorial).
I used Mikro-ORM's schema generator to initialize the table like in Fares47's Answer, but the problem still persisted.
It wasn't until I set my user to have Superuser permissions that it started working.
I am using postgresql for my data base which I downloaded using homebrew. If you have a similar set up here is what I did:
Start up psql in your terminal using psql postgres. If you want, you can view your users and check their permissions by typing \du in the shell. Then, to change the permissions for a user use the command ALTER ROLE <username> WITH SUPERUSER;. Make sure you include a semi-colon or else it will not run the command.
Check this article out for more info on psql commands.
I have the same problem i solved by install the ts-node on project
npm i -D ts-node
and set useTsNode on package.json as true.
The problem is the mikro-orm cli only add ts files paths in configPaths if the property useTsNode is true and ts-node is installed.
orther problem that i have is the regex in pattern property in mikro-orm.config.ts was wrong because a typo.
If any of the suggested steps didnt solve it for you, simply...
quit yarn watch and yarn dev
run this command from the command line
npx mikro-orm migration:up
now restart watch and dev and it you should be good.
from https://mikro-orm.io/docs/migrations/#migration-class
I also experienced this. And like Fares47 said it's possibly because I passed the username and password in ./mikro-orm.config.ts.
And my solution is simply execute the sql command that generated in ./src/migrations/Migration<NUMBERS>.ts file in postgresql terminal.
Here is the command that I execute in the database,
create table "post" ("id" serial primary key, "created_at" timestamptz(0) not null, "updated_at" timestamptz(0) not null, "title" text not null);
Just like what they suggested in the doc,
A safe approach would be generating the SQL on development server and
saving it into SQL Migration files that are executed manually on the
production server.

typeorm:migration create on New Project Does Not Recognize Entities - "No changes in database schema were found - cannot generate a migration."

I'm having trouble creating the initial migration for a nestjs-typeorm-mongo project.
I have cloned this sample project from nestjs that uses typeorm with mongodb. The project does work in that when I run it locally after putting a "Photo" document into my local mongo with db named "test" and collection "photos" then I can call to localhost:3000/photo and receive the photo documents.
Now I'm trying to create migrations with the typeorm cli using this command:
./node_modules/.bin/ts-node ./node_modules/typeorm/cli.js migration:generate -n initial
...but it's not working. I am not able to create an initial commit- even after setting "synchronize: false" in my app.module.ts file I always get the error:
No changes in database schema were found - cannot generate a migration. To create a new empty migration use "typeorm migration:create" command
when trying to generate a migration... 🤔
Other than changing synchronize to false, the only other change I made was adding an ormconfig.json file in the project root by running typeorm init --database mongodb:
{
"type": "mongodb",
"database": "test",
"synchronize": true,
"logging": false,
"entities": [
"src/**/*.entity.ts"
],
"migrations": [
"src/migration/**/*.ts"
],
"subscribers": [
"src/subscriber/**/*.ts"
],
"cli": {
"entitiesDir": "src",
"migrationsDir": "src/migration",
"subscribersDir": "src/subscriber"
}
}
Once you are using MongoDB, you don't have tables and have no need to create your collections ahead of time. Essentially, MongoDB schemas are created on the fly!
Under the hood, if the driver is MongoDB, the command typeorm migration:create is bypassed so it is useless in this case. You can check yourself the PR #3304 and Issue #2867.
However, there is an alternative called migrate-mongo which provides a way to archive incremental, reversible, and version-controlled way to apply schema and data changes. It’s well documented and actively developed.
migrate-mongo example
Run npm install -g migrate-mongo to install it.
Run migrate-mongo init to init the migrations tool. This will create a migrate-mongo-config.js configuration file and a migrations folder at the root of our project:
|_ src/
|_ migrations/
|- 20200606204524-migration-1.js
|- 20200608124524-migration-2.js
|- 20200808114324-migration-3.js
|- migrate-mongo.js
|- package.json
|- package-lock.json
Your migrate-mongo-config.js configuration file may look like:
// In this file you can configure migrate-mongo
const env = require('./server/config')
const config = {
mongodb: {
// TODO Change (or review) the url to your MongoDB:
url: env.mongo.url || "mongodb://localhost:27017",
// TODO Change this to your database name:
databaseName: env.mongo.dbname || "YOURDATABASENAME",
options: {
useNewUrlParser: true, // removes a deprecation warning when connecting
useUnifiedTopology: true, // removes a deprecating warning when connecting
// connectTimeoutMS: 3600000, // increase connection timeout up to 1 hour
// socketTimeoutMS: 3600000, // increase socket timeout up to 1 hour
}
},
// The migrations dir can be a relative or absolute path. Only edit this when really necessary.
migrationsDir: "migrations",
// The MongoDB collection where the applied changes are stored. Only edit this when really necessary.
changelogCollectionName: "changelog"
};
module.exports = config;
Run migrate-mongo create name-of-my-script to add a new migration script. A new file will be created with a corresponding timestamp.
/*
|_ migrations/
|- 20210108114324-name-of-my-script.js
*/
module.exports = {
function up(db) {
return db.collection('products').updateMany({}, { $set: { quantity: 10 } })
}
function down(db) {
return db.collection('products').updateMany({}, { $unset: { quantity: null } })
}
}
The database changelog: In order to know the current database version and which migration should apply next, there is a special collection that stores the database changelog with information such as migrations applied, and when where they applied.
To run your migrations, simply run the command: migrate-mongo up
You can find a full example in this article MongoDB Schema Migrations in Node.js

Jest: Cannot use import statement outside a module

I got an error when I run test using Jest, I tried to fix this error for 2 hours. But, I couldn't fix it. My module is using gapi-script package and error is occurred in this package. However, I don't know why this is occurred and how to fix it.
jest.config.js
module.exports = {
"collectCoverage": true,
"rootDir": "./",
"testRegex": "__tests__/.+\\.test\\.js",
"transform": {
'^.+\\.js?$': "babel-jest"
},
"moduleFileExtensions": ["js"],
"moduleDirectories": [
"node_modules",
"lib"
]
}
babel.config.js
module.exports = {
presets: [
'#babel/preset-env',
]
};
methods.test.js
import methods, { typeToActions } from '../lib/methods';
methods.js
import { gapi } from "gapi-script";
...
Error Message
C:\haram\github\react-youtube-data-api\node_modules\gapi-script\index.js:1
({"Object.":function(module,exports,require,__dirname,__filename,global,jest){import
{ gapi, gapiComplete } from './gapiScript';
SyntaxError: Cannot use import statement outside a module
What is wrong with my setting?
As of this writing, Jest is in the process of providing support for ES6 modules. You can track the progress here:
https://jestjs.io/docs/en/ecmascript-modules
For now, you can eliminate this error by running this command:
node --experimental-vm-modules node_modules/.bin/jest
instead of simply:
jest
Be sure to check the link before using this solution.
I solved this with the help of Paulo Coghi's answer to another question -
Does Jest support ES6 import/export?
Step 1:
Add your test environment to .babelrc in the root of your project:
{
"env": {
"test": {
"plugins": ["#babel/plugin-transform-modules-commonjs"]
}
}
}
Step 2:
Install the ECMAScript 6 transform plugin:
npm install --save-dev #babel/plugin-transform-modules-commonjs
Jest needs babel to work with modules.
For the testing alone, you do not need jest.config.js, just name the testfiles xxx.spec.js or xxx.test.js or put the files in a folder named test.
I use this babel.config.js:
module.exports = function (api) {
api.cache(true)
const presets = [
"#babel/preset-env"
]
return {
presets
}
}
Adding "type": "module" in package.json or using mjs as stated in other answers is not necessary when your setup is not too complicated.
I have also faced the same issue and this was resolved by adding following command-line option as a environment variable.
export NODE_OPTIONS=--experimental-vm-modules npx jest //linux
setx NODE_OPTIONS "--experimental-vm-modules npx jest" //windows
Upgrading Jest (package.json) to version 29 (latest as of now) solved this problem for me.

NestJS: Resolving dependencies in NestMiddleware?

I'm trying to combine express-session with a typeorm storage within NestJS framework. Therefore I wrote a NestMiddleware as wrapper for express-session (see below). When I start node, NestJS logs the following
Error message:
UnhandledPromiseRejectionWarning: Unhandled promise rejection
(rejection id: 1): Error: Nest can't resolve dependencies of the
SessionMiddleware (?). Please verify whether [0] argument is available
in the current context.
Express is not started (connection refused), but the Sqlite DB (where the sessions should be stored) is created (also a session table, but not the columns).
To me it looks like there is a special problem in resolving dependencies with #InjectRepository from nestjs/typorm-Module. Does anyone have a hint?
Code:
import { Middleware, NestMiddleware, ExpressMiddleware } from '#nestjs/common';
import * as expressSession from 'express-session';
import { InjectRepository } from '#nestjs/typeorm';
import { Repository } from 'typeorm';
import { TypeormStore } from 'connect-typeorm';
import { Session } from './session.entity';
#Middleware()
export class SessionMiddleware implements NestMiddleware {
constructor(
#InjectRepository(Session)
private readonly sessionRepository: Repository<Session>
) {}
resolve(): ExpressMiddleware {
return expressSession({
store: new TypeormStore({ ttl: 86400 }).connect(this.sessionRepository),
secret: 'secret'
});
}
}
It was my fault. Had the middleware in a module, but was configuring the session middleware at the app module level. On that level the following import statement was missing:
TypeOrmModule.forFeature([Session])
Moved now everything to non-app module including middleware configuration. That solved the problem.