I am trying to alter my Postgres database using Knex migrations but they are not working.
My knexfile.js looks like this:
module.exports = {
development: {
client: 'pg',
connection: {
database: 'cms_dev',
},
},
};
Then I have a db.js that looks like this:
const config = require('../knexfile.js');
const env = 'development';
const knex = require('knex')(config[env]);
module.exports = knex;
knex.migrate.latest([config]);
If I am starting with a fresh database my first migration works but if I try to update the database with a new (2nd) migration it does nothing.
When I try running knex migrate:latest --env development it says:
Using environment: development
Already up to date
*****EDIT*****
I ended up generating another new (3rd) migration and pasted in the exact same code from the previous one that was being ignored and it worked. No clue why the 2nd migration silently failed and the 3rd one worked.
I figured out the issue. I had forgotten to configure nodemon to ignore my migrations folder.
Because I have this line in db.js:
knex.migrate.latest([config]);
If I saved at all while creating the migration, my server would restart and cause the latest and incomplete migration to fire off. Since migrations only run once, Knex would think the database was already up to date.
If you're just looking to re-run any migrations from the start, delete the migrations tables in the schema/database of your choice.
Related
I know this has been asked previously but I am spinning in circles here....
I have a postgres 14 database and a Springboot application running flyway-db:7.13.0 and flyway-core:8.5.10
I am using RDS. I created a new database manually using root account. When I run my springboot my flyway migration fails with the error
Found non-empty schema(s) "public" but no schema history table. Use baseline() or set baselineOnMigrate to true to initialize the schema history table.
I understand the reason why this is happening but when I add the baselineVersion and baselineOnMigrate to my flyway gradle config it still doesn't work. I even added
flyway {
url = dbUrl
user = dbUser
password = dbPassword
schemas = ['public']
baselineVersion = '0.0'
baselineOnMigrate = true
locations = ["filesystem:${dbMigrationPath}"]
}
also I added
spring.flyway.baseline-on-migrate = true
to my application.properties
What confuses me the most is if I start the application pointing to the default postgres database with the root user. The migration works and it creates the flyway history table. but when I run it pointing to the newly created database it doesn't work.
Any idea?
I figured it out. It turned out I needed to add the baselineOnMigration flag to my configuration
Flyway.configure().baselineOnMigrate(true).dataSource(new TransactionAwareDataSourceProxy(dataSource)).load();
Whether to automatically call baseline when migrate is executed against a non-empty schema with no schema history
table. This schema will then be initialized with the baselineVersion before executing the migrations.
Only migrations above baselineVersion will then be applied.
This is useful for initial Flyway production deployments on projects with an existing DB.
Be careful when enabling this as it removes the safety net that ensures
Flyway does not migrate the wrong database in case of a configuration mistake! (default: false)
flyway.baselineOnMigrate=
flyway.baselineOnMigrate=true
I added flyway.baselineOnMigrate=true under the statement above and it worked.
I have a PostgreSQL db that is used by a Nest.Js / Prisma app.
We changed the name of a field in the Prisma schema and added a new field.
Now, when we want to update the PostreSQL structure, I'm running, as suggested by Prisma, the following commands:
npx prisma generate
and then
npx prisma migrate dev --name textSettings-added --create-only
The idea is to use the --create-only flag to review the migration before it is actually made.
However, when I run it I get a list of the changes to be made to the DB and the following message:
We need to reset the PostgreSQL database "my_database" at "my_db_name#cluster.us-east-1.rds.amazonaws.com:5432".
Do you want to continue? All data will be lost.
Of course I choose not to continue, because I don't want to lose the data. Upon inspection I see that the migration file actually contains DROP TABLE for the tables that I simply wanted to modify. How to avoid that?
So how do I run the update without affecting the data?
UPDATE:
I saw that running with --create-only creates a migration which can then be implemented on the DB level using prisma migrate dev, however, in that migration file there are still some commands that drop my previous tables because of some new parameters inside. How can I run prisma migration without deleting my PostgreSQL data?
UPDATE 2:
I don't want Prisma to drop my tables when I just updated them. The migration file generated, however, drops them and then alters them. Do you know what's the best procedure to avoid this drop? I saw somewhere I could first manually update the DB with the new options and then run the migration, so Prisma can find a way to update it, but that seems too manual to me... Maybe some other ideas?
For some cases like renaming tables or columns, Prisma's generated migration files need to be updated if they already contain data.
If that applies to your use case, Prisma's docs suggest to:
Make updates to the prisma schema
Create migration file without applying it (--create-only flag)
Update the migration script to remove the drops and instead write your custom query (e.g. RENAME <table_name> TO <new_name>)
Save and apply the migration (npx prisma migrate dev)
Note that those changes can lead to downtime (renaming a field or model), for which they have outlined the expand and contract pattern.
It might be a Prisma bug: https://github.com/prisma/prisma/issues/8053
I also recently had this situation. It probably should not try to run migration if you only want to create migration file.
But overall it is expected with Prisma to recreate your db sometimes. If you migration is breaking then it will be required to reset the data anyway when you apply it.
I suggest you to create some seeding script so you could consistently re-create the database state, it's very useful for your development environment.
More info
After removing a column from schema.prisma, and running npm prisma db push, I'm getting the error...
Error: The object 'users_role_df' is dependent on column 'role'.
I'm using Prisma 3.3.0 with the sqlserver connection.
The model looks like this...
model User {
id ....
name ...
role String #default("USER")
}
I see the initial push created the CONSTRAINT users_role_df but now when I remove it from the model, and run push, it's not handling removing the constraint first and then the column.
How can I fix this?
You could try running the command npx prisma migrate dev --create-only to see what the generated SQL looks like for the migration.
You could manually add a DROP CONSTRAINT users_role_df; inside the generated migration or change the order of the SQL commands (if such a command already exists in the generated migration).
It is fine for you to make changes to the migration file as long as you do it before applying the migration to your database.
I have a NestJS app with TypeORM, dockerized. I have synchronize turned off, using migrations instead. In the container entry point, I do yarn typeorm migration:run. It works well the first time around, and according to the logs it inserts records into the migrations table.
I noticed that when I start the project the next time it often tries to re-run migrations and fails (as expected) due to "relation already exists". At this point I can verify that entries are indeed missing from the migrations table via docker-compose exec db psql -U postgres -c 'SELECT * FROM "migrations" "migrations". The DB schema is up to date. When I insert a new record manually it gets an incremental ID after the missing records. So the records were there at some point.
I can't figure out what might cause entries in the migrations table to disappear (be rolled back?). This happens on the project linked above. It's a straightforward example project. I don't have an entity accidentally named "migrations". :)
As a workaround I currently insert into the migrations table manually:
docker-compose exec db psql -U postgres -c "INSERT INTO migrations (timestamp, name) VALUES ('1619623728180', 'AddTable1619623728180');"
Running specs that synchronized the DB was the issue.
I had a .env.test to use a different DB, but as it turns out that is not supported by dotenv. There are a few ways to make it work. I chose dotenv-flow/config and added it to my test script:
jest --collect-coverage --setupFiles dotenv-flow/config
I am brand-new to Sails and I'm looking to build my first application in javascript land! I am currently trying to connect my sails application to a postgresql database and I want to make sure that I am doing this correctly.
I started by creating a postgresql db with dbName, userName, password. I have added all this information into my connections.js file:
somePostgresqlServer: {
adapter: 'sails-postgresql',
host: 'localhost',
user: '<username>', // optional
password: '<password>', // optional
database: '<databasename>' //optional
}
I want to be sure I can migrate and perform all operations on my own so my models.js is set to migrate: safe. I ran npm install sails-postgresql. Now, to my understanding if I have migrate set to safe I will need the sails-db-migrate module (https://github.com/building5/sails-db-migrate). I followed this module step by step. I generated a User model by running sails generate api user. After this I ran grunt db:migrate. After all this, I check my psql database and no User table has been created. I know there is something I am missing, or maybe there is a more simple way to see if my postgresql db is connected.
Any advice or suggestions on how I should approach this would be greatly appreciated.
I was able to connect my PSQL DB to my application by simply:
creating db in psql
configuring it in sails app/connections.js
running npm install sails-postgresql
Adding to config/env/development.js:
models: {
connection: 'somePostgresqlServer'
}
running sails generate api user
sails lift
When asked for migration mode I chose options 2 which is alter
Opened up psequal and sure enough there was my new user table.
I'm not sure which of these actually triggered it to working from adding my db name to development.js, leaving off a PW on my DB, or changing migration mode to alter. But these steps helped it to connect. Will look further into.