I'd like to make a migration for a Flask app. I am using Alembic.
However, I receive the following error.
Target database is not up to date.
Online, I read that it has something to do with this.
http://alembic.zzzcomputing.com/en/latest/cookbook.html#building-an-up-to-date-database-from-scratch
Unfortunately, I don't quite understand how to get the database up to date and where/how I should write the code given in the link.
After creating a migration, either manually or as --autogenerate, you must apply it with alembic upgrade head. If you used db.create_all() from a shell, you can use alembic stamp head to indicate that the current state of the database represents the application of all migrations.
This Worked For me
$ flask db stamp head
$ flask db migrate
$ flask db upgrade
My stuation is like this question, When I execute "./manage.py db migrate -m 'Add relationship'", the error occused like this "
alembic.util.exc.CommandError: Target database is not up to date."
So I checked the status of my migrate:
(venv) ]#./manage.py db heads
d996b44eca57 (head)
(venv) ]#./manage.py db current
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
715f79abbd75
and found that the heads and the current are different!
I fixed it by doing this steps:
(venv)]#./manage.py db stamp heads
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running stamp_revision 715f79abbd75 -> d996b44eca57
And now the current is same to the head
(venv) ]#./manage.py db current
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
d996b44eca57 (head)
And now I can do the migrate again.
This can be solved bby many ways :
1 To fix this error, delete the latest migration file ( a python file) then try to perform a migration afresh.
If issue still persists try these commands :
$ flask db stamp head # To set the revision in the database to the head, without performing any migrations. You can change head to the required change you want.
$ flask db migrate # To detect automatically all the changes.
$ flask db upgrade # To apply all the changes.
$ flask db stamp head # To set the revision in the database to the head, without performing any migrations. You can change head to the required change you want.
$ flask db migrate # To detect automatically all the changes.
$ flask db upgrade # To apply all the changes.
You can find more info at the documentation https://flask-migrate.readthedocs.io/en/latest/
I had to delete some of my migration files for some reason. Not sure why. But that fixed the problem, kind of.
One issue is that the database ends up getting updated properly, with all the new tables, etc, but the migration files themselves don't show any changes when I use automigrate.
If someone has a better solution, please let me know, as right now my solution is kind of hacky.
I did too run into different heads and I wanted to change one of the fields from string to integer, so first run:
$ flask db stamp head # to make the current the same
$ flask db migrate
$ flask db upgrade
It's solved now!
To fix this error, delete the latest migration file ( a python file) then try to perform a migration afresh.
This can also happen if you, like myself, have just started a new project and you are using in-memory SQLite database (sqlite:///:memory:). If you apply a migration on such a database, obviously the next time you want to say auto-generate a revision, the database will still be in its original state (empty), so alembic will be complaining that the target database is not up to date. The solution is to switch to a persisted database.
I also had the same problem input with flask db migrate
I used
flask db stamp head
and then
flask db migrate
Try to drop all tables before execute the db upgrade command.
To solve this, I drop(delete) the tables in migration and run these commands
flask db migrate
and
flask db upgrade
Related
According to Alembic's docs, the migration algorithm is trying to calculate the migration "path" to target revision from the version it finds in the alembic_version table. Upon checking that in the db of the service I'm working with, I found out that the current version that Alembic operates from is not in the version history, i.e. there is no migration script in projects migrations folder with that revision ID.
It seems that for this reason, when I added a manually written migration script (with empty template from alembic revision), calling alembic upgrade with this revision's ID failed silently without carrying out the migration. I got an output ending with
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
and exit code 1.
However, when I changed the value in alembic_version table to the penultimate version ID (the one revised by my own script), alembic upgrade head worked.
It is unclear whether or not the database actually was in the state corresponding with the revision ID I set manually. The script I migrated with only creates a table like this:
...
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql as pg
# revision identifiers, used by Alembic.
revision = '09d841dcd114'
down_revision = '450e39fe439d'
branch_labels = None
depends_on = None
def upgrade():
op.create_table('table_name',
sa.Column('id', pg.UUID(), autoincrement=False, nullable=False),
sa.Column('user_id', sa.INTEGER(), autoincrement=False, nullable=False),
...
sa.PrimaryKeyConstraint('id'),
sa.ForeignKeyConstraint(['user_id'], ['other_schema.users.id'], name='schemaname_tablename_user_id_fkey'),
schema='schema_name'
)
...
so it's partly reliant on existing database state, but does not determine the state exacly.
So my question is: is it normally possible that the alembic_version in db is not among existing revisions? Was that the reason why my previous attempts to migrate the db failed silently? What would be the correct solution for this problem? Setting the alembic version manually feels like an improper way to deal with this.
The alembic commands here were mostly given using Flask-Migrate, but it seems to not affect anything in this context.
There was a hotfix revision not merged into the main branch, but used on the service. That's where the missing version was
Given a team composed of data scientists and developers.
Developers want to use schema.prisma but data scientists don't and want to freely edit the DB directly...
What happen if a data scientist alter the DB directly? Will prisma migrate dev/deploy continue to work correctly?
If the data scientists alter the database directly then there would be a drift between the prisma schema and the database schema.
So next time when someone invokes prisma migrate dev command, prisma would prompt you to reset the database.
From Documentation:
The migrate dev command will prompt you to reset the database in the
following scenarios:
Migration history conflicts caused by modified or missing migrations
The database schema has drifted away from the end-state of the migration history
It is advisable to stick to only one approach, either updating database only through migrations or just directly updating the database, combining both would lead to an undesirable state.
In addition to #Nurul Sundarani's answer, here is what I did:
$ vim .env # Points `DATABASE_URL` to the data-scientists' DB
$ npx prisma db pull # Pull changes from it and update `schema.prisma`
$ vim .env # Points `DATABASE_URL` back to localhost
$ npx prisma migrate dev # Generate the migration and apply changes tolocal DB
Name of your migration: my_sneaky_new_migration
$ git add prisma/schema.prisma prisma/migrations/20220603XXXXX_my_sneaky_new_migration
$ git commit
$ vim .env # Points `DATABASE_URL` to the data-scientists' DB
$ npx prisma migrate resolve --applied "my_sneaky_new_migration" # Mark the migration as already applied
$ vim .env # Points `DATABASE_URL` back to localhost
I have a PostgreSQL db that is used by a Nest.Js / Prisma app.
We changed the name of a field in the Prisma schema and added a new field.
Now, when we want to update the PostreSQL structure, I'm running, as suggested by Prisma, the following commands:
npx prisma generate
and then
npx prisma migrate dev --name textSettings-added --create-only
The idea is to use the --create-only flag to review the migration before it is actually made.
However, when I run it I get a list of the changes to be made to the DB and the following message:
We need to reset the PostgreSQL database "my_database" at "my_db_name#cluster.us-east-1.rds.amazonaws.com:5432".
Do you want to continue? All data will be lost.
Of course I choose not to continue, because I don't want to lose the data. Upon inspection I see that the migration file actually contains DROP TABLE for the tables that I simply wanted to modify. How to avoid that?
So how do I run the update without affecting the data?
UPDATE:
I saw that running with --create-only creates a migration which can then be implemented on the DB level using prisma migrate dev, however, in that migration file there are still some commands that drop my previous tables because of some new parameters inside. How can I run prisma migration without deleting my PostgreSQL data?
UPDATE 2:
I don't want Prisma to drop my tables when I just updated them. The migration file generated, however, drops them and then alters them. Do you know what's the best procedure to avoid this drop? I saw somewhere I could first manually update the DB with the new options and then run the migration, so Prisma can find a way to update it, but that seems too manual to me... Maybe some other ideas?
For some cases like renaming tables or columns, Prisma's generated migration files need to be updated if they already contain data.
If that applies to your use case, Prisma's docs suggest to:
Make updates to the prisma schema
Create migration file without applying it (--create-only flag)
Update the migration script to remove the drops and instead write your custom query (e.g. RENAME <table_name> TO <new_name>)
Save and apply the migration (npx prisma migrate dev)
Note that those changes can lead to downtime (renaming a field or model), for which they have outlined the expand and contract pattern.
It might be a Prisma bug: https://github.com/prisma/prisma/issues/8053
I also recently had this situation. It probably should not try to run migration if you only want to create migration file.
But overall it is expected with Prisma to recreate your db sometimes. If you migration is breaking then it will be required to reset the data anyway when you apply it.
I suggest you to create some seeding script so you could consistently re-create the database state, it's very useful for your development environment.
More info
I have made some recent changes to my models in my Flask project. I tried to apply these changes to my Postgres DB, but the script doesn't seem to have any effect. When I run the upgrade it says
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> ba60ca569e9f, empty message
but nothing changes in the DB. I dropped the database and recreated it and still nothing happened. What is going wrong?
Context impl SQLiteImpl. is a strong hint. My DB URI is determined by SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URI') or 'sqlite:///'. When I ran my project in my Docker-compose environment it worked because the DATABASE_URI was getting set correctly in a Dockerfile. When I ran it on my local environment it was not working. I could have run it on my server container and it should have worked.
I fixed this by correctly setting my DATABASE_URI to export DATABASE_URI=postgres://{USERNAME}:{PASSWORD}#127.0.0.1:5432/debateit. This let my local environment connect to the Postgres DB rather than the local SQLite.
When I try
alembic upgrade head
Alembic runs the previous migration script, obviously raising an error because my schema has changed.
In my database, I have version_num set to 48957fdfe8d5. After running
alembic revision -m '<my message>'
Alembic created the new script file—the one I want to run—with this at the top
revision = '28cc06993b73'
down_revision = '4d5f9ba76c5e'
In other words, everything looks good. So why is it clearly running the code in 4d5f9ba76c5e rather than 28cc06993b73? I have also tried
alembic upgrade 28cc06993b73
But it still runs the code in 4d5f9ba76c5e. Here is that log:
$ alembic upgrade 28cc06993b73
INFO [alembic.migration] Context impl MySQLImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
Starting in DEBUG mode
INFO [alembic.migration] Running upgrade 48957fdfe8d5 -> 4d5f9ba76c5e, Breaking up metadata into required and optional
Also, if I check Alembic's history, I see that head is on 28cc06993b73:
$ alembic history
Starting in DEBUG mode
4d5f9ba76c5e -> 28cc06993b73 (head), creating soft file table
48957fdfe8d5 -> 4d5f9ba76c5e, Breaking up metadata into required and optional
<base> -> 48957fdfe8d5, Init
Thanks in advance.
Well firstly you said that your version number is 48957fdfe8d5 in the db so an upgrade from 48957fdfe8d5 -> 4d5f9ba76c5e is expected based on your alembic history.
You also say "Alembic runs the previous migration script, obviously raising an error because my schema has changed." What error is it showing? Did you manually change the db so that it reflects what is supposed to happen in 28cc06993b73? If that is the case then you could run
alembic stamp head
to get your db in sync with alembic migrations