I'm looking to amend some tables I have in a PostgreSQL instance on Azure, but I cannot work out how to perform the upgrades with Alembic.
I have been following the tutorial here, which includes a Heroku deployment around the 12:01:00 mark. In that case, once the changes have been defined, we can run heroku run "alembic upgrade head" to perform the upgrade. However, I cannot find the equivelant process for Azure.
My postgres instance is housed in a VNet and connected to a web app. Until now, I've made code changes to a server which is running in an attached web app. I push to GitHub which then deploys the changes in Azure. Obviously, if the table already exists in postgres, changes I make to the original schema are not reflected. I considered deleting the table and stating again, but this seems a very risky strategy.
A similar question was asked here, but has remained unanswered. I've also checked the documentation for Alembic and Azure but could not find anything.
Related
In Heroku, connected to Git. I want to deploy my Dev branch, and can select it.
When I manually deploy it does it's thing (deploys my website to Heroku). But my website has Master branch code. I go back to Heroku and it's on Master.
If I select Dev as the branch for either Manual or Automatic, then reload the page, it switches back to Master. Below is a screenshot of me setting the branch to dev. If I do a browser refresh, it resets to Master.
I tried reconnecting Github. Not sure what else it could be.
Deploying Dev was working up until yesterday.
Here is a screenshot of how I manually deploy (as opposed to auto deploy) from the Heroku Deployment tab.
Edit: I should also add, I happily was on Dev, and could deploy Dev updates up until recently. I deployed Master by mistake, bat can't go back to Dev.
I ended up having a corrupt Collection / DB record. I was tipped on another forum that the symptoms I was seeing (Nighscout web app not displaying some data, not the Heroku deploy I was attempting to work around that issue) could be caused by that. So as a last resort I dropped the entire Mongo Collection and I can now deploy Master and Dev, and it sticks in Heroku.
I don't know the significance since the data should be separate from the web app source code itself.
The whole reason I wanted to try Dev was for a fix for parts of the app not working. After initialising the Mongo DB Collection, I can use Master, so Dev (and the fix it contained) is not needed.
I know this isn't the exact root cause, but I'll leave this here in case someone comes across it and hasn't thought to look at the data.
I'm working on a web project(built with the .Net framework) on a remote windows server, and this project is connected to a database my SQL server management studio, now on multiple other remote windows servers exist the same web project linked to the same database, now I change a page's code in my project or add/remove a table or stored procedure in my database, is there a way(or an already existing software) which will my to deploy the changes that I made to all the others(or to choose multiple servers if I don't want to deploy the changes to all of them)?
If it were me, I would stand up a git server somewhere (cloud or local vm), make a branch called something like Prod or Stable, and create a script (powershell if the servers are windows, bash on anything else) on a nightly or hourly job to pull from that branch. Only push to that branch after testing thoroughly. If your code requires compilation, you have the choice to compile once before committing (in which case you're probably going to commit to releases), or on each endpoint after the pull. I would have the script that does the pull also compile and restart the service (only if there was something new in the pull).
You can probably achieve this by following two things :
Create a separate publishing profile for each server.
Use git/vsts branches to keep the code separate. (as suggested by #memtha).
Let's say you have total 6 servers and two branches A and B. So, you'll have to create 6 publishing profiles. Then, you can choose which branch to deploy where. e.g. you can deploy branch B on server 1,3 and 4.
For the codebase you could use Git Hooks.
https://gist.github.com/noelboss/3fe13927025b89757f8fb12e9066f2fa
And for the database, maybe you could use migrations or something similar. You will need to provide more info about your database, do you store your database across multiple servers etc.
If the same web project is connecting to the same database and the database changes, I suspect you would need to update all the web apps to ensure the database changes don't break any of the apps and to keep all the apps updated to prevent any being left behind.
You should look at using Azure Devops to build and deploy your apps and update the database.
If you use Entity Framework, you can run the migrations on startup and have the application update the database when deployed manually or automatically using devops.
To maintain the software updated in multiple server you could use Git with hooks, post-receive hook is what you need.
The idea is to use one server as your Remote Repository and here configure the post-receive hook to update the codebase in the same server and the others.
I want to create one click deployment on azure pipelines to move Postgres Sql changes from dev to QA environment,similar to what we implement using SQL Server Database project where a Powershell script deploy the changes to the remote server.
I have tried pg_dump and psql commands which will create dump file and restore it on the remote server. It does not perform diffing ie(comparing database changes on source and destination , and only replicating the missing changes)
You've stumbled upon one of the features lacking in the Postgres ecosystem. One of the more elegant ways to solve migrations using Postgres' own tooling is to package up your migrations as a Postgres Extension. This requires you to generate the deployment scripts yourself, but it is a neat way of applying and packaging up the deployments.
There are a number of commercial tools that will assist in this process, such as Datical, Liquibase, and Flyway. Note, some of these still require you to generate the change statements yourself, some attempt to create them for you.
Generating change statements is a whole different animal and I recommend you look at schema diffing tools for Postgres to find what best suites your needs.
I have been trying to get a staging server working on Heroku where my product app runs. I thought the issue was related to a mis-configured database. There seemed to be some odd things visible in the Heroku dashboard where it showed multiple instances of Postgresql add-ons, with different names. On the production implementation of the database, it said it was also attached to my staging app - which did not seem right - so I deleted that (I want my staging app to have a clean DB and not be attached to production).
BUT... now it appears my production database is GONE. I have a support ticket in to Heroku. But would LOVE some help - I am pretty desperate at this point!
I have gone into the Activity menu for my Heroku app and rolled back to a point before the screwup - but I am pretty sure that just restores code & settings. So I am still dead as the database is not there.
To make matters worse, I quickly checked pg:backups and the list it returns is quite old. That doesn't seem right - I think I should have automatic backups - so that adds to the fear that I may have royally screwed this up.
I expected the database to detach from my staging app but remain intact for my production app. Now all that is showing is the "old" version of the database.
UPDATE: I was able to promote an instance of an older database that was still attached to my app, so the app itself is up and running (but with older data). Still waiting on Heroku support to respond to my trouble ticket and hopefully help me restore this database.
RESOLUTION: Heroku tech support responded overnight and was able to recover/restore my production database as hoped, so only minor damage done with users unable to access recent data for about 6 hours. So 2 learnings from this.
First, and fairly obvious, is to be very careful with how you manage your production DB and make sure you have regularly scheduled backups occuring (I thought I did, but after an automatic Postgres upgrade it appears those went away and I did not know to re-schedule them).
Second, that Heroku does continuously backup your app/data and can help you recover from this kind of disaster as long as you are on one of their paid plans. But you have to contact them through tech support and request this, there is no way to recover from these automatic backups via the dashboard or CLI.
I've spent the past few months developing a webApi solution that I'm ready to push up to Azure and hook into an Azure SQL Database. It was built with EF Code First.
I'm wondering what standard approaches there are to making changes to the database while in production. I've been using database initializers up to this point but they all blow away data and re-seed.
I have a feeling this question is too broad for a concise answer, so I'd like to ask: what terminology / processes / resources should a developer look into when designing a continuous integration workflow for a solution built with EF Code First and ASP.NET WebAPI, hosted as an Azure Service and hooked up to Azure SQL?
On the subject of database migration, there was an interesting article on ASP.NET about this subject: Strategies for Database Development and Deployment.
Also since you are using EF Code First you will be able to use Code First Migrations here for database changes. This will allow you to better manage the changes you make to the database.
I'm not sure how far you want to go with continuous integration but since you are using Azure it might be worth it to have a look at Continuous delivery to Windows Azure by using Team Foundation Service. Although it relies on TFS in the cloud it's of course also possible to configure it with for example Jenkins. However this does require a bit more work.
I use this technic:
1- Create a clone database for your development environment if it doesn't exist.
2- Make the necessary changes in your dev environment and dev
database.
3- Deploy to your staging environment.
4- If you added some static datas
that should also exist in your prod database, use a tool like
SQLDataExaminer to find the data differences and execute the
insert, update, deletes for according rows. Use Schema Compare in VS2012 to find differences between your dev
and prod environment by selecting source as dev and target as prod.
And execute the script in your prod.
5- Swap the environments