I have multiple desktop apps that can run DbUp at startup. And how to handle this situation if couple apps run DbUp at the same time?
I have to have automatic solution for that because there are plenty clients and DB update cannot be run by the DBA. It must be work automatically after the app update (actually it is also automatic).
DB update could be executed during the application update. But also it doesn't solve the concurrency problem because app update also could happen simultaneously. DbUp doesn't have any lock mechanism instead of transaction. I've added some lock table and added script to update value in it at every update script begin. But there is a problem that script continue to run after other DbUp instance finishes the job.
It would work if the lock table update run before schemaversion reading. Then after the lock table release it would read schemaversion table and could then realize that there are all updates already executed.
Application .Net Core based and DB is MySql.
Related
I'm using MongoDB as my database, and as a first-time back-end developer the ease with which I can delete an entire database/collection really bothers me.
Simply typing db.collection.remove() removes all records from that collection!
I know that an effective backup strategy should render this a non-issue, but I occasionally do run .remove() on some collections, and I'd hate to type in the wrong collection name by accident and (a) have to go through a backup restore, and (b) lose whatever data I had gathered between the backup and the restore, especially as my app gathers a lot of user data.
Is there any 'safeguard' I can set up my database to use, even if it's just a warning/confirmation that says
"Yo, are you sure you want to remove everything from <collectionname>? Choose: Yes/No"
User roles won't fix your problem. If your account has permissions to delete one user, you could accidentally delete them all. If your account has permissions to update an attribute for one user, you could accidentally update all of your users.
There's a simple fix for this however.
Step 0: Backup your database. And test your backups regularly. And make sure you get alerted if the backup did not run, or errored. Replica sets are not backups. I know this is obvious, but evidentally it's not obvious to everybody.
Step 1: Write a web admin GUI interface for your database. This it will only take a day or two -- and it should be simple enough that a secretary or intern could use it without fear for your data. (If you think this will take a long time, find a framework with more bells and whistles. Your admin console doesn't even need to be written in the same language as your app.)
Step 2: Data migrations (maintenance transformations of your database) should always be run from scripts checked into source control and tested on non-prod beforehand. The script could be as simple as mongo -e "foo.update(blah)", but you should run it as a script to avoid cut-n-paste errors. Ideally, you would even have a checklist for all migrations. (Check that you have a recent backup. Check the database log and system load beforehand. Write a before and after query that will tell you if the migration was successful...)
Step 3: You now no longer need to use the production Mongo console. So don't. It's a useful tool for development, but that's only needed on local development databases.
The above-mentioned Roles might be useful for read-only queries. But you can already do that against the non-master replica set member.
tl;dr: You can go pretty far using cowboy admin techniques, but eventually you're going to figure out that it's better (and not much more work) to automate everything.
There is nothing you can do in the current version to provide this functionality.
In a future version when user defined roles are available you could define a role which allows insert() and update() but not remove() or drop() etc. and therefore make yourself log-in as a different higher-role user, but that's not available in the current (2.4) version.
I am creating an application with sqlite. I am performing all kind of task on the database Insert, Update, Delete, Select.
For that I open the database every time, Then execute my query using sqlite3_step() and after the result I use sqlite3_finalize() and sqlite3_close() methods. It is working well in most cases. I am not getting when its happening but some times my database gets locked with the same process I follow and some time it works.
I need to unlock database so even in any case my database get locked then I can unlock it or Plz guide me if I can check by code that my database is locked so I can replace my database with the resource database.
I am using webservice too so I don't have issue about data loss.
Is it make sense if I replace my database if it get locked or if there is any way to unlock the database.
Open database once in the beginning. And close it in the end in AppWillterminate function. You are only consuming time by opening and closing it in every database function.
As far as database lock is concerned, it gets locked when some application is still using it and other application is trying to get its access.
This could be your app, or possibly the sqlite manager add-on of your firefox.
I faced same problem once and what i did was
Disabled the option in sqlite add-on where it remembers the previously opened database.
Restart xcode, simulator.
Make a copy of the sqlite file (desktop), delete it from the project and then add in project again from desktop.
The last solution sounds weird, but i was mad that time.
I hope this could help you.
The other day a friend suggested to play a web browser game called OGame. If you don't know it I'll tell you what it is:an rts game where you have to build things like mining factories, barracks and so on. The interesting thing that every building has a build time and you can log off while it's building because it will keep going.
Something like this I would believe is managed via dbms. I have my records where I have the end time of a costruction. How do I check when to update a building? Do I need an external application that checks every seconds what record needs to be updated? Is it possible with mysql5 to have an internal scheduler that launches a procedure on this table? And if so, is it a best practice?
I have built a similar game and I stored the construction end times (and other events to be fired) in an events table. I wrote a PHP daemon which regularly checks the events table for expired records and acts on them accordingly.
I couldn't find a way to do it in the database itself (and if I later wanted to migrate to another DB it would need rewriting). A cron'd script may overlap. A daemon can keep track of everything all the time, and output debug information if events are queuing faster than they're being processed. I also added a cron to check periodically that my daemon is still running, otherwise start it.
Creating a daemon in PHP (if you're using PHP)
Hope that helps.
Short version
If my process is terminated in the middle of a transaction, or while SQLite is committing a transaction, what are the chances that the database file will be corrupted?
Long version
My application uses an SQLite database for storage (directly, not via Core Data). I'm working on a new version of the application which will require an update to the database schema. On launch, the app will check the database and, if it needs updating, execute a series of SQL statements to do so.
Depending on the amount of data in the database, the update may be long running (on the order of seconds), so I need to consider the possibility that the process may be terminated before the update is completed. (For context, this on an iPhone, where the processor is slow and the app may be terminated by an incoming phone call.) I will, of course, wrap the upgrade SQL statements in a transaction. Will that be enough to guarantee that the database will not be corrupted?
I'm assuming that transactions work as advertised, and that if the process is terminated in the middle of the transaction, the file will be OK. But I'm also assuming there is a window of time during the COMMIT where something can go wrong.
To play it safe, I could create a backup copy of the database file before starting the update, but if the transactions are safe then that would be overkill. It would also make the update process take longer, which increases the chance it would be interrupted, and then I'd have to consider that the file copy operation might be interrupted... I'd like to keep the code as simple as possible (but no simpler).
In the course of researching this question I've started reading "Atomic Commit In SQLite", which is more detail than I probably need to know, but is giving me faith that I don't need to second-guess SQLite's ability to protect the database file. But I'd still like to hear from Stack Overflow: is a transaction good enough, or should I be more cautious?
I have read the Atomic Commit in SQLite document. It may not be overkill if you really want to understand what's going on, but in a nutshell, a transaction goes like this:
Lock the database file
Create the rollback journal
Determine what portions of the database file are going to be changing
Write copies of those pages to the journal file
Write the journal file header
Write your intended changes to the database file
Delete the rollback journal (THIS IS THE COMMIT)
When the user is done talking to mom and re-starts your app, when it tries to open the database file, if there is a rollback journal present, it will write the original data back to the datafile using a similarly safe process. Even if you lose your transaction, and lose a rollback, it will eventually be taken care of once mom's nervous breakdown is properly thwarted and he can run the app for more than a couple seconds at a time.
If it were me, I would trust the transactions. With so many users of SQLite, even in embedded apps, I think transaction commit failures would be a very hot topic all over the net if they weren't working properly.
Are you using CoreData with a SQLite backend? If so, I actually find that the best way to handle this problem is to create two separate NSManagedObjectContexts (a read-only and an editing). When the process completes, just save the "editing" context and then the two contexts will be in sync. If something happens during your operation, the editing context won't get saved, so you'll be fine.
i know this is an often asked question on these boards. And usually the question has been about how to manage the changes being made to the database before you even get around to deploy them.Mostly the answer has been to script the database and save it under sourcecontrol and then any additional updates are saved as scripts under version control too.(ex. Tool to upgrade SQL Express database after deployment)
my question is when is it best to apply the database updates , in the installer or when the new version first runs and connects to the database? note this is a WinApp that is deployed to customers each have their own databases.
One thing to add to the script: Back up the database (or at least the tables you're changing!) before applying the changes.
As a user I think I'd prefer it happens during the install, and going a little further that the installer can roll itself back in the event of a failure. My thinking here is that if I am installing an update, I'd like to know when the update is done that it actually is done and has succeeded. I don't want a message coming up the next time I run it informing me that something failed and I've potentially lost all my data. I would assume that a system admin would probably also appreciate install time feedback (of course, that doesn't matter if your web app isn't something that will be installed on a network). Also, as ראובן said, backing up the database would be a nice convenience.
You haven't said much about the architecture of the application, but since an installer is involved I assume it's a client/server application.
If you have a server installer, that's where you want to put it, since the database structure is only going to change once. Since the client installers are going to need to know about the change, it would be nice to have a way to detect the database version change, and for the old client to be able to download the client update from the server automatically and apply it.
If you only have a client installer, I still think it's better to put it there (maybe as a custom action that fires off the executable for updating the database). But it really isn't going to matter, because conceptually one installer or first-time user of the new version is going to have to fire off the changes to the database anyway. The database changes are going to put structural locks on the database so, in practical terms, everyone is going to have to be kicked off the system at that time for the database update to be applied.
Of course, this is all BS if it's not client-server.