I'm looking for a flutter web compatible solution for NoSQL data storage/retrieval such that multiple users can save/fetch data - with the catch it needs to be hosted locally with no 3rd party tools. Firebase/Firestore is 100% perfect except for the caveat data is stored remotely.
So ideally when any user launches the app, the app will parse a locally stored JSON/NoSQL-like DB (from a SharePoint directory sigh), and their edits/saves/uploads will be written back. I'm not too worried about data-access collisions at this time. What solutions do I have? Thanks!
Related
I want to add the feature of database synchronization to my project. It should be like this:
When the internet connection is available the application should transact with the online database server.
When the internet connection is not available it should keep track of the transactions and should update the local DB to online DB server when connection is available it should synchronize.
How can I do this?
Before that we are using Multi Tenant concept in our web application. Single online DB can access all clients using multiple schema.
I want to know how can my web application work in offline mode. If I use HTML5 application cache concept means the HTML and CSS file will load from cookies. How can I get the database value from database when application goes offline.
What is a good way to make my application work online and offline with database?
Most modern browsers have databases or persistence facilities similar to a database for pretty much exactly this use case.
You may end up having to dive pretty deep into javascript to get it to work though. (Especially when the online DB comes back online and you have to go back through all the offline data and send it up (post) to the server end)
THE PROBLEM
I'm writing a mobile app which will allow a user to log in, save some preferences that must be stored in a database, and display congressional bills to the user.
I've only written simple RESTful services with PHP and MySQL in the past. I'd like to take advantage of newer technologies, and am a little lost on general direction.
The bill data (formatted as JSON) can be gathered by running the scrapers found here. Using docker, I managed to set a working directory and download the files on my local machine.
I've designed a MySQL database for holding the relevant bill and user data.
I started to mess around in Google Cloud Platform, and read the doc that describes different models. I'm thinking of a few different ideas, but aren't familiar with GCP or what I can actually accomplish.
QUESTIONS
1) What are App Engine, Compute Engine, and Container Engine each for? I get the gist that Container Engine holds different instances of stuff you load up with docker, and that Compute Engine sets up a VM, but I don't really understand the relationships. How should I think of them?
2) When I run those scrapers from the shell, where are the files being stored, and how can I check on them? On my computer, I set a working directory, but how do directories work in GCP? Is it just a directory in the currently selected VM, or is this what Buckets are for?
IDEAS
1) Since my bill data already comes as JSON, should I skip the entire process of building a database for the bills and insert them into Firebase somehow? Is this even possible? If so, am I stuck using Firebase's NoSQL, or can I still set up a relational database?
2) I could schedule the scrapers to run periodically, detect new files, and run a script to parse the JSON and insert new bill data into my a database (PostgrSQL?/MySQL?). Then I would write an API.
3) Download the JSON files to a bucket, and write an API that reads from them. Not sure how the performance would compare to using a DB.
I'm open to other suggestions as well.
For your use case (stateless web application), App Engine is probably your best choice. The Google documentation has severalcomparisons of your computing options
You can use App Engine with PHP and cloud-hosted MySQL if you want, which could be a good way to get your toes wet without going in over your head.
We are working on an application that will be offered both as a web-based and as a cross-platform desktop solution by means of Electron.
Due to customer requirements, the desktop client cannot make use of "the cloud" to store data; all data should be stored in the local machine or, even better, the user should have the option to keep the database/data file on an external HDD so that another user on the same local network can use the same data file.
We've been looking at NeDB, PouchDB, etc, but all these use either Web SQL or IndexedDB on the browser itself to store the data.
NeDB can theoretically use the file system but that seems only possible for Node Webkit apps.
Another option is of course MongoDB, but it requires setting up a site on a web server. Seeing as how our users will set that up in on their own machines, that will work for one user only but would make it very hard for them to share the data (note: assume users with little technical know-how).
Is there a way to force NeDB to persist data in a file instead of the in-browser database?
Alternatively, does any one know of a file-based, compact database that plays well with electron/node?
We'd preferably like to use a NoSQL database, but options of file-based SQL databases will be considered as well.
I have some experience with NeDB in an Electron app and I can say it will definitely work on the filesystem.
How are you initializing NeDB (or whatever your database choice is)? Also, are you initializing it in the main or renderer process? If you can share that, I think we could trace the issue to a configuration issue.
This is how you start NeDB with a persistent data-store that saves to disk.
var Datastore = require('nedb')
, db = new Datastore({ filename: 'path/to/datafile', autoload: true });
I think MongoDB is going to be overkill for an Electron app (it's meant to be really a high performance, distributed database running in the cloud).
Another option you could consider is LevelDB (a key/value store that can persist to the filesystem) which is popular in the node community. (EDIT 4/17/17 IndexedDB uses LevelDB underneath the hood, so if you go that route, may as well just use that)
One aspect I would definitely evaluate carefully is: How difficult is this database going to be to package and distribute? How do I integrate it into my build system? Level and NeDB can be included simply via npm install and any native code compiling is handled seamlessly with node-gyp, which is as simple as it gets. However, bundling Mongo, for example, will require some work to get a working build for each different platform.
I am writing a Chrome Packaged App that uses the IndexedDB for data storage. Chrome allows me to view the contents of the database, but I can't find any way to manually change the data. I need to update this data from time to time because, you know, I'm still writing the app. Any idea how to manually change the data in the database?
Any changes to the IndexedDB database have to be performed via the IndexedDB API. There are no utilities, data editors, query apps, loaders, importers, or any other kind of external utility, such as there is for MySQL, SQLite, Oracle, or any other such database.
Furthermore, it's not even theoretically possible to write such a utility, because an IndexedDB database is sandboxed inside a single app, and no other app can access it.
What I do is incorporate the needed update forms and commands (delete database, create database, count rows, etc.) as modules inside the app, perhaps accessible from a Maintenance or Admin menu item. Obviously, this is a lot of work, but there is no other way if you're using IndexedDB.
In addition, I have a "load database" menu item that loads it from JSON in an external file. I do that from time to time when I want the app to have some initial data, or test data. But, this is just an example of what I said in the first sentence, above.
HTML5 Storage Manager All in One folks promise they'll have indexedDB support soon.
They use some tricks to open extension window inside the same domain as debugged page, thus making indexedDb accessible.
Doesn't work successfully at the time of this writing, though.
I've been ramping up on Azure Mobile Services over the past week. There are definitely some PROs and CONs in using them over a standard Azure Web Site where I can write APIs that hit SQL DB, etc.
One of the biggest negatives I see is developing the server side code and DB structures ON THE SERVER. I've watched lots of videos from launch and beyond, read lots of blog posts about tips and tricks around WAMS, but nobody seems to talk about the downside of developing the code (server scripts) and database structures on the server, at your live URL.
This is all great for developing the first version of your mobile app and associated mobile services. But once it's all deployed, how do you ever build version 2? Real apps hitting real APIs and data, but now I want to develop/change/play with the server scripts and database schema?
With Azure Web Sites, I can develop locally and only publish code and DB changes to the server on my schedule.
Have any of you seen or heard of the "v2 development story" around Azure Mobile Services?
Only thing I can think of would be to create another set of tables and APIs around them, most likely "virtual tables" that allow me to write APIs against the original set of data. Seems like a huge hassle, since the client code would now have to know about the original set of tables and the new set of tables... that's only for v2...
Thanks for any thoughts / insight.
You should have two services, one dev and one production and use scripts to promote your code from dev to production (pretty similar to how most workflows go, in moving from a test setup to a production one).
http://channel9.msdn.com/Events/Build/2013/3-511