I'm developing a small NodeJS application, which uses data from a remote PostgreSQL database.
Everything works, but when scaling the lookup and round-trip time to/from the PostgreSQL are a hurdle.
I would like to implement a local cache of my table/view from PostgreSQL.
I've thought of using an In-Memory database like SQLite, and synchronize it on-the-go if something changes in PostgreSQL.
But... it seems a lot development and a bit cumbersome.
Are there more simple and straight-forward solutions?
Related
Is it possible to have a real time sync between a heavy Oracle database and mongodb? Has any one tried this?
I saw a site - Keep MongoDB and Oracle in sync
Here they have mentioned having triggers on the oracle tables, my doubt is that will this slow the applications already running on the Oracle database. Will this replication cause the applications to slow down or bring down the oracle database's performance?
The right solution would involve Change Data Capture from Oracle. This does not require triggers on oracle and thus won't effect performance. There are several tools you can use such as Striim and Attunity. Striim supports change data capture from Oracle and writing to MongoDB.
https://striim.com
https://attunity.com
I am working on a project using Ionic for the mobile side, I have a web app as well linked to a Cassandra database.
I need data synchronization between the mobile device (local storage) and the server-hosted Cassandra database. I use the cassandra-driver to connect to the database but then I realize how problematic it is to convert the data to an other type of database (SQLite for example).
Should I rather use an other database than Cassandra to make the synchronization easier ? (I need a noSQL solution)
Choice of the database depends on type of data you want to store. Cassandra is a column oriented database. It has great performance when you have to deal with large amount of data, but has many limitations related to the queries you need in order to pull data. For that reason, it might require additional efforts to develop something that you could easily do with some other database. So, the real question is do you really need Cassandra.
If you are using it only for mobile application, I don't think you will have so much data to exploit Cassandra benefits.
In your place, I would rather consider some other databases, such as MongoDB in case JSON is appropriate format for your data or Redis if you data is key/value pairs.
I would like to give a web app with a PostgreSQL database 100% offline functionality. In an ideal case the database should be completely replicated in the browser per user, and synchronized when online. So that the same code can be used to talk to both the offline and online database. I know this is possible with PouchDB and CouchDB, but have not found a solution that works with PostgreSQL. Is this at all possible?
Short answer: I don't know of anything like this that currently exists.
However, in theory, this could be made to work...(long answer:)
Write a PostgreSQL backend for levelup (one exists for MySQL: https://github.com/kesla/mysqldown)
Wire up pouch-server to read/write from your PostgreSQL db using pouchdb's existing leveldb adapter (which in turn will have to be configured to use your postgres backend). Congrats, you can now sync data using PouchDB!
Whether an approach like this is practical in reality for your application is a different question you'll have to answer.
You may be wondering, for example, "will I be able to sync an existing complex schema with multiple tables to the client with this approach?" The answer is probably not - the mysqldown implementation of leveldown uses a single MySQL table with three fields: id, key, and value (source), and I imagine any general-purpose PostgreSQL adapter would be similar (nothing says you can't do a special-purpose adapter just for your app though!).
On the other hand, if you were to implement a couchdb-compatible API (or a subset- you may not need attachments, for example) over your existing database schema, there's nothing stopping you from using PouchDB on the client to talk directly to that as if it were an actual CouchDB - just pop in the URL and call replicate()! Implementing the replication protocol might be a fair bit of work, since you'd need to track revisions and so on somewhere - but again, technically not impossible!
There are also implementations of levelup's backend storage that are designed for browsers. See level.js, which could be another way to sync between a server-side Postgres levelup backend and the browser.
TL;DR: There's tons of work being done around Javascript databases right now. Is syncing with Postgres impossible? probably not. Would it be a lot of work? Definitely. Worth it? Who knows, but it would be cool.
Without installing PostgreSQL on the client? No. Obviously you can cache data for offline use, but an entire RDBMS+procedural languages in Javscript, no.
Can you store an sqlite database remotely on a server? I am building an iphone app and all the tutorials seem to store the sqlite database on the iphone itself...
That is not advisable, as you will quickly run into concurrency problems. I would use a traditional RDMS such as MySQL or PostegreSQL in such a case.
SQLite has a useful page explaining when to use it. It says:
If you have many client programs
accessing a common database over a
network, you should consider using a
client/server database engine instead
of SQLite.
I agree with Matthew Flaschen, but if you really want to, you can connect to a sqlite db throuth an ssh tunnel. If multiple clients prefer mysql postgresql.
It's obvious that I'm not an expert on Cassandra. So the question may sound silly.
Given an existing SQL-based project does it give any benefit or is it even possible to apply a no-SQL database(e.g. Cassandra) as an additional layer between business logic and SQL database to speed up our queries or inserts.
It's relatively new technology and I'm trying to find its place.
Cassandra will work fine, but if you don't care if you have to rebuild your data memcached will be faster.
But if you want a persistent cache, Cassandra is probably your best option -- reddit started by using Cassandra like this and is working on moving more functionality to it.
I would go with Windows Server AppFabric aka Velocity distributed cache with SQL Server, assuming you are on the .NET platform.
Scott Hanselman has a bunch of posts on AppFabric.