When is it a good idea to use more than one DBMS? What are the possible repercussions, and how do you decide when to do so?
I'm currently building an application which runs an analysis on our users' websites and stores it. This allows me to analyze all the data and give them analytics.
Since the data collected from each site is static and varies greatly from site to site, CouchDB seemed like a great fit. But in order to create this system, I'd need to build a user account system which couch is quite horrible at (reserving names, emails, etc has all sorts of problems).
My first thought was to use MySQL to handle the user accounts and CouchDB for the massive amounts of data. Essentially, trying to use a hammer for a nail and a screwdriver for a screw.
Is this a time when more than one DBMS is a good idea?
I don't see anything wrong with using MySQL for users accounts and CouchDB for crawled information.
For the users, you might even consider something simpler, like GDBM
Related
So I am fairly new and I have a question about data storing. I know about many ways to store local data on individuals devices. But, what do people generally use when making a app that many users access that data that is created.
For example if User 1 creates a list of things, how can that data be stored so many users can look at that list and interact with that list?
Look forward to hearing about data storing answers and tips!
Thanks
You're going to need some sort of backend. I personally use AWS DynamoDB and Cognito to achieve this. There are many tutorials to help you get started but the documentation is a little outdated and may not make sense for beginners.
Nonetheless, if you want different users to access some sort of shared data, it will need to be stored somewhere that can be accessed by each user (a website, an online database, etc.)
You can use Firebase - extremely easy to use and if you are just displaying strings and integers or images (which most everything is) then it's really user friendly (in my opinion) and it's free as long as you're not hosting 10GB? (I think) or so of data (that's a lot of strings).
I am creating an inventory app currently for iPhone using Parse for companies to keep track of all of their tools, supplies, inventory. Now I'd like to allow for the user/company when adding a new item to their database for them to have the option to search from a pre-made database of items such as for a construction company when adding a simple Dewalt Drill Battery to their inventory would search the pre-made database for "Dewalt #DC9096 18V XRP 2.4A Battery" or an office would search for pencils by brand/serial number/name. I am looking for a simple way to make a database or even a table containing multiple brands products including their prices, product specifications, website for ordering more, company website, warranty phone number, etc... I have considered parsing all of the retail websites for information but don't know the legalities behind it and if the websites change then I'd need to update code. If there is ANY (easier/better) way to do this then assistance or direction would be great!
Thanks always
I would not go down the route of trying to parse websites, that will be a huge pain in the neck and impossible to maintain unless you have extensive resources (and as you mention it probably violates most site's terms of service anyway). Your best bet would be to hook into existing product databases via an API, such as Google's Search API for shopping, or maybe Amazon's API. Here's where you can start if you wanted to use Google:
https://developers.google.com/shopping-search/
Hopefully that gets you going in the right direction.
Edit: Here's a list of a lot more shopping APIs that could be good options:
http://www.programmableweb.com/apis/directory/1?apicat=Shopping
If you did find yourself needing to parse many different vendor websites (we'd call this "screen scraping") and you have the legal right to do so, you should use a tool like SelectorGadget to get your XPaths, it's much faster, easier and less error-prone than doing it by hand.
If you're doing more than a couple websites, though, you'll probably find that you'll have to update the scraping rules pretty often, it definitely won't be a set-and-forget operation.
I will be constructing an ecommerce site, and would like to use a no-sql database, which will fit well with the plans for the app. But when it comes to which database would fit the job, im not sure. After comparing various DB's, the ones that seem best might be either mongo, couch, or even orientdb. I have seen arguments for all of them to be used or not used compared to something like MySQL. But between themselves (nosql databases), which one would fit well with an ecommerce solution?
Note, for the use case, i wont be having thousands of transactions a second. Or similarly high write rates. they will be moderate sure, but at a level that any established database could handle.
CouchDB: Has master to master replication, which I could really use. If not, I will still have to implement the same functionality in code anyways. I need to be able to have a users database, sync with the mothership. (users will have their own, potentially localhost database, that could sync with the main domains server). Couch is also fast, once your queries have been stored in the db.As i will probably have a higher need for read performance. Though not by a lot.
MongoDB: queries are very easy and user friendly. Also, with the fact that end users may need to query for certain things at a given time that I may not be able to account for ahead of time, this seems like it may be a better fit. I dont have to pre-store my queries in the db. Does support atomic transactions, though only when writing to a single document at a time.
OrientDB: A graph database. much different that most people are used to, but with the needs, it could fit very well too. Orient has the benefits of being schemaless, as well as having support for ACID transactions. There is a lot of customer, and product relationships that a graph database could be great with. Orient also support master to master replication, similar to couchdb.
Dont get me wrong, I can see how to build this traditionally with something like MySQL, but the ease and simplicity of a nosql solution, is very attractive. Although, in my case, needing a schemaless solution, would be much easier in nosql rather than mysql. a given product may have more or less items, than another. and avoiding recreating a table whenever a new field is added, is preferrable.
So between these 3 (or even others you think may be better), what features in each could potentially work for, or against me in regards to an ecommerce based site, when dealing with customer transactions?
Edit: The reason I am not using an existing solution, is because with the integrated features I need, there are no solutions available out there. We are also aiming to use this as a full product for our company. There will be a handful of other integrations than just sales. It is also going to be working with a store's POS system.
Since e-commerce can encompass everything from shopping carts through to membership and recurring subscriptions, it is hard to guess exactly what requirements and complexity you are envisioning.
When constructing an e-commerce site, one of the early considerations should be investigating whether there is already an established e-commerce product or toolkit that could meet your requirements. There are many subtleties to processes like ordering, invoicing, payments, products, and customer relationships even when your use case appears to be straightforward. It may also be possible to separate your application into the catalog management aspects (possibly more custom) versus the billing (potentially third party, perhaps even via a hosted billing/payment API).
Another consideration should be who you are developing the e-commerce site for: is this to scratch your own itch, or for a client? Time, budget, and features for a custom build can be difficult to estimate and schedule .. and a niche choice of technology may make it difficult to find/hire additional development expertise.
A third consideration is what your language(s) of choice are for developing your application. Some languages will have more complete/mature/documented drivers and/or framework abstractions for the different databases.
That said, writing an e-commerce system appears to be a rite of passage for many developers ;-).
Edit: a lot has changed since this answer was originally posted in 2012 and you should definitely refer to current product information. For example, MongoDB has had support for Decimal128 values since MongoDB 3.4 (2016) and multi-document transactions since MongoDB 3.6 (2017).
Check the comparison of different available NoSql databases here. Suit your requirement as per that.
MongoDB 4 now multi-document ACID transactions! That makes it suitable for e-Commerce!
Check out: https://www.mongodb.com/transactions
I'm trying to figure out how to make an iPhone application allow multiple users (that have it installed) share data among them. Now, the tricky part is that I don't want to host a server at my place (very poor ISP services), so I would opt for an online hosting solution. Next, by data I understand them to be able to, let's say, post a comment that would become readable to all the other users and to see what other users have said.
So, in my mind, I'm thinking of either having a file remotely hosted that could be accessed by multiple users at the same time, or a database of some sort or anything like that.
You haven't given us much to go on -- it's not even clear what your question is. If you're just asking how to go about this, I'd suggest the following steps:
Figure out what, specifically, you want the app to do, what data it will share, and who the data will be shared with. Is this an app that you're going to distribute publicly? Will all users share the same data, or will groups of users share with each other but not outside the group? How big is the data, and how is it structured? Can any part of the data change at any time (like a shared document) or will the data just be updated (like a SMS conversation)?
Decide how you want to host the data. If you'll need to serve a lot of users, you'll want some sort of database. If you'll need to serve a LOT of users, you'll want to make sure that your solution will scale easily. There are lots of hosting companies that provide access to databases like Oracle or MySQL, and that may be enough for your purposes. Or, you might want to look into some of the web services options, such as those offered by Google and Amazon. These can be fairly easy to use and have the advantage that they'll scale very well.
Get to work. You'll probably want to build a very basic version of your app around the same time that you're getting the server side working, so that it's easier to test. Once the server side is working and reliable, you can shift the focus back to building out the rest of your app.
I would like to get feedback from all you seasoned developers as to what methodology would be the more "correct" or "efficient" way of implementing my solution.
I have a 4.5 MB flat file that is some 16,000 rows with 13 columns. I know I can import this into SQLite and create my data model but would it be more iPhone efficient to use this file locally on the iPhone or have the application read the data from a web service?
Thanks.
If you are not going to update the data (or only update it when you are updating the app) the local sqlitedb is going to simpler and more responsive. You would probably be even better off importing the data into CoreData, that way you won't need to directly manipulate sqlite or deal with things like synchronous read APIs.
If you want to be able to have the app download updated data the choice because a lot more difficult, depending on the quantity of data, the frequency of updates, how large the changes tend to be, etc.
a local database should always be more efficient in terms of user experience than a web service
I'd use both.
A remote source allowing for a dynamic datastore, and a local datastore with local cacheing seems like a pretty safe bet.
As for the web service. Unless there is any server-side only business logic, maybe give a cloud solution a try. Something like Amazon's SimpleDB comes to mind.
It of course really depends on how static your data is. As everyone has mentioned already if you don't need many updates the most effective solution is a sole local datastore.
Cheers
I guess it depends a bit on how much of the data you need at any one time. If your users need to download a lot of data just to use your application, that would make your app potentially very slow and also unusable without a network connection.
How often do you need to update the data? Frequent updates would favour a web service solution. Otherwise you'd need to update your app and resubmit every time a bit of your data changes.
Another thing to think about: how much do you pay for web traffic for your website? It could become quite expensive if a lot of users constantly need to download data. Unless you use some kind of subscription you only get money once, when you sell the app.
Personally, I'd probably lean towards putting the data on the phone and not using a web service.