I want to develop a website that will serve millions of pages everyday including the mobile devices. Site will have strong social features and thus would require lots of reads/writes. It will also suggest things to users based on their social behaviors (likes, dislikes etc) and their friends' behaviors. After considering many elements I have come up with
NoSQL (MongoDB or Cassandra) Database. Not sure which one is the right one.
memcached
Varnish or squid for http acceleration
php and python (Not sure if php is that scalable)
nginx or Apache web server
Any recommendations?
There are NoSQL databases that has an integrated web service that can handle much more web requests per second (including database transaction time) compared to traditional web services requesting data from an external data source. Using this kind of solution increases the performance, save a lot of time in implementation and simplify scaling your website.
The recommendation depends on how you plan on implementing the solution: a server side rendering solution or a client rendered solution? Will you have any MVVM style implementation making the communication talkative? Also what server side environment do you have in mind? Microsoft/Linux?
Take a look at Starcounter database that has a web server component integrated into the database engine and see if that could help you.
Related
I've been looking all over the web for a solution to a client problem. They refuse to retire their 16-bit dbase database, but still want us to development real-time PWA web applications to read/write to the dbase system.
If it is even possible to get a RESTful API server working with 16-bit dBase?
Yes. You can create a PWA app with dBase or any DB, which has option to communicate to modern programming languages via APIs like JDBC, ODBC. Then that programming language of your choice, say JAVA talking to dBase via JDBC, can expose the functionality via RESTFul service.
For a PWA app, underlying back-end service tech stack is masked over plain HTTP requests. So it doesn't matter what back-end service you use and what DB that talks to in turn, for a PWA
As an update on your “real time “ need, not sure what kind of real-time functionality you are planning and on expecting what ETA/SLA. While no doubt you can build a PWA with any dB as stated above, it can certainly impact its performance. If you need something like googles auto suggest which pulls up results as you type, you need something like “in-memory” DB.
So the question is more of what performance your DB can support and what is expected. I do not see any feasibility challenge with respect to PWA.
I know this has been asked a few times, but there seems to be no clear answer ... am searching on this for the past 3 days or more.
There seem to be 2 ways to connect to an Oracle database from an iOS App :
ODBC Client
I need to compile ODBC (which ODBC?) using gcj for ARM. I think this is the hard way, wrought with errors, but possible with quite an effort.
USING WEB SERVICE
Connect from App to webservice and from web service to Oracle DB.
Are these the 2 methods available or any other?
Few questions on the two methods:
a. Which is more secure?
b. Will my company's security department oppose to any of the above?
c. Which is more performant?
d. Which of the above does one normally use?
Webservices are the answer, you do not want people connecting directly to the database from a mobile device. A Webserver will add one extra layer of security as well as the ability to handle simultaneous request without stressing the database directly
a. Which is more secure?
Webservices as explained above
b. Will my company's security department oppose to any of the above?
Yes, security department will insist not to open the oracle port to connect directly, unless they have it already open.
c. Which is more performant?
Webservices, setting up the right cache policies in a webserver can save resources to the database.
d. Which of the above does one normally use?
Webservices, because they offer you great advantages in security and performance, not only that, webservices are reusable and can be accessed by many different platforms, think on the future you might want to serve your application later on Android devices and Webservices will save you a lot of development time.
Many of today's top applications in the market use webservices, think about it.
Google Maps is a great example of how powerful webservices are!
It's not a good idea to connect to your database directly from your app. It can be secure if you create an account that can do nothing but SELECT, but there are some other things to consider.
Why burden the app with the Oracle client?
If you have many users you have to worry about Oracle handling a huge number of simultaneous connections. With a Restful API requests are stateless.
If you decide to change your schema. You'll also have to change your app. When you place a service in between, the app is no longer dependent on the schema.
ODBC connection will require that the Oracle port is open to the Internet, which in vast majority of cases will not be allowed for security and performance reasons. Even if it were, or even if you establish a secure VPN, a direct database access requires that the connection is kept open, which can be problematic when a mobile device can go in and out of the network coverage.
HTTP is far more tolerant to unreliable networks and can be encrypted using SSL (HTTPS). The problem with HTTP is that database do not have direct support for this transport so most people develop dedicated web services.
I work on a project called SlashDB, which automatically constructs RESTful APIs out of databases. For public APIs you would install /db in so called DMZ (a network segment between two firewalls) as described in this blog post.
SlashDB can be configured to allow restricted data access to public users or you can define specific users with varying privileges to data. It is designed as stateless service, which means that you can easily set up multiple nodes behind a load balancer and reverse HTTP proxy for high availability web scale deployments.
Regardless whether you develop the web service by hand or use our product you will achieve better scalablity, performance and security for your solution than by using direct client/server approach. I would even argue that REST APIs should be used internal enterprise data integration solutions but that's a whole new topic.
I am going to repeat what everyone else said, Rest API is the way to go. Do not connect to the database directly. However, there might be a way to connect to your database which I never tried my self.
http://odbcrouter.com/iosvsweb#hn_iOS_Open_Database_Connectivity_SDK
i am working on a ipad app, i need to write some data to a remote online database , can i do this with out using web service,,, i need some advice,,,
thanx in advance
Technically this is possible, there are remote database drivers for the iPhone platform, for example Flipper.
However, I'd strongly recommend use some kind of "Service" to do your database access. This could be a full SOAP/HTTP WebService, a RESTful Service, or even just a little bit of php that you invoke over http or https. Don't be concerned that developing this "Service" will be lots of work, it need take no more than an hour or two. In fact with a product such as Worklight it took me literally 15 minutes using the Worklight SQL adapter. (Disclaimer I work for IBM, we recently acquired Worklight.)
There are several reasons to prefer using an intermediary service rather than direct access to the DB from the client. Here's a couple:
Scalability. Each user's connection to the DB consumes server side resources, if your app is widely used then you could end up with many tens of thousands of simultaneous connections. The service approach uses Web-facing connections to the phone, using (for example) web containers designed for high numbers of concurrent sessions, and then funnels down to a few database connections. Even very busy web sites tend to use (and reuse) only a small number (a few 10s) of database connections.
Security. It is strongly recommended to avoid making databases directly accessible to the internet. It's a big topic, but if the database contains any kind of valuable data then a pattern of fronting the database by a service greatly reduces vulnerability.
I recommend using the service Parse. Their service is built specifically to solve the iOS/Android backend problem. I just wrote a blog post about them: Parse, The Best Backend for iPhone SDK.
In search of a 'Cloud Content Management System' like http://osmek.com/,
I could not find a single other CCMS that does what I want it to do :)
Basically, what I need is content management without a website frontend attached.
Just basic storage of data, documents, images, etc. etc. with a simple API to access, like Osmek. Just NoSQL or SQL based services won't do, because there can be images or documents attached. And, ofcourse, I'd like to have a backend to manage the data (like a typical CMS does) without writing a backend myself (if it's just the service)
Osmek is great, and it works most awesome in conjunction with Actionscript 3, but I'm just looking / searching for alternatives (if there even are any yet).
I need this form of hosted content management for content-manageing a mobile application.
So the question is: Is there anything else out there that does the same as osmek that you know of? OR, how do you manage application specific content?
Thanks!
I'd encourage you to take a look at Cloud CMS (http://www.cloudcms.com).
Cloud CMS is a JSON content management (CMS) platform built on top of MongoDB with a REST API and drivers for a variety of languages. You just drop in a driver and call methods to query, create, update and delete content.
The platform provides everything you need to power the back-end for mobile and HTML5 applications - from managing your content to managing users and groups, credentials, security tokens (OAuth2), Git-like collaborative workspaces, real-time analytics, activities, data transformations and more.
Everything runs in the cloud on an elastic back-end. It's probably more akin to Parse than a traditional CMS. You just make calls to the APIs. We keep the costs low by letting you only pay for what you use (almost like a utility). You just pay for storage and data transfer.
Disclaimer: I'm one of the founders of Cloud CMS. So I'm a pretty lousy reference in terms of its objective value. However, a couple of us worked at traditional "ECM" companies in the past and we think we've built something that puts a genuine beating on those guys.
Okay, so I'm currently developing an iphone app that I plan to take into production and scale. I'm a bit lost on the whole subject.
What is better to use: core data or sqlite? (as the local DB)
Also, can sqlite be used exclusively to communicate with my remote server as well? At first I thought it could but I've been reading that sqlite isn't great to use on servers that get a massive amount of hits.
I've read that oracle, mysql, or mssql may be better to use on a remote server and that I can communicate with these servers via REST or SOAP.
I plan to be able to both read and write to a remote server. The files transferred will mostly be small data objects and pictures. Speed is of the essence, so I'd like to know which options are my fastest routes. Of course, I want the option to scale and not have performance take too much of a hit as well.
On the subject of Core Data vs sqlite see this question.
SQLite is a small and lite embedded SQL database engine. It's not meant to used in server environments. In general, it's not a good idea to communicate directly over the Internet. It's more common to have some sort of process logic between the client code and the database to do a range of things like validate input, process business logic, security, etc. You can implement this sort of layer in REST, SOAP, or whatever you like. Since your clients will be mobile devices, a http based web service (like REST or SOAP) is a good idea as all mobility platforms have inbuilt API support for http messaging. There are lots and lots of options on the server depending on what type of server you want to setup and run with.
If your new to this, maybe you should read something like 'Patterns of Enterprise Application Architecture' by Martin Fowler to get a idea of what sort of design patterns people use to implement the server side layering.