I need some help with deciding on the architecture of my project (a web app for unlocking discounts). I am first planning on creating the website (React for the front-end & Django for the back-end, PostgreSQL database). In the future, I may create a mobile app too for Android & iOS (unsure what front-end framework yet).
So I have decided I want the front-end and back-end to be completely separated so the back-end is a REST api. This will allow me to not have to create multiple back-ends for mobile apps.
But, after researching, I have found that this could be quite expensive in terms of server costs. This is a new business and I am the only developer so funding isn't high. So I was thinking that I could deploy the front-end & back-end on the same server but as separate apps that talk via nginx?
I have 4 questions about this:
If I do this, would it still be possible to reuse the back-end as a REST api for the mobile apps or is that a no because it's linked to the web front-end?
If it is possible, would I be able to host the mobile front-end in the same server (so have everything hosted on 1 server)?
Is this a stupid idea - would I just be better off deploying everything into separate servers in the long-run (to reduce load)?
Should I just worry about this in the future? And for now just deploy the separated web front-end & back-end to the same server.
I have never really deployed anything into a real life production environment so I'm sorry if my questions seem silly. I haven't started development yet but I want to think about scalability & future extensibility before I start. Thank you.
Nowadays I'd go with a serverless approach. Instead of having servers to maintain you can focus on your app functionalities.
There are a lot of options. You can check, for example, AWS Amplify (https://aws.amazon.com/amplify/) or Netlify (https://www.netlify.com/) for a more "full-stack" approach.
In AWS, you also can keep separated projects, having your backend in lambdas and your frontend served through S3 + CloudFront. You also don't have servers to care about.
There are only examples of how you can solve your problem without servers, but answering your questions:
You can reuse your APIs regardless of the way your app is deployed. It will be more related to how you designed them;
Yes, you can host everything in a single server if you want, but I really don't recommend that;
If you don't want to pay for 24/7 servers, you can go for a serverless approach;
As I told you before, you can do what you want without worrying about servers.
Your main point of focus is to keep the cost lower and to implement a good solution also. My suggestion would be to look for AWS Lightsail. Lightsail offers fixed price VM which you can configure yourself, and it starts from $3.5 / month at the time of writing this answer.
My answers to your questions
If I do this, would it still be possible to reuse the back-end as a REST api for the mobile apps or is that a no because it's linked to the web front-end?
Yes, it's possible. Keep the frontend and backend in different repo, and you can deploy it as docker instances on the same server. You will have 1 frontend docker container and 1 backend docker container, and they can communicate with each other.
If it is possible, would I be able to host the mobile front-end in the same server (so have everything hosted on 1 server)?
For mobile, you will develop a mobile application which you can publish to playstore or deploy to smartphone. Your app can then call the backend service and get the JSON in response. So you have to design your backend in such a way that it can serve data to both requests.
Is this a stupid idea - would I just be better off deploying everything into separate servers in the long-run (to reduce load)?
For long term and design perspective, you need to consider factors like scalability, maintainability, security etc.., so its always better to have multiple server to avoid single point of failure.
Should I just worry about this in the future? And for now just deploy the separated web front-end & back-end to the same server.
My advice to you will be to think carefully now, so you don't get nightmares in the future. Invest your time now and design a stable solution which could help you in long-term. As you mentioned that its a small business, but your solution should be able to easy handle growth.
My suggestion
As suggested by the Paulo, S3 + CloudFront looks good for frontend. You can get 1 year free CDN using Lightsail.
For Backend, you should at least have 2 (I will suggest minimum 3) servers and deploy backend docker containers. You can use docker compose to automate the deployment. If you want to orchestrate then Docker Swarm Mode is best. With this you will avoid single point of failure. You can get very affordable servers from Amazon Lightsail
For database, you need to make it scalable. To ensure scalability and High Avalability we should have replicated DB. Minimum 3 DB instances will be good starting point. MongoDB is a good choice. With simple configuration you can enable DB replication. 1 Master 2 slaves instances.
1 Load-balancer in front of your servers to distribute the load. To save the cost you can configure the Load-balancer yourself but this will add learning curve and you will have to spent time and understanding the details. The better solution is to use a managed load balancer. Lightsail offers Load Balancer for $18 / month at the time writing this answer.
The above mentioned solution is cost-effective and will give you long-term benefit and also you can estimate the cost based on your solution.
Obviously, this can still be improved but I tried to cover the necessary aspects of the question asked.
Related
Background:
I'm building an API service app. The app is just like any other, you send an HTTP request and receive a response. This seems simple up until I start thinking about user registration, payments, authentication, logging and so on.
Application:
tl;dr simple app diagram
Endpoints listening for HTTP requests and doing all the request related work. This is the core of the service, what the service user would use this app for. Directly not accessible to the end user (unless somehow it knows the url). Python flask server, deployed on google cloud RUN.
API gateway acting like a proxy and a single access point forwarding the requests to the endpoints. This is the service access point for the end users. This part will also be responsible for authentication, limitations, logging and tracking the use of the API endpoints. Python flask server, deployed on google cloud RUN.
Website including documentation, demo and show off of API calls through API gateway, registration, payment (thinking of Stripe) etc. VueJS app on NodeJS server on google cloud compute VM.
Database storing credentials of registered users, payment information and auth keys. Not implemented yet.
Problems:
Is this architecture proper? What could be done differently or improved? How could I further simplify all the interactions between separate parts of the app? Am I not missing any essential parts?
Haven't yet implemented the database part and I'm not sure what should I
use? There are plenty of options on google cloud. Also I could go with something simple and just install a DB with http/JSON interface on google cloud compute VM. How do I chose the DB? Given such an app, what would be the best choice?
Please recommend literature/blogs/other sources of info on similar app
architecture for new developers not familiar with it?
This is pretty open ended, but here are some general comments:
Think about how your UI will work. Are you setting up a static app served directly from cloud storage or do you need something rendered on the server? Personally I prefer separating UI from API when I can but you need to be aware of things like search engine optimization. Even if you need to render some content dynamically your site can still be static. Take a look at static site generators like Gatsby. I haven't had to implement a server rendered UI in years and that makes me happy.
API gateway might be fine, but you don't really need it for anything. It might be simpler to start without it and concentrate on what actually matters. If your APIs are being called by an external client you can't trust the calls anyways and any API key you might be using will be exposed. I'd say don't worry about it for a single app. That being said, if you definitely want to use a GW then use one, just be aware that it is mostly a glorified proxy and not some core part of your architecture.
Make sure your API implementations don't store any local state so you can rely on Cloud Run scaling your services up and down. Definitely don't ever store state directly inside your containers. If you need state on the server it needs to be in some external data store.
Use JWTs or an external IDM (that will generate JWTs) for authentication. Keep session data on the client side as much as possible and pass the JWT in every API call to authenticate the caller. If you are implementing login on your own the only APIs you need to expose without tokens are for auth and password recovery, which you can separate into their own service.
Database selection depends on how well you understand your processes, how transactional your services are and your existing skillset. Overall I would use what you are comfortable with, you can probably succeed with a lot of things. Certain NoSQL flavors can seem simple on the surface but if you don't have a clear understanding on the types of queries you need to run they can get tedious to work with. Generally you should stick to relational databases for OLAP style implementations and consider NoSQL for OLTP. Personally I like MongoDB and it is very popular, probably because it sort of sits in the middle of the pack which makes it fit a lot of applications. Using MongoDB also makes you cloud agnostic since it is available on every platform. Using platform specific database flavors can lock you down to a specific vendor.
Whatever you do, don't start installing things on VMs. You can be almost 100% sure you are doing it wrong if this comes up. Remember, the services you consume don't all have to be managed by Google or even run on GCP. You can get MongoDB capacity directly from MongoDB who manage it on your behalf on all of the Big3 cloud vendors.
At least think about the long term, even if you don't necessarily need to have it impact your architecture right now. If you are expecting your app to be up for years try to make it more platform agnostic than less. This might mean sticking away from some really platform specific serverless features that will force you to jump a couple of extra hoops. If you are using Cloud Run you are using containers which already makes your app pretty portable, don't lock it to one platform by using a lot of platform specific features. That being said, don't stay away from them either. You should always go for the low hanging fruit, so don't try to avoid using things like secrets manager etc. If your app has a short lifespan and you need really fast time to market then don't worry about it.
Just my 2c, what you are doing is very generic and can be done in a lot of different ways.
I've been ramping up on Azure Mobile Services over the past week. There are definitely some PROs and CONs in using them over a standard Azure Web Site where I can write APIs that hit SQL DB, etc.
One of the biggest negatives I see is developing the server side code and DB structures ON THE SERVER. I've watched lots of videos from launch and beyond, read lots of blog posts about tips and tricks around WAMS, but nobody seems to talk about the downside of developing the code (server scripts) and database structures on the server, at your live URL.
This is all great for developing the first version of your mobile app and associated mobile services. But once it's all deployed, how do you ever build version 2? Real apps hitting real APIs and data, but now I want to develop/change/play with the server scripts and database schema?
With Azure Web Sites, I can develop locally and only publish code and DB changes to the server on my schedule.
Have any of you seen or heard of the "v2 development story" around Azure Mobile Services?
Only thing I can think of would be to create another set of tables and APIs around them, most likely "virtual tables" that allow me to write APIs against the original set of data. Seems like a huge hassle, since the client code would now have to know about the original set of tables and the new set of tables... that's only for v2...
Thanks for any thoughts / insight.
You should have two services, one dev and one production and use scripts to promote your code from dev to production (pretty similar to how most workflows go, in moving from a test setup to a production one).
http://channel9.msdn.com/Events/Build/2013/3-511
I want to develop a website that will serve millions of pages everyday including the mobile devices. Site will have strong social features and thus would require lots of reads/writes. It will also suggest things to users based on their social behaviors (likes, dislikes etc) and their friends' behaviors. After considering many elements I have come up with
NoSQL (MongoDB or Cassandra) Database. Not sure which one is the right one.
memcached
Varnish or squid for http acceleration
php and python (Not sure if php is that scalable)
nginx or Apache web server
Any recommendations?
There are NoSQL databases that has an integrated web service that can handle much more web requests per second (including database transaction time) compared to traditional web services requesting data from an external data source. Using this kind of solution increases the performance, save a lot of time in implementation and simplify scaling your website.
The recommendation depends on how you plan on implementing the solution: a server side rendering solution or a client rendered solution? Will you have any MVVM style implementation making the communication talkative? Also what server side environment do you have in mind? Microsoft/Linux?
Take a look at Starcounter database that has a web server component integrated into the database engine and see if that could help you.
I know this has been asked a few times, but there seems to be no clear answer ... am searching on this for the past 3 days or more.
There seem to be 2 ways to connect to an Oracle database from an iOS App :
ODBC Client
I need to compile ODBC (which ODBC?) using gcj for ARM. I think this is the hard way, wrought with errors, but possible with quite an effort.
USING WEB SERVICE
Connect from App to webservice and from web service to Oracle DB.
Are these the 2 methods available or any other?
Few questions on the two methods:
a. Which is more secure?
b. Will my company's security department oppose to any of the above?
c. Which is more performant?
d. Which of the above does one normally use?
Webservices are the answer, you do not want people connecting directly to the database from a mobile device. A Webserver will add one extra layer of security as well as the ability to handle simultaneous request without stressing the database directly
a. Which is more secure?
Webservices as explained above
b. Will my company's security department oppose to any of the above?
Yes, security department will insist not to open the oracle port to connect directly, unless they have it already open.
c. Which is more performant?
Webservices, setting up the right cache policies in a webserver can save resources to the database.
d. Which of the above does one normally use?
Webservices, because they offer you great advantages in security and performance, not only that, webservices are reusable and can be accessed by many different platforms, think on the future you might want to serve your application later on Android devices and Webservices will save you a lot of development time.
Many of today's top applications in the market use webservices, think about it.
Google Maps is a great example of how powerful webservices are!
It's not a good idea to connect to your database directly from your app. It can be secure if you create an account that can do nothing but SELECT, but there are some other things to consider.
Why burden the app with the Oracle client?
If you have many users you have to worry about Oracle handling a huge number of simultaneous connections. With a Restful API requests are stateless.
If you decide to change your schema. You'll also have to change your app. When you place a service in between, the app is no longer dependent on the schema.
ODBC connection will require that the Oracle port is open to the Internet, which in vast majority of cases will not be allowed for security and performance reasons. Even if it were, or even if you establish a secure VPN, a direct database access requires that the connection is kept open, which can be problematic when a mobile device can go in and out of the network coverage.
HTTP is far more tolerant to unreliable networks and can be encrypted using SSL (HTTPS). The problem with HTTP is that database do not have direct support for this transport so most people develop dedicated web services.
I work on a project called SlashDB, which automatically constructs RESTful APIs out of databases. For public APIs you would install /db in so called DMZ (a network segment between two firewalls) as described in this blog post.
SlashDB can be configured to allow restricted data access to public users or you can define specific users with varying privileges to data. It is designed as stateless service, which means that you can easily set up multiple nodes behind a load balancer and reverse HTTP proxy for high availability web scale deployments.
Regardless whether you develop the web service by hand or use our product you will achieve better scalablity, performance and security for your solution than by using direct client/server approach. I would even argue that REST APIs should be used internal enterprise data integration solutions but that's a whole new topic.
I am going to repeat what everyone else said, Rest API is the way to go. Do not connect to the database directly. However, there might be a way to connect to your database which I never tried my self.
http://odbcrouter.com/iosvsweb#hn_iOS_Open_Database_Connectivity_SDK
i am working on a ipad app, i need to write some data to a remote online database , can i do this with out using web service,,, i need some advice,,,
thanx in advance
Technically this is possible, there are remote database drivers for the iPhone platform, for example Flipper.
However, I'd strongly recommend use some kind of "Service" to do your database access. This could be a full SOAP/HTTP WebService, a RESTful Service, or even just a little bit of php that you invoke over http or https. Don't be concerned that developing this "Service" will be lots of work, it need take no more than an hour or two. In fact with a product such as Worklight it took me literally 15 minutes using the Worklight SQL adapter. (Disclaimer I work for IBM, we recently acquired Worklight.)
There are several reasons to prefer using an intermediary service rather than direct access to the DB from the client. Here's a couple:
Scalability. Each user's connection to the DB consumes server side resources, if your app is widely used then you could end up with many tens of thousands of simultaneous connections. The service approach uses Web-facing connections to the phone, using (for example) web containers designed for high numbers of concurrent sessions, and then funnels down to a few database connections. Even very busy web sites tend to use (and reuse) only a small number (a few 10s) of database connections.
Security. It is strongly recommended to avoid making databases directly accessible to the internet. It's a big topic, but if the database contains any kind of valuable data then a pattern of fronting the database by a service greatly reduces vulnerability.
I recommend using the service Parse. Their service is built specifically to solve the iOS/Android backend problem. I just wrote a blog post about them: Parse, The Best Backend for iPhone SDK.