I'm in the process of designing a cloud deployed website for a new solution my company is looking to provide. I have been attempting to answer a few questions and haven't had any luck, so when in rome.
First, I don't want the website to be stuck to any one particular framework. I know there is no way to completely future proof a website, but I would rather not put all of our eggs in one basket.
Secondly, I want to have a separation between the front and back end entirely. I have a list of reasons why I'm looking to do this, don't necessarily want to get into the conversation of what they are. Server Side rendering for the most part is out of the question.
So where does that leave me?
My initial thoughts on the design are to have a REST API that can be accessed for any API calls (this may be turned to GraphQL in the future).
The design decisions that I'm mostly wresting with are for the front end. The website will be a dashboard type system, where tenants can log in and see screens for them.
I was thinking that I would have a sort of shell, that hooks on to the index.html. This would have it's own routing, that would render micro-applications that are completely separate from the shell logic.
So for example, if I load index.html, path being "/"
It has some routes that it's responsible for, lets say
"/todos"
"/account"
If I accessed the /todos route, my shell application would then render that micro app. This application would be completely separate from the shell, except some data that might be loaded via the window. Once this application is rendered via the shell application.
So my todos route, for example, could be a redux application that's independent. It could have it's own routing, etc.
Is this is a common architecture? Are there any examples of this? Is there a better way of going about this?
Thanks for any insight!
Sounds like your well and truly over engineering this beast.
You may take on such an architecture for a HUGE build with many dev teams all working separately. Small agile team, the above would create so much overhead in boilerplate and brain ache in context switching between each "app"
Micro-service architecture is seriously great. Just don't break it up too small, read your use case well and break your services up accordingly.
For example: we are a team of 3. We have a pretty large-ish app devised into:
Php API
Backend management interface (redux)
Frontend website (html, react, php)
Search service (elastic search)
Cache (redis)
Data store (mysql)
All on running in multiple docker containers across multiple hosts. Pull down the backend.. Fine the frontend website is still up and running!
Related
I am developing monolithic applications since many years and want to try microservices and containers now.
For learning microservices and containers I am planning a small calendar application (as a web application), where you can create dates and invite others to them. I identified three services that I have to implement:
Authentication service -> handles login, gives back a JWT
UserData services -> handles registration, shows user data, handles edits of user data (profile picture, name, short description)
Calendar service -> for creating, editing and deleting dates, inviting others to dates, viewing dates and so on.
This is the part I feel confident about, but if I did already something wrong, please correct me.
Where I am not sure what to do is how to implement the database and frontend part.
Database
I never used Neo4j or any graphbased database before, but they seem to work well for this use case and I want to learn something new. Maybe my database choice may not be relevant here, but my question is:
Should every microservice have its own database or should they access the same database?
Some data sets are connected, like every date has a creator and multiple participants.
And should the database run in a container or should it be hosted on the server directly? (I want to self-host the database)
For storing profile pictures I decided to use a shared container volume, so multiple instances of a service can access the same files. But I never used containers before, so if you have a better idea, I am open to hear it.
Frontend
Should I build a single (monolithic) frontend application, or is it useful to build some kind of micro frontend, which contains only a navigation and loads other parts like the calendar view or user data view from the above defined services? And if yes, should I pack the UserData frontend and the UserData service into one container?
MVP
I want the whole application to be useable by others, so they can install it on their own servers/cloud. Is it possible to pack the whole application (all services and frontend) into one package and make it installable in kubernetes with a single step, but in a way, that each service still has its own container? In my head this sounds necessary, because the calendar service won’t work without the userdata service, because every date needs a user who creates it.
Additional optional services
Imagine I want to add some additional but optional features like real time chat. What is the best way to check if these features are installed? Should I add a management service which checks which services exist and tells the frontend which navigation links it should show, or should the frontend application ping all possible services to check if they are installed?
I know, these are many questions, but I think they are tied together, because choices on one part can influence others.
I am looking forward for your answers.
I'm further along than you but far from "microservice expert". But I'll try my best:
Should every microservice have its own database or should they access the same database?
The rule of thumb is that a microservice should have its own database. This decouples them, making it so that your contract between services is just the API. Furthermore it keeps the logic simpler within a service in that there's less types for data that your service is responsible for handling.
And should the database run in a container or should it be hosted on the server directly?
I'm not a fan of running a database in a container. In general, a database:
can consume a lot of resources (depending on the query)
sustains long-lived connections
is something you vertically scale, rather than horizontally
These qualities reflect a poor case for containerization, imho.
For storing profile pictures I decided to use a shared container volume
Nothing wrong with this, but using cloud storage like Amazon S3 is a popular move here, just so you don't have to manage the state of the volume. i.e. if the volume goes away, do you still want your pictures around?
Should I build a single (monolithic) frontend application?
I would say yes: this is would be the simplest approach. The other big motivation for microservices is to allow separate development teams to work independently. If that's not the case here, I think you should avoid yourself the headache for micro-frontends.
And if yes, should I pack the UserData frontend and the UserData service into one container?
I'm not sure I perfectly understand here. I think it's fine to have the frontend and backend service as part of the same codebase, which can get "deployed" together. How you want to serve the frontend is completely up to you and how it's implemented. Some like to serve it through a CDN to minimize latency, but I've never seen the need.
Is it possible to pack the whole application (all services and frontend) into one package and make it installable in kubernetes with a single step?
This is use-case for Helm charts: A package manager for kubernetes.
Imagine I want to add some additional but optional features like real time chat. What is the best way to check if these features are installed? Should I add a management service which checks which services exist and tells the frontend which navigation links it should show, or should the frontend application ping all possible services to check if they are installed?
There's a thin line between what you're describing and monitoring. If it were me, I'd pick the service that radiates this information and just set some booleans in a config file or something. That's because this state is isn't going to change until reinstall, so there's no need to check it on an interval.
--
Those were my thoughts but, as with all things architecture, there's no universal truth. Some times exceptions are warranted depending on your actual circumstances.
Background:
I'm building an API service app. The app is just like any other, you send an HTTP request and receive a response. This seems simple up until I start thinking about user registration, payments, authentication, logging and so on.
Application:
tl;dr simple app diagram
Endpoints listening for HTTP requests and doing all the request related work. This is the core of the service, what the service user would use this app for. Directly not accessible to the end user (unless somehow it knows the url). Python flask server, deployed on google cloud RUN.
API gateway acting like a proxy and a single access point forwarding the requests to the endpoints. This is the service access point for the end users. This part will also be responsible for authentication, limitations, logging and tracking the use of the API endpoints. Python flask server, deployed on google cloud RUN.
Website including documentation, demo and show off of API calls through API gateway, registration, payment (thinking of Stripe) etc. VueJS app on NodeJS server on google cloud compute VM.
Database storing credentials of registered users, payment information and auth keys. Not implemented yet.
Problems:
Is this architecture proper? What could be done differently or improved? How could I further simplify all the interactions between separate parts of the app? Am I not missing any essential parts?
Haven't yet implemented the database part and I'm not sure what should I
use? There are plenty of options on google cloud. Also I could go with something simple and just install a DB with http/JSON interface on google cloud compute VM. How do I chose the DB? Given such an app, what would be the best choice?
Please recommend literature/blogs/other sources of info on similar app
architecture for new developers not familiar with it?
This is pretty open ended, but here are some general comments:
Think about how your UI will work. Are you setting up a static app served directly from cloud storage or do you need something rendered on the server? Personally I prefer separating UI from API when I can but you need to be aware of things like search engine optimization. Even if you need to render some content dynamically your site can still be static. Take a look at static site generators like Gatsby. I haven't had to implement a server rendered UI in years and that makes me happy.
API gateway might be fine, but you don't really need it for anything. It might be simpler to start without it and concentrate on what actually matters. If your APIs are being called by an external client you can't trust the calls anyways and any API key you might be using will be exposed. I'd say don't worry about it for a single app. That being said, if you definitely want to use a GW then use one, just be aware that it is mostly a glorified proxy and not some core part of your architecture.
Make sure your API implementations don't store any local state so you can rely on Cloud Run scaling your services up and down. Definitely don't ever store state directly inside your containers. If you need state on the server it needs to be in some external data store.
Use JWTs or an external IDM (that will generate JWTs) for authentication. Keep session data on the client side as much as possible and pass the JWT in every API call to authenticate the caller. If you are implementing login on your own the only APIs you need to expose without tokens are for auth and password recovery, which you can separate into their own service.
Database selection depends on how well you understand your processes, how transactional your services are and your existing skillset. Overall I would use what you are comfortable with, you can probably succeed with a lot of things. Certain NoSQL flavors can seem simple on the surface but if you don't have a clear understanding on the types of queries you need to run they can get tedious to work with. Generally you should stick to relational databases for OLAP style implementations and consider NoSQL for OLTP. Personally I like MongoDB and it is very popular, probably because it sort of sits in the middle of the pack which makes it fit a lot of applications. Using MongoDB also makes you cloud agnostic since it is available on every platform. Using platform specific database flavors can lock you down to a specific vendor.
Whatever you do, don't start installing things on VMs. You can be almost 100% sure you are doing it wrong if this comes up. Remember, the services you consume don't all have to be managed by Google or even run on GCP. You can get MongoDB capacity directly from MongoDB who manage it on your behalf on all of the Big3 cloud vendors.
At least think about the long term, even if you don't necessarily need to have it impact your architecture right now. If you are expecting your app to be up for years try to make it more platform agnostic than less. This might mean sticking away from some really platform specific serverless features that will force you to jump a couple of extra hoops. If you are using Cloud Run you are using containers which already makes your app pretty portable, don't lock it to one platform by using a lot of platform specific features. That being said, don't stay away from them either. You should always go for the low hanging fruit, so don't try to avoid using things like secrets manager etc. If your app has a short lifespan and you need really fast time to market then don't worry about it.
Just my 2c, what you are doing is very generic and can be done in a lot of different ways.
I'm confused about Firebase and serverless in general, as I was just introduced to this concept recently (note that I'm still studying computer science and I'm just exploring on my own now).
In the past I've been part of a project that had the following structure:
Front End in a Single-page-app
Back End built as a REST API
Now let's say I want to build a product, that might have a website and mobile apps. It also has to have backend logic as there are accounts, objects owned by users, and possible integration with a payment service.
What I initially expected from this, before knowning about serverless, is that you build a backend using something like Go (was my case), where you handle all the database data and third-party integrations, build a front end with something like Vue, and then use the backend's REST API to communicate between both of them.
Is this still the case with serverless? Do you build the whole backend server code, or does it work in another way?
I don't need/want you to explain all of it to me. I just need some insight on what is done and common so I can investigate further.
There are a ton of online CMS services out there. And a ton of (new) backend-as-a-service products too. But I can't seem to find what I am looking for.
I am building an app for a client. The app contains data about shops, products, and more. The client must be able to update this data (and not just one person: each shop manager needs to be able to log in and edit the data for their own shop). And of course the app must be able to access this data.
Client edits data online
This has to be extremely user-friendly and completely online. I don't want to sell my client something where they need to install stuff on their server. I don't want to sell them something that's accessible online but looks like phpMyAdmin.
I want a shop owner to be able to go to a webpage, log in, and then see a pretty UI where they can edit the data for their shop. The back-end needs to have a pretty front-end that's auto-generated for whatever data this particular shop owner is allowed to edit.
So there are two bits: storing data in the cloud in such a way that it can be accessed by the app (which I am building with Titanium), and allowing the client to log into the backend and edit the data in a non-tech, user-friendly way.
Here's a list of things I tried...
Backend-as-a-service
Services with a great back-end, but without easy auto-generated data editing website:
Appcelerator (Titanium) Cloud Service
Amazon EC2
Stackmob
BackBeam
WebVanta
Parse
API o Mat
ShepHertz Cloud42
Kii
Online CMS
Services that provide a nice way for clients to edit data, but no easy way for apps to connect:
CloudCMS
(and many others I'm sure)
It's insane that no-one seems to be providing the cross-breed of BaaS and online CMS. So many people are building apps for clients, and so many clients are not tech-savvy and are reluctant to get a special server and host database software they don't understand. Why does this not exist? What am I missing?
With apiOmat it's easy to create your own data-editing app for e.g. with JavaScript SDK and HTML. Or you send a feature request so that they build a module for your preferred CMS.
As you mentioned, Cloud CMS is a really good option (disclaimer: I'm one of the founders). The product provides an enterprise content management backend and an API that lets you plug in some really powerful features right into your mobile apps.
This month, we released a brand new user interface which provides much of what you're asking about. Instant forms, document libraries, search and workflow all in one place.
You can check out Cloud CMS here: http://www.cloudcms.com
I completely agree with your assessment particularly with respect to the last mile (getting the final app built). It's kind of the wild west out there and the strong technologies are still proving out.
You mentioned Titanium - that's a good choice. I also quite like the Ionic Framework (http://www.drifty.com/). It's a step in the right direction.
In search of a 'Cloud Content Management System' like http://osmek.com/,
I could not find a single other CCMS that does what I want it to do :)
Basically, what I need is content management without a website frontend attached.
Just basic storage of data, documents, images, etc. etc. with a simple API to access, like Osmek. Just NoSQL or SQL based services won't do, because there can be images or documents attached. And, ofcourse, I'd like to have a backend to manage the data (like a typical CMS does) without writing a backend myself (if it's just the service)
Osmek is great, and it works most awesome in conjunction with Actionscript 3, but I'm just looking / searching for alternatives (if there even are any yet).
I need this form of hosted content management for content-manageing a mobile application.
So the question is: Is there anything else out there that does the same as osmek that you know of? OR, how do you manage application specific content?
Thanks!
I'd encourage you to take a look at Cloud CMS (http://www.cloudcms.com).
Cloud CMS is a JSON content management (CMS) platform built on top of MongoDB with a REST API and drivers for a variety of languages. You just drop in a driver and call methods to query, create, update and delete content.
The platform provides everything you need to power the back-end for mobile and HTML5 applications - from managing your content to managing users and groups, credentials, security tokens (OAuth2), Git-like collaborative workspaces, real-time analytics, activities, data transformations and more.
Everything runs in the cloud on an elastic back-end. It's probably more akin to Parse than a traditional CMS. You just make calls to the APIs. We keep the costs low by letting you only pay for what you use (almost like a utility). You just pay for storage and data transfer.
Disclaimer: I'm one of the founders of Cloud CMS. So I'm a pretty lousy reference in terms of its objective value. However, a couple of us worked at traditional "ECM" companies in the past and we think we've built something that puts a genuine beating on those guys.