I have a 2 gateways which connects to the server where user details are logged.
I can think of two ways to log users accessing the server through gateway.
First way:
The logging is done on the server side i.e
Server(port 5001) Code:
au:([user:`$()]; tim:`timestamp$()); /- Table to maintain logged users
.z.pw:{`au upsert (x;.z.n); show y; 1b}
Gateway 1:
h:hopen `::5001:a:uts1
Gateway 2:
h: hopen `::5001:b:uts2
Second way:
The logging is done from the gateway i.e
Server(port 5001) Code:
au:([user:`$()]; tim:`timestamp$()); /- Table to maintain logged users
Gateway 1:
q)h:hopen `::5001:a:uts1
q)h"`au upsert (`a;.z.p)"
Gateway 2:
q)h: hopen `::5001:b:uts2
q)h"`au upsert (`b;.z.p)"
Hence, is it good to write the user logging code on the server side(Server) or at the client side(Gateway in this case) or is there is better/standard way to do the same?
EDIT - What if we add a middleware(user manager) between multiple gateways and multiple servers, in that case will it be good to write the user logging code on the middleware(user manager) or at the client side(Gateway in this case)?
If users are connecting through a gateway to a number of servers I would implement the logging of users and authentication at the GW level. This would further abstract sensitive data away from potentially unauthorized users, and simplify keeping a master record account activity as all users need to go through the GW. Performing the account activity on the server-side would then need aggregation over multiple servers to get the full picture of system activity.
Related
I've been reading about instances and tenants and in the Saas architecture. My questions are as follows (please correct anything that you notice I've gotten wrong with any of the following terms):
1) Instance: Is an instance of a piece of software just a copy of that software with its own database? Is there anything more to it than that?
2) Tenant: Is a tenant a user / group of users that share a common set of access privileges to an individual instance?
3) Single-instance: If a Saas provider offers single-instance service, does this mean that they create only a single instance of their software? Or does it mean that there could be multiple instances, but that each instance can serve multiple tenants? If so, is single-instance the same as multi-tenant?
4) Multi-instance: Does this mean that each instance can serve only one tenant, or can there be multiple instances that each serve multiple tenants? ie. Can a multi-instance service be either single-tenant or multi-tenant?
5) Single-tenant: Does this just mean that an individual instance can serve only one tenant, or does it also imply that there are multiple instances? ie. Can a single-tenant service be both single-instance and multi-instance?
6) Multi-tenant: Does this just mean that an individual instance can serve multiple tenants, or does it imply that there is only a single instance? ie. Can a multi-tenant service be both single-instance and multi-instance?
7) To sum up: Can you have single-instance+single-tenant, single-instance+multi-tenant, multi-instance+single-tenant, and multi-instance+multi-tenant?
I'm going to write from my direct experience:
1) simple answer is 'yes'.
2) nearly yes: there will probably be refined access rights, say an administrator or two, and general users.
3) they're providing you with just one instance of that module, which will be single tenant.
4) they're providing you with multiple instances of that module, which will be single tenant.
5) I would use single-tenant to refer to the server hosting the instances is used by only one tenant. This might be done for perceived security benefits, or the server is running on a time zone that is non-standard for SaaS provider, like staying on UTC all year round.
6) I would use multi-tenant to refer to the server hosting the instances is used by more than one tenant. This tends to be more cost effective and probably just as secure as single-tenant.
7) yes, no, yes, yes.
Let's say there are two (or more) RESTful microservices serving JSON. Service (A) stores user information (name, login, password, etc) and service (B) stores messages to/from that user (e.g. sender_id, subject, body, rcpt_ids).
Service (A) on /profile/{user_id} may respond with:
{id: 1, name:'Bob'}
{id: 2, name:'Alice'}
{id: 3, name:'Sue'}
and so on
Service (B) responding at /user/{user_id}/messages returns a list of messages destined for that {user_id} like so:
{id: 1, subj:'Hey', body:'Lorem ipsum', sender_id: 2, rcpt_ids: [1,3]},
{id: 2, subj:'Test', body:'blah blah', sender_id: 3, rcpt_ids: [1]}
How does the client application consuming these services handle putting the message listing together such that names are shown instead of sender/rcpt ids?
Method 1: Pull the list of messages, then start pulling profile info for each id listed in sender_id and rcpt_ids? That may require 100's of requests and could take a while. Rather naive and inefficient and may not scale with complex apps???
Method 2: Pull the list of messages, extract all user ids and make bulk request for all relevant users separately... this assumes such service endpoint exists. There is still delay between getting message listing, extracting user ids, sending request for bulk user info, and then awaiting for bulk user info response.
Ideally I want to serve out a complete response set in one go (messages and user info). My research brings me to merging of responses at service layer... a.k.a. Method 3: API Gateway technique.
But how does one even implement this?
I can obtain list of messages, extract user ids, make a call behind the scenes and obtain users data, merge result sets, then serve this final result up... This works ok with 2 services behind the scenes... But what if the message listing depends on more services... What if I needed to query multiple services behind the scenes, further parse responses of these, query more services based on secondary (tertiary?) results, and then finally merge... where does this madness stop? How does this affect response times?
And I've now effectively created another "client" that combines all microservice responses into one mega-response... which is no different that Method 1 above... except at server level.
Is that how it's done in the "real world"? Any insights? Are there any open source projects that are built on such API Gateway architecture I could examine?
The solution which we used for such problem was denormalization of data and events for updating.
Basically, a microservice has a subset of data it requires from other microservices beforehand so that it doesn't have to call them at run time. This data is managed through events. Other microservices when updated, fire an event with id as a context which can be consumed by any microservice which have any interest in it. This way the data remain in sync (of course it requires some form of failure mechanism for events). This seems lots of work but helps us with any future decisions regarding consolidation of data from different microservices. Our microservice will always have all data available locally for it process any request without synchronous dependency on other services
In your case i.e. for showing names with a message, you can keep an extra property for names in Service(B). So whenever a name update in Service(A) it will fire an update event with id for the updated name. The Service(B) then gets consumes the event, fetches relevant data from Service(A) and updates its database. This way even if Service(A) is down Service(B) will function, albeit with some stale data which will eventually be consistent when Service(A) comes up and you will always have some name to be shown on UI.
https://enterprisecraftsmanship.com/2017/07/05/how-to-request-information-from-multiple-microservices/
You might want to perform response aggregation strategies on your API gateway. I've written an article on how to perform this on ASP.net Core and Ocelot, but there should be a counter-part for other API gateway technologies:
https://www.pogsdotnet.com/2018/09/api-gateway-response-aggregation-with.html
You need to write another service called Aggregator which will internally call both services and get the response and merge/filter them and return the desired result. This can be easily achieved in non-blocking using Mono/Flux in Spring Reactive.
An API Gateway often does API composition.
But this is typical engineering problem where you have microservices which is implementing databases per service pattern.
The API Composition and Command Query Responsibility Segregation (CQRS) pattern are useful ways to implement queries .
Ideally I want to serve out a complete response set in one go
(messages and user info).
The problem you've described is what Facebook realized years ago in which they decided to tackle that by creating an open source specification called GraphQL.
But how does one even implement this?
It is already implemented in various popular programming languages and maybe you can give it a try in the programming language of your choice.
As I know REST architecture consists of "frontend" - server which receives requests from clients, and services which are actual "backend" - all business logic and data stored in services(which are exactly same application, which runs at same time on multiple servers). If every service has it's own data storage then how data synchronized?
Each service has a data set and the synchronization can be established with differents ways. Please view the example bellow.
Service A - Products - ProductDB
Service B - Users - UserDB
If a user wants to get all the products for the users with black hairs (for example) this will be a common process that will system follow:
1) Receive the request from client in Gateway
2) Go to user service and the users with black hairs.
select * from users where hair_color='black'
3) Now the user service sends the users IDs to the product service. So the product service executes the following query.
select * from product where user_id IN $(userIds)
4) Finally, the products return to the client.
Please read about gateway https://www.nginx.com/blog/building-microservices-using-an-api-gateway/
This is still a theory in my mind.
I'm rebuilding my backend by splitting things into microservices. The microservices I'm imagining for starting off are:
- Order (stores order details and status of each order)
- Customer (stores customer details, addresses, orders booked)
- Service Provider (stores service provider details, status & location of each service provider, order(s) currently being processed by the service provider, etc.)
- Payment (stores payment info for each order)
- Channel (communicates with customers via email / SMS / mobile push)
I hope to be able to use PUB/SUB to create a message with corresponding data, which can be used by any other microservice subscribing to that message.
First off, I understand the concept that each microservice should have complete code & data isolation (thus, on different instances / VMs); and that all microservices should communicate strictly using HTTP REST API contracts.
My doubts are as follows:
To show a list of orders, I'll be using the Order DB to get all orders. In each Order document (I'll be using MongoDB for storage), I'll be having a customer_id Foreign Key. Now the issue of resolving customer_name by using customer_id.
If I need to show 100 orders on the page and go with the assumption that each order has a unique customer_id associated with it, then will I need to do a REST API call 100 times so as to get the names of all the 100 customer_ids?
Or, is data replication a good solution for this problem?
I am envisioning something like this w.r.t. PUB/SUB: The business center personnel mark an order as assigned & select the service provider to allot to that order. This creates a message on the cross-server PUB/SUB channel.
Then, the Channel microservice (which is on a totally different instance / VM) captures this message & sends a Push message & SMS to the service provider's device using the data within the message's contents.
Is this possible at all?
UPDATE TO QUESTION 2: I want the Order microservice to be completely independent of any other microservices that will be built upon / side-by-side it. Channel microservice is an example of a microservice that depends upon events taking place within Order microservice.
Also, please guide me as to what all technologies / libraries to use.
What I'll be developing on:
Java
MongoDB
Amazon AWS instances for each microservice.
Would appreciate anyone's help on this.
Thanks!
#1
If I need to show 100 orders and each order has a unique customer_id, will I need to do 100 REST API call?
No, just make 1 request with 100 order_id(s) and return a dictionary of order_id <=> customer_id
#2
It's a single request
POST
/orders/new
{
"selected_service_provider_id" : "123"
...
}
Which can return you order_id and you can print it locally for the customer or track progress or what have you.
On the server side, you receive an order and process it. Processing can include sending an SMS at some stage. This functionality can be implemented inside original service that received this request or as a separate call to another dedicated service.
To your first question, you don't need to do 100 queries, just one with the array of your 100 documents, like the following:
db.collection.find( { _id : { $in : [1,2,3,4] } } );
https://stackoverflow.com/a/7713461/1384539
I know this question is 1 year old, but I would like to add my answer to the first point.
One option would be to use some form of CQRS and store on the OrderDB also some of the user details when creating an order. This way when you have to show the list of orders you already have all the details you need. Also, the order document would represent a photograph of the user state at the moment of the order creation.
Of course, in case you don't have the user details when storing the order, you just need to make a GET call to the User Service, but that would be 1 call, not 100.
I'm developing an Azure application using this stack:
(Client) Angular/Breeze
(Server) Web API/Breeze Server/Entity Framework/SQL Server
With every request I want to ensure that the user actually has the authorization to execute that action using server-side code. My question is how to best implement this within the Breeze/Web API context.
Is the best strategy to:
Modify the Web API Controller and try to analyze the contents of the
Breeze request before passing it further down the chain?
Modify the EFContextProvider and add an authorization test to
every method exposed?
Move the security all into the database layer and make sure that a User GUID and Tenant GUID are required parameters for every query and only return relevant data?
Some other solution, or some combination of the above?
If you are using Sql Azure then one option is to use Azure Federation to do exactly that.
In a very simplistic term if you have TenantId in your table which stores data from multiple tenants then before you execute a query like SELECT Col1 FROM Table1, you execute USE FEDERATION... statement to restrict the query results to a particular TenantId only, and you don't need to add WHERE TenantId=#TenantId to your query,
USE FEDERATION example: http://msdn.microsoft.com/en-us/library/windowsazure/hh597471.aspx
Note that use of Sql Azure Federation comes with lots of strings attached when it comes to Building a DB schema one of the best blog I have found about it is http://blogs.msdn.com/b/cbiyikoglu/archive/2011/04/16/schema-constraints-to-consider-with-federations-in-sql-azure.aspx.