For multi tenancy support, I would like to create like a datastore concept limiting the storage per tenant across all namespaces in the tenant. Kubernetes website [1] says "given namespace", but I want to be able to set resource quota based on storage class but not limited to "per namespace". How can I do that ?
Not directly. The quota system is per namespace only. But of course you can make a validating webhook yourself which implements whatever logic you want.
Related
I have come across a need that I need to serve application users based on their geo-location.
One possibility, I could think of it to have application installed on multiple k8s clusters hosted in different region and then load-balance the traffic based on geo-location of the users.
While exploring this idea, I came across several articles on "Kubernetes Cluster Federation" (e.g. https://kubernetes.io/blog/2016/10/globally-distributed-services-kubernetes-cluster-federation/). But seems like this functionality has been retired as mentioned in https://github.com/kubernetes-retired/federation.
Does someone know:
If there is any alternative for "Kubernetes Cluster Federation"?
Is there any other solution/s to address the need of serving users based on their geo-location?
If we leave the application part, is there any way to store the data in same geo-location?
Thanks!
https://github.com/kubernetes-sigs/kubefed is a successor to the "Kubernetes Cluster Federation", though I am not sure what is its current state. If you want to deploy a global loadbalancer, I suggest to have a look into https://www.k8gb.io/ .
...k8s clusters hosted in different region and then load-balance the traffic based on geo-location of the users
If you determine the user location simply by the network location, you can use DNS geolocation routing capability such as Route 53 to reach nearest services. In this context k8s federation is not required.
If we leave the application part, is there any way to store the data in same geo-location?
Apart from global scale database solution such as Aurora, Spanner, your application can point to a centralize database that resides in one of the region; if the increase latency is acceptable.
I’m looking to build a distributed Access Control system for a microservice platform. I’m considering using Mongodb as my database technology. My system design objectives are as follows:
Policy Enforcement should be distributed - If any given Policy
Enforcement Point (PEP) experiences downtime, only the application
that the PEP serves should be affected.
Policy Decisions should be
distributed - We don’t want the whole platform to experience downtime
because a central Policy Decision Point (PDP) is experiencing
downtime. We only want it to affect the application that it serves.
Policy Administration should be centralized - Creating a centralized
policy administration interface provides the ability for any system
(including a UI) to understand what rights an individual has, and by
establishing a common interface it allows us to more easily audit
changes to access across a whole platform.
Policy Information (context) is distributed - We don’t get to choose this if we are
building a distributed microservice platform. We can centralize the
retrieval of additional context by aggregating data that is needed to
make access control decisions into a single place, but the data
sources are still distributed.
I’m considering building a system like the one shown below. The idea is that Access Policies are administered by a central Policy Admin API. This API manages Policies that are persisted to a mongodb cluster with a 3 member replica set backing it. I would like other APIs in the platform to have a dedicated policy-query-api (Policy Decision Point) that is deployed along side it to make Access Control decisions pertinent to the API. The idea is that if any one of the policy-query-apis goes down, only the API that it serves will be affected.
I want changes to Policies to be governed by the Policy Admin API and I would like the changes to be replicated across each mongo instance that is used by each of the policy-query-apis.I don’t want the mongo replicas for each policy-query-api to affect a write to the primaries.
I also don’t need immediate data consistency (less than 5 sec latency), but I would like the data replication to be handled at the database layer if possible. The technology is already built to handle this and I don’t want to reinvent the wheel at the application layer if possible.
I’ve looked at the documentation on Replica Set Members and I’ve pretty thoroughly reviewed the documentation on Replica Sets in Mongo. It seems like having a Hidden Member or Delayed Member would be a good fit for my use case. Do you agree? Also, I’m concerned about the 50 member replica set limit 1. Since each one of these replicas would serve an API in my platform, if there exceeded more than 50 microservices (which is quite likely) how would I manage replication like this?
Just so that I understand, you are asking about:
one standalone (?? your picture suggests standalone but you are asking about 50 node RS limit) node per application, data mirrored to standalone from the master RS
the application only queries its local standalone
MongoDB provides read preference nearest for the use case of reading data from local nodes. Importantly the nearest read preference still provides availability if your local node is unavailable - the next closest (roughly) node will be used in this case. Your proposed architecture would take the application down every time its local database node needs to be restarted for version upgrades.
You may also look into tag sets.
Additionally, MongoDB allows specifying priorities on nodes for election purposes. If you put all of your MongoDB nodes into the same RS, you can use priorities to have one of the 3 designated "main" servers be primaries if any of them are available.
So I am doing some research into using Service Fabric for a very large application. One thing I need to have is a service that is partitioned by name, which seems fairly trivial at the application manifest level.
However, I really would like to be able to add and remove named partitions on the fly without having to republish the application.
Each partition represents our equivalent of a tenant, and we want to have a backend management app to add new tenants.
Each partition will be a long-running application that fires up a TCP server that uses a custom protocol, and I'll need to be able to query for the address by name from the cluster.
Is this possible with Service Fabric, and if so is there any documentation on this, or something I should be looking for?
Each partition represents our equivalent of a tenant, and we want to have a backend management app to add new tenants.
You need to rethink your model. Partitioning is for distributing data so it accessible fast, for read and write. But within the same logical container.
If you want to do some multitenant in Service Fabric you can deploy an Application multiple times to the cluster.
From Visual Studio it seems you can only have one instance of an Application. This is because in the ApplicationManifest.xml there are DefaultServices defined. This is okay for developing on the local Service Fabric cluster. For production you might want to consider deploying the application with powershell, this will open up the possibility to deploy the same application multiple times with settings for each instance(like: tenant name, security, ... )
And not only Applications can be deployed multiple times, stateful/stateless services as well. So you could have one application and for each tenant you deploy a service of a certain type. Services are findable via the naming service inside Service Fabric, see the FabricClient class for more info on that.
It is not possible to change the partition count for an existing application.
From https://azure.microsoft.com/en-us/documentation/articles/service-fabric-concepts-partitioning/#plan-for-partitioning (emphasis mine):
In rare cases, you may end up needing more partitions than you have initially chosen. As you cannot change the partition count after the fact, you would need to apply some advanced partition approaches, such as creating a new service instance of the same service type. You would also need to implement some client-side logic that routes the requests to the correct service instance, based on client-side knowledge that your client code must maintain.
You are encouraged to do up-front capacity planning to determine the maximum number of partitions you will need - and if you end up needing more, you'll need to implement some special client side handling to cope.
We had the same problem and ended up creating an instance of the service for each tenant. This is pretty easy to do and will scale to any number of tenants.
Is it possible to create a Google Cloud Storage bucket and restrict it's access to one IP? I plan on using a bucket to store data that only I would ever need to upload/download from.
Check out VPC Service Controls. This no cost feature allows you to restrict client access to project resources based on a variety of attributes, including source IP address, and includes support for Cloud Storage buckets.
No, that's not available at this time.
Looking at the design docs for Resource Quotas, it appears they can be set on a namespace level. That makes sense, to limit total resource usage of a Namespace.
In my organization, there are many teams, each working on their own microservice. The microservices need to communicate with one another, so I want them all to be in the same namespace, for ease of service discovery. However, I'd like to put some protections in place so that one of the microservices doesn't accidentally take over all the resources in the namespace.
I was planning on giving each microservice its own Service Account. Ideally, I would like to be able to set resource quotas on those service accounts. But I can't find any way in the documentation to do that.
Is that possible? If not, is something like this planned for the future?
Thanks.
Quota is per namespace. I don't think it is likely to be updated to subdivide a namespace by user (or service account). Right now, resources don't have the concept of an "owner", so it isn't possible to attribute a pod in a namespace to a particular creator. It gets even muddier when replication controllers create pods automatically. If one person creates a replication controller and another person scales it up, which one of them "owns" those pods? Those are the sorts of questions that would need clear answers before ownership could factor into quota.