Is it possible to configure k8 so that based on some data in a request it is always routed to the same hostname?
Perhaps something using stateful sets? Can we do a modulo based routing so that a request like
POST /myroute
{
_id : 1
}
will always go to one hostname,
and a request like
POST /myroute
{
_id : 2
}
will always go to another??
Or is there away so that each host can ignore requests based on the payload. This is a pretty standard routing method, similar to a distributed hash table.
Sorry if I'm not explaining this well. But does anyone have any example of this type of k8 configuration?
You can use vulcand as Ingress controller.It has very good routing-language
Also nginx plus as Ingress can be a solution
Related
Are there any recommendations on querying remote state stores between application instances that are deployed in Kubernetes? Our application instances are deployed with 2 or more replicas.
Based on documentation
https://kafka.apache.org/10/documentation/streams/developer-guide/interactive-queries.html#id7
streams.allMetadataForStore("word-count")
.stream()
.map(streamsMetadata -> {
// Construct the (fictituous) full endpoint URL to query the current remote application instance
String url = "http://" + streamsMetadata.host() + ":" + streamsMetadata.port() + "/word-count/alice";
// Read and return the count for 'alice', if any.
return http.getLong(url);
})
.filter(s -> s != null)
.findFirst();
will streamsMetadata.host() result in the POD IP? And if it does, will the call from this pod to another be allowed? Is this the correct approach?
streamsMetadata.host()
This method returns whatever you configured via application.server configuration parameter. I.e., each application instance (in your case each POD), must set this config to provide the information how it is reachable (e.g., its IP and port). Kafka Streams distributes this information for you to all application instances.
You also need to configure your PODs accordingly to allow sending/receiving query request via the specified port. This part is additional code you need to write yourself, i.e., some kind of "query routing layer". Kafka Streams has only built-in support to query local state and to distribute the metadata about which state is hosted where; but there is no built-in remove query support.
An example implementation (WordCountInteractiveQueries) of a query routing layer can be found on Github: https://github.com/confluentinc/kafka-streams-examples
I would also recommend to checkout the docs and blog post:
https://docs.confluent.io/current/streams/developer-guide/interactive-queries.html
https://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/
I'm using the Kubernetes golang operator sdk to implement an operator that manages RabbitMQ queues. I'm wondering if there's a way for k8s to enforce immutability of particular spec fields on my custom resource. I have the following golang struct which represents a rabbitMQ queue and some parameters to have it bind to a rabbitMQ exchange:
type RmqQueueSpec struct {
VHost string `json:"vhost,required"`
Exchange string `json:"exchange,required"`
RoutingKey string `json:"routingKey"`
SecretConfig map[string]string `json:"secretConfig"`
}
The reason why I want immutability, specifically for the VHost field, is because it's a parameter that's used to namespace a queue in rabbitMQ. If it were changed for an existing deployed queue, the k8s reconciler will fail to query rabbitMQ for the intended queue since it will be querying with a different vhost (effectively a different namespace), which could cause the creation of a new queue or an update of the wrong queue.
There are a few alternatives that I'm considering such as using the required ObjectMeta.Name field to contain both the concatenated vhost and the queuename to ensure that they are immutable for a deployed queue. Or somehow caching older specs within the operator (haven't figured out exactly how to do this yet) and doing a comparison of the old and current spec in the reconciler returning an error if VHost changes. However neither of these approaches seem ideal. Ideally if the operator framework could enforce immutability on the VHost field, that would be a simple approach to handling this.
This validation is possible by using the ValidatingAdmissionWebhook with future support coming via CRD's OpenAPI validation.
https://github.com/operator-framework/operator-sdk/issues/1587
https://github.com/kubernetes/kubernetes/issues/65973
AFAIK this is not yet available to CRDs. Our approach is generally to use the object name as the default name of the object being controlled (vhost name in this case) so it just naturally works out okay.
The Specification (Part 3: Address Space Model) of OPC UA says
5.2.2 NodeId
... A Server shall persist the NodeId of a Node, that is, it shall not
generate new NodeIds when rebooting.
but how can this be?
NodeId is a combination from a NamespceIndex and Identifier. NamespceIndex can be changed when the Server is restarting. see:
http://documentation.unified-automation.com/uasdkhp/1.0.0/html/_l2_ua_node_ids.html
For this reason, a Client should not persist the namespace index without storing the namespace URI as well, because a namespace URI represented by index “2” during one session could be represented by index “5” during the next session
Also the use of FolderType with e.g. "Files" as Items speak again this, or should the server store the NodeId it uses for File-X to assign it right again after restart?
What for is "GenericModelChangeEventType" if no NodeId can be created?
Client: I thought useing BrowsePath-Path (e.g. "Objects.Server.ServerStatus.CurrentTime" (* ) ) for addressing NodeIds and then using the NodeId while the clinet session to access the nodes is a good approach. Also because Companion Specifications defines the browsename so I might by save. Is this a good idea? ( *need attention on collisions caused by different namespaces)
Server: How should the Server behave when it needs to generate/create new NodeIds. Need the NodeIds to be unambiguous all the time or just for the Server runtime. I know some Servers are using NodeIds with String-Typed Identifiers and this String-Identifiers are made from the BrowsePath e.g. "ns=1;s=Server.ServerStatus.CurrentTime". But I don't like this...
What the OPC UA spec means when it says " A Server shall persist the NodeId of a Node, that is, it shall not generate new NodeIds when rebooting." is as follows: The NodeIds, when seen as a combination of namespace URI and identifier, must not change. The server may or may not reassign namespace indices after reboot - but the resulting namespaceURI/Identifier must not change. So, if on the first run I had a node with Identifier 1234 and namespace index 7, and that namespace index corresponded to "http://mynamespace.mycompany.com" in the namespace table, on the second run the same node may have Identifier 1234, but the namespace index 8, as long as in the new NamespaceTable index 8 now corresponds to "http://mynamespace.mycompany.com".
I think the Unified Automation SDK technically violates the spec in this regard. The recommendation it suggests is good practice for client implementations either way, but as you pointed out, shouldn't strictly be necessary.
Also the use of FolderType with e.g. "Files" as Items speak again this, or should the server store the NodeId it uses for File-X to assign it right again after restart?
I'm not sure what you're asking here.
What for is "GenericModelChangeEventType" if no NodeId can be created?
That's not what is being said here. Nodes can be created and deleted and the structure of objects and variables can change. All the spec is saying is that given Node "Foo" with NodeId "ns=1;s=Foo" it should have the same NodeId if the server reboots.
I thought useing BrowsePath-Path (e.g. "Objects.Server.ServerStatus.CurrentTime" (* ) ) for addressing NodeIds and then using the NodeId while the clinet session to access the nodes is a good approach.
Browse paths are for programming against types. The approach suggested by the Unified Automation SDK docs is the safe one for persisting NodeIds in your client.
How should the Server behave when it needs to generate/create new NodeIds. Need the NodeIds to be unambiguous all the time or just for the Server runtime. I know some Servers are using NodeIds with String-Typed Identifiers and this String-Identifiers are made from the BrowsePath e.g. "ns=1;s=Server.ServerStatus.CurrentTime". But I don't like this...
Create them however you like in the Namespaces you control, it's up to you. Using string-based NodeIds allows you to easily "derive" the NodeId from certain other sources, though, e.g. from the address of a variable in a PLC or something similar.
I have a url path that looks like this:
/{identifier}/rest/of/resource/path
If the identifier is A then the request should go to service_I. If the identifier is B then the request should also go to service_I. If the identifier is C, then the request should go to service_II, and so on.
Later on a new identifiers M and N is added to the system and their requests should be routed to service_IV.
Is it possible to dynamically configure a Spring cloud zuul proxy to perform the tasks described above?
Edit
This question offered contains a different way to examine the question.
In it Zuul has the following configuration:
zuul:
routes:
<service_id>:
path: /path/**
Zuul will collaborate with Eureka to find the service-id and return the host parameters so that the service can be accessed. What if instead of /path we have /{userID} and the userID instances are distributed across several service_id hosts?
Can Zuul / the DiscoveryClient query Eureka for both the service_id and the userID to figure out which host is hosting the particular userID?
You would need to write a custom ZuulFilter to accomplish this. Take a look at the PreDecorationFilter for some hints as this is the filter responsible for handling /path where the path is a service-id (among other things).
I've seen tons of examples on how to structure URL's for basic CRUD operations but seen very little when talking about more Command-like operations, or application service calls.
For example, let's say in my application service I have a call like RemoveOldOrders(int customerId) that would remove any order from the system that is over 2 years old for a customer with id "customerId". What would the URL look like on my restful service? What would the payload of the call look like? What HTTP method (POST?) would I use?
My thought is it would be something like this:
/Customer/1/RemoveOldOrders as a POST, with an empty body (as the customerID would come from the url).
Are there good guidelines on something like this?
Update: I feel like I need to clarify my question a bit in lieu of the comment about a possible duplicate post (yes, that post is asking essentially the same thing but I don't really feel the question was answered well).
What if I want to perform an operation against a resource but that operation doesn't fit into the standard HTTP verbs?
Another example: my application is hooked into an ESB and there needs to be a way to force a projection of my resource onto the ESB for processing? In my current SOAP-based web service I'd have a method like:
ExportCustomer(int customerId)
Now, in the case of a RESTful service, how could I represent this action in a uri? Option 1 from Brian Kelly's answer seems like the most logical, something like:
POST http://someapp/api/customer/1/export
or would:
POST http://someapi/api/customer/export/1
be better?
Anytime you want to model verbs like "remove", you should think of DELETE. Similarly, for "create" think POST (and/or maybe PUT), for "read" think GET and for "update" think PUT (or maybe PATCH).
So for your example of "remove old orders", you should definitely use DELETE. Now your only remaining challenge is how to identify the orders that should be removed. Once you figure that out, the URI scheme will fall into place with it.
Here are some options:
DELETE http://your-api.com/old-orders
Here, the meaning and range of old-orders will be determined by the server receiving this request. This frees the client from having to do so, but removes their ability to change that range.
GET http://your-api.com/order-query?days-older-than=730
This returns a Location URI of http://your-api.com/order-query-result/{some ID} that represents the set of old appointments. Then you can simply issue a DELETE on that URI to purge the old records in one fell swoop.
Instead of forcing the client to remember to issue deletion commands of this type, offer some kind of configuration resource that can be manipulated via your API to set some field like purgeRecordsOlderThanDays=730, and just let the server do it automatically for you in a cron-like fashion. That would be my preferred approach.
For exporting, then you need to remove the "export" verb and replace it with an ESB representation of the resource: -
GET http://someapp/customer/{id}
The response includes a link to export: GET http://someapp/customer/{id}/ESB
Getting on the export link returns the ESB representation (with appropriate content type)
For example, let's say in my application service I have a call like
RemoveOldOrders(int customerId) that would remove any order from the
system that is over 2 years old for a customer with id "customerId".
What would the URL look like on my restful service? What would the
payload of the call look like? What HTTP method (POST?) would I use?
RemoveOldOrders(int customerId)
DELETE /oldOrders {customerId: id}
DELETE /customer/{id}/orders?old=true
etc...
You should read more about uniform interface / resource identifiers and the HTTP method specification.
The URL does not really matters. What matters that you should have resource with resource identifiers (URLs) and you have to manipulate them by using an uniform (standard) interface, like the call of HTTP methods.