Taken from
http://outworkers.com/blog/post/a-series-on-phantom-part-1-getting-started-with-phantom
I'm trying to connect to a Cassandra cluster that has multiple nodes like this:
object Defaults {
val hosts = Seq("Cassnode1.company.com", "Cassnode2.company.com", "Cassnode3.company.com")
val Connector = ContactPoints(hosts).keySpace("whatever")
}
If for some reason, one of the nodes does not exist, I get:
Caused by: java.lang.IllegalArgumentException: Cassnode3.company.com: unknown error
If I remove this node from the hosts Seq everything works fine.
I am using phantom dsl version "1.28.12" and I was wondering if this is the expected behavior as I assumed that whenever one of the listed hosts does not exist/is not available the application would use the remaining ones.
Is there a way to test connectivity to the nodes before passing the list to the ContactPoints?
Thanks!
There isn't, the whole point of ContactPoints which just leverage the underlying ClusterBuilder is to deal "with that kind of thing" for you. You can also pass in an error handling function to deal with some of the issues, so that may make things easier.
By gut feeling is that the rest of the nodes have some kind of IP mapping in your /etc/hosts, but you are missing the one for Cassnode3.company.com. Remember they all need to be resolved to an IP address otherwise they are no good to the ClusterBuilder.
I would strongly recommend upgrading to a 2.1.3 version of phantom, but in this particular case the culprit is almost entirely your local dev setup, which just needs an IP mapping for that third URL.
Related
This question is for pg-promise, its recommended usage pattern & based on following assumption,
It does-not make sense to create more than a single pgp instance, if they are connecting to same DB(also enforced by the good warning message of "Creating a duplicate database object for the same connection.")
Given:
I have 2 individual packages which need DB connection, currently they take connection string in constructor from outside and create connection object inside them, which leads to the warning of duplicate connection object and is fair as they both talk to same DB and there is a possibility for optimisation here(since i am in control of those packages).
Then: To prevent this, i thought of implementing dependency injection, for which i pass a resolve function in libraries constructor which gives them the DB connection object.
Issue: There are some settings which are at top level like parsers and helpers and transaction modes which may be different for each of these packages what is the recommendation for such settings or is there a better patterns to address these issues.
EG:
const pg = require('pg-promise');
const instance = pg({"schema": "public"});
instance.pg.types.setTypeParser(1114, str => str);//UTC Date which one library requires other doesnt
const constring = "";
const resolveFunctionPackage1 = ()=>instance(constring);
const resolveFunctionPackage2 = ()=>instance(constring);
To sum up: What is the best way to implement dependency injection for pg-promise?
I have 2 individual packages which need DB connection, currently they take connection string in constructor from outside and create connection object inside them
That is a serious design flaw, and it's is never gonna work well. Any independent package that uses a database must be able to reuse an existing connection pool, which is the most valuable resource when it comes to connection usage. Head-on duplication of a connection pool inside an independent module will use up existing physical connections, and hinder performance of all other modules that need to use the same physical connection.
If a third-party library supports pg-promise, it should be able to accept instantiated db object for accessing the database.
And if the third-party library supports the base driver only, it should at least accept an instantiated Pool object. In pg-promise, db object exposes the underlying Pool object via db.$pool.
what happens when they want to set conflicting typeparsers?
There will be a conflict, because pg.types is a singleton from the underlying driver, so it can only be configured in one way. It is an unfortunate limitation.
The only way to avoid it, is for reusable modules to never re-configure the parsers. It should only be done within the actual client application.
UPDATE
Strictly speaking, one should avoid splitting a database-access layer of an application into multiple modules, there can be a number of problems to follow that.
But specifically for separation of type parsers, the library supports setting custom type parsers on the pool level. See example here. Note that the update is just for TypeScript, i.e. in JavaScript clients it has been working for awhile.
So you still can have your separate module create its own db object, but I would advise that you limit its connection pool size to the minimum then, like 1:
const moduleDb = pgp({
// ...connection details...
max: 1, // set pool size to just 1 connection
types: /* your custom type parsers */
});
I'm using the Kubernetes golang operator sdk to implement an operator that manages RabbitMQ queues. I'm wondering if there's a way for k8s to enforce immutability of particular spec fields on my custom resource. I have the following golang struct which represents a rabbitMQ queue and some parameters to have it bind to a rabbitMQ exchange:
type RmqQueueSpec struct {
VHost string `json:"vhost,required"`
Exchange string `json:"exchange,required"`
RoutingKey string `json:"routingKey"`
SecretConfig map[string]string `json:"secretConfig"`
}
The reason why I want immutability, specifically for the VHost field, is because it's a parameter that's used to namespace a queue in rabbitMQ. If it were changed for an existing deployed queue, the k8s reconciler will fail to query rabbitMQ for the intended queue since it will be querying with a different vhost (effectively a different namespace), which could cause the creation of a new queue or an update of the wrong queue.
There are a few alternatives that I'm considering such as using the required ObjectMeta.Name field to contain both the concatenated vhost and the queuename to ensure that they are immutable for a deployed queue. Or somehow caching older specs within the operator (haven't figured out exactly how to do this yet) and doing a comparison of the old and current spec in the reconciler returning an error if VHost changes. However neither of these approaches seem ideal. Ideally if the operator framework could enforce immutability on the VHost field, that would be a simple approach to handling this.
This validation is possible by using the ValidatingAdmissionWebhook with future support coming via CRD's OpenAPI validation.
https://github.com/operator-framework/operator-sdk/issues/1587
https://github.com/kubernetes/kubernetes/issues/65973
AFAIK this is not yet available to CRDs. Our approach is generally to use the object name as the default name of the object being controlled (vhost name in this case) so it just naturally works out okay.
There is a Java and CLI-interface to query Yarn RM for node-to-nodelabel (and inverse) mappings. Is there a way to do this via the REST-API as well?
An initial RM-API search revealed only node-label based job submissions as an option.
Sadly that is actually broken in MapR-Hadoop (6.1 as of 6/6/19), so my code has to work around that, by implementing the correct scheduling itself. This works (barely - more broken APIs here as well) using the YarnClient Java API.
But as I want to schedule jobs against different resource managers at the same time, behind firewalls, the REST-API is the most compelling option to achieve this, and the YarnClient API's RPC backend can't be easily transported.
My current worst-case solution would be to parse the YARN-WebUI in some way.
The only solution I found so far:
Request /ws/v1/cluster/nodes - this gets you all nodes.
FlatMap/Distinct on each node's nodeLabels, if you need just the list of node labels. Filter by nodeLabel, if you need all nodes for a specified label.
This does mean, that you always have to query all nodes, then sort/filter/arrange by NodeLabels, which is a lot of client-side magic. But apparently there's no GetNodesToLabel or even GetClusterNodeLabels to help us out.
I assume getLabelsToNodes is just a client-side implementation, as the protocol doesn't define the API, so that's right out the window for REST, unless implemented in the WebService.
The Specification (Part 3: Address Space Model) of OPC UA says
5.2.2 NodeId
... A Server shall persist the NodeId of a Node, that is, it shall not
generate new NodeIds when rebooting.
but how can this be?
NodeId is a combination from a NamespceIndex and Identifier. NamespceIndex can be changed when the Server is restarting. see:
http://documentation.unified-automation.com/uasdkhp/1.0.0/html/_l2_ua_node_ids.html
For this reason, a Client should not persist the namespace index without storing the namespace URI as well, because a namespace URI represented by index “2” during one session could be represented by index “5” during the next session
Also the use of FolderType with e.g. "Files" as Items speak again this, or should the server store the NodeId it uses for File-X to assign it right again after restart?
What for is "GenericModelChangeEventType" if no NodeId can be created?
Client: I thought useing BrowsePath-Path (e.g. "Objects.Server.ServerStatus.CurrentTime" (* ) ) for addressing NodeIds and then using the NodeId while the clinet session to access the nodes is a good approach. Also because Companion Specifications defines the browsename so I might by save. Is this a good idea? ( *need attention on collisions caused by different namespaces)
Server: How should the Server behave when it needs to generate/create new NodeIds. Need the NodeIds to be unambiguous all the time or just for the Server runtime. I know some Servers are using NodeIds with String-Typed Identifiers and this String-Identifiers are made from the BrowsePath e.g. "ns=1;s=Server.ServerStatus.CurrentTime". But I don't like this...
What the OPC UA spec means when it says " A Server shall persist the NodeId of a Node, that is, it shall not generate new NodeIds when rebooting." is as follows: The NodeIds, when seen as a combination of namespace URI and identifier, must not change. The server may or may not reassign namespace indices after reboot - but the resulting namespaceURI/Identifier must not change. So, if on the first run I had a node with Identifier 1234 and namespace index 7, and that namespace index corresponded to "http://mynamespace.mycompany.com" in the namespace table, on the second run the same node may have Identifier 1234, but the namespace index 8, as long as in the new NamespaceTable index 8 now corresponds to "http://mynamespace.mycompany.com".
I think the Unified Automation SDK technically violates the spec in this regard. The recommendation it suggests is good practice for client implementations either way, but as you pointed out, shouldn't strictly be necessary.
Also the use of FolderType with e.g. "Files" as Items speak again this, or should the server store the NodeId it uses for File-X to assign it right again after restart?
I'm not sure what you're asking here.
What for is "GenericModelChangeEventType" if no NodeId can be created?
That's not what is being said here. Nodes can be created and deleted and the structure of objects and variables can change. All the spec is saying is that given Node "Foo" with NodeId "ns=1;s=Foo" it should have the same NodeId if the server reboots.
I thought useing BrowsePath-Path (e.g. "Objects.Server.ServerStatus.CurrentTime" (* ) ) for addressing NodeIds and then using the NodeId while the clinet session to access the nodes is a good approach.
Browse paths are for programming against types. The approach suggested by the Unified Automation SDK docs is the safe one for persisting NodeIds in your client.
How should the Server behave when it needs to generate/create new NodeIds. Need the NodeIds to be unambiguous all the time or just for the Server runtime. I know some Servers are using NodeIds with String-Typed Identifiers and this String-Identifiers are made from the BrowsePath e.g. "ns=1;s=Server.ServerStatus.CurrentTime". But I don't like this...
Create them however you like in the Namespaces you control, it's up to you. Using string-based NodeIds allows you to easily "derive" the NodeId from certain other sources, though, e.g. from the address of a variable in a PLC or something similar.
we have two distinct jax-ws #WebServices with different target namespaces. however, these two services each declare a single #WebMethod with same name, but different parameter types (which in both cases is a single custom java class). we deploy them to websphere servers and have our WSDL/XSD generated dynamically.
apparently, we end up with two sets of WSDL/XSD where we have different ports, different namespaces for everything, but same operation name and same request/response type names and element names (again, namespaces are different).
when we call these webservices, one of them works, the other is not able to unmarshall the message because it magically expects the request type from the first one, which is clearly said in the exception that we receive.
i was able to reproduce this locally, using only jaxb and the issue appears to be as follows: SOAP messages like "soap:Body/ns1:request1" and "soap:Body/ns2:request1" somehow conflict with eachother when they are unmarshalled within the same JAXBContext and the one jaxb-annotated class referenced last wins.
WSDL spec says nothing about this (as ports are different), and even the internet doesn't know about this too much at all, so i wonder if there's something really wrong with what we do. common sense suggests this must be entirely possible.
the issue is very similar to this [unanswered] post: http://www.coderanch.com/t/463070/Web-Services/java/method-classes-conflict. and is exactly the same as this [unanswered really] post: Jaxb objects with same name (the guy doesn't really explain, why this style of doing things is not recommended).
the question really is: is it possible to declare two jax-ws services with methods having same names but different arguments and then deploy them in one web application?