I'm using scala akka-camel with http4 component (2.12.2 version).
I'm creating a Camel producer with endpoint:
def endpointUri = "https4://host-path" +
"?bridgeEndpoint=true" +
"&httpClientConfigurer=#configurer" +
"&clientConnectionManager=#manager"
where configurer is an HttpClientConfigurer registered in Camel context registry (the same principle applies to manager).
When I'm sending a CamelMessage to that endpoint I can see at akka logs this:
DEBUG o.a.c.component.http4.HttpComponent - Creating endpoint uri https4://host-path?bridgeEndpoint=true&httpClientConfigurer=#configurer&clientConnectionManager=#manager
DEBUG o.a.camel.util.IntrospectionSupport - Configured property: clientConnectionManager on bean: Endpoint["https4://host-path?bridgeEndpoint=true&httpClientConfigurer=#configurer&clientConnectionManager=#manager"] with value: org.apache.http.impl.conn.PoolingClientConnectionManager#3da3d36f
DEBUG o.a.camel.util.IntrospectionSupport - Configured property: bridgeEndpoint on bean: Endpoint["https4://host-path?bridgeEndpoint=true&httpClientConfigurer=#configurer&clientConnectionManager=#manager"] with value: true
INFO o.a.c.component.http4.HttpComponent - Registering SSL scheme https on port 443
INFO o.a.c.component.http4.HttpComponent - Registering SSL scheme https4 on port 443
So httpClientConfigurer is not configured and I don't know why it's ignoring this parameter. I've been looking for any related issue at Apache Camel issue tracker but I have found nothing similar.
Any idea?
Thanks in advance.
Finally, it's resolved. I haven't used the none of clientConnectionManager or httpClientConfigurer. I've used SSLContextParams and a trait called TlsConfigurer that is meant to be mixed-in with a Producer.
I want to use different X509 certificates, so, as Camel suggests:
Important: Only one instance of org.apache.camel.util.jsse.SSLContextParameters is supported per HttpComponent. If you need to use 2 or more different instances, you need to define a new HttpComponent per instance you need.
Therefore, TlsConfigurer configure method must be able to get a http4 component instance from camel context and then apply the SSLContextParams and add the modified instance as a new component to camel context.
This is how it looks:
import org.apache.camel.component.http4.HttpComponent
import org.apache.camel.util.jsse._
trait TlsConfigurer {
self: {val camel: akka.camel.Camel} =>
def configure(
componentName: String,
keyStorePath:String,
trustStorePath:String,
password: String) {
val ksp = new KeyStoreParameters
ksp.setResource(keystorePath)
ksp.setPassword(password)
val kmp = new KeyManagersParameters
kmp.setKeyStore(ksp)
kmp.setKeyPassword(password)
val scp = new SSLContextParameters
scp.setKeyManagers(kmp)
val httpComponent =
camel.context.getComponent("http4",classOf[HttpComponent])
httpComponent.setSslContextParameters(scp)
camel.context.addComponent(componentName, httpComponent)
}
}
This way I can create two different end-points: http-client1://... and http-client2://... and manage their certificates in a separate way.
httpClientConfigurer is not set to the HttpEndpoint by using IntrospectionSupport, so you don't see the debug log.
I think we about to find out the configurer is called when you add some log in the customer configurer.
Related
I am using flink 1.8.0 and I am trying to query my job state.
val descriptor = new ValueStateDescriptor("myState", Types.CASE_CLASS[Foo])
descriptor.setQueryable("my-queryable-State")
I used port 9067 which is the default port according to this, my client:
val client = new QueryableStateClient("127.0.0.1", 9067)
val jobId = JobID.fromHexString("d48a6c980d1a147e0622565700158d9e")
val execConfig = new ExecutionConfig
val descriptor = new ValueStateDescriptor("my-queryable-State", Types.CASE_CLASS[Foo])
val res: Future[ValueState[Foo]] = client.getKvState(jobId, "my-queryable-State","a", BasicTypeInfo.STRING_TYPE_INFO, descriptor)
res.map(_.toString).pipeTo(sender)
but I am getting :
[ERROR] [06/25/2019 20:37:05.499] [bvAkkaHttpServer-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(bvAkkaHttpServer)] Error during processing of request: 'org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:9067'. Completing with 500 Internal Server Error response. To change default exception handling behavior, provide a custom ExceptionHandler.
java.util.concurrent.CompletionException: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /127.0.0.1:9067
what am I doing wrong ?
how and where should I define QueryableStateOptions
So If You want to use the QueryableState You need to add the proper Jar to Your flink. The jar is flink-queryable-state-runtime, it can be found in the opt folder in Your flink distribution and You should move it to the lib folder.
As for the second question the QueryableStateOption is just a class that is used to create static ConfigOption definitions. The definitions are then used to read the configurations from flink-conf.yaml file. So currently the only option to configure the QueryableState is to use the flink-conf file in the flink distribution.
EDIT: Also, try reading this]1 it provides more info on how does Queryable State works. You shouldn't really connect directly to the server port but rather You should use the proxy port which by default is 9069.
Compiler error when using example provided in Flink documentation. The Flink documentation provides sample Scala code to set the REST client factory parameters when talking to Elasticsearch, https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html.
When trying out this code i get a compiler error in IntelliJ which says "Cannot resolve symbol restClientBuilder".
I found the following SO which is EXACTLY my problem except that it is in Java and i am doing this in Scala.
Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
I tried copy pasting the solution code provided in the above SO into IntelliJ, the auto-converted code also has compiler errors.
// provide a RestClientFactory for custom configuration on the internally created REST client
// i only show the setMaxRetryTimeoutMillis for illustration purposes, the actual code will use HTTP cutom callback
esSinkBuilder.setRestClientFactory(
restClientBuilder -> {
restClientBuilder.setMaxRetryTimeoutMillis(10)
}
)
Then i tried (auto generated Java to Scala code by IntelliJ)
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
import org.apache.http.auth.AuthScope
import org.apache.http.auth.UsernamePasswordCredentials
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClientBuilder
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
esSinkBuilder.setRestClientFactory((restClientBuilder) => {
def foo(restClientBuilder) = restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = { // elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}
})
foo(restClientBuilder)
})
The original code snippet produces the error "cannot resolve RestClientFactory" and then Java to Scala shows several other errors.
So basically i need to find a Scala version of the solution described in Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
Update 1: I was able to make some progress with some help from IntelliJ. The following code compiles and runs but there is another problem.
esSinkBuilder.setRestClientFactory(
new RestClientFactory {
override def configureRestClientBuilder(restClientBuilder: RestClientBuilder): Unit = {
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
// elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
httpClientBuilder.setSSLContext(trustfulSslContext)
}
})
}
}
The problem is that i am not sure if i should be doing a new of the RestClientFactory object. What happens is that the application connects to the elasticsearch cluster but then discovers that the SSL CERT is not valid, so i had to put the trustfullSslContext (as described here https://gist.github.com/iRevive/4a3c7cb96374da5da80d4538f3da17cb), this got me past the SSL issue but now the ES REST Client does a ping test and the ping fails, it throws an exception and the app shutsdown. I am suspecting that the ping fails because of the SSL error and maybe it is not using the trustfulSslContext i setup as part of new RestClientFactory and this makes me suspect that i should not have done the new, there should be a simple way to update the existing RestclientFactory object and basically this is all happening because of my lack of Scala knowledge.
Happy to report that this is resolved. The code i posted in Update 1 is correct. The ping to ECE was not working for two reasons:
The certificate needs to include the complete chain including the root CA, the intermediate CA and the cert for the ECE. This helped get rid of the whole trustfulSslContext stuff.
The ECE was sitting behind an ha-proxy and the proxy did the mapping for the hostname in the HTTP request to the actual deployment cluster name in ECE. this mapping logic did not take into account that the Java REST High Level client uses the org.apache.httphost class which creates the hostname as hostname:port_number even when the port number is 443. Since it did not find the mapping because of the 443 therefore the ECE returned a 404 error instead of 200 ok (only way to find this was to look at unencrypted packets at the ha-proxy). Once the mapping logic in ha-proxy was fixed, the mapping was found and the pings are now successfull.
I have Spring Boot Camel application where rest apis are exposed using camel-restlet
Sample route
#Component
public class AppRoute extends RouteBuilder{
public void configure(CamelContext context){
from("restlet:employee?restletMethods=GET").log("${body}");
}
}
The App runs perfect ( spring-boot:run ). but am unable to locate under which path the API is exposed. Log has no information.
Every API i hit returns 404. Log shows the route has been started. Under which path is it running. And how do I change it?
Note: Please dont suggest any XML based configuration. Anything that I can put under #Configuration would be perfect
I would go with the Rest DSL which is supported by the camel-restlet component like this
restConfiguration().component("restlet").port(8080);
rest("/rest")
.get("employee")
.route().log("${body}")
.endRest();
And this route will listen to the following url
http://localhost:8080/rest/employee
EDIT:
I guess you could do something like without using the Rest DSL
String host = InetAddress.getLocalHost().getHostName();
from("restlet:http://" + host + contextPath + "/employee?restletMethods=GET").log("${body}")
The port and context path are configurable with the following properties
camel.component.restlet.port=8686
server.servlet.context-path=/my-path
The context path can be injected in the routeBuilder with
#Value("${server.servlet.context-path}")
private String contextPath;
According to the documentation, the format of the URI in a restlet endpoint definition should be the following:
restlet:restletUrl[?options]
Where restletUrl should have the following format:
protocol://hostname[:port][/resourcePattern]
So in your case you could define the URI in the following way:
from("restlet:http://localhost/employee?restletMethods=GET")
This should make the endpoint available under the following URL:
http://localhost/employee
Which you can test e.g. in a web browser.
Use the first of the three configuration methods described here:
https://restlet.com/open-source/documentation/javadocs/2.0/jee/ext/org/restlet/ext/servlet/ServerServlet.html
You should be able to customize it using the Component:
https://restlet.com/open-source/documentation/javadocs/2.0/jee/api/org/restlet/Component.html?is-external=true
See in particular setServers() methods (or XML equivalent) to change the hostname and port.
I'm trying to use websudos phantom
Does anyone know how to specify connection arguments to Cassandra such as username and password?
Thanks
Phantom doesn't yet support Kerberos authentication in the phantom-connectors framework but it is on the roadmap for the coming weeks.
However, when using connectors it is possible to override the createCluster method and connect to the cluster in any way you want.
object CustomCassandraManger extends DefaultCasandraManager {
override protected[this] def createCluster: Cluster = {
val inets = hosts.toSeq.map(_.getAddress)
Cluster.builder()
.addContactPoints(inets: _*)
...
.withUsernameAndPassword(..)
}
}
I may not have used the correct methods, but that's how you control the way a cluster is created. Then all you need is to inject this manager in a connector:
trait MyConnector extends SimpleCassandraConnector {
override val manager = CustomCassandraMananger
}
And then you mix in this connector into all your tables, just as you would normally when using phantom.
Recent versions of phantom
In more recent versions of phantom, the API is based on ContactPoints, where you can specify whatever ClusterBuilder options you want, using the following DSL.
import com.datastax.driver.core.PlainTextAuthProvider
lazy val local = ContactPoint.local.withClusterBuilder(
_.withAuthProvider(new PlainTextAuthProvider("user", "pass"))
).keySpace("phantom")
I have service in which i am accessing few configuration properties from grailsApplication
I am injecting it like this
class MyWebService{
def grailsApplication
WebService webService = new WebService()
def getProxy(url, flag){
return webService.getClient(url)
}
def getResponse(){
def proxy = getProxy(grailsApplication.config.grails.wsdlURL, true)
def response = proxy.getItem(ItemType)
return response
}
}
When i call getProxy() method, i see this in tomcat logs
No signature of method: org.example.MyWebService.getProxy() is applicable for argument types: (groovy.util.ConfigObject, java.lang.Boolean) values: [[:], true]
Possible solutions: getProxy(), getProxy(java.lang.String, boolean), setProxy(java.lang.Object)
which means grailsApplication is not getting injected into the service, is there any alternate way to access configuration object ? according to burtbeckwith's post configurationholder has been deprecated, can't think of anything else.
Interestingly the very same service works fine in my local IDE(GGTS 3.1.0), that means locally grailsApplication is getting injected, but when i create a war to deploy to a standalone tomcat, it stops getting injected.
I seem to have figured out the problem, actually grailsApplication is getting injected properly otherwise it would have thrown a null pointer exception, i feel the configuration properties are not getting added. Actually the scenario is like, i have a separate custom configuration file which holds configuration data for different environments, my application listens to the environement type(a variable which is set from tomcat) and based on that merges the corresponding config properties from my custom configuration file. i think those propreties are probably not getting added