Spray HTTPS Inbound closed before receiving peer's close_notify - scala

Currently I am implementing a web service with Scala using Spray i/o. Looking to use SSL to secure my requests. However I am having difficulty configuring the SSL. When an https call is initiated there is an error thrown related to the handshake
fatal error: 80: Inbound closed before receiving peer's close_notify: possible truncation attack?
javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
Created a cert using this
keytool -genkey -keyalg RSA -alias mykey -dname "CN=dev.site.com,OU=app" -keystore keystore.jks -storepass pas -validity 365
Created an ssl trait like this
trait SSLConfiguration {
implicit def sslContext: SSLContext = {
val keystore = keystore.jks
val password = pas
val keyStore = KeyStore.getInstance("jks")
val in = getClass.getClassLoader.getResourceAsStream(keystore)
require(in != null, "Bad java key storage file: " + keystore)
keyStore.load(in, password.toCharArray)
val keyManagerFactory = KeyManagerFactory.getInstance("SunX509")
keyManagerFactory.init(keyStore, password.toCharArray)
val trustManagerFactory = TrustManagerFactory.getInstance("SunX509")
trustManagerFactory.init(keyStore)
val context = SSLContext.getInstance("TLS")
context.init(keyManagerFactory.getKeyManagers, trustManagerFactory.getTrustManagers, new SecureRandom)
context
}
implicit def sslEngineProvider: ServerSSLEngineProvider = {
ServerSSLEngineProvider { engine =>
engine.setEnabledCipherSuites(Array("TLS_RSA_WITH_AES_256_CBC_SHA"))
engine.setEnabledProtocols(Array("SSLv3", "TLSv1"))
engine
}
}
}
Set up my boot to use the trait.
object Boot extends App with SSLConfiguration
//bind to io interface. set ssl engine providor
IO(Http) ! Http.Bind(service, interface = interface, port)(sslEngineProvider)
}

Do you enable SSL in the configuration file?
spray.can {
server {
ssl-encryption = on
}
}
I tried your code and slightly changed it, it worked in my laptop.
I deleted the whole
implicit def sslEngineProvider
and use default one
IO(Http) ! Http.Bind(service, interface = "0.0.0.0", port = 8080).
And do you put your keystore file in the project's resource folder (project/src/main/resources)?

The issue could be related to the java version installed.
I faced this issue when I was installing open JDK version of java on linux machine, when I changed the java version to Oracle JDK the issue disappeared.
The exact application that threw this exception is Information Workbench (fluid ops product) and java version was 8
Using which version of java wasn't mentiond in system prerequists by fluid ops people.

Related

Proxy authentication issue using Jsoupbrowser in container

I'm having an issue trying to authenticate my proxy inside a Docker.
That's what I did :
Authenticator.setDefault(new Authenticator() {
override def getPasswordAuthentication = new PasswordAuthentication(<USERNAME>, <PASSWORD>.toCharArray)
})
val browser = new JsoupBrowser(ua,proxy) {
override def requestSettings(conn: Connection) = conn.timeout(5000)
}
// Step 1: We __scrape__ the current page.
val doc = browser.get(baseUrl)
It works locally, but when I deploy it on my server I get an Error 407
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
I also tried upgrading the configuration to container level but it didn't work.
I found a solution.
The problem was only at the deploy, so I concluded that the problem came with the build of the docker.
I add this JVM parameter in my Dockerfile :
-Djdk.http.auth.tunneling.disabledSchemes=
To the CMD
CMD java -Dhttp.port=${port} -Djdk.http.auth.tunneling.disabledSchemes= -Dplay.crypto.secret=secret $* -jar ./app-assembly.jar
It works now.

How to access a confluent schema registry server secured with a password using Spring cloud stream?

I'm using spring cloud stream alongside Aiven's schema registry which uses confluent's schema registry. Aiven's schema registry is secured with a password. Based on these instructions, these two config parameters need to be set to successfully access the schema registry server.
props.put("basic.auth.credentials.source", "USER_INFO");
props.put("basic.auth.user.info", "avnadmin:schema-reg-password");
Everything is fine when I only use vanilla java's kafka drivers, but if I use Spring cloud stream, I don't know how to inject these two parameters. At the moment, I'm putting "basic.auth.user.info" and "basic.auth.credentials.source" under "spring.cloud.stream.kafka.binder.configuration" in the application.yml file.
Doing this, I'm getting "401 Unauthorized" on the line where the schema wants to get registered.
Update 1:
Based on 'Ali n's suggestion, I updated the way SchemaRegistryClient's bean was configured so that it becomes aware of the SSL context.
#Bean
public SchemaRegistryClient schemaRegistryClient(
#Value("${spring.cloud.stream.schemaRegistryClient.endpoint}") String endpoint) {
try {
final KeyStore keyStore = KeyStore.getInstance("PKCS12");
keyStore.load(new FileInputStream(
new File("path/to/client.keystore.p12")),
"secret".toCharArray());
final KeyStore trustStore = KeyStore.getInstance("JKS");
trustStore.load(new FileInputStream(
new File("path/to/client.truststore.jks")),
"secret".toCharArray());
TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
SSLContext sslContext = SSLContextBuilder
.create()
.loadKeyMaterial(keyStore, "secret".toCharArray())
.loadTrustMaterial(trustStore, acceptingTrustStrategy)
.build();
HttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).build();
ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(
httpClient);
ConfluentSchemaRegistryClient schemaRegistryClient = new ConfluentSchemaRegistryClient(
new RestTemplate(requestFactory));
schemaRegistryClient.setEndpoint(endpoint);
return schemaRegistryClient;
} catch (Exception ex) {
ex.printStackTrace();
return null;
}
}
This helped getting rid of the error on app's startup and registered the schema. However, whenever the app wanted to push a message to Kafka, a new error was thrown again. Finally this was also fixed by mmelsen's answer.
I ran into the same problem as the situation I was in was to connect to a secured schema registry hosted by aiven and secured by basic auth. In order for me to make it work I had to configure the following properties:
spring.kafka.properties.schema.registry.url=https://***.aiven***.com:port
spring.kafka.properties.basic.auth.credentials.source=USER_INFO
spring.kafka.properties.basic.auth.user.info=username:password
the other properties for my binder are:
spring.cloud.stream.binders.input.type=kafka
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.brokers=https://***.aiven***.com:port <-- different from the before mentioned port
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.security.protocol=SSL
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.truststore.location=truststore.jks
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.truststore.password=secret
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.keystore.type=PKCS12
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.keystore.location=clientkeystore.p12
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.keystore.password=secret
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.ssl.key.password=secret
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.binder.configuration.value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
spring.cloud.stream.binders.input.environment.spring.cloud.stream.kafka.streams.binder.autoCreateTopics=false
what actually happens is that Spring cloud stream will add the spring.kafka.properties.basic* to the DefaultKafkaConsumerFactory and that will add the config to the KafkaConsumer. At some point during the initialization of the spring kafka, a CachedSchemaRegistryClient is created that is provisioned with these properties. This Client contains a method called configureRestService that will check if a map of properties contains "basic.auth.credentials.source". As we provide this through the spring.kafka.properties it will find this property and will take care of creating the appropriate headers when accessing the schema registry's endpoint.
hope this works out for you as well.
I'm using spring cloud version Greenwich.SR1, spring-boot-starter 2.1.4.RELEASE, avro-version 1.8.2 and confluent.version 5.2.1
The binder configuration only handles well-known consumer and producer properties.
You can set arbitrary properties at the binding level.
spring.cloud.stream.kafka.binding.<binding>.consumer.configuration.basic.auth...
Since Aiven uses SSL for Kafka security protocol, it is required to use certificates for the authentication.
You can follow this page to understand how it works. In the nutshell, you need to run the following command to generate certificates and import them:
openssl pkcs12 -export -inkey service.key -in service.cert -out client.keystore.p12 -name service_key
keytool -import -file ca.pem -alias CA -keystore client.truststore.jks
Then you can use the following properties to make use of the certificates:
spring.cloud.stream.kafka.streams.binder:
configuration:
security.protocol: SSL
ssl.truststore.location: client.truststore.jks
ssl.truststore.password: secret
ssl.keystore.type: PKCS12
ssl.keystore.location: client.keystore.p12
ssl.keystore.password: secret
ssl.key.password: secret
key.serializer: org.apache.kafka.common.serialization.StringSerializer
value.serializer: org.apache.kafka.common.serialization.StringSerializer

How to fix Keberos Authentication prompt while running Java application

I wrote a Java application to connect with hive-metastore. It works fine but when I run the jar file on Linux it asks for Kerberos Username and password
. I already specified the Kerberos principal and keytab file in my code. And I don't want to use additional jass.config file. Is there any way to resolve that problem?
Here is my source code
HiveConf conf = new HiveConf();
MetastoreConf.setVar(conf, ConfVars.THRIFT_URIS, "thrift://HOSTNAME:9083");
MetastoreConf.setBoolVar(conf, ConfVars.USE_THRIFT_SASL, true);
MetastoreConf.setVar(conf, ConfVars.KERBEROS_PRINCIPAL, "hive/HOSTNAME#example.com");
MetastoreConf.setVar(conf, ConfVars.KERBEROS_KEYTAB_FILE, "hive.keytab");
System.setProperty("java.security.krb5.realm", "EXAMPLE.COM");
System.setProperty("java.security.krb5.kdc", "HOSTNAME");
System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
HiveMetaStoreClient client = new HiveMetaStoreClient(conf);
client.close();
Expected Result-
It should verify the connection successfully
Actual result-
java -jar Application
Kerberos username [root]:
Kerberos password [root]:
I want to bypass this kerberos terminal prompt authentication. Is there any way to do that?
final String user = "hive/HOSTNAME#EXAMPLE.COM";
final String keyPath = "hive.keytab";
Configuration conf = new Configuration();
System.out.println("In case of kerberos authentication");
conf.set("hadoop.security.authentication", "kerberos");
System.setProperty("java.security.krb5.kdc", "HOSTNAME");
System.setProperty("java.security.krb5.realm", "EXAMPLE.COM");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab(user, keyPath);
HiveConf conf1 = new HiveConf(); MetastoreConf.setVar(conf1,
ConfVars.THRIFT_URIS, "thrift://HOSTNAME:9083"); HiveMetaStoreClient
client = new HiveMetaStoreClient(conf1);
client.close();
It works fine now

Kerberos Exception launching Spark locally

I am trying to set up a Spark Testng unit test:
#Test
def testStuff(): Unit = {
val sc = new SparkContext(new SparkConf().setAppName("test").setMaster("local"))
...
}
The code fails with: IllegalArgumentException: Can't get Kerberos realm
What am I missing?
The error suggests that your JVM is unable to locate the kerberos config (krb5.conf file).
Depending on your company's environment/infrastruture you have a few options:
Check if your company has standard library to set kerberos authentication.
Alternatively try:
set JVM property: -Djava.security.krb5.conf=/file-path/for/krb5.conf
Put the krb5.conf file into the <jdk-home>/jre/lib/security folder

Disable SSL with Scala Dispatch Library

I am currently in the process of moving all our Rest tests to the CI server and have noticed that all tests are failing due to the an SSL handshake, now I have successfully disabled this with the TrustManager with our Java test suite, but am unsure how to do it with Scala dispatch library, and havent been able to find many examples that could apply in this scenario.
val JSONstr = "{samplekey:samplevalue}"
val response:String = Http(url("https://www.host.com/path/to/post")
<< (checkInJSONstr, "application/json") as_str)
The following exception is occuring as expected:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352)
...
Is there a way to do it cleanly syntactically ignore SSL with the dispatch library?
import dispatch._
val http = new Http with HttpsLeniency
val response:String = http(url("https://www.host.com/path/to/post")
<< (checkInJSONstr, "application/json") as_str)
viktortnk's answer doesn't work anymore with the newest version of Dispatch (0.13.2). For the newest version, you can use the following to create an http client that accepts any certificate:
val myHttp = Http.withConfiguration(config => config.setAcceptAnyCertificate(true))
Then you can use it for GET requests like this:
myHttp(url("https://www.host.com/path").GET OK as.String)
I found this out here: Why does dispatch throw "java.net.ConnectException: General SSLEngine ..." and "unexpected status" exceptions for a particular URL?