Akka. How to set a pem certificate in a https request - scala

I'm using Akka (version 2.5.18) to send JSON strings to a specific server via https. I have used a poolRouter (balancing-pool with 10 instances) in order to create a pool of actors that are going to send JSONs (generated from different customers) to a single server:
val router: ActorRef = system.actorOf(
FromConfig.props(Props(new SenderActor(configuration.getString("https://server.com"), this.self))),
"poolRouter"
)
The project specification says that the requests can also be sent using curl:
curl -X PUT --cert certificate.pem --key private.key -H 'Content-Type: application / json' -H 'cache-control: no-cache' -d '[{"id" : "test"}] 'https://server.com'
Where "certificate.pem" is the tls certificate of the customer and "private.key" is the private key used to generate the CSR of the customer.
I'm using a balancing-pool because I will have a very big set of certificates (one for each customer) and I need to send the requests concurrently.
My approach is to have a "SenderActor" class that will be created by the balancing pool. Each actor, upon the reception of a message with a "customerId" and the JSON data generated by this customer, will send a https request:
override def receive: Receive = {
case Data(customerId, jsonData) =>
send(customerId(cid, jsonData))
Each SenderActor will read the certificate (and the private key) based on a path using the customerId. For instance, the customerId: "cust1" will have their certificate and key stored in "/home/test/cust1". This way, the same actor class can be used for all the customers.
According to the documentation, I need to create a HttpsConnectionContext in order to send the different requests:
def send(customerId: String, dataToSend): Future[HttpResponse] = {
// Create the request
val req = HttpRequest(
PUT,
uri = "https://server.com",
entity = HttpEntity(`application/x-www-form-urlencoded` withCharset `UTF-8`, dataToSend),
protocol = `HTTP/1.0`)
val ctx: SSLContext = SSLContext.getInstance("TLS")
val permissiveTrustManager: TrustManager = new X509TrustManager() {
override def checkClientTrusted(chain: Array[X509Certificate], authType: String): Unit = {}
override def checkServerTrusted(chain: Array[X509Certificate], authType: String): Unit = {}
override def getAcceptedIssuers(): Array[X509Certificate] = Array.empty
}
ctx.init(Array.empty, Array(permissiveTrustManager), new SecureRandom())
val httpsConnContext: HttpsConnectionContext = ConnectionContext.https(ctx)
// Send the request
Http(system).singleRequest(req, httpsConnContext)
}
The problem I have is that I don't have any clue about how to "set the certificate and the key" in the request, so that the server accepts them.
For instance, I can read the certificate using the following code:
import java.util.Base64
val certificate: String => String = (customer: String) => IO {
Source.fromInputStream(getClass.getClassLoader
.getResourceAsStream("/home/test/".concat(customer).concat("_cert.pem")))
.getLines().mkString
}.unsafeRunSync()
val decodedCertificate = Base64.getDecoder.decode(certificate(customerId)
.replaceAll(X509Factory.BEGIN_CERT, "").replaceAll(X509Factory.END_CERT, ""))
val cert: Certificate = CertificateFactory.getInstance("X.509")
.generateCertificate(new ByteArrayInputStream(decodedCertificate))
But I don't know how to "set" this certificate and the private key in the request (which is protected by a passphrase), so that the server accepts it.
Any hint or help would be greatly appreciated.

The following allows making a https request and identifying yourself with a private key from a x.509 certificate.
The following libraries are used to manage ssl configuration and to make https calls:
ssl-config
akka-http
Convert your pem certificate to pks12 format as defined here
openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt
Define key-store in your application.conf. It supports only pkcs12 and because of
this step 1 is required.
ssl-config {
keyManager {
stores = [
{
type = "pkcs12"
path = "/path/to/pkcs12/cetificate"
password = changeme //the password is set when using openssl
}
]
}
}
Load ssl config using special akka trait DefaultSSLContextCreation
import akka.actor.ActorSystem
import akka.actor.ExtendedActorSystem
import akka.http.scaladsl.DefaultSSLContextCreation
import com.typesafe.sslconfig.akka.AkkaSSLConfig
import com.typesafe.sslconfig.ssl.SSLConfigFactory
class TlsProvider(val actorSystem: ActorSystem) extends DefaultSSLContextCreation {
override protected def sslConfig: AkkaSSLConfig =
throw new RuntimeException("Unsupported behaviour when creating new sslConfig")
def httpsConnectionContext() = {
val akkaSslConfig =
new AkkaSSLConfig(system.asInstanceOf[ExtendedActorSystem], SSLConfigFactory.parse(system.settings.config))
createClientHttpsContext(akkaSslConfig)
}
}
Create a https context and use in http connection pool.
Http(actorSystem).cachedHostConnectionPoolHttps[RequestContext](
host = host,
port = portValue,
connectionContext = new TlsProvider(actorSystem).httpsConnectionContext()
)
Or set connection context to Http(actorSystem).singleRequest method.
In summary, I used ssl-config library to manage certificates instead of doing it programmatically yourself. By defining a keyManager in a ssl-config, any http request done with help of custom httpsConnectionContext will use the certificate to identify the caller/client.
I focused on describing how to establish a https connection using client certificate. Any dynamic behavior for managing multiple certificates is omitted. But I hope this code should be able give you understanding how to proceed.

Related

Mutual Authentication in Scala with Akka

I would create a TLS Session in Scala using Akka with mutual authentication between a client and a server. I have created two CA certificate that have to trust the respective certificates incoming from the other part.
Could you give me an exemple of how implement this?
Thank you.
I created a github project which demonstrates mutual authentication with different kind of clients, including Akka. Please have a look here: https://github.com/Hakky54/mutual-tls-ssl
It contains a full example of loading the ssl material into the client and server
A summary what you need to do is:
For the client
Create a key and a certificate and load it into a keystore
Export the certificate
Create a separate keystore for trusted certificates and import server certificate
Load the two keystores into your http client
For the server
Create a key and a certificate and load it into a keystore
Export the certificate
Create a separate keystore for trusted certificates and import client certificate
Load the two keystores into your server
It is not really clear to me what kind of server you are using, but if you are using spring-boot the example configuration would be:
server:
port: 8443
ssl:
enabled: true
key-store: classpath:identity.jks
key-password: secret
key-store-password: secret
trust-store: classpath:truststore.jks
trust-store-password: secret
client-auth: need
Akka requires a pre-configured instance of SSLContext to be able to configure HTTPS. An example of creating a client with https options would be the code snippet below.
import akka.actor.ActorSystem;
import akka.http.javadsl.ConnectionContext;
import akka.http.javadsl.Http;
import akka.http.javadsl.HttpsConnectionContext;
import com.typesafe.config.ConfigFactory;
import javax.net.ssl.SSLContext;
import java.util.Optional;
class App {
public static void main(String[] args) {
ActorSystem actorSystem = ActorSystem.create(
App.class.getSimpleName(),
ConfigFactory.defaultApplication(App.class.getClassLoader())
);
SSLContext sslContext = ...; //Initialized SSLContext
Http http = Http.get(actorSystem);
HttpsConnectionContext httpsContext = ConnectionContext.https(
sslContext,
Optional.empty(),
Optional.empty(),
Optional.empty(),
Optional.of(sslContext.getDefaultSSLParameters()));
http.setDefaultClientHttpsContext(httpsContext);
}
}
There are couple of libraries which provides easy to use utility/factory/builder classes to help you to create a SSLContext.
Apache SSLContextBuilder
Jetty SslContextFactory
SSLContext-Kickstart
There could be a bunch other libraries which provide similar functionality, but I am only aware of these three. By the way the sslcontext-kickstart is a library which is maintained by me.
Below is an overview of four ways to load the keystores and create an SSLContext. Vanilla Java and by using the three libraries.
import io.netty.handler.ssl.SslContextBuilder;
import nl.altindag.sslcontext.SSLFactory;
import org.apache.http.ssl.SSLContextBuilder;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import javax.net.ssl.*;
import java.io.File;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.util.Objects;
class SslExample {
public static void main(String[] args) throws Exception {
//Traditional flow of creating sslContext
String keyStorePath = "keystore.p12";
String trustStorePath = "truststore.p12";
char[] keyStorePassword = "secret".toCharArray();
char[] trustStorePassword = "secret".toCharArray();
KeyStore keyStore = KeyStore.getInstance("PKCS12");
KeyStore trustStore = KeyStore.getInstance("PKCS12");
try(InputStream keyStoreInputStream = SslExample.class.getClassLoader().getResourceAsStream(keyStorePath);
InputStream trustStoreInputStream = SslExample.class.getClassLoader().getResourceAsStream(trustStorePath)) {
Objects.requireNonNull(keyStoreInputStream);
Objects.requireNonNull(trustStoreInputStream);
keyStore.load(keyStoreInputStream, keyStorePassword);
trustStore.load(trustStoreInputStream, trustStorePassword);
}
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(keyStore, keyStorePassword);
KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();
TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(trustStore);
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();
SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(keyManagers, trustManagers, new SecureRandom());
//creating sslContext with Apache SSLContextBuilder
SSLContext sslContext1 = SSLContextBuilder.create()
.loadKeyMaterial(new File("keystore.p12"), "secret".toCharArray(), "secret".toCharArray())
.loadTrustMaterial(new File("truststore.p12"), "secret".toCharArray())
.build();
//creating sslContext with Jetty SslContextFactory
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
sslContextFactory.setKeyStorePath("keystore.p12");
sslContextFactory.setKeyStorePassword("secret");
sslContextFactory.setTrustStorePath("truststore.p12");
sslContextFactory.setTrustStorePassword("secret");
sslContextFactory.start();
SSLContext sslContext2 = sslContextFactory.getSslContext();
//creating sslContext with sslcontext-kickstart
SSLFactory sslFactory = SSLFactory.builder()
.withIdentity("keystore.p12", "secret".toCharArray())
.withTrustStore("truststore.p12", "secret".toCharArray())
.build();
SSLContext sslContext3 = sslFactory.getSslContext();
}
}
It is in java, but IntelliJ Idea provides a handy translate function to scala when pasting the code snippet.

How to set HTTP password in gerrit while making a REST call using python-requests?

Using requests module want to query Gerrit server and for this need set authentication in below code. And this needs to be done without using any other module like pygerrit2. And if this can be done using HTTP password generated from gerrit. If this is not doable, share an example if auth has to be passed along with Get request.
import requests
import json
import os
GERRIT_SERVER = 'https://gerrit.abc.com'
class GerritServer():
def __init__(self):
self._base_url = GERRIT_SERVER
self._endpoints = {
'CHANGES': 'changes',
}
def _get_endpoint(self, token):
try:
endpoint = self._endpoints[token]
except ValueError:
raise Exception('endpoint not defined for {0}'.format(token))
return '{0}/{1}'.format(self._base_url, endpoint)
def get_endpoint_changes(self):
return self._get_endpoint('CHANGES')
Gerrit's REST API uses digest auth.
username = 'foo'
httpassword = 'bar'
api = 'baz'
auth = requests.auth.HTTPDigestAuth(username, httpassword)
r = requests.get(api, auth=auth)

Using Finagle Http client for https requests

I am trying to get some data from a REST web service. So far I can get the data correctly if I don't use HTTPS with this code working as expected -
val client = Http.client.newService(s"$host:80")
val r = http.Request(http.Method.Post, "/api/search/")
r.host(host)
r.content = queryBuf
r.headerMap.add(Fields.ContentLength, queryBuf.length.toString)
r.headerMap.add("Content-Type", "application/json;charset=UTF-8")
val response: Future[http.Response] = client(r)
But when I am trying to get the same data from https request (Following this link)
val client = Http.client.withTls(host).newService(s"$host:443")
val r = http.Request(http.Method.Post, "/api/search/")
r.headerMap.add("Cookie", s"_elfowl=${authToken.elfowlToken}; dc=$dc")
r.host(host)
r.content = queryBuf
r.headerMap.add(Fields.ContentLength, queryBuf.length.toString)
r.headerMap.add("Content-Type", "application/json;charset=UTF-8")
r.headerMap.add("User-Agent", authToken.userAgent)
val response: Future[http.Response] = client(r)
I get the error
Remote Info: Not Available at remote address: searchservice.com/10.59.201.29:443. Remote Info: Not Available, flags=0x08
I can curl the same endpoint with 443 port and it returns the right result. Can anyone please help me troubleshoot the issue ?
Few things to check:
withTls(host)
needs to be the host name that is in the certificate of server (as opposed to the the ip for instance)
you can try:
Http.client.withTlsWithoutValidation
to verify the above.
Also you might want to verify if the server checks that the host header is set, and if so, you might want to include it:
val withHeader = new SimpleFilter[http.Request, http.Response] {
override def apply(request: http.Request, service: HttpService): Future[http.Response] = {
request.host_=(host)
service(request)
}
}
withHeader.andThen(client)
more info on host header:
What is http host header?

AWS: InvalidSignature exception while adding record

InvalidSignatureException occurs when trying to add user record using Kinesis Producer library.
AWS_JAVA_SDK_VERSION=1.10.26
AWS_KINESIS_PRODUCER_VERSION=0.10.1
ERROR:
PutRecords failed: {"__type":"InvalidSignatureException","message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method.
SCALA KINESIS PRODUCER CODE
private val configuration: KinesisProducerConfiguration = new KinesisProducerConfiguration
val credentialsProvider: AWSCredentialsProvider = AwsUtil.getAwsCredentials(config.awsAccessKey, config.awsSecretKey)
configuration.setCredentialsProvider(credentialsProvider)
configuration.setRecordMaxBufferedTime(config.timeLimit)
configuration.setAggregationMaxCount(1)
configuration.setRegion(config.streamRegion)
configuration.setMetricsLevel("none")
private val kinesisProducer = new KinesisProducer(configuration)
kinesisProducer.addUserRecord(streamName, key, eventBytes)`
The above code is not working. But its possible for me to add records to kinesis stream through aws cli from terminal and KinesisClient in code which is specified below.
private def createKinesisClient = {
val accessKey = config.awsAccessKey
val secretKey = config.awsSecretKey
val credentialsProvider: AWSCredentialsProvider = AwsUtil.getAwsCredentials(accessKey, secretKey)
val client = new AmazonKinesisClient(credentialsProvider)
client.setEndpoint(config.streamEndpoint)
client
}
This happens because your VM/PC/Server clock might by skewed.
If you're running ubuntu, try updating your system time:
sudo ntpdate ntp.ubuntu.com
If you are using docker-machine on Mac, you can resolve with this command:
docker-machine ssh default 'sudo ntpclient -s -h pool.ntp.org'

grails - RestClientBuilder

I am using the current version of rest client builder plugin. I tested out the uri via curl:
curl --user username:password https://localhost:8085/rest/api/latest/plan.json?os_authType=basic
I get the expected json in return. When I try to translate this to grails using the plugin like this:
RestBuilder rb = new RestBuilder()
def response = rb.get("https://localhost:8085/rest/api/latest/plan.json?os_authType=basic"){
auth 'username', 'password'
}
response.json instanceof JSONObject
I get this error:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Why does it work in curl and not with the plugin? How do I get this to work?
Thanks!
You need to add the root certificate to the store of the trusted ones.
http://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html
Import the Certificate as a Trusted Certificate
Before you can grant the signed code permission to read a specified file, you need to import Susan's certificate as a trusted certificate in your keystore.
Suppose that you have received from Susan
the signed JAR file sCount.jar, which contains the Count.class file, and
the file Example.cer, which contains the public key certificate for the public key corresponding to the private key used to sign the JAR file.
Even though you created these files and they haven't actually been transported anywhere, you can simulate being someone other than the creater and sender, Susan. Pretend that you are now Ray. Acting as Ray, you will create a keystore named exampleraystore and will use it to import the certificate into an entry with an alias of susan.
A keystore is created whenever you use a keytool command specifying a keystore that doesn't yet exist. Thus we can create the exampleraystore and import the certificate via a single keytool command. Do the following in your command window.
Go to the directory containing the public key certificate file Example.cer. (You should actually already be there, since this lesson assumes that you stay in a single directory throughout.)
Type the following command on one line:
keytool -import -alias susan
-file Example.cer -keystore exampleraystore
Since the keystore doesn't yet exist, it will be created, and you will be prompted for a keystore password; type whatever password you want.
The keytool command will print out the certificate information and ask you to verify it, for example, by comparing the displayed certificate fingerprints with those obtained from another (trusted) source of information. (Each fingerprint is a relatively short number that uniquely and reliably identifies the certificate.) For example, in the real world you might call up Susan and ask her what the fingerprints should be. She can get the fingerprints of the Example.cer file she created by executing the command
keytool -printcert -file Example.cer
If the fingerprints she sees are the same as the ones reported to you by keytool, the certificate has not been modified in transit. In that case you let keytool proceed with placing a trusted certificate entry in the keystore. The entry contains the public key certificate data from the file Example.cer and is assigned the alias susan.
You can just disable SSL check for RestBuilder.
See an example of code:
static Scheme disableSSLCheck() {
def sslContext = SSLContext.getInstance("SSL")
sslContext.init(null, [new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
#Override
X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0]
}
}] as TrustManager[], new SecureRandom())
def sf = new SSLSocketFactory(sslContext, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
def httpsScheme = new Scheme("https", sf, 443)
httpsScheme
}
And register this Scheme to the RestClient:
Scheme httpsScheme = disableSSLCheck()
restClient.client.connectionManager.schemeRegistry.register(httpsScheme)
Mb too late but have a look here.
https://gist.github.com/thomastaylor312/80fcb016020e4115aa64320b98fb0017
I do have it as separate method in my Integration test
def static disableSSLCheck() {
def nullTrustManager = [
checkClientTrusted: { chain, authType -> },
checkServerTrusted: { chain, authType -> },
getAcceptedIssuers: { null }
]
def nullHostnameVerifier = [
verify: { hostname, session -> true }
]
SSLContext sc = SSLContext.getInstance("SSL")
sc.init(null, [nullTrustManager as X509TrustManager] as TrustManager[], null)
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory())
HttpsURLConnection.setDefaultHostnameVerifier(nullHostnameVerifier as HostnameVerifier)
}
And then just
void "test authentication"(){
given:
String url = "j_spring_security_check"
MultiValueMap<String, String> form = new LinkedMultiValueMap<String, String>()
form.add("grant_type", "password")
form.add("j_username", "vadim#ondeviceresearch.com")
form.add("j_password", "notSecure")
form.add("_spring_security_remember_me", "true")
//TODO SET username and pass
//todo get token back
disableSSLCheck()
when:
RestResponse response = rest.post(host + url){
accept("application/json")
contentType("application/x-www-form-urlencoded")
body(form)
}
response
then:
response.status == 200
}