I want to get my all key aliases from hsm, but i get this error message :
Exception in thread "main" java.io.IOException: load failed
at sun.security.pkcs11.P11KeyStore.engineLoad(P11KeyStore.java:778)
at java.security.KeyStore.load(KeyStore.java:1445)
at eracom.provider.ProCrypt.main(ProCrypt.java:99)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception:
CKR_OBJECT_HANDLE_INVALID
at sun.security.pkcs11.wrapper.PKCS11.C_GetAttributeValue(Native Method)
at sun.security.pkcs11.P11KeyStore.mapLabels(P11KeyStore.java:2415)
at sun.security.pkcs11.P11KeyStore.engineLoad(P11KeyStore.java:770)
If i create just one secret key on hsm. there is no problem, i can get
the alia from hsm, but if i have 2 or more, i get the error.
How to solve it?
KeyStore keyStore = KeyStore.getInstance("PKCS11", pkcs11);
keyStore.load(null, slotPIN);
SecretKey key = (SecretKey) keyStore.getKey("KEY2", slotPIN);
System.out.println("key's attributes: " + key.toString());
java.util.Enumeration<String> aliases = keyStore.aliases();
String alias = null;
while (aliases.hasMoreElements()) {
alias = aliases.nextElement();
System.out.println("key label name : " + alias);
}
Related
I'm trying to connect to a SSL enabled mongo cluster from a spark application. I'm trying to use self signed cert and getting the following error.
Exception in monitor thread while connecting to server CLUSTER_NAME
com.mongodb.MongoSocketWriteException: Exception sending message
at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:525)
at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:413)
at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:269)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:253)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching CLUSTER_NAME found
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
My read config uri looks something like this:
val uri: String = "mongodb://" + URLEncoder.encode(Login, "UTF-8") + ":" + URLEncoder.encode(Password, "UTF-8") + "#" + cluster + ":27017/" + database + "." + collection + "?authSource=" + (if (authenticationDatabase != "") authenticationDatabase else "admin") + (if (replicaset == null) "" else "&replicaSet=" + replicaset) + "&ssl=true"
I want to use self signed cert something like :
class TrustAllX509TrustManager extends X509TrustManager {
override def getAcceptedIssuers = new Array[X509Certificate](0)
override def checkClientTrusted(certs: Array[X509Certificate], authType: String): Unit = {
}
override def checkServerTrusted(certs: Array[X509Certificate], authType: String): Unit = {
}
}
The version of the env's I'm using:
Spark: 2.2.0
Mongo: 3.4
Any help will be appreciated.
Thanks!
This is same as making any other SSL connection. Import your cert in keystore and refer to that key store using below code
System.setProperty("javax.net.ssl.trustStore", "keystoreFilefullpath")
System.setProperty("javax.net.ssl.trustStorePassword", "password")
Once these params are set then Kafka SSL should work. If you are publishing from Spark then keystore file must be uploaded to driver/executor using --files option
I'm following sample provided here https://console.bluemix.net/docs/services/cloud-object-storage/libraries/java.html#java
Created the following code:
SDKGlobalConfiguration.IAM_ENDPOINT = "https://iam.bluemix.net/oidc/token";
String bucketName = "mybucket_name";
String api_key = "api_key_taken_fro_the_service_credential_json";
String service_instance_id = "service_id_taken_from_the_bucket_policies_page";
String endpoint_url = "http://s3.mel01.objectstorage.softlayer.net";
String location = "mel01";
System.out.println("Current time: " + new Timestamp(System.currentTimeMillis()).toString());
_cos = createClient(api_key, service_instance_id, endpoint_url, location);
listObjects(bucketName, _cos);
listBuckets(_cos);
the listObjects and listBuckets are exact c&p from that referred page.
I'm getting Invalid argument exception, as follows:
Listing objects in bucket mybucket_name
[INFO ] FFDC1015I: An FFDC Incident has been created: "com.ibm.cloud.objectstorage.services.s3.model.AmazonS3Exception: Invalid Argument (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: 3bb36c4c-8de4-4f39-98f4-a1bfd3aa8972), S3 Extended Request ID: null com.ibm.ws.webcontainer.servlet.ServletWrapper.init 181" at ffdc_18.04.12_13.02.15.0.log
[ERROR ] SRVE0271E: Uncaught init() exception created by servlet [gas.servlet.BlmxS3Servlet] in application [s3]: com.ibm.cloud.objectstorage.services.s3.model.AmazonS3Exception: Invalid Argument (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: 3bb36c4c-8de4-4f39-98f4-a1bfd3aa8972), S3 Extended Request ID: null
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.ibm.cloud.objectstorage.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.ibm.cloud.objectstorage.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3635)
at com.ibm.cloud.objectstorage.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3582)
at com.ibm.cloud.objectstorage.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3576)
at com.ibm.cloud.objectstorage.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:761)
at gas.servlet.BlmxS3Servlet.listObjects(BlmxS3Servlet.java:94)
at gas.servlet.BlmxS3Servlet.init(BlmxS3Servlet.java:63)
at javax.servlet.GenericServlet.init(GenericServlet.java:244)
at [internal classes]
What argument might be invalid? No clue from the log nor ffdc.
UPDATE
The bucket name is correct, otherwise I'm getting:
[ERROR ] SRVE0271E: Uncaught init() exception created by servlet [gas.servlet.BlmxS3Servlet] in application [s3]: com.ibm.cloud.objectstorage.services.s3.model.AmazonS3Exception: The specified bucket does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchBucket; Request ID: bf299e50-1aac-4af7-89df-188eb4b7c60f), S3 Extended Request ID: null
your endpoint_url should be over https instead of http
There were several issues during configuring this:
1) You need to create credential with additional properties of {"HMAC":true}
2) For the configuration use:
apikey - apikey entry from the credential
service_instance_id - this is a bit misleading - as it is your Object servcie name, not id - in my case it was string Cloud Object Storage-abc
endpoint_url - bucket endpoint - in my case http://s3.mel01.objectstorage.softlayer.net - make sure that it is https and that you imported certificates to your truststore.
Then is should work.
Listing buckets is not working for me still due to "Access Denied", so I have to check the permissions for the service ID.
I’m trying to download several files from a local network device:
http file directory
I want to write a code that will automatically download all those .avi files to my pc drive.
I have 2 problems:
Problem 1: AUTHENTICATING using WebClient class only.
If I use WebClient class only to connect, I get a 401 Unauthorized error.
Code:
try
{
using (WebClient myWebClient = new WebClient())
{
myWebClient.UseDefaultCredentials = false;
myWebClient.Credentials = new NetworkCredential("user", "pword");
String userName = "user";
String passWord = "pword";
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(userName + ":" + passWord));
myWebClient.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
Console.WriteLine("Header AUTHORIZATION: "+ myWebClient.Headers[HttpRequestHeader.Authorization].ToString());
// Download the Web resource and save it into the current filesystem folder.
Console.WriteLine("Start DL");
myWebClient.DownloadFile("http://192.168.2.72:81/sd/20170121/record000/P170121_000000_001006.avi", "P170121_000000_001006.avi");
Console.WriteLine("End DL");
}
}
catch(Exception ex)
{
Console.WriteLine("DOWNLOAD ERROR: " + ex.ToString());
}
Error Message: Failure to authenticate
401 Unauthorized Error
Problem 2: Was able to authenticate using WebProxy class but can’t download . Getting 403 Not found error.
Code:
try
{
using (WebClient myWebClient = new WebClient())
{
WebProxy wp = new WebProxy("http://192.168.2.72:81/sd/20170121/record000/",false);
wp.Credentials = new NetworkCredential("user","pword");
Console.WriteLine("Web Proxy: " + wp.Address);
myWebClient.UseDefaultCredentials = false;
myWebClient.Credentials = wp.Credentials;
myWebClient.Proxy = wp;
Console.WriteLine("Downloading File \"{0}\" from \"{1}\"\n\n", filename, wp.Address);
// Download the Web resource and save it into the current filesystem folder.
Console.WriteLine("Start DL");
myWebClient.DownloadFile("http://192.168.2.72:81/sd/20170121/record000/P170121_000000_001006.avi", "P170121_000000_001006.avi");
Console.WriteLine("End DL");
}
}
catch(Exception ex)
{
Console.WriteLine("DOWNLOAD ERROR: " + ex.ToString());
}
Error Message: 403 Not Found
DOWNLOAD ERROR: System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.WebClient.DownloadFile(Uri address, String fileName)
at System.Net.WebClient.DownloadFile(String address, String fileName)
at ConsoleApplication2.Program.Main(String[] args) in C:\Users\Gordon\documents\visual studio 2015\Projects\ConsoleApplication2\ConsoleApplication2\Program.cs:line 139
Please help me identify if there are any mistakes in my code or is there a better way to submit credentials and download all the files.
Thanks in advance!
I'm not Dot Net developer, I'm just sharing my opinion.
In the second point you have mentioned that you are getting 403 which is the Http status code for Acces Denied. I feel your credentials are not valid or you don't have privilege to do the operation.
I have object DictionarySqlLiteDao which should create table of it not exist during its initialization. But I have exception despite the fact that the database is created in file system. Help me to figure out the reason of it. Here is my code:
object DictionarySqlLiteDao extends Dao[Word] {
private val CREATE_WORD_TABLE_SQL = "CREATE TABLE IF NOT EXISTS thelper.WORD " +
"(ID INTEGER PRIMARY KEY AUTOINCREMENT, " +
"WORD TEXT NOT NULL, " +
"UKR TEXT NOT NULL, " +
"ENG TEXT NOT NULL, " +
"EXAMPLE TEXT)"
private val SAVE_WORD_SQL = "INSERT INTO WORD(WORD, UKR, ENG, EXAMPLE" +
"VALUES(?, ?, ?, ?)"
private val connection: Connection = {
Class.forName("org.sqlite.JDBC")
DriverManager.getConnection("jdbc:sqlite:thelper")
}
{
connection.createStatement().executeUpdate(CREATE_WORD_TABLE_SQL)
}
override def save(word: Word): Word = {
val statement = connection.prepareStatement(SAVE_WORD_SQL, Statement.RETURN_GENERATED_KEYS)
statement.setString(1, word.word)
statement.setString(2, word.translation("ukr"))
statement.setString(3, word.translation("eng"))
statement.setString(4, word.example.orNull)
statement.executeUpdate()
val id = statement.getGeneratedKeys.getInt(1)
word.copy(id = id)
}
def main(args: Array[String]) {
println(connection == null) // here I get false
println(connection.createStatement().execute("select date('now')")) // here I get true
val w = save(Word(-1, "germanWord", Map("ukr" -> "ukrTranslation", "eng" -> "engTranslation"), Option.empty[String])) // and here I get an Exception
println(w)
}
}
And here is an exception I get:
Exception in thread "main" java.lang.ExceptionInInitializerError
at thelper.Dictionary.dao.DictionarySqlLiteDao.main(DictionarySqlLiteDao.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.sql.SQLException: unknown database thelper
at org.sqlite.core.NativeDB.throwex(NativeDB.java:397)
at org.sqlite.core.NativeDB._exec(Native Method)
at org.sqlite.jdbc3.JDBC3Statement.executeUpdate(JDBC3Statement.java:116)
at thelper.Dictionary.dao.DictionarySqlLiteDao$.<init>(DictionarySqlLiteDao.scala:23)
at thelper.Dictionary.dao.DictionarySqlLiteDao$.<clinit>(DictionarySqlLiteDao.scala)
... 6 more
As far as I remember you should specify full path to database, in your in-memory case it should be thelper.db like:
DriverManager.getConnection("jdbc:sqlite:thelper.db")`
P.S. After that you can omit database name: "CREATE TABLE IF NOT EXISTS WORD " + ....
P.P.S you can use triple quotes to write multiline strings like
"""CREATE TABLE IF NOT EXISTS WORD(
| ID INTEGER PRIMARY KEY AUTOINCREMENT,
| WORD TEXT NOT NULL,
| UKR TEXT NOT NULL,
| ENG TEXT NOT NULL,
| EXAMPLE TEXT)""".stripMargin
I hava hadoop-2.7 cluster, oozie-4.0.1 running in secure mode(with kerberos).
All are well. I can use cli commands submit job as follow:
Kinit myuser
oozie job -oozie https://10.1.130.10:21003/oozie -config job.properties -run
but I use oozie java api submit job, kerberos exception occur.
Exception in thread "main" AUTHENTICATION : Could not authenticate, GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at org.apache.oozie.client.AuthOozieClient.createConnection(AuthOozieClient.java:150)
at org.apache.oozie.client.OozieClient.getSupportedProtocolVersions(OozieClient.java:577)
at org.apache.oozie.client.OozieClient.validateWSVersion(OozieClient.java:538)
at org.apache.oozie.client.OozieClient.createURL(OozieClient.java:651)
at org.apache.oozie.client.OozieClient.access$100(OozieClient.java:103)
at org.apache.oozie.client.OozieClient$ClientCallable.call(OozieClient.java:803)
at org.apache.oozie.client.OozieClient.run(OozieClient.java:999)
at com.huawei.oozie.OozieMain.main(OozieMain.java:47)
Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:334)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:206)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.oozie.client.AuthOozieClient.createConnection(AuthOozieClient.java:144)
... 7 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:313)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:288)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:288)
... 10 more
my java code as follow:
System.setProperty("java.security.auth.login.config", System.getProperty("user.dir") + File.separator + "conf"
+ File.separator + "jaas.conf ");
System.setProperty("java.security.krb5.conf", System.getProperty("user.dir") + File.separator + "conf"
+ File.separator + "krb5.conf ");
String url = "https://10.137.60.60:21003/oozie";
AuthOozieClient wc = new AuthOozieClient(url);
wc.setDebugMode(1);
Properties conf = wc.createConfiguration();
FileReader fr = new FileReader("conf/job.properties");
conf.load(fr);
System.out.println(conf.toString());
String jobId = wc.run(conf);
System.out.println("Workflow job submitted");
while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING)
{
System.out.println("Workflow job running ...");
Thread.sleep(3 * 1000);
}
System.out.println("Workflow job completed ...");
System.out.println(wc.getJobInfo(jobId));
my conf/jaas.conf as follow:
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="D:/workspace/4.4-billing/Oozie/conf/oozie.keytab"
principal="oozie#HADOOP.COM"
useTicketCache=false
storeKey=true
debug=true;
};
can anyone help me ? I know oozie use hadoop-auth jar. but how to set keytab, write authenticate code, I cannot.
set your kerberos user account as
conf.setProperty(OozieClient.USER_NAME, "xyz");
oozieClient.run(conf);