There are a couple of examples on the AWS SDK how to get the credentials, e.g.:
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/prog-services-sts.html
https://docs.aws.amazon.com/AmazonS3/latest/dev/AuthUsingTempSessionTokenJava.html
But when I run these snippets I cannot import AWSSecurityTokenServiceClientBuilder:
// note that the AWS SDK is pretty brittle across versions.
import $ivy.`com.amazonaws:aws-java-sdk:1.7.4`
import $ivy.`org.apache.hadoop:hadoop-aws:2.7.3`
import com.amazonaws.auth.profile.ProfileCredentialsProvider
import com.amazonaws.services.securitytoken.model.AssumeRoleRequest
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder
var clientRegion = "*** Client region ***";
var roleARN = "*** ARN for role to be assumed ***";
var roleSessionName = "*** Role session name ***";
var stsClient = AWSSecurityTokenServiceClientBuilder.standard()
.withCredentials(new ProfileCredentialsProvider())
.withRegion(clientRegion)
.build()
var roleRequest = new AssumeRoleRequest()
.withRoleArn(roleARN)
.withRoleSessionName(roleSessionName)
var roleResponse = stsClient.assumeRole(roleRequest)
var sessionCredentials = roleResponse.getCredentials()
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder
^cmd16.sc:13: not found: value AWSSecurityTokenServiceClientBuilder
var stsClient = AWSSecurityTokenServiceClientBuilder.standard()
^Compilation Failed
Compilation Failed
scala version 2.11.12
spark version 2.3.4, that means that I am tight to hadoop-aws 2.7.3, which depends on aws-java-sdk 1.7.4 :/
AFAICS, this class is not part of the AWS Java SDK until version 1.11.0.
You have to instantiate the AWSSecurityTokenServiceClient class yourself instead, without using the builder.
Call this constructor:
val stsClient = new AWSSecurityTokenServiceClient(new ProfileCredentialsProvider())
stsClient.setRegion(clientRegion)
Related
I am using httpClient lib to do some REST API calls using scala. I am able to post the data.
I am using following code to read the content. However, When I run on Spark Databricks Cluster it give me error.
val entity = response.getEntity
var content = ""
if (entity != null) {
val inputStream = entity.getContent
content = io.Source.fromInputStream(inputStream).getLines.mkString
inputStream.close()
}
Error
error: object Source is not a member of package io
content = io.Source.fromInputStream(inputStream).getLines.mkString
is there a way I can fix this error, or a different way to read HTTP response content.
Please try to import scala.io.Source
OR use the completed package name like this:
content = scala.io.Source.fromInputStream(inputStream).getLines.mkString
I'm currently using Grails 2.4.5 and used AmazonWebService plugin for grails 2.4.5
I'm trying to create a new recordset on route53 using this plugin.
On my BuildConfig.groovy I used this plugin fro aws web services.
compile ":aws-sdk:1.10.44"
I need your help guys regarding route53 change resource record sets.
I got an error below when I tried to change route53 resource record sets.
Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found more than one in Change with [Action=CREATE, Name=app.sample.com., Type=A, SetIdentifier=null] (Service: AmazonRoute53; Status Code: 400; Error Code: InvalidInput; Request ID: 2ca80154-78a7-11e9-b5e7-f7bc7c79e5e6). Stacktrace follows:
Message: Invalid request: Expected exactly one of [AliasTarget, all of [TTL, and ResourceRecords], or TrafficPolicyInstanceId], but found more than one in Change with [Action=CREATE, Name=app.sample.com., Type=A, SetIdentifier=null] (Service: AmazonRoute53; Status Code: 400; Error Code: InvalidInput; Request ID: 2ca80154-78a7-11e9-b5e7-f7bc7c79e5e6)
This is my code.
import com.amazonaws.services.route53.AmazonRoute53Client
import com.amazonaws.services.route53.model.AliasTarget
import com.amazonaws.services.route53.model.Change
import com.amazonaws.services.route53.model.ChangeAction
import com.amazonaws.services.route53.model.ChangeBatch
import com.amazonaws.services.route53.model.ChangeResourceRecordSetsRequest
import com.amazonaws.services.route53.model.ChangeResourceRecordSetsResult
import com.amazonaws.services.route53.model.RRType
import com.amazonaws.services.route53.model.ResourceRecord
import com.amazonaws.services.route53.model.ResourceRecordSet
import grails.plugin.awssdk.AmazonWebService
import grails.transaction.Transactional
#Transactional
class AwsRoute53Service {
AmazonWebService amazonWebService
ChangeResourceRecordSetsResult changeRecordSet() {
AmazonRoute53Client route53Client = amazonWebService.route53
AliasTarget target = new AliasTarget('hostedZoneIDHere', 'app.sample.com.')
target.setEvaluateTargetHealth(true)
List<ResourceRecord> resourceRecords = new ArrayList<>()
resourceRecords.add(new ResourceRecord('dNSNameHere'))
ResourceRecordSet recordSet = new ResourceRecordSet('app.sample.com.', RRType.A)
recordSet.setAliasTarget(target)
recordSet.setResourceRecords(resourceRecords)
recordSet.setTrafficPolicyInstanceId('simple')
List<Change> changes = new ArrayList<>()
changes.add(new Change(ChangeAction.CREATE, recordSet))
ChangeBatch changeBatch = new ChangeBatch(changes)
ChangeResourceRecordSetsRequest request = new ChangeResourceRecordSetsRequest('hostedZoneIDHere', changeBatch)
return route53Client.changeResourceRecordSets(request)
}
}
Can you tell me what is the problem with the setup?
I would be glad if you can help me with my problem right now.
Thank you guys.
I already solve this problem. below is the working code.
import com.amazonaws.services.route53.AmazonRoute53Client
import com.amazonaws.services.route53.model.AliasTarget
import com.amazonaws.services.route53.model.Change
import com.amazonaws.services.route53.model.ChangeAction
import com.amazonaws.services.route53.model.ChangeBatch
import com.amazonaws.services.route53.model.ChangeResourceRecordSetsRequest
import com.amazonaws.services.route53.model.ChangeResourceRecordSetsResult
import com.amazonaws.services.route53.model.RRType
import com.amazonaws.services.route53.model.ResourceRecord
import com.amazonaws.services.route53.model.ResourceRecordSet
import grails.plugin.awssdk.AmazonWebService
import grails.transaction.Transactional
#Transactional
class AwsRoute53Service {
private static final String DOMAIN_NAME_SERVER = "${System.env.DOMAIN_NAME_SERVER}"
private static final String HOSTED_ZONE_ID = "${System.env.HOSTED_ZONE_ID}"
AmazonWebService amazonWebService
ChangeResourceRecordSetsResult changeRecordSet() {
AmazonRoute53Client route53Client = amazonWebService.route53
GetHostedZoneResult hostedZoneResult = route53Client.getHostedZone(new GetHostedZoneRequest(HOSTED_ZONE_ID))
HostedZone hostedZone = hostedZoneResult.getHostedZone()
ResourceRecordSet resourceRecordSet = new ResourceRecordSet()
.withName('dNSName')
.withType(RRType.CNAME)
.withTTL(60)
.withResourceRecords([
new ResourceRecord().withValue(DOMAIN_NAME_SERVER)
])
ChangeResourceRecordSetsRequest request = new ChangeResourceRecordSetsRequest()
.withHostedZoneId(hostedZone.id)
.withChangeBatch(
new ChangeBatch()
.withChanges([
new Change()
.withAction(ChangeAction.CREATE)
.withResourceRecordSet(resourceRecordSet)
])
)
return route53Client.changeResourceRecordSets(request)
}
}
Swift Object storage allow you to create a temporary URL for any resource with an expiry date. This can be achieved with swift CLI command line. To make use of this functionality in an web application, I need to achieve the creation of temporary URL using API call, So that I can make a rest CALL and get the temp URL which can later be embedded in HTML and resource downloaded by the we browser directly.
From the documentation I dont see any API mentioned for this ? Do anyone know how i can get it from Java using API call.
Thanks
Manoj
There is no direct API available to generate temporary URL for Swift objects. Instead it has to generated from client side with the help of X-Account-Meta-Temp-URL-Key secret key as per described in this document
Here is the python version of code to generate it. Refer this to re-implement it in Java and then it can be embedded anywhere.
import hmac
from hashlib import sha1
from time import time
method = 'GET'
duration_in_seconds = 60*60*24
expires = int(time() + duration_in_seconds)
path = '/v1/AUTH_a422b2-91f3-2f46-74b7-d7c9e8958f5d30/container/object'
key = 'mykey'
hmac_body = '%s\n%s\n%s' % (method, expires, path)
sig = hmac.new(key, hmac_body, sha1).hexdigest()
s = 'https://{host}/{path}?temp_url_sig={sig}&temp_url_expires={expires}'
url = s.format(host='swift-cluster.example.com', path=path, sig=sig, expires=expires)
Here is an another reference, which is a customization done to Openstack Horizon to provide an UI feature to generate swift objects temp URL.
For other people looking for the answer in java, Below is the code snippet to get the hmac in java
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SignatureException;
import java.util.Formatter;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
private static String toHexString(byte[] bytes) {
Formatter formatter = new Formatter();
for (byte b : bytes) {
formatter.format("%02x", b);
}
return formatter.toString();
}
public static String calculateRFC2104HMAC(String data, String key)
throws SignatureException, NoSuchAlgorithmException, InvalidKeyException
{
SecretKeySpec signingKey = new SecretKeySpec(key.getBytes(), HMAC_SHA1_ALGORITHM);
Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac.init(signingKey);
return toHexString(mac.doFinal(data.getBytes()));
}
The above code is taken from https://gist.github.com/ishikawa/88599
Use the hmac to create the temporary URL as per the below code
Long expires = (System.currentTimeMillis()/1000)+ <expiry in seconds>;
String tempURL=""+baseURL+path+"?temp_url_sig="+hmac+"& temp_url_expires="+expires;
Thanks
I need to override the DS.Store.findQuery in Ember cli. that is no problem in itself.
The problem is importing the _findQuery method from the 'finder' file -- in that new app/store.js file
this._findQuery doesnt work
https://github.com/emberjs/data/blob/master/packages/ember-data/lib/system/store.js
in the 'shimmed' component/ember-data
the prototype is
function ember$data$lib$system$store$finders$$_findQuery(adapter, store, typeClass, query, recordArray
Has anyone some advice on the required import statement.
here is some failed attempts
import DS from 'ember-data';
import Ember from 'ember';
//import _findQuery from 'ember-data/lib/system/store/finders'; NOPE
//import _findQuery from 'ember-data'; NOPE
export default DS.Store.extend({
findQuery: function(typeName, query) {
var type = this.modelFor(typeName);
var array = this.recordArrayManager
.createAdapterPopulatedRecordArray(type, query);
var adapter = this.adapterFor(type);
Ember.assert("You tried to load a query but you have no adapter (for " + type + ")", adapter);
Ember.assert("You tried to load a query but your adapter does not implement `findQuery`", typeof adapter.findQuery === 'function');
var x = _findQuery(adapter, this, type, query, array); // <-- URGH HERE
return promiseArray(x);
},
I'm not sure that you are able to import it in the way you describe, you could do it on the Adapter though.
You should be able to override it per Adapter, or if you want to do it everywhere, override on your application Adapter.
Like this
import DS from 'ember-data';
import Ember from 'ember';
export default DS.ActiveModelAdapter.extend({
findQuery (typeName, query) {
// do your stuff here
}
});
I have a Scala class, User:
class User {
#OId var id: String = _
var email: String = _
var password_hash: String = _
var password_salt: String = _
var admin : Boolean = _
#OVersion var version: String = _
}
I can successfully create and store User objects in the DB, but when I query for a User, e.g. db.queryBySql[User]("select * from User where email = ?", username)
I get a User object back, but all fields are null.
When stepping the code I can see the correct result from the DB, the POJO conversions is what fails.
I am using scala 2.9.2 and Orient DB 1.1.0.
What am I doing wrong?
I did the exemple explain at : http://code.google.com/p/orient/wiki/ScalaLanguage
This exemple uses the local connexion
var db: ODatabaseObjectTx = new ODatabaseObjectTx("local:/tmp/dbtmp")
I change a little the main app.
I change import to match with organisation of OrientDb packages.
import com.orientechnologies.orient.core.id.ORecordId
import com.orientechnologies.orient.core.sql.query.OSQLSynchQuery
import scala.collection.JavaConverters._
import scala.collection.JavaConversions._
import com.orientechnologies.orient.core.db.`object`.ODatabaseObject
import com.orientechnologies.orient.`object`.db.ODatabaseObjectTx
And, I change the way to open connexion.
Be careful ODatabaseObjectTx is deprecated.
I use db2 instead db after this snippet.
var db2:ODatabaseObject =
if (!db.exists) {
db.create()
} else {
db.open("admin", "admin")
}
I import these packages :
orient-commons:1.1.0
orientdb-core:1.1.0
orientdb-object:1.1.0
hibernate-jpa-2.0-api:1.0.0.Final
scala-library:2.9.2