Please note: even though I'm using Groovy here, I think my exception is really about using the Jersey/JAX-RS API correctly.
Given the following code:
ClientConfig clientConfig = new DefaultClientConfig()
clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE)
Client jerseyClient = Client.create(clientConfig)
WebResource webResource = jerseyClient.resource("http://localhost:8080/location/")
Long id = 5L
Address address = webResource.path("address").path(id)
.accept(MediaType.APPLICATION_JSON)
.get(Long)
I am getting the following exception:
groovy.lang.MissingMethodException: No signature of method: com.sun.jersey.api.client.WebResource.path() is applicable for argument types: (java.lang.Long) values: [5]
Possible solutions: path(java.lang.String), put(), wait(long), put(com.sun.jersey.api.client.GenericType), put(java.lang.Class), put(java.lang.Object)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.me.myapp.Driver.run(Driver.groovy:43)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<rest omitted for brevity>
I am trying to hit the following REST endpoint:
GET http://localhost:8080/location/address/{id}
Where am I going wrong?
You're calling the path method with a long, but it can only take a String. Your id is a long, but since the error message says the value is 1, I assume that LocationResourcePaths.ADDRESS_PATH is also a long with value 1, is that the case?
Related
I need to write to external HDFS cluster whose authentication details are available for both simple as well as kerberos authentication. For the sake of simplicity, lets assume we are dealing with simple authentication.
This is what I have:
External HDFS cluster connection details (host, port)
Authentication details (user for simple auth)
HDFS location where files need to be written (hdfs://host:port/loc)
Also, other details like format, etc.
Please note SPARK user is not same as user specified for HDFS auth.
Now, using the spark programming API, this is what I am trying to do:
val hadoopConf = new Configuration()
hadoopConf.set("fs.defaultFS", fileSystemPath)
hadoopConf.set("hadoop.job.ugi", userName)
val jConf = new JobConf(hadoopConf)
jConf.setUser(user)
jConf.set("user.name", user)
jConf.setOutputKeyClass(classOf[NullWritable])
jConf.setOutputValueClass(classOf[Text])
jConf.setOutputFormat(classOf[TextOutputFormat[NullWritable, Text]])
outputDStream.foreachRDD(r => {
val rdd = r.mapPartitions { iter =>
val text = new Text()
iter.map { x =>
text.set(x.toString)
println(x.toString)
(NullWritable.get(), text)
}
}
val rddCount = rdd.count()
if(rddCount > 0) {
rdd.saveAsHadoopFile(config.outputPath, classOf[NullWritable], classOf[Text], classOf[TextOutputFormat[NullWritable, Text]], jConf)
}
})
Here, I was assuming that if we pass JobConf with correct details, it should be used for authentication and write should be done using the user specified in JobConf.
However, write still happens as the spark user ("root") irrespective of the auth details present in JobConf ("hdfs" as user). Below is the exception that I get:
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=root, access=WRITE, inode="/spark-deploy/out/_temporary/0":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1698)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1682)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1665)
at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3900)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:978)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
at org.apache.hadoop.ipc.Client.call(Client.java:1475)
at org.apache.hadoop.ipc.Client.call(Client.java:1412)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy40.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:558)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy41.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3000)
... 45 more
Please let me know, if there are any suggestions.
This is probably more a comment than an answer but as it is too long I put it here. I haven't tried this because I have no environment to test it. Please try and let me know if this works (and if it doesn't I'll remove this answer).
Looking a bit into the code it looks like DFSClient creates a proxy using createProxyWithClientProtocol that uses UserGroupInformation.getCurrentUser() (I haven't traced the createHAProxy branch down but I suspect the same logic there). Then this info is sent to the server for authentication.
It means that you need to change what UserGroupInformation.getCurrentUser() returns in the context of your particular call. This is what UserGroupInformation.doAs is supposed to do so you just need to get a proper UserGroupInformation instance. And in the case of simple authentication UserGroupInformation.createRemoteUser might actually work.
So I suggest trying something like this:
...
val rddCount = rdd.count()
if(rddCount > 0) {
val remoteUgi = UserGroupInformation.createRemoteUser("hdfsUserName")
remoteUgi.doAs(() => { rdd.saveAsHadoopFile(config.outputPath, classOf[NullWritable], classOf[Text], classOf[TextOutputFormat[NullWritable, Text]], jConf) })
}
I am using play-framework 2.3.x with reactivemongo-extension JSON type. following is my code for fetch the data from db as below:
def getStoredAccessToken(authInfo: AuthInfo[User]) = {
println(">>>>>>>>>>>>>>>>>>>>>>: BEFORE"); //$doc("clientId" $eq authInfo.user.email, "userId" $eq authInfo.user._id.get)
var future = accessTokenService.findRandom(Json.obj("clientId" -> authInfo.user.email, "userId" -> authInfo.user._id.get));
println(">>>>>>>>>>>>>>>>>>>>>>: AFTER: "+future);
future.map { option => {
println("*************************** ")
println("***************************: "+option.isEmpty)
if (!option.isEmpty){
var accessToken = option.get;println(">>>>>>>>>>>>>>>>>>>>>>: BEFORE VALUE");
var value = Crypto.validateToken(accessToken.createdAt.value)
println(">>>>>>>>>>>>>>>>>>>>>>: "+value);
Some(scalaoauth2.provider.AccessToken(accessToken.accessToken, accessToken.refreshToken, authInfo.scope,
Some(value), new Date(accessToken.createdAt.value)))
}else{
Option.empty
}
}}
}
When i using BSONDao and BsonDocument for fetching the data, this code successfully run, but after converting to JSONDao i getting the following error:
Note: Some time this code will run but some it thrown an exception after converting to JSON
play - Cannot invoke the action, eventually got an error: java.lang.IllegalArgumentException: bound must be positive
application -
Following are the logs of application full exception strack trace as below:
>>>>>>>>>>>>>>>>>>>>>>: BEFORE
>>>>>>>>>>>>>>>>>>>>>>: AFTER: scala.concurrent.impl.Promise$DefaultPromise#7f4703e3
play - Cannot invoke the action, eventually got an error: java.lang.IllegalArgumentException: bound must be positive
application -
! #6m1520jff - Internal server error, for (POST) [/oauth2/token] ->
play.api.Application$$anon$1: Execution exception[[IllegalArgumentException: bound must be positive]]
at play.api.Application$class.handleError(Application.scala:296) ~[play_2.11-2.3.8.jar:2.3.8]
at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
at scala.Option.map(Option.scala:146) [scala-library-2.11.6.jar:na]
Caused by: java.lang.IllegalArgumentException: bound must be positive
at java.util.Random.nextInt(Random.java:388) ~[na:1.8.0_40]
at scala.util.Random.nextInt(Random.scala:66) ~[scala-library-2.11.6.jar:na]
The problem is solve, but i am not sure, why this produce, I think there is problem with reactivemongo-extension JSONDao library. because when i use findOne instead of findRandom the code is run successfully, but the findRandom is run good on BSON dao. Still not found what the exact problem is that, but following is the resolved code.
def getStoredAccessToken(authInfo: AuthInfo[User]) = {
println(authInfo.user.email+" ---- "+authInfo.user._id.get)
var future = accessTokenService.findOne($doc("clientId" $eq authInfo.user.email, "userId" $eq authInfo.user._id.get)); //user findOne instead of findRandom in JsonDao
future.map { option => {
if (!option.isEmpty){
var accessToken = option.get;
var value = Crypto.validateToken(accessToken.createdAt.value)
Some(scalaoauth2.provider.AccessToken(accessToken.accessToken, accessToken.refreshToken, authInfo.scope,
Some(value), new Date(accessToken.createdAt.value)))
}else{
Option.empty
}
}}
}
i am using the following code to get the balancesheet and profilt and loss data from quickbooks.
OAuthRequestValidator oauth = new OAuthRequestValidator(accessToken, accessTokenSecret,
consumerKey, consumerSecret);
ServiceContext serviceContext = new ServiceContext(realmId, IntuitServicesType.QBO, oauth);
serviceContext.IppConfiguration.Message.Request.SerializationFormat =
Intuit.Ipp.Core.Configuration.SerializationFormat.Json;
ReportService reportService = new ReportService(serviceContext);
reportService.accounting_method = "Accrual";
reportService.start_date = "2014-01-01";
reportService.end_date = "2014-06-01";
Report report = reportService.ExecuteReport("BalanceSheet");
it compiles well, but when it runs it gives the following error.
"
**Ids service endpoint was not found.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: Intuit.Ipp.Exception.EndpointNotFoundException: Ids service endpoint was not found.
Source Error:
Line 133: reportService.start_date = "2014-01-01";
Line 134: reportService.end_date = "2014-06-01";
Line 135: Report report = reportService.ExecuteReport("BalanceSheet");
Line 136:
Line 137:"**
Please use the correct date format YYYY-MM-DD.
Please use response format as JSON as xml is not supported. Your code will work then.
serviceContext.IppConfiguration.Message.Response.SerializationFormat = Intuit.Ipp.Core.Configuration.SerializationFormat.Json;
I have a simple REST client using the Apache CXF library. Here is the code snippet:
val wc = WebClient.create(address(host, port) + "/" + resource).`type`("text/plain")
requestParam match {
case Some(reqParams) => reqParams.foreach((param: (String, String)) => {
wc.query(param._1, param._2)
})
case None => wc
}
println("Acturl url is " + wc.getCurrentURI)
wc.get(classOf[String])
}
What I'm trying to fetch is a simple file named test.txt. I want to render this as a plain String and that is what I do at the last line on the code snippet above. But I get the following error:
javax.ws.rs.NotAuthorizedException was thrown.
javax.ws.rs.NotAuthorizedException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.cxf.jaxrs.client.AbstractClient.convertToWebApplicationException(AbstractClient.java:462)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:860)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:831)
at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:394)
at org.apache.cxf.jaxrs.client.WebClient.get(WebClient.java:573)
How can I return the call to a GET endpoint as a plain String?
I was a bit stupid. My original code actually worked and returned a String that I wanted!
wc.get(classOf[String])
I am having trouble using Steam as OpenId provider. Everything works fine until the callback to my site is made, I see the steam login web page and I can login with my user, but when the calback executes I get an exception.
I use play 2.2 and Scala. The code is very similar to the one found on the play docs
def loginPost = Action.async { implicit request =>
OpenID.redirectURL("http://steamcommunity.com/openid",
routes.Application.openIDCallback.absoluteURL(),
realm = Option("http://mydomain.com/"))
.map(url => Redirect(url))
.recover { case error => Redirect(routes.Application.login) }
}
def openIDCallback = Action.async { implicit request =>
OpenID.verifiedId.map(info => Ok(info.id + "\n" + info.attributes))
.recover {
case error =>
println(error.getMessage()) //prints null
Redirect(routes.Application.login)
}
}
Stacktrace:
Internal server error, for (GET) [/steam/login?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.mode=error&openid.error=Invalid+claimed_id+or+identity] ->
play.api.Application$$anon$1: Execution exception[[BAD_RESPONSE$: null]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:165) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:162) [play_2.10.jar:2.2.1]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.3.jar:na]
at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.3.jar:na]
Caused by: play.api.libs.openid.Errors$BAD_RESPONSE$: null
at play.api.libs.openid.Errors$BAD_RESPONSE$.<clinit>(OpenIDError.scala) ~[play_2.10.jar:2.2.1]
at play.api.libs.openid.OpenIDClient.verifiedId(OpenID.scala:111) ~[play_2.10.jar:2.2.1]
at play.api.libs.openid.OpenIDClient.verifiedId(OpenID.scala:92) ~[play_2.10.jar:2.2.1]
at controllers.Application$$anonfun$openIDCallback$1.apply(Application.scala:29) ~[classes/:2.2.1]
at controllers.Application$$anonfun$openIDCallback$1.apply(Application.scala:28) ~[classes/:2.2.1]
at play.api.mvc.Action$.invokeBlock(Action.scala:357) ~[play_2.10.jar:2.2.1]
I see in the returned URL this error message openid.error=Invalid+claimed_id+or+identity but couldnt find anything related.
What am I missing? Thanks.
It's because the Play Framework OpenID classes are not properly generating the redirect URL. Print out the value of the url variable from this line in your code:
.map(url => Redirect(url))
It most likely looks something like this:
https://steamcommunity.com/openid/login?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0
&openid.mode=checkid_setup
&openid.claimed_id=http%3A%2F%2Fsteamcommunity.com%2Fopenid
&openid.identity=http%3A%2F%2Fsteamcommunity.com%2Fopenid
&openid.return_to=http%3A%2F%2Fwww.mydomain.com%2Fsteam%2Flogin
&openid.realm=http%3A%2F%2Fwww.mydomain.com
This is incorrect per the OpenID 2.0 spec, specifically at http://openid.net/specs/openid-authentication-2_0.html#discovered_info:
If the end user entered an OpenID Provider (OP) Identifier, there is no Claimed Identifier. For the purposes of making OpenID Authentication requests, the value "http://specs.openid.net/auth/2.0/identifier_select" MUST be used as both the Claimed Identifier and the OP-Local Identifier when an OP Identifier is entered.
Based on that, the generated redirect url variable should be:
https://steamcommunity.com/openid/login?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0
&openid.mode=checkid_setup
&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select
&openid.return_to=http%3A%2F%2Fwww.mydomain.com%2Fsteam%2Flogin
&openid.realm=http%3A%2F%2Fwww.mydomain.com
I've written up an issue for this on the Play Framework issue tracker:
https://github.com/playframework/playframework/issues/3740
In the meantime as a temporary hack/fix, you can use any number of string replacement techniques on your url variable to set the openid.claimed_id and openid.identity parameters to the correct http://specs.openid.net/auth/2.0/identifier_select value.