Mirth SQL Server Database reader source, DBMaxRetries error - mirth

I am using Mirth Connect Server 3.4.2.8129. I created a test channel, a very simple one which should just select the records from a table (SQL Server) and send them to a destination of type javascript writer.
I defined a database reader source and I am trying to select the records of a table. I will paste the code below.
The Javascript code I use in the Source is this (I replaced sensitive information with ***):
var dbConn;
try
{
dbConn = DatabaseConnectionFactory.createDatabaseConnection('net.sourceforge.jtds.jdbc.Driver','jdbc:jtds:sqlserver://***:***;InstanceName=***;DatabaseName=***','***','***');
var results = executeCachedQuery('select * from [***].[dbo].[***]');
return results;
}
finally
{
if (dbConn)
{
dbConn.close();
}
}
I get the following error:
ERROR 2019-10-14 12:41:00,164 [Database Reader Polling Thread on TestMedisOrders (fba0f4c7-0a2d-47ec-b638-acb586925c92) < fba0f4c7-0a2d-47ec-b638-acb586925c92_Worker-1] com.mirth.connect.connectors.jdbc.DatabaseReceiver: Failed to poll for messages from the database in channel "TestMedisOrders"
com.mirth.connect.connectors.jdbc.DatabaseReceiverException: Error executing script 4bf7e588-202a-4aea-9d5e-be0155a4fa6a.
at com.mirth.connect.connectors.jdbc.DatabaseReceiverScript.poll(DatabaseReceiverScript.java:128)
at com.mirth.connect.connectors.jdbc.DatabaseReceiver.poll(DatabaseReceiver.java:108)
at com.mirth.connect.donkey.server.channel.PollConnectorJob.execute(PollConnectorJob.java:49)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: com.mirth.connect.server.MirthJavascriptTransformerException:
CHANNEL: TestMedisOrders
CONNECTOR: Source
SOURCE CODE:
125: }
126:
127: function executeOperation(source, operation, expression, parameters) {
128: var dbConn = getDBConnection(source);
129: var attempts = 0;
130: var maxAttempts = java.lang.Integer.parseInt($('DBMaxRetries'));
131:
132: while (attempts < maxAttempts) {
133: attempts++;
134:
LINE NUMBER: 130
DETAILS: Wrapped java.lang.NumberFormatException: For input string: ""
at 4bf7e588-202a-4aea-9d5e-be0155a4fa6a:130 (executeOperation)
at 4bf7e588-202a-4aea-9d5e-be0155a4fa6a:67 (executeCachedQuery)
at 4bf7e588-202a-4aea-9d5e-be0155a4fa6a:249 (doScript)
at 4bf7e588-202a-4aea-9d5e-be0155a4fa6a:259
at com.mirth.connect.server.util.javascript.JavaScriptUtil.executeScript(JavaScriptUtil.java:527)
at com.mirth.connect.connectors.jdbc.DatabaseReceiverScript$SelectTask.doCall(DatabaseReceiverScript.java:168)
at com.mirth.connect.server.util.javascript.JavaScriptTask.call(JavaScriptTask.java:113)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(Unknown Source)
at java.lang.Integer.parseInt(Unknown Source)
at java.lang.Integer.parseInt(Unknown Source)
at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1479)
at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815)
at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3280)
at org.mozilla.javascript.InterpretedFunction.exec(InterpretedFunction.java:120)
at com.mirth.connect.server.util.javascript.JavaScriptTask.executeScript(JavaScriptTask.java:142)
at com.mirth.connect.server.util.javascript.JavaScriptUtil.executeScript(JavaScriptUtil.java:522)
... 6 more
I tried searching for a setting in all the configuration files in mirth installation folder, but I didn't find any file containing it. I also unarchived the SQL Server driver (jtds 1.3.1.jar) and searched there, too, but no. I did not find anything about a DBMaxRetries.
Has anyone else face this?
Thank you.

You likely meant to call:
var results = dbConn.executeCachedQuery('select * from [***].[dbo].[***]');
According to the stack trace, you must have javascript functions executeCachedQuery and executeOperation defined, likely through a code template if you check your channel dependencies. You are calling the executeCachedQuery javascript function instead of the DatabaseConnection.executeCachedQuery API method.
$('DBMaxRetries') is attempting to pull the value from one of the mirth maps. globalMap, globalChannelMap, and configurationMap are the only ones available from a Database Reader context. If you haven't defined DBMaxRetries in at least one of these maps, it will return null.

Related

spark: SAXParseException while writing to parquet on s3

I'm trying to read in some json, infer a schema, and write it out again as parquet to s3 (s3a). For some reason, about a third of the way through the writing portion of the run, spark always errors out with the error included below. I can't find any obvious reasons for the issue: it isn't out of memory; there are no long GC pauses. There don't seem to be any additional error messages in the logs of the individual executors.
The script runs fine on another set of data that I have, which is of a very similar structure, but several orders of magnitude smaller.
I am running spark 2.0.1-hadoop-2.7 and am using the FileOutputCommitter. The algorithm version doesn't seem to matter.
Edit:
This does not appear to be a problem in badly formed json or corrupted files. I have unzipped and read in each file individually with no error.
Here's a simplified version of the script:
object Foo {
def parseJson(json: String): Option[Map[String, Any]] = {
if (json == null)
Some(Map())
else
parseOpt(json).map((j: JValue) => j.values.asInstanceOf[Map[String, Any]])
}
}
}
// read in as text and parse json using json4s
val jsonRDD: RDD[String] = sc.textFile(inputPath)
.map(row -> Foo.parseJson(row))
// infer a schema that will encapsulate the most rows in a sample of size sampleRowNum
val schema: StructType = Infer.getMostCommonSchema(sc, jsonRDD, sampleRowNum)
// get documents compatibility with schema
val jsonWithCompatibilityRDD: RDD[(String, Boolean)] = jsonRDD
.map(js => (js, Infer.getSchemaCompatibility(schema, Infer.inferSchema(js)).toBoolean))
.repartition(partitions)
val jsonCompatibleRDD: RDD[String] = jsonWithCompatibilityRDD
.filter { case (js: String, compatible: Boolean) => compatible }
.map { case (js: String, _: Boolean) => js }
// create a dataframe from documents with compatible schema
val dataFrame: DataFrame = spark.read.schema(schema).json(jsonCompatibleRDD)
It completes the earlier schema inferring steps successfully. The error itself occurs on the last line, but I suppose that could encompass at least the immediately preceding statemnt, if not earlier:
org.apache.spark.SparkException: Task failed while writing rows
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.writeRows(WriterContainer.scala:261)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(InsertIntoHadoopFsRelationCommand.scala:143)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(InsertIntoHadoopFsRelationCommand.scala:143)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:86)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Failed to commit task
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.org$apache$spark$sql$execution$datasources$DefaultWriterContainer$$commitTask$1(WriterContainer.scala:275)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply$mcV$sp(WriterContainer.scala:257)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply(WriterContainer.scala:252)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$1.apply(WriterContainer.scala:252)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1345)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.writeRows(WriterContainer.scala:258)
... 8 more
Suppressed: java.lang.NullPointerException
at org.apache.parquet.hadoop.InternalParquetRecordWriter.flushRowGroupToStore(InternalParquetRecordWriter.java:147)
at org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:113)
at org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:112)
at org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetFileFormat.scala:569)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.org$apache$spark$sql$execution$datasources$DefaultWriterContainer$$abortTask$1(WriterContainer.scala:282)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer$$anonfun$writeRows$2.apply$mcV$sp(WriterContainer.scala:258)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1354)
... 9 more
Caused by: com.amazonaws.AmazonClientException: Unable to unmarshall response (Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler). Response Code: 200, Response Text: OK
at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:738)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:399)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3480)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:604)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:962)
at org.apache.hadoop.fs.s3a.S3AFileSystem.deleteUnnecessaryFakeDirectories(S3AFileSystem.java:1147)
at org.apache.hadoop.fs.s3a.S3AFileSystem.finishedWrite(S3AFileSystem.java:1136)
at org.apache.hadoop.fs.s3a.S3AOutputStream.close(S3AOutputStream.java:142)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
at org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:400)
at org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:117)
at org.apache.parquet.hadoop.ParquetRecordWriter.close(ParquetRecordWriter.java:112)
at org.apache.spark.sql.execution.datasources.parquet.ParquetOutputWriter.close(ParquetFileFormat.scala:569)
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.org$apache$spark$sql$execution$datasources$DefaultWriterContainer$$commitTask$1(WriterContainer.scala:267)
... 13 more
Caused by: com.amazonaws.AmazonClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:150)
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:279)
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:75)
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:72)
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:712)
... 29 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 2; XML document structures must start and end within the same entity.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.endEntity(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl.endEntity(Unknown Source)
at org.apache.xerces.impl.XMLEntityManager.endEntity(Unknown Source)
at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)
at org.apache.xerces.impl.XMLEntityScanner.skipChar(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:141)
... 35 more
Here's my conf:
spark.executor.extraJavaOptions -XX:+UseG1GC -XX:MaxPermSize=1G -XX:+HeapDumpOnOutOfMemoryError
spark.executor.memory 16G
spark.executor.uri https://s3.amazonaws.com/foo/spark-2.0.1-bin-hadoop2.7.tgz
spark.hadoop.fs.s3a.impl org.apache.hadoop.fs.s3a.S3AFileSystem
spark.hadoop.fs.s3a.buffer.dir /raid0/spark
spark.hadoop.fs.s3n.buffer.dir /raid0/spark
spark.hadoop.fs.s3a.connection.timeout 500000
spark.hadoop.fs.s3n.multipart.uploads.enabled true
spark.hadoop.parquet.block.size 2147483648
spark.hadoop.parquet.enable.summary-metadata false
spark.jars.packages com.databricks:spark-avro_2.11:3.0.1
spark.local.dir /raid0/spark
spark.mesos.coarse false
spark.mesos.constraints priority:1
spark.network.timeout 600
spark.rpc.message.maxSize 500
spark.speculation false
spark.sql.parquet.mergeSchema false
spark.sql.planner.externalSort true
spark.submit.deployMode client
spark.task.cpus 1
I can think for three possible reasons for this problem.
JVM version. AWS SDK checks for the following ones. "1.6.0_06",
"1.6.0_13", "1.6.0_17", "1.6.0_65", "1.7.0_45". If you are using one
of them, try upgrading.
Old AWS SDK. Refer to
https://github.com/aws/aws-sdk-java/issues/460 for a workaround.
If you lots of files in the directory where you are writing these files, you might be hitting https://issues.apache.org/jira/browse/HADOOP-13164. Consider increasing the timeout to larger values.
A SAXParseException may indicate a badly formatted XML file. Since the job fails roughly a third of the way through consistently, this means it's probably failing in the same place every time (a file whose partition is roughly a third of the way through the partition list).
Can you paste your script? It may be possible to wrap the Spark step in a try/catch loop that will print out the file if this error occurs, which will let you easily zoom in on the problem.
From the logs:
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 2; XML document structures must start and end within the same entity.
and
Caused by: com.amazonaws.AmazonClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
It looks like you have a corrupted/incorrectly formatted file, and your error is actually occurring during the read portion of the task. You could confirm this by trying another operation that will force the read such as count().
If confirmed, the goal would then be to find the corrupted file. You could do this by listing the s3 files, sc.parallelize() that list, and then trying to read the files in a custom function using map().
import boto3
from pyspark.sql import Row
def scanKeys(startKey, endKey):
bucket = boto3.resource('s3').Bucket('bucketName')
for obj in bucket.objects.filter(Prefix='prefix', Marker=startKey):
if obj.key < endKey:
yield obj.key
else:
return
def testFile(s3Path):
s3obj = boto3.resource('s3').Object(bucket_name='bucketName', key=key)
body = s3obj.get()['Body']
...
logic to test file format, or use a try/except and attempt to parse it
...
if fileFormatedCorrectly == True:
return Row(status='Good', key = s3Path)
else:
return Row(status='Fail', key = s3Path)
keys = list(scanKeys(startKey, endKey))
keyListRdd = sc.parallelize(keys, 1000)
keyListRdd.map(testFile).filter(lambda x: x.asDict.get('status') == 'Fail').collect()
This will return the s3 paths for the incorrectly formatted files
For Googlers:
If you:
have a versioned bucket
use s3a://
see ListBucketHandler and listObjects in your error message
Quick solution:
use s3:// instead of s3a://, which will use S3 driver provided by EMR
You may see this error because s3a:// in older versions uses S3::ListObjects (v1) API instead of S3::ListObjectsV2. The former would return extra info like owner, and is not robust against large number of deletion markers. Newer versions of the s3a:// driver solved this problem, but you could always use the s3:// driver instead.
Quote:
the V1 list API experience always returns 5000 entries (as set in fs.s3a.paging.maximum
except for the final entry
if you have versioning turned on in your bucket, deleted entries retain tombstone markers with references to their versions
which will surface in the S3-side of list calls, but get stripped out from the response
so...for a very large tree, you may end up S3 having to keep a channel open while is skips of thousands to millions of deleted
objects before it can find actual ones to return.
which can time out connections.
Quote:
Introducing a new version of the ListObjects (ListObjectsV2) API that allows listing objects with a large number of delete markers.
Quote:
If there are thousands of delete markers, the list operation might timeout。

Talend: REST Call results in either "URI is not absolute" or "UnknownHostException"

I am trying to get data from a REST API to our database with the help of Talend Jaspersoft ETL Express Version 5.6.
To make sure that my problem is not related to our server configuration i installed the tool locally too and got stuck. Since I am a student who is not well experienced with java but needs to get this run for his company i hope you can help me. I have only some basic knowledge so i did not try to find a solution in the "Code" tab, only worked with the Designer.
In the end I wanted to try out the following tutorial:
http://dwetl.com/2015/08/11/trest-use-case-example-use-rest-api-in-talend/
but that don't works too for me. I can get the data when i use the call in my browser but with that tool i am getting only errors.
With "https://api.github.com/users/mralexgray/followers" as URL I get an UnknownHostException:
Starte Job Test am 11:20 21/07/2016.
[statistics] connecting to socket on port 3941
[statistics] connected
Exception in component tREST_1
com.sun.jersey.api.client.ClientHandlerException: java.net.UnknownHostException: api.github.com
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:131)
at com.sun.jersey.api.client.Client.handle(Client.java:616)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:559)
at com.sun.jersey.api.client.WebResource.get(WebResource.java:182)
at testetl.test_0_1.Test.tREST_1Process(Test.java:572)
at testetl.test_0_1.Test.runJobInTOS(Test.java:929)
at testetl.test_0_1.Test.main(Test.java:786)
Caused by: java.net.UnknownHostException: api.github.com
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
[statistics] disconnected
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:218)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:129)
... 6 more
Job Test endet am 11:20 21/07/2016. [exit code=1]
If I cut off that "https://" part and use "api.github.com/users/mralexgray/followers" as Url I get the error "Uri is not absolute"
Starte Job Test am 11:31 21/07/2016.
[statistics] connecting to socket on port 3809
[statistics] connected
Exception in component tREST_1
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:131)
at com.sun.jersey.api.client.Client.handle(Client.java:616)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:559)
at com.sun.jersey.api.client.WebResource.get(WebResource.java:182)
at testetl.test_0_1.Test.tREST_1Process(Test.java:572)
at testetl.test_0_1.Test.runJobInTOS(Test.java:929)
at testetl.test_0_1.Test.main(Test.java:786)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(Unknown Source)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:140)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:129)
... 6 more
[statistics] disconnected
Job Test endet am 11:31 21/07/2016. [exit code=1]
Just to give you as many information as possible, here is a little part out of the auto generated java code which seems to be about this REST Call for me, but maybe I am completly wrong and this dont helps at all since I have only basic java knowledge:
/**
* [tREST_1 begin ] start
*/
ok_Hash.put("tREST_1", false);
start_Hash.put("tREST_1", System.currentTimeMillis());
currentComponent = "tREST_1";
int tos_count_tREST_1 = 0;
String endpoint_tREST_1 = "api.github.com/users/mralexgray/followers";
String trustStoreFile_tREST_1 = System
.getProperty("javax.net.ssl.trustStore");
String trustStoreType_tREST_1 = System
.getProperty("javax.net.ssl.trustStoreType");
String trustStorePWD_tREST_1 = System
.getProperty("javax.net.ssl.trustStorePassword");
String keyStoreFile_tREST_1 = System
.getProperty("javax.net.ssl.keyStore");
String keyStoreType_tREST_1 = System
.getProperty("javax.net.ssl.keyStoreType");
String keyStorePWD_tREST_1 = System
.getProperty("javax.net.ssl.keyStorePassword");
com.sun.jersey.api.client.config.ClientConfig config_tREST_1 = new com.sun.jersey.api.client.config.DefaultClientConfig();
javax.net.ssl.SSLContext ctx_tREST_1 = javax.net.ssl.SSLContext
.getInstance("SSL");
javax.net.ssl.TrustManager[] tms_tREST_1 = null;
if (trustStoreFile_tREST_1 != null
&& trustStoreType_tREST_1 != null) {
char[] password_tREST_1 = null;
if (trustStorePWD_tREST_1 != null)
password_tREST_1 = trustStorePWD_tREST_1.toCharArray();
java.security.KeyStore trustStore_tREST_1 = java.security.KeyStore
.getInstance(trustStoreType_tREST_1);
trustStore_tREST_1.load(new java.io.FileInputStream(
trustStoreFile_tREST_1), password_tREST_1);
javax.net.ssl.TrustManagerFactory tmf_tREST_1 = javax.net.ssl.TrustManagerFactory
.getInstance(javax.net.ssl.KeyManagerFactory
.getDefaultAlgorithm());
tmf_tREST_1.init(trustStore_tREST_1);
tms_tREST_1 = tmf_tREST_1.getTrustManagers();
}
javax.net.ssl.KeyManager[] kms_tREST_1 = null;
if (keyStoreFile_tREST_1 != null
&& keyStoreType_tREST_1 != null) {
char[] password_tREST_1 = null;
if (keyStorePWD_tREST_1 != null)
password_tREST_1 = keyStorePWD_tREST_1.toCharArray();
java.security.KeyStore keyStore_tREST_1 = java.security.KeyStore
.getInstance(keyStoreType_tREST_1);
keyStore_tREST_1.load(new java.io.FileInputStream(
keyStoreFile_tREST_1), password_tREST_1);
javax.net.ssl.KeyManagerFactory kmf_tREST_1 = javax.net.ssl.KeyManagerFactory
.getInstance(javax.net.ssl.KeyManagerFactory
.getDefaultAlgorithm());
kmf_tREST_1.init(keyStore_tREST_1, password_tREST_1);
kms_tREST_1 = kmf_tREST_1.getKeyManagers();
}
ctx_tREST_1.init(kms_tREST_1, tms_tREST_1, null);
config_tREST_1
.getProperties()
.put(com.sun.jersey.client.urlconnection.HTTPSProperties.PROPERTY_HTTPS_PROPERTIES,
new com.sun.jersey.client.urlconnection.HTTPSProperties(
new javax.net.ssl.HostnameVerifier() {
public boolean verify(
String hostName,
javax.net.ssl.SSLSession session) {
return true;
}
}, ctx_tREST_1));
com.sun.jersey.api.client.Client restClient_tREST_1 = com.sun.jersey.api.client.Client
.create(config_tREST_1);
com.sun.jersey.api.client.WebResource restResource_tREST_1;
if (endpoint_tREST_1 != null && !("").equals(endpoint_tREST_1)) {
restResource_tREST_1 = restClient_tREST_1
.resource(endpoint_tREST_1);
} else {
throw new IllegalArgumentException("url can't be empty!");
}
com.sun.jersey.api.client.ClientResponse errorResponse_tREST_1 = null;
String restResponse_tREST_1 = "";
try {
restResponse_tREST_1 = restResource_tREST_1
.get(String.class);
} catch (com.sun.jersey.api.client.UniformInterfaceException ue) {
errorResponse_tREST_1 = ue.getResponse();
}
// for output
row1 = new row1Struct();
if (errorResponse_tREST_1 != null) {
row1.ERROR_CODE = errorResponse_tREST_1.getStatus();
} else {
row1.Body = restResponse_tREST_1;
}
/**
* [tREST_1 begin ] stop
*/
/**
* [tREST_1 main ] start
*/
Any help would be very appreciated :). All my examples are about that tutorial but while I was trying to get our real API to work with ETL I just got the exact same problems and I guess this tutorial is easier to give you guys as an example.
We finally could figure out a way to solve our connection issue.
I don't know why Talend is not using our proxy correctly even if its manually set up in the preferences but if we use the tSetProxy component it works and gets the expected JSON object back from the server.
java.net.UnknownHostException : Did you check if you are behind a proxy ? The previous error should happen if, from your studio position, the site is not accessible.
You have to know that your browser doesn't necessarily have the same configuration as Talend Studio. So, please check your "Internet Settings" for the existing of any proxy. If it is the case, then it is possible to change the proxy settings in menu: Window -> Preferences, under General -> Network.

Strange exception using Jersey with embedded Jetty

I have created an embedded Jetty application which utilizes Jersey in order to implement several RESTful services. I am using some standard code as described here in Stack Overflow as well as other websites:
public static void main(String[] args)
{
Server server = new Server(8080);
ServletContextHandler ctx = new ServletContextHandler(
ServletContextHandler.SESSIONS);
ctx.setContextPath("/");
ServletHolder holder = ctx.addServlet(
"org.glassfish.jersey.servlet.ServletContainer.class", "/*");
holder.setInitOrder(0);
holder.setInitParameter("jersey.config.server.provider.classnames",
RestfulClass.class.getCanonicalName());
server.setHandler(ctx);
try
{
server.start();
server.join();
}
catch(Exception exe)
{
exe.printStackTrace();
}
}
I've used all the recommended jar files, and several other jar files that the various blogs and sites failed to mention. When running the Jetty application, I get the following exception:
java.lang.ClassNotFoundException: org.glassfish.jersey.servlet.ServletContainer.class
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.eclipse.jetty.util.Loader.loadClass(Loader.java:86)
at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:95)
<several lines omitted for brevty>
Caused by: javax.servlet.UnavailableException: org.glassfish.jersey.servlet.ServletContainer.class
at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:102)
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:361)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:874)
... 11 more
It is the "UnavailableException" that I do not understand. The ServletContainer class is actually in one of the Jar files (in jersey-container-servlet-core.jar, to be precise), but for some reason it is identified as "unavailable". This is causing a class that is actually in a referenced Jar file to be "not found"!
Can anyone tell me what is causing this UnavailableException and (more importantly) how I can prevent it from being thrown?
Someone please advise...
ServletHolder holder = ctx.addServlet(
"org.glassfish.jersey.servlet.ServletContainer.class", "/*");
You are using a String for the class name. When doing this, you don't use the .class suffix. That at only when you want to get the actual Class object. You have two options
Remove the .class from the String
ServletHolder holder = ctx.addServlet(
"org.glassfish.jersey.servlet.ServletContainer", "/*");
Remove the "" (double quotes) and just use the Class object1.
ServletHolder holder = ctx.addServlet(
org.glassfish.jersey.servlet.ServletContainer.class, "/*");
1 - See addServlet(Class, String)

Invalid Cursor State - No current row

I am very new to Java, and I am trying to complete one simple task of reading data from a table and passing it to form variables. However, I am getting this frustrating exception:
Invalid cursor state - no current row;
I tried everything but still no luck. I looked into Google but I'm not getting an answer that will fix this issue. Hope you guys can help.
public class MainFrame extends javax.swing.JFrame {
Connection con;
Statement stm;
ResultSet rs;
/**
* Creates new form MainFrame
*/
public MainFrame() {
initComponents();
DoConnect();
}
public void DoConnect() {
try{
String host = "jdbc:derby:studentinfo;create=true";
String uname = "sudeep";
String upass = "sunny";
con = DriverManager.getConnection(host, uname, upass);
stm = con.createStatement();
String sql = "SELECT * FROM APP.STUDENTID";
rs = stm.executeQuery(sql);
System.out.println("Query executed");
if(rs.next())
{
// This is where the exception is occurring!!!!!!
String id = rs.getString("STUDENTID");
String first = rs.getString("FIRST_NAME");
String second = rs.getString("SECOND_NAME");
ID.setText(id);
fnam1.setText(first);
lnam.setText(second);
System.out.println(id);
}
}
catch ( SQLException err) {
JOptionPane.showMessageDialog(MainFrame.this, err.getMessage());
System.out.println("ERROR GETCONNECTION");}
}
}
Here is the trace generated by the program
run:
[EL Info]: 2013-09-03 21:14:15.732--ServerSession(552364977)--EclipseLink, version: Eclipse Persistence Services - 2.5.0.v20130507-3faac2b
[EL Info]: connection: 2013-09-03 21:14:16.299--ServerSession(552364977)--file:/Users/vasundhra_sudeep/NetBeansProjects/StudentForm/build/classes/_jdbc:derby:studentinfo;create=truePU login successful
java.sql.SQLException: Invalid cursor state - no current row.
Invalid cursor state - no current row.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.newSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.checkOnRow(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.getColumnType(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.getString(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedResultSet.getString(Unknown Source)
at studentform.MainFrame.DoConnect(MainFrame.java:57)
at studentform.MainFrame.(MainFrame.java:35)
at studentform.MainFrame$7.run(MainFrame.java:579)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:715)
at java.awt.EventQueue.access$400(EventQueue.java:82)
at java.awt.EventQueue$2.run(EventQueue.java:676)
at java.awt.EventQueue$2.run(EventQueue.java:674)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:86)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:685)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Caused by: java.sql.SQLException: Invalid cursor state - no current row.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 27 more
BUILD SUCCESSFUL (total time: 7 seconds)
It's likely that APP.STUDENTID has no data in it. Insert at least one record (aka row) of data into that table, run your program again, and see if the same exception occurs.

Business Objects Enterprise reporting using SDK client gives exception

We have a client that is using the SDK for invoking reports on the Business Objects Embedded Report Server. We can login, but when calling the openDocument method, something goes wrong.
code:
//LOGON
IEnterpriseSession session = sessionMgr.logon(username, password, clusterNode, authType);
IInfoStore infoStore = (IInfoStore)session.getService("InfoStore");
//GET REPORT OBJECT
String queryForFolder = "Select SI_ID, SI_NAME From CI_INFOOBJECTS Where SI_NAME = '" + folderName + "'";
IInfoObjects queryForFolderResult = infoStore.query(queryForFolder);
if (queryForFolderResult.isEmpty())
{
throw new Exception("No Folder Found");
}
//report folder found
IInfoObject reportFolder = (IInfoObject)queryForFolderResult.get(0);
String queryForFile = "Select SI_ID, SI_NAME From CI_INFOOBJECTS Where SI_NAME = '" + reportFile + "'" + " and SI_PARENTID = " + reportFolder ;
IReportAppFactory reportAppFactory = (IReportAppFactory)session.getService("RASReportFactory");
IInfoObjects queryForFileResult = infoStore.query(queryForFile);
if (queryForFileResult.isEmpty())
{
throw new Exception("Report file not found");
}
//report found
IReport report = (IReport)queryForFileResult.get(0);
//OPEN REPORT
clientDoc = reportAppFactory.openDocument(report, 0, locale); /*row 58 in exception*/
exception:
com.crystaldecisions.sdk.occa.report.lib.ReportSDKServerException: Unable to connect to the server: . - Server not found or server may be down---- Error code:-2147217387 Error code name:connectServer
at com.crystaldecisions.sdk.occa.managedreports.ras.internal.RASReportAppFactory.a(Unknown Source)
at com.crystaldecisions.sdk.occa.managedreports.ras.internal.RASReportAppFactory.a(Unknown Source)
at com.crystaldecisions.sdk.occa.managedreports.ras.internal.RASReportAppFactory.a(Unknown Source)
at com.crystaldecisions.sdk.occa.managedreports.ras.internal.RASReportAppFactory.openDocument(Unknown Source)
at com.reportclient.MyReportClient.getReportFromInfoStore(MyReportClient.java:58)
... 28 more
Caused by: com.crystaldecisions.sdk.occa.report.lib.ReportSDKServerException: Unable to connect to the server: . - Server not found or server may be down---- Error code:-2147217387 Error code name:connectServer
at com.crystaldecisions.sdk.occa.report.lib.ReportSDKServerException.throwReportSDKServerException(Unknown Source)
at com.crystaldecisions.sdk.occa.managedreports.ras.internal.CECORBACommunicationAdapter.connect(Unknown Source)
... 32 more
Caused by: com.crystaldecisions.enterprise.ocaframework.OCAFrameworkException$NotFoundInDirectory: Server not found or server may be down
at com.crystaldecisions.enterprise.ocaframework.j.find(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.AbstractServerHandler.buildServerInfo(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.AbstractServerHandler.buildClusterInfo(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.aa.for(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.for(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.o.a(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.p.a(Unknown Source)
at com.crystaldecisions.enterprise.ocaframework.ServiceMgr.getManagedService(Unknown Source)
... 33 more
The communication obviously works when logging in. Please let me know if you got any ideas or know where I can go and look for the answer. :)
Regards,
Karl
After some further research, this is what I found out. The first error was cased by usage of an earlier version of the BO SDK. The second error, "CORBA communication failure: reason[error number WSAETIMEDOUT]" occurs when the iiop-port is not opened. I solved this by setting the SDK listener port (described in the document http://www.sdn.sap.com/irj/boc/go/portal/prtroot/docs/library/uuid/0047e5f4-3140-2b10-1bae-de175e4c741c?QuickLink=index&overridelayout=true) and triple check that the correct firewall opening was made.
I'm going to guess that the ReportEngine that you are using, your variable reportAppFactory, is not correct for the type of document you are attempting to open.
The other possibility is that this is a DeskI report and it is looking for the Connection Server component to be able to open the document.
I will attempt to help, if you could provide some more details.