The below code returns an exception even though the provided REST endpoint is up and valid. It is processed correctly as the DEBUG prompt shows the link of the GET request. It fails at the http.request line regardless of the REST endpoint provided. This is being executed by hitting Run 'MyScript' on intelliJ 14.1 .. Is there something missing to this piece of code? Source: github.com/jgritman/httpbuilder/wiki/GET-Examples
11:42:30.897 [main] DEBUG groovyx.net.http.HTTPBuilder - GET http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Calvin+and+Hobbes
Caught: java.net.UnknownHostException: ajax.googleapis.com
MyScript.groovy
import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.RESTClient
import groovyx.net.http.Method
def http = new HTTPBuilder()
http.request( 'http://ajax.googleapis.com', Method.GET, ContentType.JSON ) { req ->
uri.path = '/ajax/services/search/web'
uri.query = [ v:'1.0', q: 'Calvin and Hobbes' ]
headers.'User-Agent' = "Mozilla/5.0 Firefox/3.0.4"
headers.Accept = 'application/json'
response.success = { resp, reader ->
assert resp.statusLine.statusCode == 200
println "Got response: ${resp.statusLine}"
println "Content-Type: ${resp.headers.'Content-Type'}"
println reader.text
}
response.'404' = {
println 'Not found'
}
}
Run MyScript.groovy exceptions
11:52:32.133 [main] DEBUG groovyx.net.http.HTTPBuilder - GET http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Calvin+and+Hobbes
11:52:32.395 [main] DEBUG o.a.h.i.c.BasicClientConnectionManager - Get connection for route {}->http://ajax.googleapis.com
11:52:32.413 [main] DEBUG o.a.h.i.conn.DefaultClientConnection - Connection org.apache.http.impl.conn.DefaultClientConnection#2b175c00 closed
11:52:32.413 [main] DEBUG o.a.h.i.conn.DefaultClientConnection - Connection org.apache.http.impl.conn.DefaultClientConnection#2b175c00 shut down
11:52:32.414 [main] DEBUG o.a.h.i.c.BasicClientConnectionManager - Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl#3eb81efb
11:52:32.414 [main] DEBUG o.a.h.i.c.BasicClientConnectionManager - Connection can be kept alive for 9223372036854775807 MILLISECONDS
Caught: java.net.UnknownHostException: ajax.googleapis.com
java.net.UnknownHostException: ajax.googleapis.com
Related
In my project, I can compile and run without problems, my project is a project in akka-http and I provide services of type json, but when calling the service this error appears:
> run
[info] Running net.claritysales.api.Main
[DEBUG] [11/02/2017 11:08:32.360] [run-main-0] [EventStream(akka://default)] logger log1-Logging$DefaultLogger started
[DEBUG] [11/02/2017 11:08:32.361] [run-main-0] [EventStream(akka://default)] Default Loggers started
[DEBUG] [11/02/2017 11:08:33.253] [run-main-0] [AkkaSSLConfig(akka://default)] Initializing AkkaSSLConfig extension...
[DEBUG] [11/02/2017 11:08:33.258] [run-main-0] [AkkaSSLConfig(akka://default)] buildHostnameVerifier: created hostname verifier: com.typesafe.sslconfig.ssl.DefaultHostnameVerifier#5160a3bd
[DEBUG] [11/02/2017 11:08:34.687] [default-akka.actor.default-dispatcher-5] [akka://default/system/IO-TCP/selectors/$a/0] Successfully bound to /127.0.0.1:9000
[DEBUG] [11/02/2017 11:08:40.607] [default-akka.actor.default-dispatcher-5] [akka://default/system/IO-TCP/selectors/$a/0] New connection accepted
[ERROR] [11/02/2017 11:08:41.795] [default-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(default)] Error during processing of request: 'No configuration setting found for key 'c
ors''. Completing with 500 Internal Server Error response.
[ERROR] [11/02/2017 11:08:43.467] [default-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(default)] Error during processing of request: 'No configuration setting found for key 'c
ors''. Completing with 500 Internal Server Error response.
[DEBUG] [11/02/2017 11:09:49.232] [default-akka.actor.default-dispatcher-5] [akka://default/user/StreamSupervisor-0/flow-1-0-unknown-operation] Aborting tcp connection to /127.0.0.1:64741 beca
use of upstream failure: HTTP idle-timeout encountered, no bytes passed in the last 1 minute. This is configurable by akka.http.[server|client].idle-timeout.
since insomnia this appears:
My application.conf is:
akka {
loglevel = DEBUG
}
http {
interface = "localhost"
interface = ${?HTTP_INTERFACE}
port = 9000
port = ${?HTTP_PORT}
}
database = {
cs {
profile = "slick.jdbc.PostgresProfile$"
db {
url = "jdbc:postgresql://localhost:5432/cs2_company"
url = ${?PSQL_URL}
user = "postgres"
user = ${?PSQL_USER}
password = "qwerty"
password = ${?PSQL_PASSWORD}
}
}
}
I read my configuration from a trait called Config, it has:
package net.cs.api.utils
import com.typesafe.config.ConfigFactory
trait Config {
private val config = ConfigFactory.load()
private val httpConfig = config.getConfig("http")
private val databaseConfig = config.getConfig("database.cs")
val httpHost = httpConfig.getString("interface")
val httpPort = httpConfig.getInt("port")
val jdbcUrl = databaseConfig.getString("db.url")
val dbUser = databaseConfig.getString("db.user")
val dbPassword = databaseConfig.getString("db.password")
}
additionally I have another application.conf file in the test resources, because I want to perform the tests with H2 and execute the application with Postgres
My application.conf for the test is:
akka {
loglevel = WARNING
}
database = {
cs {
profile = "slick.jdbc.H2Profile$"
db {
driver = "org.h2.Driver"
url = "jdbc:h2:mem:cs2_company;MODE=PostgreSQL;DB_CLOSE_DELAY=-1;IGNORECASE=TRUE;DATABASE_TO_UPPER=false"
url = ${?PSQL_TEST_URL}
user = "sales"
user = ${?PSQL_TEST_USER}
password = "sales"
password = ${?PSQL_TEST_PASSWORD}
}
}
}
What is the error? any help I would appreciate it. Thanks!
it seems that something was looking for a key called cors in the configuration and can not find it, in my configuration I added:
cors.allowed-origin = "*"
everything works!
I'm developing a system that consists of an arduino mkr1000 that I want to send data via wifi to a java server program running in my local network.
Everything works except the main part: data sent by the arduino is sometimes not received by the server...
I'm using the arduino Wifi101 library to connect to my wifi, get a WiFiClient and send data.
The following code is just a example to demonstrate the problem:
for (int i = 0; i < 3; ++i) {
Serial.println(F("Connecting to wifi"));
const auto status = WiFi.begin("...", "...");
if (status != WL_CONNECTED) {
Serial.print(F("Could not connect to WiFi: "));
switch (status) {
case WL_CONNECT_FAILED:
Serial.println(F("WL_CONNECT_FAILED"));
break;
case WL_DISCONNECTED:
Serial.println(F("WL_DISCONNECTED"));
break;
default:
Serial.print(F("Code "));
Serial.println(status, DEC);
break;
}
} else {
Serial.println(F("WiFi status: WL_CONNECTED"));
WiFiClient client;
if (client.connect("192.168.0.102", 1234)) {
delay(500);
client.print(F("Test "));
client.println(i, DEC);
client.flush();
Serial.println(F("Data written"));
delay(5000);
client.stop();
} else {
Serial.println(F("Could not connect"));
}
WiFi.end();
}
delay(2000);
}
The java server is based on Netty but the same thing with manually creating and reading from a Socket yields the same result.
The testing code is pretty standard with only a simple output (note: in Kotlin):
val bossGroup = NioEventLoopGroup(1)
val workerGroup = NioEventLoopGroup(6)
val serverFuture = ServerBootstrap().run {
group(bossGroup, workerGroup)
channel(NioServerSocketChannel::class.java)
childHandler(object : ChannelInitializer<NioSocketChannel>() {
override fun initChannel(ch: NioSocketChannel) {
ch.pipeline()
.addLast(LineBasedFrameDecoder(Int.MAX_VALUE))
.addLast(StringDecoder())
.addLast(object : ChannelInboundHandlerAdapter() {
override fun channelRead(ctx: ChannelHandlerContext, msg: Any) {
println("msg = $msg")
ctx.close()
}
})
}
})
bind(port).sync()
}
The arduino tells that everything is OK (i.e. writing Data written to the serial console for each iteration) but the server sometimes skips individual messages.
Adding the LoggingHandler from Netty tells me in these cases:
11:28:48.576 [nioEventLoopGroup-3-1] WARN i.n.handler.logging.LoggingHandler - [id: 0x9991c251, L:/192.168.0.20:1234 - R:/192.168.0.105:63845] REGISTERED
11:28:48.577 [nioEventLoopGroup-3-1] WARN i.n.handler.logging.LoggingHandler - [id: 0x9991c251, L:/192.168.0.20:1234 - R:/192.168.0.105:63845] ACTIVE
In the cases where the message is received it tells me:
11:30:01.392 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 - R:/192.168.0.105:59927] REGISTERED
11:30:01.394 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 - R:/192.168.0.105:59927] ACTIVE
11:30:01.439 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 - R:/192.168.0.105:59927] READ: 8B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 54 65 73 74 20 32 0d 0a |Test 2.. |
+--------+-------------------------------------------------+----------------+
11:30:01.449 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 - R:/192.168.0.105:59927] CLOSE
11:30:01.451 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 ! R:/192.168.0.105:59927] READ COMPLETE
11:30:01.453 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 ! R:/192.168.0.105:59927] INACTIVE
11:30:01.464 [nioEventLoopGroup-3-6] WARN i.n.handler.logging.LoggingHandler - [id: 0xd51b7bc3, L:/192.168.0.20:1234 ! R:/192.168.0.105:59927] UNREGISTERED
With my understanding this means that the TCP packets are indeed received but in the faulty cases the IO thread from Netty is waiting to read the TCP data but does never continue...
The same problem exists when trying with a rudimentary python server (just waiting for a connection and printing the received data).
I confirmed the data is sent by using tcpflow on Arch Linux with the arguments -i any -C -g port 1234.
I even tried the server on a Windows 7 machine with the same results (TCP packets confirmed with SmartSniff).
Strangely using a java server to send the data always and reproducibly is received...
Does anybody have any idea to solve the problem or at least how to diagnose?
PS: Maybe it is important to note that with tcpflow (i.e. on linux) I could watch the TCP packets being resent to the server.
Does this mean the server is receiving the packets but not sending an ACK?
SmartSniff didn't show the same behavior (but maybe I used wrong options to display the resent packets).
In the meantime I send messages to acknowledge receiving another message. If the acknowledgement is not received the message is sent again.
For anyone with the same problem:
While testing something different I updated the wifi firmware of the board to the latest version 19.5.2. Since then I haven't noticed any lost data. Maybe this was the problem.
See Check WiFi101 Firmware Version and Firmware and certificates Updater.
Note: I couldn't get the sketches to run with the Arduino IDE but with PlatformIO.
I'm using IdleManager, in Scala, to listen to a Gmail folder.
I already have this props.setProperty("mail.imaps.usesocketchannels", "true")
The main part of my code is like this:
folder.addMessageCountListener(new MessageCountAdapter() {
override def messagesAdded(ev: MessageCountEvent) {
Logger.info("Got " + ev.getMessages.length + " new messages")
idleManager.watch(folder)
}
})
// timeLength = 20 minutes
system.scheduler.schedule(initialDelay = timeLength, interval = timeLength) {
try {
folder.asInstanceOf[IMAPFolder].doCommand(new IMAPFolder.ProtocolCommand() {
def doCommand(p: IMAPProtocol) = {
p.simpleCommand("NOOP", null)
null
}
})
Logger.debug("Continue after sending NOOP")
idleManager.watch(folder)
} catch {
case e: Exception => Logger.error(s"MailHelper: ${e.getMessage}")
}
}
idleManager.watch(folder)
You can see that I let the idleManager continue watching the folder after I get new messages and after I send a NOOP command. A scheduler is created to periodically (currently, once in 20 minutes) send a NOOP command to the server, to keep the connection. My program worked fine, but just for a while.
14 hours after the first call idleManager.watch(folder) and about 12.5 hours after the last email received, I still got the log Continue after sending NOOP, but right after that is an error log MailHelper: Folder is not using SocketChannels.
Could you please help me with this?
Edited:
Thanks #BillShannon for your quick reply. I have updated from v1.5.2 to v1.5.6 and turned on the debug output. I'm sure the Properties object and the "store" instance (created from a Session with the "imaps" protocol) are unchanged.
The error has appeared again. After a call to idleManager.watch(folder), here is the log ([folder] is the imaps protocol string for my folder)
DEBUG IMAP: IdleManager watching [folder]
A385 IDLE
+ idling
DEBUG IMAP: startIdle: set to IDLE
DEBUG IMAP: startIdle: return true
DEBUG IMAP: IdleManager.watch startIdle succeeded for [folder]
DEBUG IMAP: IdleManager selected 0 channels
DEBUG IMAP: IdleManager adding [folder] to selector
DEBUG IMAP: IdleManager waiting...
DEBUG IMAP: IdleManager selected 1 channels
DEBUG IMAP: IdleManager selected folder: [folder]
DEBUG IMAP: handleIdle: set to RUNNING
DEBUG IMAP: IdleManager got exception for folder: [folder], THROW:
javax.mail.FolderClosedException: * BYE JavaMail Exception: java.io.IOException: Connection dropped by server?
at com.sun.mail.imap.IMAPFolder.handleIdle(IMAPFolder.java:3199)
at com.sun.mail.imap.IdleManager.processKeys(IdleManager.java:370)
at com.sun.mail.imap.IdleManager.select(IdleManager.java:281)
at com.sun.mail.imap.IdleManager.access$200(IdleManager.java:137)
at com.sun.mail.imap.IdleManager$1.run(IdleManager.java:164)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
DEBUG IMAP: IdleManager waiting...
20 minutes later, the program sent another "NOOP", the status returned was "OK". And the program called idleManager.watch(folder) once again, here the error log reappeared Folder is not using SocketChannels.
Do you need anything else? Is this an issue with the library?
Sorry, this is kind of long, because I need to include various files.
Problem
I am not sure what is going on in my setup [using spray 1.3.3]. I am trying to do file uploads using chunked requests which seemed to work as expected once or twice, but for some reason most of the time the actor never receives the chunks after the initial registration of the chunk handler is finished. It just disappears into oblivion and my request and logs just keep waiting the whole time. The 2 times out of 50 it did work were when I ran through the debugger. However even with the debugger it doesn't work mostly.
Based on various examples and discussions related to DemoService and FileUploadHandler, I am using spray-can to check the received HttpMessage for chunks, and at that point spawn off a separate route. I use curl with chunked encoding to test my output.
Please help! I have spent too many hours trying to get chunked requests mixed with routes working for my use case.
Code
Here is the code I have:
TestApp.scala
object TestApp extends App with GlobalConfig {
implicit val system = ActorSystem("TestApp")
implicit val ec = system.dispatcher
val healthActor = system.actorOf(Props[HealthStateActor])
val routes = new HealthcheckController(healthActor).route ~
new ResourceController().route
val requestRouter = system.actorOf(Props(new HttpRequestCustomHandler(routes)))
IO(Http) ! Http.Bind(requestRouter, "0.0.0.0", HttpBindPort)
}
FileUploadActor.scala
class FileUploadActor(client: ActorRef, requestMetadata: RequestMetadata, request: HttpRequest, ctx: RequestContext)
extends Actor with ActorLogging with GlobalConfig {
import request._
var bytesWritten = 0L
var bytes: Array[Byte] = "".getBytes
// client ! CommandWrapper(SetRequestTimeout(Duration.Inf)) // cancel timeout
def receive = {
case c: MessageChunk =>
log.info(s"Got ${c.data.length} bytes of chunked request $method $uri")
bytes ++= c.data.toByteArray
bytesWritten += c.data.length
case e: ChunkedMessageEnd =>
log.info(s"Got end of chunked request $method $uri. Writing $bytesWritten bytes for upload: $requestMetadata")
Try(saveFile(requestMetadata)) match {
case Success(_) => ctx.complete(HttpResponse(StatusCodes.Created, entity = "success"))
case Failure(f) => f.printStackTrace(); ctx.complete(HttpResponse(StatusCodes.InternalServerError, entity = "failure"))
}
// client ! CommandWrapper(SetRequestTimeout(UploadRequestTimeout.seconds)) // reset timeout to original value
context.stop(self)
}
}
FileUploadService.scala
The RegisterChunkHandler message is the last step where I see the debugger stop at break points, and where the logs go quiet. When it does work I can see MessageChunk messages being received by FileUploadActor.
trait FileUploadService extends Directives {
this: Actor with ActorLogging with GlobalConfig =>
def chunkedRoute() = {
path(resourceAPI / "upload" / "resource" / Segment) { resourceId =>
put {
detach() {
ctx => {
val request = ctx.request
val client = sender()
val handler = context.actorOf(Props(new FileUploadActor(client,
RequestMetadata(....),
request, ctx)))
sender ! RegisterChunkHandler(handler)
}
}
}
}
}
}
HttpRequestCustomHandler.scala
class HttpRequestCustomHandler(routes: Route, resourceProviderRef: ResourceProvider)
extends HttpServiceActor
with FileUploadService
with ActorLogging
with GlobalConfig {
val normal = routes
val chunked = chunkedRoute()
def resourceProvider = resourceProviderRef
val customReceive: Receive = {
// clients get connected to self (singleton handler)
case _: Http.Connected => sender ! Http.Register(self)
case r: HttpRequest =>
normal(RequestContext(r, sender(), r.uri.path).withDefaultSender(sender()))
case s#ChunkedRequestStart(HttpRequest(PUT, path, _, _, _)) =>
chunked(RequestContext(s.request, sender(), s.request.uri.path).withDefaultSender(sender()))
}
override def receive: Receive = customReceive
}
HttpRequestHandler.scala
abstract class HttpRequestHandler(routes: Route) extends HttpServiceActor{
override def receive: Receive = runRoute(routes)
}
application.conf:
spray.can.server {
request-timeout = 20 s
pipelining-limit = disabled
reaping-cycle = infinite
stats-support = off
request-chunk-aggregation-limit = 0
parsing.max-content-length = 100000000
parsing.incoming-auto-chunking-threshold-size = 15000000
chunkless-streaming = on
verbose-error-messages = on
verbose-error-logging = on
}
curl Success:
curl -vvv -X PUT -H "Content-Type: multipart/form-data" -d
'#/Users/abc/Documents/test.json'
'http://localhost:8180/upload/resource/02521081'
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8100 (#0)
> PUT /upload/resource/02521081 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8100
> Accept: */*
> Content-Type: multipart/form-data
> Content-Length: 82129103
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 201 Created
* Server spray-can/1.3.3 is not blacklisted
< Server: spray-can/1.3.3
< Date: Mon, 17 Aug 2015 07:45:58 GMT
< Content-Type: text/plain; charset=UTF-8
< Content-Length: 7
<
* Connection #0 to host localhost left intact
success
Failure with same curl (waits forever):
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8100 (#0)
> PUT /upload/resource/02521081 HTTP/1.1
> User-Agent: curl/7.37.1
> Host: localhost:8100
> Accept: */*
> Transfer-Encoding: chunked
> Content-Type: multipart/form-data
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
^C
Failure (waits forever):
resource 2015-08-17 01:33:09.374 [Resource] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
resource 2015-08-17 01:33:09.396 08:33:09.382UTC [Resource] DEBUG akka.event.EventStream main EventStream(akka://resource) - logger log1-Slf4jLogger started
resource 2015-08-17 01:33:09.404 08:33:09.383UTC [Resource] DEBUG akka.event.EventStream main EventStream(akka://resource) - Default Loggers started
resource 2015-08-17 01:33:10.160 08:33:10.159UTC [Resource] INFO spray.can.server.HttpListener Resource-akka.actor.default-dispatcher-4 akka://resource/user/IO-HTTP/listener-0 - Bound to /0.0.0.0:8100
Success (logs edited for clarity):
resource 2015-08-17 00:42:12.283 [Resource] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
resource 2015-08-17 00:42:12.295 07:42:12.290UTC [Resource] DEBUG akka.event.EventStream main EventStream(akka://resource) - logger log1-Slf4jLogger started
resource 2015-08-17 00:42:12.308 07:42:12.291UTC [Resource] DEBUG akka.event.EventStream main EventStream(akka://resource) - Default Loggers started
resource 2015-08-17 00:42:13.007 07:42:13.005UTC [Resource] INFO spray.can.server.HttpListener Resource-akka.actor.default-dispatcher-4 akka://resource/user/IO-HTTP/listener-0 - Bound to /0.0.0.0:8100
resource 2015-08-17 00:43:47.615 07:43:47.529UTC [Resource] DEBUG c.l.resource.actor.FileUploadActor Resource-akka.actor.default-dispatcher-7 akka://resource/user/$b/$b - Got 131072 bytes of chunked request PUT http://localhost:8100/resourcesvc/0.2/api/upload/resource/02521081-20e5-483a-929f-712a9e11d117/content/5adfb5-561d-4577-b6ad-c6f42eef98
resource 2015-08-17 00:43:49.220 07:43:49.204UTC [Resource] DEBUG c.l.resource.actor.FileUploadActor Resource-akka.actor.default-dispatcher-7 akka://resource/user/$b/$b - Got 131072 bytes of chunked request PUT http://localhost:8100/resourcesvc/0.2/api/upload/resource/02521081-20e5-483a-929f-712a9e11d117/content/5adfb5-561d-4577-b6ad-c6f42eef98
.
.
.
resource 2015-08-17 00:44:05.605 07:44:05.605UTC [Resource] DEBUG c.l.resource.actor.FileUploadActor Resource-akka.actor.default-dispatcher-7 akka://resource/user/$b/$b - Got 45263 bytes of chunked request PUT http://localhost:8100/resourcesvc/0.2/api/upload/resource/02521081-20e5-483a-929f-712a9e11d117/content/5adfb5-561d-4577-b6ad-c6f42eef98
resource 2015-08-17 00:44:05.633 07:44:05.633UTC [Resource] INFO c.l.resource.actor.FileUploadActor Resource-akka.actor.default-dispatcher-7 akka://resource/user/$b/$b - Got end of chunked request PUT http://localhost:8100/resourcesvc/0.2/api/upload/resource/02521081-20e5-483a-929f-712a9e11d117/content/5adfb5-561d-4577-b6ad-c6f42eef98. Writing 82129103 bytes for upload: RequestMetadata(...,multipart/form-data)
resource 2015-08-17 00:44:05.634 07:44:05.633UTC [Resource] DEBUG c.l.resource.actor.FileUploadActor Resource-akka.actor.default-dispatcher-7 akka://resource/user/$b/$b - actor is akka://resource/user/$b/$b, sender is Actor[akka://resource/temp/$a], client is Actor[akka://resource/temp/$a]
resource 2015-08-17 00:45:58.445 [Resource] DEBUG com.abc.resource.io.FileClient$ - UploadResult#109a69fb
resource 2015-08-17 00:45:58.445 [Resource] DEBUG com.abc.resource.io.FileClient$ - upload is done: true
Please let me know if you see anything weird. What is usually the reason an actor would vanish like this? Thanks in advance for your help!
UPDATE:
I added further logging and see that the from and to actors are apparently both turning into deadLetters even though that is clearly not the case in the log line above that. This is when message 'RegisterChunkHandler' is sent to sender in FileUploadService.scala.
sender ! RegisterChunkHandler(handler)
Related log:
resource 2015 - 0 8 - 17 21: 14: 32.173 20: 14: 32.173 UTC[Resource] DEBUG c.l.a.io.HttpRequestCustomHandler Resource - akka.actor.default - dispatcher - 3 akka :// Resource / user / httpcustomactor - sender is Actor[akka :// Resource / temp / $a]
resource 2015 - 0 8 - 17 21: 14: 32.175 20: 14: 32.175 UTC[Resource] INFO akka.actor.DeadLetterActorRef A4Resource-akka.actor.default-dispatcher-6 akka://A4Resource/deadLetters - Message [spray.can.Http$RegisterChunkHandler] from Actor[akka://A4Resource/user/httpcustomactor#-1286373908] to Actor[akka://A4Resource/deadLetters] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
resource 2015 - 0 8 - 17 21: 14: 32.176 20: 14: 32.176 UTC[Resource] DEBUG c.l.resource.actor.FileUploadActor Resource - akka.actor.default - dispatcher - 7 akka :// Resource / user / httpcustomactor / $a - pre - start
Any idea how this can be avoided?
I'm trying to send one very big http post request to stand alone neo4j server (neo4j-community-1.9.M01).
Bellow is the sample code that I'm using.
The input file: ideas/src/test/test-rest.txt contains one very big json string (more than (250k).
The problem is that I cannot control a lot of settings of the web server launched by the standalone version of neo4j. Probably there is some limit on the web server that prevent me to send big post requests. Could someone help me to find out this setting.
Alternative : could some one tell me how to replace the default webserver used by neo4j for example with tomcat.
The error that I receive on the client is:
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at org.apache.commons.httpclient.WireLogOutputStream.write(WireLogOutputStream.java:86)
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor$3.writeRequest(DefaultApacheHttpMethodExecutor.java:186)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:210)
Here are some logs from the web client:
11-11#08:28:01 INFO (HttpMethodDirector.java:445) - Retrying request
11-11#08:28:01 DEBUG (HttpConnection.java:692) - Open connection to localhost:7474
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "POST /db/data/batch HTTP/1.1[\r][\n]"
11-11#08:28:01 DEBUG (HttpMethodBase.java:1352) - Adding Host request header
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Accept: application/json; stream=true[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "X-Stream: true[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Content-Type: application/json[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Content-Length: 246292[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Host: localhost:7474[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "[\r][\n]"
11-11#08:31:22 DEBUG (HttpMethodDirector.java:404) - Closing the connection.
11-11#08:31:22 DEBUG (HttpMethodDirector.java:434) - Method retry handler returned false. Automatic recovery will not be attempted
11-11#08:31:22 DEBUG (HttpConnection.java:1178) - Releasing connection back to connection manager.
Here is the client code:
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.client.apache.ApacheHttpClient;
import org.apache.commons.io.FileUtils;
import org.neo4j.rest.graphdb.RequestResult;
import org.neo4j.rest.graphdb.UserAgent;
import javax.ws.rs.core.MediaType;
import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.Map;
import static javax.ws.rs.core.MediaType.APPLICATION_JSON_TYPE;
public class Main {
private static final UserAgent userAgent = new UserAgent();
public static final MediaType STREAMING_JSON_TYPE = new MediaType(APPLICATION_JSON_TYPE.getType(),APPLICATION_JSON_TYPE.getSubtype(), stringMap("stream","true"));
public static void main(String[] args) throws URISyntaxException, IOException {
String data = FileUtils.readFileToString(new File("ideas/src/test/test-rest.txt"));
Client client = createClient();
WebResource resource = client.resource(new URI("http://localhost:7474/db/data/batch"));
WebResource.Builder builder = resource.accept(STREAMING_JSON_TYPE).header("X-Stream", "true");
builder.entity( toInputStream(data), APPLICATION_JSON_TYPE );
System.out.println(RequestResult.extractFrom(builder.post(ClientResponse.class)));
}
private static InputStream toInputStream(String data) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(4024 * 1024);
outputStream.write(data.getBytes());
byte[] buf = outputStream.toByteArray();
ByteArrayInputStream inputStream = new ByteArrayInputStream(buf);
return inputStream;
}
protected static Client createClient1() {
Client client = Client.create();
client.setConnectTimeout(800);
client.setReadTimeout(800);
client.setChunkedEncodingSize(80 * 1024);
userAgent.install(client);
return client;
}
protected static Client createClient() {
Client client = ApacheHttpClient.create();
return client;
}
private static Map<String, String> stringMap(String stream, String aTrue) {
HashMap<String, String> result = new HashMap<String, String>();
result.put(stream, aTrue);
return result;
}
}
Thanks in advance.
I found the reason of the problem.
I was running under Virtual BOX 4.2.0 with Windows XP guest machine and windows 7 host machine machine.
After putting the server on another machine, the problem disappeared.
I investigated with wireshark, before the failure I saw a lot of tcp packages that were not acknowledged and therefore retransmitted.
My question is still opened:how to replace the default webserver used by neo4j for example with tomcat?