Neo4j cannot accept big http rest requests - rest

Question

I'm trying to send one very big http post request to stand alone neo4j server (neo4j-community-1.9.M01).
Bellow is the sample code that I'm using.
The input file: ideas/src/test/test-rest.txt contains one very big json string (more than (250k).
The problem is that I cannot control a lot of settings of the web server launched by the standalone version of neo4j. Probably there is some limit on the web server that prevent me to send big post requests. Could someone help me to find out this setting.
Alternative : could some one tell me how to replace the default webserver used by neo4j for example with tomcat.
The error that I receive on the client is:
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at org.apache.commons.httpclient.WireLogOutputStream.write(WireLogOutputStream.java:86)
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor$3.writeRequest(DefaultApacheHttpMethodExecutor.java:186)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at com.sun.jersey.client.apache.DefaultApacheHttpMethodExecutor.executeMethod(DefaultApacheHttpMethodExecutor.java:210)
Here are some logs from the web client:
11-11#08:28:01 INFO (HttpMethodDirector.java:445) - Retrying request
11-11#08:28:01 DEBUG (HttpConnection.java:692) - Open connection to localhost:7474
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "POST /db/data/batch HTTP/1.1[\r][\n]"
11-11#08:28:01 DEBUG (HttpMethodBase.java:1352) - Adding Host request header
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Accept: application/json; stream=true[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "X-Stream: true[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Content-Type: application/json[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Content-Length: 246292[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "Host: localhost:7474[\r][\n]"
11-11#08:28:01 DEBUG ( Wire.java:70) - >> "[\r][\n]"
11-11#08:31:22 DEBUG (HttpMethodDirector.java:404) - Closing the connection.
11-11#08:31:22 DEBUG (HttpMethodDirector.java:434) - Method retry handler returned false. Automatic recovery will not be attempted
11-11#08:31:22 DEBUG (HttpConnection.java:1178) - Releasing connection back to connection manager.
Here is the client code:
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.client.apache.ApacheHttpClient;
import org.apache.commons.io.FileUtils;
import org.neo4j.rest.graphdb.RequestResult;
import org.neo4j.rest.graphdb.UserAgent;
import javax.ws.rs.core.MediaType;
import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.Map;
import static javax.ws.rs.core.MediaType.APPLICATION_JSON_TYPE;
public class Main {
private static final UserAgent userAgent = new UserAgent();
public static final MediaType STREAMING_JSON_TYPE = new MediaType(APPLICATION_JSON_TYPE.getType(),APPLICATION_JSON_TYPE.getSubtype(), stringMap("stream","true"));
public static void main(String[] args) throws URISyntaxException, IOException {
String data = FileUtils.readFileToString(new File("ideas/src/test/test-rest.txt"));
Client client = createClient();
WebResource resource = client.resource(new URI("http://localhost:7474/db/data/batch"));
WebResource.Builder builder = resource.accept(STREAMING_JSON_TYPE).header("X-Stream", "true");
builder.entity( toInputStream(data), APPLICATION_JSON_TYPE );
System.out.println(RequestResult.extractFrom(builder.post(ClientResponse.class)));
}
private static InputStream toInputStream(String data) throws IOException {
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(4024 * 1024);
outputStream.write(data.getBytes());
byte[] buf = outputStream.toByteArray();
ByteArrayInputStream inputStream = new ByteArrayInputStream(buf);
return inputStream;
}
protected static Client createClient1() {
Client client = Client.create();
client.setConnectTimeout(800);
client.setReadTimeout(800);
client.setChunkedEncodingSize(80 * 1024);
userAgent.install(client);
return client;
}
protected static Client createClient() {
Client client = ApacheHttpClient.create();
return client;
}
private static Map<String, String> stringMap(String stream, String aTrue) {
HashMap<String, String> result = new HashMap<String, String>();
result.put(stream, aTrue);
return result;
}
}
Thanks in advance.

Answer 1

I found the reason of the problem.
I was running under Virtual BOX 4.2.0 with Windows XP guest machine and windows 7 host machine machine.
After putting the server on another machine, the problem disappeared.
I investigated with wireshark, before the failure I saw a lot of tcp packages that were not acknowledged and therefore retransmitted.
My question is still opened:how to replace the default webserver used by neo4j for example with tomcat?