I am using Netty to setup a simple http/2 server. I am using this example as http/2 server.
To test this server, I am using netty example client.
My client code where I am sending the request to server:
Complete code : http://netty.io/5.0/xref/io/netty/example/http2/client/package-summary.html
HttpResponseHandler responseHandler = initializer.responseHandler();
int streamId = 3;
HttpScheme scheme = SSL ? HttpScheme.HTTPS : HttpScheme.HTTP;
AsciiString hostName = new AsciiString(HOST + ':' + PORT);
System.out.println("Sending request(s)...");
if (URL != null) {
System.out.println("with url");
// Create a simple GET request.
FullHttpRequest request = new DefaultFullHttpRequest(HTTP_1_1, GET, URL);
request.headers().add(HttpHeaderNames.HOST, hostName);
request.headers().add(HttpConversionUtil.ExtensionHeaderNames.SCHEME.text(), scheme.name());
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP);
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.DEFLATE);
responseHandler.put(streamId, channel.writeAndFlush(request), channel.newPromise());
streamId += 2;
}
Above code works fine with stream id 3,5 and so on.
But when i change the stream id to any other number like 4,6,8 etc, above code doesn't work. From server I still get the messages for stream id 3,5,7 etc. I am unable to find the logic for these stream id inside example server
Stream numbering is mandated by the HTTP/2 specification.
Related
I have a scenario where I am trying to send multiple messages in one and trying to stream the one message from the gRPC server to the gRPC client side.
My proto files on server side look like this:
Service Greeter{
rpc AccumulateEvents (EventRequest) returns (stream EventsMessage);
}
message EventsMessage{
FirstEvent firstEvents =1;
SecondEvents secondEvent = 2;
}
message EventRequest{
//sending empty request
}
The service method is as follows:
public override async Task AccumulateEvents(EventRequest eventRequest, IServerStreamWriter < EventsMessage > responseStream, ServerCallContext context) {
IDisposable disposable4 = service.SubscribeToEvents(OnEvents);
service.execute();
await responseStream.WriteAsync(new EventsMessage {
FirstEvent = firstEvent, SecondEvents = secondEvents
});
}
When I am trying to fetch and parse the stream from the client side,i am getting null for secondEvent part of the message EventsMessage. Only firstEvents was returned from the server to the client. I tried debugging and could see secondEvent getting populated but then it became null when the streaming started from the server.
Also, secondEvent is a repeated field. I am not sure if that is the reason of it becoming null.
Please let me know what i might be missing here.
The vertx application runs in Docker containers, on two EC2 instances and is clustered.
Clustering is achieved with the hazelcast-aws plugin and the application is started like this:
docker run --name ... -p ... \
--network ... \
-v ... \
-d ... \
-c 'exec java \
-Dvertx.eventBus.options.setClustered=true \
-Dvertx.eventBus.options.setClusterPort=15701 \
-jar ... -conf ... \
-cluster'
Nothing cluster-related is set programmatically.
Client opens a socket on the first request and uses it for future similar requests.
Each request will:
initiate an async request with the server by publishing a message to the event bus
register a consumer on the event bus which will handle the result of the above,
and which is passed a reference to the socket connection where it should send the result to
Since vertx does round robin by default when clustered and there are two instances, this means any instance gets every other message (from 1., above) and makes the client, which connects to one instance only, receive exactly half of all expected responses.
I suppose this is because, even though the registered consumer has a reference to the socket object, it can't use it because it was created on a different node/webserver.
Would that be correct and is there a way to get 100% of messages to the client, connected to just one node, without introducing things like RabbitMQ?
Here's the SockJS handler code:
SockJSHandler sockJSHandler = SockJSHandler.create(vertx, new SockJSHandlerOptions());
sockJSHandler.socketHandler(socket -> {
SecurityService securityService = (SecurityService) ServiceFactory.getService(SecurityService.class);
if (securityService.socketHeadersSecurity(socket)) {
socket.handler(socketMessage -> {
try {
LOGGER.trace("socketMessage: " + socketMessage);
Socket socket = Json.decodeValue(socketMessage.toString(), Socket.class);
Report report = socket.getReport();
if (report != null) {
Account accountRequest = socket.getAccount();
Account accountDatabase = accountRequest == null ? null
: ((AccountService) ServiceFactory.getService(AccountService.class)).getById(accountRequest.getId());
Response result = securityService.socketReportSecurity(accountRequest, accountDatabase, report) ?
((ReportService) ServiceFactory.getService(ReportService.class)).createOrUpdateReport(report, accountDatabase)
: new Response(Response.unauthorized);
if (Response.success.equals(result.getResponse())) {
//register a consumer
String consumerName = "report.result." + Timestamp.from(ClockFactory.getClock().instant());
vertx.eventBus().consumer(consumerName, message -> {
Response executionResult;
if ("success".equals(message.body())) {
try {
Path csvFile = Paths.get(config.getString(Config.reportPath.getConfigName(), Config.reportPath.getDefaultValue())
+ "/" + ((Report) result.getPayload()).getId() + ".csv");
executionResult = new Response(new JsonObject().put("csv", new String(Files.readAllBytes(csvFile))));
} catch (IOException ioEx) {
executionResult = new Response(new Validator("Failed to read file.", ioEx.getMessage(), null, null));
LOGGER.error("Failed to read file.", ioEx);
}
} else {
executionResult = new Response(new Validator("Report execution failed", (String)message.body(), null, null));
}
//send second message to client
socket.write(Json.encode(executionResult));
vertx.eventBus().consumer(consumerName).unregister();
});
//order report execution
vertx.eventBus().send("report.request", new JsonObject()
.put("reportId", ((Report) result.getPayload()).getId())
.put("consumerName", consumerName));
}
//send first message to client
socket.write(Json.encode(result));
} else {
LOGGER.info("Insufficient data sent over socket: " + socketMessage.toString());
socket.end();
}
} catch (DecodeException dEx) {
LOGGER.error("Error decoding message.", dEx);
socket.end();
}
});
} else {
LOGGER.info("Illegal socket connection attempt from: " + socket.remoteAddress());
socket.end();
}
});
mainRouter.route("/websocket/*").handler(sockJSHandler);
Interestingly, when running two nodes clustered on localhost the client gets 100% of the results.
EDIT:
This was not a SockJS but a configuration issue.
Since vertx does round robin by default when clustered and there are
two instances, this means any instance gets every other message (from
1., above) and makes the client, which connects to one instance only, receive exactly half of all expected responses.
This assumption is only partially correct. Vert.x does round-robin, yes, but this means each instance will get half of the connections, not half of the messages.
Once connection is established, all its messages will arrive to a single instance.
So this:
Would that be correct and is there a way to get 100% of messages to
the client, connected to just one node, without introducing things
like RabbitMQ?
Already happens.
I'm working on a UWP application which should communicate via TCP/UDP to a remote device. My issue is that the UWP app successfully sends UDP messages to the remote device, but does not receive the replies.
Here follows the code extracted from the app (simplified):
async Task TestUdpIP()
{
// Writer to the DatagramSocket
DataWriter writer;
using (var udpClient = new DatagramSocket())
{
try
{
// UDP Socket binding
udpClient.MessageReceived += UdpClient_MessageReceived;
var controllerName = new Windows.Networking.HostName(controllerIpAddress.ToString());
await udpClient.BindEndpointAsync(controllerName, controllerIpPort.ToString());
var remoteHostName = new Windows.Networking.HostName(hostIpAddres.ToString());
await udpClient.ConnectAsync(remoteHostName, remoteHostPort.ToString());
// Create a message to send
string message = "Some message";
// Reset the counter of messages received back from the remote robot
messagesReceived = 0;
// Send the message
writer = new DataWriter(udpClient.OutputStream);
writer.WriteString(message);
await writer.StoreAsync();
// Wait for robot status messages
await Task.Delay(5000);
}
catch
{
}
}
}
void UdpClient_MessageReceived(DatagramSocket sender, DatagramSocketMessageReceivedEventArgs args)
{
// Just increment the number of messages received
messagesReceived++;
}
However the UdpClient_MessageReceived handler does not fire. I'm sure that UDP messages are correctly sent from the UWP app and that the remote device replies back as shown in the following screenshot from Wireshark (the test has been taken on the same PC where the UWP app is running)
.
(IP Address, Port) details are shown in the following list to better explain the picture above
UWP application: (192.168.1.108, 19000) to send and receive.
Remote device: (192.168.1.152, 15999) to receive and (192.168.1.152, 54697) to send
Note: this is a similar question, where the answer says that for whatever reason the DatagramSocket should fire some messages before being able to receive. In my example a message is sent out however the message received handler does not fire anyway.
Note: The UWP app has been granted the internet (client) and the internet (client and server) capabilities
Note: I've tried also this (more readable) approach to bind the inbound/outbound datagram socket on the UWP app obtaining the same results:
// UDP Socket binding
var controllerName = new HostName(controllerIpAddress.ToString());
var remoteHostName = new HostName(hostIpAddres.ToString());
EndpointPair endpointpar = new EndpointPair(controllerName,
controllerIpPort.ToString(),
remoteHostName,
remoteHostPort.ToString());
udpClient.MessageReceived += UdpClient_MessageReceived;
await udpClient.ConnectAsync(endpointpar);
Where is the issue? Many thanks!
Please try to also add the Private Networks (Client & Server) capability in your app's manifest.
On other hand, you can try the Official DatagramSocket sample to see whether the both devices can communicate with each other.
After some experimenting, I came out with the idea of using two different DatagramSocket instances: one to send out UDP messages, and one to listen for incoming messages. With this code, I've been able to send UDP messages to the remote device (as before) and also I've been able to receive UDP messages from the remote device.
async Task TestUdpIP_DifferentPorts()
{
// Writer to the DatagramSocket
DataWriter writer;
// Inbound and outbound DatagramSocket
DatagramSocket udpListener = new DatagramSocket();
DatagramSocket udpSender = new DatagramSocket();
try
{
// String containing the serializaed message
string serializedMessage = "Some message";
var controllerName = new HostName(controllerIpAddress.ToString());
var remoteHostName = new HostName(hostIpAddres.ToString());
// Bind listener
udpListener.MessageReceived += UdpClient_MessageReceived;
await udpListener.BindEndpointAsync(controllerName, controllerIpPort.ToString());
// Connect sender
await udpSender.ConnectAsync(remoteHostName, remoteHostPort.ToString());
// Reset the counter of messages received back from the remote robot
messagesReceived = 0;
// Send the message
writer = new DataWriter(udpSender.OutputStream);
writer.WriteString(JsonConvert.SerializeObject(message));
await writer.StoreAsync();
// Wait for robot status messages
await Task.Delay(1000);
}
catch (Exception ex)
{
// Some exception handling
}
udpSender.Dispose();
udpListener.Dispose();
}
void UdpClient_MessageReceived(DatagramSocket sender, DatagramSocketMessageReceivedEventArgs args)
{
// Just increment the number of messages received
messagesReceived++;
}
With this approach only the IP port (or service name) of the inbound listener needs to be specified. The framework will choose the next available outbound IP port.
Note: If I understand well the DatagramSocket.ConnectAsync(EndpointPair) documentation, the same DatagramSocket instance can be used to send and listen for incoming message, so I could not figure out the need for two different instances. From the documentation:
This ConnectAsync(EndPointPair) method on a DatagramSocket is used to define the local and remote endpoint where datagrams will be sent when using the OutputStream property. This method also restricts remote IP addresses of packets that will be accepted to the remote hostname in the endpointPair parameter. Only incoming packets that match the remote endpoint in the endpointPair parameter will trigger the MessageReceived event on the DatagramSocket.
Now am working on an exception for our uploading image service, the scenario below:
We have a web page, user from all over the world can upload their images to our server, the image normally keep about 3MB. Now we held a promotion, so the images number uploading to our server is extremely huge, which, however caused the server throws the exception as "org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool".
We use apache httpclient as the core uploading middleware, its version is 4.5.*, we correctly handled the response by using the method mentioned in this article.
The code like below:
if (returnType != StorageHttpResponse.class && response != null) {
EntityUtils.consumeQuietly(response.getEntity());
httpRequest.abort();
}
Also, the max connection pool for the service is 128 and the max connection time out is 50000 ms. We upload the images by using stream mode, not directly upload the image file.
So here, I correctly handled the response entity by consuming it in finally code block, but I still can't stop the service throw connection pool timeout exception.
Any other stuffs that I need to add to my service? Do I really using redis to make a queue to user's uploading requests and post handling?
Whole code here:
public <T> T excute(Request request, Class<T> returnType) {
Preconditions.checkState(!isShutDown, "JSSHttpClient is destory!");
HttpRequestBase httpRequest = new HttpRequestBuild(this.credential).build(request);
HttpResponse response = null;
try {
response = this.client.execute(httpRequest);
if (errorHandler.hasError(request, response)) {
int statusCode = response.getStatusLine().getStatusCode();
log.warn("Unexpected response," + request + " http code [" + statusCode + "]");
errorHandler.handleError(response);
}
if (returnType != null && returnType != StorageHttpResponse.class) {
return JsonMessageConverter.read(returnType, response);
}
if (returnType == StorageHttpResponse.class) {
return (T) new StorageHttpResponse(response);
}
} catch (IOException e) {
Throwables.propagate(e);
} finally {
if (returnType != StorageHttpResponse.class && response != null) {
EntityUtils.consumeQuietly(response.getEntity());
httpRequest.abort();
}
}
return null;
}
you can set the parameters in either properties or yml file like below.
http:
pool:
size: 100
sockettimeout: 20000
defaultMaxPerRoute: 200
maxPerRoutes:
-
scheme: http
host: localhost
port: 8080
maxPerRoute: 100
-
scheme: https
host: {{URL}}
port: -1
maxPerRoute: 200
Finally, we solved this not by using code. Because we all know if the response not consumed directly, the connection of a request will not released. So in our code, we offen consume the response first.
We solved this problem not by using better code but slightly modify some parameters like maxconnectionpoolsize, maxconnectionperroute and maxconnectiontimeout based on our business scenario. Then running it and all seems ok now. Hope this helps you.
Using J2ME I need to send a JPEG image to a server using Socket Communication. Can anyone send me a sample code for my application?
Take a look at this article from Sun Developer Network. It has some minimalistic examples and should give you some ideas.
...
SocketConnection client = (SocketConnection) Connector.open("socket://" + hostname + ":" + port);
// set application-specific options on the socket. Call setSocketOption to set other options
client.setSocketOption(DELAY, 0);
client.setSocketOption(KEEPALIVE, 0);
InputStream is = client.openInputStream();
OutputStream os = client.openOutputStream();
// send something to server
os.write("some string".getBytes());
// read server response
int c = 0;
while((c = is.read()) != -1) {
// do something with the response
}
// close streams and connection
is.close();
os.close();
client.close();
...