I have a scenario where I am trying to send multiple messages in one and trying to stream the one message from the gRPC server to the gRPC client side.
My proto files on server side look like this:
Service Greeter{
rpc AccumulateEvents (EventRequest) returns (stream EventsMessage);
}
message EventsMessage{
FirstEvent firstEvents =1;
SecondEvents secondEvent = 2;
}
message EventRequest{
//sending empty request
}
The service method is as follows:
public override async Task AccumulateEvents(EventRequest eventRequest, IServerStreamWriter < EventsMessage > responseStream, ServerCallContext context) {
IDisposable disposable4 = service.SubscribeToEvents(OnEvents);
service.execute();
await responseStream.WriteAsync(new EventsMessage {
FirstEvent = firstEvent, SecondEvents = secondEvents
});
}
When I am trying to fetch and parse the stream from the client side,i am getting null for secondEvent part of the message EventsMessage. Only firstEvents was returned from the server to the client. I tried debugging and could see secondEvent getting populated but then it became null when the streaming started from the server.
Also, secondEvent is a repeated field. I am not sure if that is the reason of it becoming null.
Please let me know what i might be missing here.
Related
The vertx application runs in Docker containers, on two EC2 instances and is clustered.
Clustering is achieved with the hazelcast-aws plugin and the application is started like this:
docker run --name ... -p ... \
--network ... \
-v ... \
-d ... \
-c 'exec java \
-Dvertx.eventBus.options.setClustered=true \
-Dvertx.eventBus.options.setClusterPort=15701 \
-jar ... -conf ... \
-cluster'
Nothing cluster-related is set programmatically.
Client opens a socket on the first request and uses it for future similar requests.
Each request will:
initiate an async request with the server by publishing a message to the event bus
register a consumer on the event bus which will handle the result of the above,
and which is passed a reference to the socket connection where it should send the result to
Since vertx does round robin by default when clustered and there are two instances, this means any instance gets every other message (from 1., above) and makes the client, which connects to one instance only, receive exactly half of all expected responses.
I suppose this is because, even though the registered consumer has a reference to the socket object, it can't use it because it was created on a different node/webserver.
Would that be correct and is there a way to get 100% of messages to the client, connected to just one node, without introducing things like RabbitMQ?
Here's the SockJS handler code:
SockJSHandler sockJSHandler = SockJSHandler.create(vertx, new SockJSHandlerOptions());
sockJSHandler.socketHandler(socket -> {
SecurityService securityService = (SecurityService) ServiceFactory.getService(SecurityService.class);
if (securityService.socketHeadersSecurity(socket)) {
socket.handler(socketMessage -> {
try {
LOGGER.trace("socketMessage: " + socketMessage);
Socket socket = Json.decodeValue(socketMessage.toString(), Socket.class);
Report report = socket.getReport();
if (report != null) {
Account accountRequest = socket.getAccount();
Account accountDatabase = accountRequest == null ? null
: ((AccountService) ServiceFactory.getService(AccountService.class)).getById(accountRequest.getId());
Response result = securityService.socketReportSecurity(accountRequest, accountDatabase, report) ?
((ReportService) ServiceFactory.getService(ReportService.class)).createOrUpdateReport(report, accountDatabase)
: new Response(Response.unauthorized);
if (Response.success.equals(result.getResponse())) {
//register a consumer
String consumerName = "report.result." + Timestamp.from(ClockFactory.getClock().instant());
vertx.eventBus().consumer(consumerName, message -> {
Response executionResult;
if ("success".equals(message.body())) {
try {
Path csvFile = Paths.get(config.getString(Config.reportPath.getConfigName(), Config.reportPath.getDefaultValue())
+ "/" + ((Report) result.getPayload()).getId() + ".csv");
executionResult = new Response(new JsonObject().put("csv", new String(Files.readAllBytes(csvFile))));
} catch (IOException ioEx) {
executionResult = new Response(new Validator("Failed to read file.", ioEx.getMessage(), null, null));
LOGGER.error("Failed to read file.", ioEx);
}
} else {
executionResult = new Response(new Validator("Report execution failed", (String)message.body(), null, null));
}
//send second message to client
socket.write(Json.encode(executionResult));
vertx.eventBus().consumer(consumerName).unregister();
});
//order report execution
vertx.eventBus().send("report.request", new JsonObject()
.put("reportId", ((Report) result.getPayload()).getId())
.put("consumerName", consumerName));
}
//send first message to client
socket.write(Json.encode(result));
} else {
LOGGER.info("Insufficient data sent over socket: " + socketMessage.toString());
socket.end();
}
} catch (DecodeException dEx) {
LOGGER.error("Error decoding message.", dEx);
socket.end();
}
});
} else {
LOGGER.info("Illegal socket connection attempt from: " + socket.remoteAddress());
socket.end();
}
});
mainRouter.route("/websocket/*").handler(sockJSHandler);
Interestingly, when running two nodes clustered on localhost the client gets 100% of the results.
EDIT:
This was not a SockJS but a configuration issue.
Since vertx does round robin by default when clustered and there are
two instances, this means any instance gets every other message (from
1., above) and makes the client, which connects to one instance only, receive exactly half of all expected responses.
This assumption is only partially correct. Vert.x does round-robin, yes, but this means each instance will get half of the connections, not half of the messages.
Once connection is established, all its messages will arrive to a single instance.
So this:
Would that be correct and is there a way to get 100% of messages to
the client, connected to just one node, without introducing things
like RabbitMQ?
Already happens.
I'm working on a UWP application which should communicate via TCP/UDP to a remote device. My issue is that the UWP app successfully sends UDP messages to the remote device, but does not receive the replies.
Here follows the code extracted from the app (simplified):
async Task TestUdpIP()
{
// Writer to the DatagramSocket
DataWriter writer;
using (var udpClient = new DatagramSocket())
{
try
{
// UDP Socket binding
udpClient.MessageReceived += UdpClient_MessageReceived;
var controllerName = new Windows.Networking.HostName(controllerIpAddress.ToString());
await udpClient.BindEndpointAsync(controllerName, controllerIpPort.ToString());
var remoteHostName = new Windows.Networking.HostName(hostIpAddres.ToString());
await udpClient.ConnectAsync(remoteHostName, remoteHostPort.ToString());
// Create a message to send
string message = "Some message";
// Reset the counter of messages received back from the remote robot
messagesReceived = 0;
// Send the message
writer = new DataWriter(udpClient.OutputStream);
writer.WriteString(message);
await writer.StoreAsync();
// Wait for robot status messages
await Task.Delay(5000);
}
catch
{
}
}
}
void UdpClient_MessageReceived(DatagramSocket sender, DatagramSocketMessageReceivedEventArgs args)
{
// Just increment the number of messages received
messagesReceived++;
}
However the UdpClient_MessageReceived handler does not fire. I'm sure that UDP messages are correctly sent from the UWP app and that the remote device replies back as shown in the following screenshot from Wireshark (the test has been taken on the same PC where the UWP app is running)
.
(IP Address, Port) details are shown in the following list to better explain the picture above
UWP application: (192.168.1.108, 19000) to send and receive.
Remote device: (192.168.1.152, 15999) to receive and (192.168.1.152, 54697) to send
Note: this is a similar question, where the answer says that for whatever reason the DatagramSocket should fire some messages before being able to receive. In my example a message is sent out however the message received handler does not fire anyway.
Note: The UWP app has been granted the internet (client) and the internet (client and server) capabilities
Note: I've tried also this (more readable) approach to bind the inbound/outbound datagram socket on the UWP app obtaining the same results:
// UDP Socket binding
var controllerName = new HostName(controllerIpAddress.ToString());
var remoteHostName = new HostName(hostIpAddres.ToString());
EndpointPair endpointpar = new EndpointPair(controllerName,
controllerIpPort.ToString(),
remoteHostName,
remoteHostPort.ToString());
udpClient.MessageReceived += UdpClient_MessageReceived;
await udpClient.ConnectAsync(endpointpar);
Where is the issue? Many thanks!
Please try to also add the Private Networks (Client & Server) capability in your app's manifest.
On other hand, you can try the Official DatagramSocket sample to see whether the both devices can communicate with each other.
After some experimenting, I came out with the idea of using two different DatagramSocket instances: one to send out UDP messages, and one to listen for incoming messages. With this code, I've been able to send UDP messages to the remote device (as before) and also I've been able to receive UDP messages from the remote device.
async Task TestUdpIP_DifferentPorts()
{
// Writer to the DatagramSocket
DataWriter writer;
// Inbound and outbound DatagramSocket
DatagramSocket udpListener = new DatagramSocket();
DatagramSocket udpSender = new DatagramSocket();
try
{
// String containing the serializaed message
string serializedMessage = "Some message";
var controllerName = new HostName(controllerIpAddress.ToString());
var remoteHostName = new HostName(hostIpAddres.ToString());
// Bind listener
udpListener.MessageReceived += UdpClient_MessageReceived;
await udpListener.BindEndpointAsync(controllerName, controllerIpPort.ToString());
// Connect sender
await udpSender.ConnectAsync(remoteHostName, remoteHostPort.ToString());
// Reset the counter of messages received back from the remote robot
messagesReceived = 0;
// Send the message
writer = new DataWriter(udpSender.OutputStream);
writer.WriteString(JsonConvert.SerializeObject(message));
await writer.StoreAsync();
// Wait for robot status messages
await Task.Delay(1000);
}
catch (Exception ex)
{
// Some exception handling
}
udpSender.Dispose();
udpListener.Dispose();
}
void UdpClient_MessageReceived(DatagramSocket sender, DatagramSocketMessageReceivedEventArgs args)
{
// Just increment the number of messages received
messagesReceived++;
}
With this approach only the IP port (or service name) of the inbound listener needs to be specified. The framework will choose the next available outbound IP port.
Note: If I understand well the DatagramSocket.ConnectAsync(EndpointPair) documentation, the same DatagramSocket instance can be used to send and listen for incoming message, so I could not figure out the need for two different instances. From the documentation:
This ConnectAsync(EndPointPair) method on a DatagramSocket is used to define the local and remote endpoint where datagrams will be sent when using the OutputStream property. This method also restricts remote IP addresses of packets that will be accepted to the remote hostname in the endpointPair parameter. Only incoming packets that match the remote endpoint in the endpointPair parameter will trigger the MessageReceived event on the DatagramSocket.
I am using Netty to setup a simple http/2 server. I am using this example as http/2 server.
To test this server, I am using netty example client.
My client code where I am sending the request to server:
Complete code : http://netty.io/5.0/xref/io/netty/example/http2/client/package-summary.html
HttpResponseHandler responseHandler = initializer.responseHandler();
int streamId = 3;
HttpScheme scheme = SSL ? HttpScheme.HTTPS : HttpScheme.HTTP;
AsciiString hostName = new AsciiString(HOST + ':' + PORT);
System.out.println("Sending request(s)...");
if (URL != null) {
System.out.println("with url");
// Create a simple GET request.
FullHttpRequest request = new DefaultFullHttpRequest(HTTP_1_1, GET, URL);
request.headers().add(HttpHeaderNames.HOST, hostName);
request.headers().add(HttpConversionUtil.ExtensionHeaderNames.SCHEME.text(), scheme.name());
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP);
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.DEFLATE);
responseHandler.put(streamId, channel.writeAndFlush(request), channel.newPromise());
streamId += 2;
}
Above code works fine with stream id 3,5 and so on.
But when i change the stream id to any other number like 4,6,8 etc, above code doesn't work. From server I still get the messages for stream id 3,5,7 etc. I am unable to find the logic for these stream id inside example server
Stream numbering is mandated by the HTTP/2 specification.
I'm using MassTransit with Reactive Extensions to stream messages from the queue in batches. Since the behaviour isn't the same as a normal consumer I need to be able to send a message to the error queue if it fails an x number of times.
I've looked through the MassTransit source code and posted on the google groups and can't find an anwser.
Is this available on the ConsumeContext interface? Or is this even possible?
Here is my code. I've removed some of it to make it simpler.
_busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.UseInMemoryScheduler();
cfg.ReceiveEndpoint(host, "customer_update_queue", e =>
{
var _observer = new ObservableObserver<ConsumeContext<Customer>>();
_observer.Buffer(TimeSpan.FromMilliseconds(1000)).Subscribe(OnNext);
e.Observer(_observer);
});
});
private void OnNext(IList<ConsumeContext<Customer>> messages)
{
foreach (var consumeContext in messages)
{
Console.WriteLine("Content: " + consumeContext.Message.Content);
if (consumeContext.Message.RetryCount > 3)
{
// I want to be able to send to the error queue
consumeContext.SendToErrorQueue()
}
}
}
I've found a work around by using the RabbitMQ client mixed with MassTransit. Since I can't throw an exception when using an Observable and therefore no error queue is created. I create it manually using the RabbitMQ client like below.
ConnectionFactory factory = new ConnectionFactory();
factory.HostName = "localhost";
factory.UserName = "guest";
factory.Password = "guest";
using (IConnection connection = factory.CreateConnection())
{
using (IModel model = connection.CreateModel())
{
string exchangeName = "customer_update_queue_error";
string queueName = "customer_update_queue_error";
string routingKey = "";
model.ExchangeDeclare(exchangeName, ExchangeType.Fanout);
model.QueueDeclare(queueName, false, false, false, null);
model.QueueBind(queueName, exchangeName, routingKey);
}
}
The send part is to send it directly to the message queue if it fails an x amount of times like so.
consumeContext.Send(new Uri("rabbitmq://localhost/customer_update_queue_error"), consumeContext.Message);
Hopefully the batch feature will be implemented soon and I can use that instead.
https://github.com/MassTransit/MassTransit/issues/800
I'm using SmartGWT with RestDataSource. Whenever I lose internet connection, I get an SC.warn window with:
Server returned TRANSPORT_ERROR with no error message
I have tried to intercept this message to create a more friendly message, by adding a Callback handler for RPCManager like so:
RPCManager.setHandleTransportErrorCallback(new HandleTransportErrorCallback() {
public void handleTransportError(int transactionNum, int status,
int httpResponseCode, String httpResponseText) {
System.err.println("Transaction number: "+transactionNum);
System.err.println("Status: "+status);
System.err.println("Response code: "+httpResponseCode);
System.err.println("Response text:"+httpResponseText);
SC.warn("You have no internet connection.");
}
});
However, the Error messages print, and my warn message shows, but so does the system warn message above!
Keep in mind that I'm using a RestDataSource and not directly connecting to server with RPCManager.
How can I intercept this error message?
Thanks
Try instead RPCManager.setHandleErrorCallback(..).