I have a project where I send through AMQP a message with a signed JSON object. On the server side I have the following:
public class MyServerHandler implements MessageListener{
...
#Override
public void onMessage(Message msg)
{
String _payload = new String(msg.getBody(), StandardCharsets.UTF_8);
System.out.println(_payload);
...
Now, if I use the method convertSendAndReceive like this:
String msg = ...deo // JSON object (String)
Object response = getRabbitOperations().convertSendAndReceive(_myExchange, _myRoutingKey, msg,
new MessagePostProcessor()
{
public Message postProcessMessage(Message message) throws AmqpException
{
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.NON_PERSISTENT);
message.getMessageProperties().setRedelivered(false);
message.getMessageProperties().setUserId(_myUser);
message.getMessageProperties().setType(_myType);
return message;
}
}
);
When I get the contents of the AMQP message I get the character string sent but in quotation marks:
String _payload = new String(msg.getBody(), StandardCharsets.UTF_8);
System.out.println(_payload);
"eyJ4NWMiOlsiTUlJRHV6Q0NBcU9nQXdJQkFnSUNDMEV3RFFZSktvWklodmNOQVFFR....."
But if I use the sendAndReceive method, I don't get quotation marks:
MessageProperties mp = new MessageProperties();
mp.setDeliveryMode(MessageDeliveryMode.NON_PERSISTENT);
mp.setRedelivered(false);
mp.setUserId(_myUser);
mp.setType(_myType);
Message _m = new Message(msg.getBytes(), mp);
Object _response = getRabbitOperations().sendAndReceive(_myExchange, _myRoutingKey, _m);
On the server side I get(with the same code):
eyJ4NWMiOlsiTUlJRHV6Q0NBcU9nQXdJQkFnSUNDMEV3RFFZSktvWklodmNOQVFFR.....
Why is that? Can I change it to work the same way? And which would be the right one?
Thank you
The problem with sendAndReceive that it deals with message directly and bypasses MessageConverter. According to your quotation experience it looks like there is a JSON converter.
You can use SimpleMessageConverter. But that doesn't matter really: if you want clear byte[] to send, just do the same msg.getBytes() for the convertSendAndReceive.
Related
I'm using Flink to process my streaming data.
The streaming is coming from some other middleware, such as Kafka, Pravega, etc.
Saying that Pravega is sending some word stream, hello world my name is....
What I need is three steps of process:
Map each word to my custom class object MyJson.
Map the object MyJson to String.
Write Strings to files: one String is written to one file.
For example, for the stream hello world my name is, I should get five files.
Here is my code:
// init Pravega connector
PravegaDeserializationSchema<String> adapter = new PravegaDeserializationSchema<>(String.class, new JavaSerializer<>());
FlinkPravegaReader<String> source = FlinkPravegaReader.<String>builder()
.withPravegaConfig(pravegaConfig)
.forStream(stream)
.withDeserializationSchema(adapter)
.build();
// map stream to MyJson
DataStream<MyJson> jsonStream = env.addSource(source).name("Pravega Stream")
.map(new MapFunction<String, MyJson>() {
#Override
public MyJson map(String s) throws Exception {
MyJson myJson = JSON.parseObject(s, MyJson.class);
return myJson;
}
});
// map MyJson to String
DataStream<String> valueInJson = jsonStream
.map(new MapFunction<MyJson, String>() {
#Override
public String map(MyJson myJson) throws Exception {
return myJson.toString();
}
});
// output
valueInJson.print();
This code will output all of results to Flink log files.
My question is how to write one word to one output file?
I think the easiest way to do this would be with a custom sink.
stream.addSink(new WordFileSink)
public static class WordFileSink implements SinkFunction<String> {
#Override
public void invoke(String value, Context context) {
// generate a unique name for the new file and open it
// write the word to the file
// close the file
}
}
Note that this implementation won't necessarily provide exactly once behavior. You might want to take care that the file naming scheme is both unique and deterministic (rather than depending on processing time), and be prepared for the case that the file may already exist.
I am sending a string with 2000 character on TCP socket by netty, both side of client and server,I set this pipeline :
socketChannel.pipeline().addLast(
new StringEncoder(),
new StringDecoder(),
new LineBasedFrameDecoder(Integer.MAX_VALUE),
myHandler);
on server side, I have a ChannelInboundHandlerAdapter.
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
// some code
ctx.writeAndFlush(line);
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER)
.addListener(ChannelFutureListener.CLOSE);
}
on client side I have a SimpleChannelInboundHandler. but I get my string incomplete. why !? How can I solve that?
protected void channelRead0(ChannelHandlerContext channelHandlerContext, Object msg) throws Exception {//msg is incomplete}
I found solution.
new LineBasedFrameDecoder(Integer.MAX_VALUE) is nonsense. because maximum of characters in each String in java is 65536, so try new LineBasedFrameDecoder(65536) or any size you need.
In server side each msg should end with \n or \r\n. so for example if you read a line from a text file you need add \n in the end of that.
I am exposing a rest service using "CamelHttpTransportServlet" that receive orders and place in jms queue. The code works fine on happy path and returns 200 response.
I have written Processor to validate the input JSON, and set http_response_code based on the input.
The issue is - for invalid requests though failure response code - 400 is set, the flow continues to the next route and pushes the data to the queue instead of sending the 400 response back to the calling app.
rest("/ordermanagement")
.post("/order").to("direct:checkInput");
from("direct:checkInput")
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
String requestBody = exchange.getIn().getBody(String.class);
if(requestBody == "" || requestBody== null) {
exchange.getIn().setBody("{ "error": Bad Request}");
exchange.getIn().setHeader(Exchange.CONTENT_TYPE, "application/json");
exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 400);
}
}
})
.to("direct:sendToQ");
from("direct:sendToQ")
.to("jms:queue:orderReceiver")
.log("Sent to JMS");
Can someone advise what is missing here and provide a sample if possible?
Trying to implement onException approach:
rest("/ordermanagement")
.post("/order").to("direct:checkInput");
onException(CustomException.class).handled(true)
.setHeader(Exchange.HTTP_RESPONSE_CODE, code)
.setBody(jsonObject);
from("direct:checkInput")
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
String requestBody = exchange.getIn().getBody(String.class);
if(requestBody == "" || requestBody== null) {
throw CustomException(code, jsonObject)
}
}
})
.to("direct:sendToQ");
from("direct:sendToQ")
.to("jms:queue:orderReceiver")
.log("Sent to JMS");
However I could not figure out how to pass the parameters - code,jsonObject from processor to onException block.
Any help on this? Is this feasible?
I'd use something along the lines of the code example below:
onException(CustomException.class)
.handled(true)
.bean(PrepareErrorResponse.class)
.log("Error response processed");
rest("/ordermanagement")
.post("/order")
.to("direct:checkInput");
from("direct:checkInput")
.process((Exchange exchange) -> {
String requestBody = exchange.getIn().getBody(String.class);
if(requestBody == "" || requestBody== null) {
throw new CustomException(code, jsonObject);
}
})
.to("direct:sendToQ");
from("direct:sendToQ")
.to("jms:queue:orderReceiver")
.log("Sent to JMS");
Camel will store any exception caught in the exchange's property and should be therefore obtainable via the Exchange.EXCEPTION_CAUGHT property key. The sample below illustrates how such a custom error message bean can look like:
public class PrepareErrorResponse {
#Handler
public void prepareErrorResponse(Exchange exchange) {
Throwable cause = exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Throwable.class);
if (cause instanceof CustomException) {
CustomException validationEx = (CustomException) cause;
// ...
}
Message msg = exchange.getOut();
msg.setHeader(Exchange.CONTENT_TYPE, MediaType.APPLICATION_JSON);
msg.setHeader(Exchange.HTTP_RESPONSE_CODE, 400);
JsonObject errorMessage = new JsonObject();
errorMessage.put("error", "Bad Request");
errorMessage.put("reason", cause.getMessage());
msg.setBody(errorMessage.toString());
// we need to do the fault=false below in order to prevent a
// HTTP 500 error code from being returned
msg.setFault(false);
}
}
Camel provides a couple of ways actually to deal with exceptions. The presented way here is just one example. The proposed code however allows to use custom redelivery strategies for different caught exceptions as well as additional stuff. If the error could get resolved within the exception handler, the route is proceeded at the point the exception occurred (i.e. temporary network issue with a redelivery strategy applied). If the error could not get fixed within the handler, the exchange will be stopped. Usually one would then send the currently processed message to a DLQ and log something about the error.
Note that this example will assume that CustomException is an unchecked exception as the processor is replaced with a simpler lambda. If you can't or don't want to use such an exception (or lambda expressions) replace the lambda-processor with new Processor() { #Override public void process(Exchange exchange) throws Exception { ... } } construct.
Here is one way to do it. You can use choice
rest("/ordermanagement")
.post("/order").to("direct:checkInput");
from("direct:checkInput")
.process(exchange -> {
String requestBody = exchange.getIn().getBody(String.class);
if(requestBody == null || requestBody.equals("")) {
exchange.getIn().setBody("{ "error": Bad Request}");
exchange.getIn().setHeader(Exchange.CONTENT_TYPE, "application/json");
exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 400);
}
})
.choice()
.when(exchange -> {
Object header = exchange.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE);
return header != null && header.equals(400);
})
.stop()
.otherwise()
.to("direct:sendToQ")
.endChoice();
from("direct:sendToQ")
.to("jms:queue:orderReceiver")
.log("Sent to JMS");
Setting ROUTE_STOP property to true in the processor should prevent further flow and return your response:
...
exchange.getIn().setBody("{ "error": Bad Request}");
exchange.getIn().setHeader(Exchange.CONTENT_TYPE, "application/json");
exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 400);
exchange.setProperty(Exchange.ROUTE_STOP, Boolean.TRUE);
...
Could someone explain why this httpunit test case keeps failing in wc.getResponse with "bad file descriptor". I added the is.close() as a guess and moved it before and after the failure but that had no effect. This tests put requests to a Dropwizard app.
public class TestCircuitRequests
{
static WebConversation wc = new WebConversation();
static String url = "http://localhost:8888/funl/circuit/test.circuit1";
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
WebRequest rq = new PutMethodWebRequest(url, is, "application/json");
wc.setAuthentication("FUNL", "foo", "bar");
WebResponse response = wc.getResponse(rq);
is.close();
}
No responses? So I'll try myself based on what I learned fighting this.
Httpunit is an old familiar tool that I'd use if I could. But it hasn't been updated in more than two years, so I gather its support for #PUT requests isn't right.
So I converted to Jersey-client instead. After a bunch of struggling I wound up with this code which does seem to work:
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
String circuit = StreamUtil.readFully(is);
is.close();
Authenticator.setDefault(new MyAuthenticator());
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
com.sun.jersey.api.client.WebResource service = client.resource(url);
Builder builder = service.accept(MediaType.APPLICATION_JSON);
builder.entity(circuit, MediaType.APPLICATION_JSON);
builder.put(String.class, circuit);
return;
}
This intentionally avoids JAX-RS automatic construction of beans from JSON strings.
This is my method to send message to a private Q
using (MessageQueue msgQ = new MessageQueue(MessageQueueName))
{
using (System.Messaging.Message newMessage = new System.Messaging.Message(MessageBody,
new System.Messaging.ActiveXMessageFormatter()))
{
newMessage.Label = MessageLabel;
newMessage.Priority = Priority;
msgQ.Send(newMessage);
}
}
I have an order object which i serialize and send as message body. The serialized object is
<?xml version="1.0"?>
<OrderInfo>
<OrderID>11111</OrderID>
<OrderDetails>
<LineItem>
<ProductDetails>
<Name>qwqwqw</Name>
<Manufacturer>asasas</Manufacturer>
<UPC>12222222222</UPC>
<sku>2132</sku>
<Price>12.21</Price>
</ProductDetails>
<Quantity>1</Quantity>
</LineItem>
</OrderDetails>
</OrderInfo>
This is my method to receive that message in a windows service
void queue_ReceiveCompleted(object sender, ReceiveCompletedEventArgs asyncResult)
{
// Connect to the queue.
MessageQueue mq = (MessageQueue)sender;
// End the asynchronous Receive operation.
Message m = mq.EndReceive(asyncResult.AsyncResult);
m.Formatter = new System.Messaging.ActiveXMessageFormatter()
//Get Filedata from body
OrdrInfo qMessage = (OrdrInfo)XMLUtil.Deserialize(m.Body.ToString(), typeof(OrdrInfo));
}
when I try to look at m.Body in quickwatch this is what it states
m.Body.Message = Cannot find a formatter capable of reading this message.
m.Body.StackTrace = at System.Messaging.Message.get_Body()
Hopefully you're not still stuck on this, but as it came up top of my search when running into the same problem.
As no one had answered it, here is one answer that I've just found else where (thanks TechRepublic). This code assume that "MyType" is a typically basic message that can be read by XML Serialisation - this means it is marked as serializable and all data to be sent/reconstructed is in public get/set properties.
Code is:
MessageQueue msgQ = new MessageQueue(#".\private$\CreateNewEntity");
msgQ.Formatter = new XmlMessageFormatter(new []{typeof(MyType)});
var msg = msgQ.Receive();
msgQ.Close();
return msg.Body as MyType;