How get original message after get an errorHandler and write a file - email

I've been building a Spring integration Service Email using Java DSL.
This service must have a recovery policy in order to retry sending the emails but I'm not getting success.
A brief story: The application recieve a Payload and Header and try to send to email server. It tries 3 times and in case of failure, it creates a new file with Header and Body of message.
How could I get the original Message(Header and Payload) and put the information pair in a json file, in case of failure to send the email?
Thanks.
This is my beans and the service:
/**
* #################
* MESSAGE ENDPOINTS
* #################
*/
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers
.fixedRate(NumberUtils.createLong(QUEUE_RATE))
.maxMessagesPerPoll(NumberUtils.createLong(QUEUE_CAPACITY))
.errorHandler(e -> LOG.error("Exception : " + e.getMessage()))
.get();
}
#Bean
public MessageChannel recoveryChannel() {
return MessageChannels.direct().get();
}
#MessagingGateway
public static interface MailService {
#Gateway(requestChannel = "mail.input")
void sendMail(String body, #Headers Map<String,String> headers);
}
#Bean
public RetryPolicy retryPolicy() {
final Map<Class<? extends Throwable>, Boolean> map =
new HashMap<Class<? extends Throwable>, Boolean>() {
{
put(MailSendException.class,true);
put(RuntimeException.class, true);
}
private static final long serialVersionUID = -1L;
};
final RetryPolicy ret = new SimpleRetryPolicy(3, map, true);
return ret;
}
#Bean
public RetryTemplate retryTemplate() {
final RetryTemplate ret = new RetryTemplate();
ret.setRetryPolicy(retryPolicy());
ret.setThrowLastExceptionOnExhausted(false);
return ret;
}
#Bean
public Advice retryAdvice() {
final RequestHandlerRetryAdvice advice = new RequestHandlerRetryAdvice();
advice.setRetryTemplate(retryTemplate());
RecoveryCallback<Object> recoveryCallBack = new ErrorMessageSendingRecoverer(recoveryChannel());
advice.setRecoveryCallback(recoveryCallBack);
return advice;
}
private MailSendingMessageHandlerSpec mailOutboundAdapter(){
MailSendingMessageHandlerSpec msmhs =
Mail.outboundAdapter(emailServerHost())
.port(serverPort())
.credentials(MAIL_USER_NAME, MAIL_PASSWORD)
.protocol(emailProtocol())
.javaMailProperties(p -> p
.put("mail.debug", "true")
.put("mail.smtp.ssl.enable",enableSSL())
.put("mail.smtp.connectiontimeout", 5000)
.put("mail.smtp.timeout", 5000));
return msmhs;
}
#Bean
public FileWritingMessageHandler fileOutboundAdapter(){
FileWritingMessageHandler fwmhs = Files
.outboundAdapter(new File("logs/errors/"))
.autoCreateDirectory(true)
.get();
return fwmhs;
}
/**
* ################
* FLOWS
* ################
*/
#Bean
public IntegrationFlow smtp(){
return IntegrationFlows.from("mail.input")
.channel(MessageChannels.queue())
.handle(this.mailOutboundAdapter(),
e -> e.id("smtpOut")
.advice(retryAdvice())
)
.get();
}
#Bean
public IntegrationFlow errorFlow(){
return IntegrationFlows.from(recoveryChannel())
.transform(Transformers.toJson())
.enrichHeaders(c -> c.header(FileHeaders.FILENAME, "emailErrors"))
.handle(this.fileOutboundAdapter())
.get();
}
}

The error message has a payload MessagingException. It has two properties cause and failedMessage.
The failed message is the message at the point of failure, with headers and payload.

Related

Spring framework integration TCP IP - Client application SSL not working and posting incomplete requests

I am new to Spring framework. We have a requirement where our application is acting as a client and needs to integrate with another application using TCP. We will be sending them fixed length requests and we will receive response for the same. We have been asked to use the same TCP connection for each request. Using the same open connection, our application will also be receiving heartbeat messages from server application and we do not need to send any response for them.
The request messages that we need to send is header + body where header has message type and length details.
We will be using SSL. When we try to test with SSL, it does not show any exception during getConnection but is not able to receive any heartbeat messages.
When we test without SSL, it is able to send requests and receive response as well as heartbeat messages. But after the first request response, it sends partial request text to server application for subsequent messages which is causing issues and connections are being reset by peer due to unexpected message received at their end.
I have tried many things referring to online documents available but not able to successfully implement the requirement.
Please find below code. Thanks in advance.
public class ClientConfig implements ApplicationEventPublisherAware{
protected String port;
protected String host;
protected String connectionTimeout;
protected String keyStorePath;
protected String trustStorePath;
protected String keyStorePassword;
protected String trustStorePassword;
protected String protocol;
private ApplicationEventPublisher applicationEventPublisher;
#Override
public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
this.applicationEventPublisher = applicationEventPublisher;
}
#Bean
public DefaultTcpNioSSLConnectionSupport connectionSupport() {
if("SSL".equalsIgnoreCase(getProtocol())) {
DefaultTcpSSLContextSupport sslContextSupport =
new DefaultTcpSSLContextSupport(getKeyStorePath(),
getTrustStorePath(), getKeyStorePassword(), getTrustStorePassword());
sslContextSupport.setProtocol(getProtocol());
DefaultTcpNioSSLConnectionSupport tcpNioConnectionSupport =
new DefaultTcpNioSSLConnectionSupport(sslContextSupport);
return tcpNioConnectionSupport;
}
return null;
}
#Bean
public AbstractClientConnectionFactory clientConnectionFactory() {
if(StringUtils.isNullOrEmptyTrim(getHost()) || StringUtils.isNullOrEmptyTrim(getPort())) {
return null;
}
TcpNioClientConnectionFactory tcpNioClientConnectionFactory =
new TcpNioClientConnectionFactory(getHost(), Integer.valueOf(getPort()));
tcpNioClientConnectionFactory.setApplicationEventPublisher(applicationEventPublisher);
tcpNioClientConnectionFactory.setSoKeepAlive(true);
tcpNioClientConnectionFactory.setDeserializer(new CustomSerializerDeserializer());
tcpNioClientConnectionFactory.setSerializer(new CustomSerializerDeserializer());
tcpNioClientConnectionFactory.setLeaveOpen(true);
tcpNioClientConnectionFactory.setSingleUse(false);
if("SSL".equalsIgnoreCase(getProtocol())) {
tcpNioClientConnectionFactory.setSslHandshakeTimeout(60);
tcpNioClientConnectionFactory.setTcpNioConnectionSupport(connectionSupport());
}
return tcpNioClientConnectionFactory;
}
#Bean
public MessageChannel outboundChannel() {
return new DirectChannel();
}
#Bean
public PollableChannel receiverChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "outboundChannel")
public TcpSendingMessageHandler outboundClient
(AbstractClientConnectionFactory clientConnectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(clientConnectionFactory);
if(!StringUtils.isNullOrEmpty(getConnectionTimeout())) {
long timeout = Long.valueOf(getConnectionTimeout());
outbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
}
outbound.setClientMode(true);
return outbound;
}
#Bean
public TcpReceivingChannelAdapter inboundClient(TcpNioClientConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
if(!StringUtils.isNullOrEmpty(getConnectionTimeout())) {
long timeout = Long.valueOf(getConnectionTimeout());
inbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
}
inbound.setOutputChannel(receiverChannel());
inbound.setClientMode(true);
return inbound;
}
}
public class CustomSerializerDeserializer implements Serializer<String>, Deserializer<String> {
#Override
public String deserialize(InputStream inputStream) throws IOException {
int i = 0;
byte[] lenbuf = new byte[8];
String message = null;
while ((i = inputStream.read(lenbuf)) != -1) {
String messageType = new String(lenbuf);
if(messageType.contains(APP_DATA_LEN)){
byte byteResp[] = new byte[RESP_MSG_LEN-8];
inputStream.read(byteResp, 0, RESP_MSG_LEN-8);
String readMsg = new String(byteResp);
message = messageType + readMsg;
}else {
byte byteResp[] = new byte[HANDSHAKE_LEN-8];
inputStream.read(byteResp, 0, HANDSHAKE_LEN-8);
String readMsg = new String(byteResp);
message = messageType + readMsg;
}
}
return message;
}
#Override
public void serialize(String object, OutputStream outputStream) throws IOException {
outputStream.write(object.getBytes());
outputStream.flush();
}
}
#Override
public String sendMessage(String message) {
Message<String> request = MessageBuilder.withPayload(message).build();
DirectChannel outboundChannel = (DirectChannel) applicationContext.getBean(DirectChannel.class);
outboundChannel.send(request);
}
//Below code is being used to open connection
TcpNioClientConnectionFactory cf = (TcpNioClientConnectionFactory) applicationContext.getBean(AbstractClientConnectionFactory.class);
if(cf != null) {
TcpNioConnection conn = (TcpNioConnection) cf.getConnection();
}

How to read the Header values in the Batch listener error handling scenario

I am trying to handle the exception at the listener
#KafkaListener(id = PropertiesUtil.ID,
topics = "#{'${kafka.consumer.topic}'}",
groupId = "${kafka.consumer.group.id.config}",
containerFactory = "containerFactory",
errorHandler = "errorHandler")
public void receiveEvents(#Payload List<ConsumerRecord<String, String>> recordList,
Acknowledgment acknowledgment) {
try {
log.info("Consuming the batch of size {} from kafka topic {}", consumerRecordList.size(),
consumerRecordList.get(0).topic());
processEvent(consumerRecordList);
incrementOffset(acknowledgment);
} catch (Exception exception) {
throwOrHandleExceptions(exception, recordList, acknowledgment);
.........
}
}
The Kafka container config:
#Bean
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, String>>
containerFactory() {
ConcurrentKafkaListenerContainerFactory<String, String> factory =
new ConcurrentKafkaListenerContainerFactory<>();
factory.setConcurrency(this.numberOfConsumers);
factory.getContainerProperties().setAckOnError(false);
factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL);
factory.setConsumerFactory(getConsumerFactory());
factory.setBatchListener(true);
return factory;
}
}
the listener error handler impl
#Bean
public ConsumerAwareListenerErrorHandler errorHandler() {
return (m, e, c) -> {
MessageHeaders headers = m.getHeaders();
List<String> topics = headers.get(KafkaHeaders.RECEIVED_TOPIC, List.class);
List<Integer> partitions = headers.get(KafkaHeaders.RECEIVED_PARTITION_ID, List.class);
List<Long> offsets = headers.get(KafkaHeaders.OFFSET, List.class);
Map<TopicPartition, Long> offsetsToReset = new HashMap<>();
for (int i = 0; i < topics.size(); i++) {
int index = i;
offsetsToReset.compute(new TopicPartition(topics.get(i), partitions.get(i)),
(k, v) -> v == null ? offsets.get(index) : Math.min(v, offsets.get(index)));
}
...
};
}
when i try to run the same without the batching processing then i am able to fetch the partition,topic and offset values but when i enable batch processing and try to test it then i am getting only two values inside the headers i.e id and timestamp and other values are not set. Am i missing anything here??
What version are you using? I just tested it with Boot 2.2.4 (SK 2.3.5) and it works fine...
#SpringBootApplication
public class So60152179Application {
public static void main(String[] args) {
SpringApplication.run(So60152179Application.class, args);
}
#KafkaListener(id = "so60152179", topics = "so60152179", errorHandler = "eh")
public void listen(List<String> in) {
throw new RuntimeException("foo");
}
#Bean
public ConsumerAwareListenerErrorHandler eh() {
return (m, e, c) -> {
System.out.println(m);
return null;
};
}
#Bean
public ApplicationRunner runner(KafkaTemplate<String, String> template) {
return args -> {
template.send("so60152179", "foo");
};
}
#Bean
public NewTopic topic() {
return TopicBuilder.name("so60152179").partitions(1).replicas(1).build();
}
}
spring.kafka.listener.type=batch
spring.kafka.consumer.auto-offset-reset=earliest
GenericMessage [payload=[foo], headers={kafka_offset=[0], kafka_nativeHeaders=[RecordHeaders(headers = [], isReadOnly = false)], kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer#2f2e787f, kafka_timestampType=[CREATE_TIME], kafka_receivedMessageKey=[null], kafka_receivedPartitionId=[0], kafka_receivedTopic=[so60152179], kafka_receivedTimestamp=[1581351585253], kafka_groupId=so60152179}]

netty SimpleChannelInboundHandler<String> channelRead0 only occasionally invoked

I know that there are several similar questions that have either been answered or still outstanding, however, for the life of me...
Later Edit 2016-08-25 10:05 CST - Actually, I asked the wrong question.
The question is the following: given that I have both a netty server (taken from DiscardServer example) and a netty client - (see above) what must I do to force the DiscardServer to immediately send the client a request?
I have added an OutboundHandler to the server and to the client.
After looking at both the DiscardServer and PingPongServer examples, there is an external event occurring to kick off all the action. In the case of Discard server, it is originally waiting for a telnet connection, then will transmit whatever was in the telnet msg to the client.
In the case of PingPongServer, the SERVER is waiting on the client to initiate action.
What I want is for the Server to immediately start transmitting after connection with the client. None of the examples from netty seem to do this.
If I have missed something, and someone can point it out, much good karma.
My client:
public final class P4Listener {
static final Logger LOG;
static final String HOST;
static final int PORT;
static final Boolean SSL = Boolean.FALSE;
public static Dto DTO;
static {
LOG = LoggerFactory.getLogger(P4Listener.class);
HOST = P4ListenerProperties.getP4ServerAddress();
PORT = Integer.valueOf(P4ListenerProperties.getListenerPort());
DTO = new Dto();
}
public static String getId() { return DTO.getId(); }
public static void main(String[] args) throws Exception {
final SslContext sslCtx;
if (SSL) {
LOG.info("{} creating SslContext", getId());
sslCtx = SslContextBuilder.forClient().trustManager(InsecureTrustManagerFactory.INSTANCE).build();
} else {
sslCtx = null;
}
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap b = new Bootstrap();
b.group(group).channel(NioSocketChannel.class)
.handler(new LoggingHandler(LogLevel.INFO))
.handler(new P4ListenerInitializer(sslCtx));
// Start the connection attempt.
LOG.debug(" {} starting connection attempt...", getId());
Channel ch = b.connect(HOST, PORT).sync().channel();
// ChannelFuture localWriteFuture = ch.writeAndFlush("ready\n");
// localWriteFuture.sync();
} finally {
group.shutdownGracefully();
}
}
}
public class P4ListenerHandler extends SimpleChannelInboundHandler<String> {
static final Logger LOG = LoggerFactory.getLogger(P4ListenerHandler.class);
static final DateTimeFormatter DTFormatter = DateTimeFormatter.ofPattern("yyyyMMdd-HHMMss.SSS");
static final String EndSOT;
static final String StartSOT;
static final String EOL = "\n";
static final ClassPathXmlApplicationContext AppContext;
static {
EndSOT = P4ListenerProperties.getEndSOT();
StartSOT = P4ListenerProperties.getStartSOT();
AppContext = new ClassPathXmlApplicationContext(new String[] { "applicationContext.xml" });
}
private final RequestValidator rv = new RequestValidator();
private JAXBContext jaxbContext = null;
private Unmarshaller jaxbUnmarshaller = null;
private boolean initialized = false;
private Dto dto;
public P4ListenerHandler() {
dto = new Dto();
}
public Dto getDto() { return dto; }
public String getId() { return getDto().getId(); }
Message convertXmlToMessage(String xml) {
if (xml == null)
throw new IllegalArgumentException("xml message is null!");
try {
jaxbContext = JAXBContext.newInstance(p4.model.xml.request.Message.class, p4.model.xml.request.Header.class,
p4.model.xml.request.Claims.class, p4.model.xml.request.Insurance.class,
p4.model.xml.request.Body.class, p4.model.xml.request.Prescriber.class,
p4.model.xml.request.PriorAuthorization.class,
p4.model.xml.request.PriorAuthorizationSupportingDocumentation.class);
jaxbUnmarshaller = jaxbContext.createUnmarshaller();
StringReader strReader = new StringReader(xml);
Message m = (Message) jaxbUnmarshaller.unmarshal(strReader);
return m;
} catch (JAXBException jaxbe) {
String error = StacktraceUtil.getCustomStackTrace(jaxbe);
LOG.error(error);
throw new P4XMLUnmarshalException("Problems when attempting to unmarshal transmission string: \n" + xml,
jaxbe);
}
}
#Override
public void channelActive(ChannelHandlerContext ctx) {
LOG.debug("{} let server know we are ready", getId());
ctx.writeAndFlush("Ready...\n");
}
/**
* Important - this method will be renamed to
* <code><b>messageReceived(ChannelHandlerContext, I)</b></code> in netty 5.0
*
* #param ctx
* #param msg
*/
#Override
protected void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception {
ChannelFuture lastWriteFuture = null;
LOG.debug("{} -- received message: {}", getId(), msg);
Channel channel = ctx.channel();
Message m = null;
try {
if (msg instanceof String && msg.length() > 0) {
m = convertXmlToMessage(msg);
m.setMessageStr(msg);
dto.setRequestMsg(m);
LOG.info("{}: received TIMESTAMP: {}", dto.getId(), LocalDateTime.now().format(DTFormatter));
LOG.debug("{}: received from server: {}", dto.getId(), msg);
/*
* theoretically we have a complete P4(XML) request
*/
final List<RequestFieldError> errorList = rv.validateMessage(m);
if (!errorList.isEmpty()) {
for (RequestFieldError fe : errorList) {
lastWriteFuture = channel.writeAndFlush(fe.toString().concat(EOL));
}
}
/*
* Create DBHandler with message, messageStr, clientIp to get
* dbResponse
*/
InetSocketAddress socketAddress = (InetSocketAddress) channel.remoteAddress();
InetAddress inetaddress = socketAddress.getAddress();
String clientIp = inetaddress.getHostAddress();
/*
* I know - bad form to ask the ApplicationContext for the
* bean... BUT ...lack of time turns angels into demons
*/
final P4DbRequestHandler dbHandler = (P4DbRequestHandler) AppContext.getBean("dbRequestHandler");
// must set the requestDTO for the dbHandler!
dbHandler.setClientIp(clientIp);
dbHandler.setRequestDTO(dto);
//
// build database request and receive response (string)
String dbResponse = dbHandler.submitDbRequest();
/*
* create ResponseHandler and get back response string
*/
P4ResponseHandler responseHandler = new P4ResponseHandler(dto, dbHandler);
String responseStr = responseHandler.decodeDbServiceResponse(dbResponse);
/*
* write response string to output and repeat exercise
*/
LOG.debug("{} -- response to be written back to server:\n {}", dto.getId(), responseStr);
lastWriteFuture = channel.writeAndFlush(responseStr.concat(EOL));
//
LOG.info("{}: response sent TIMESTAMP: {}", dto.getId(), LocalDateTime.now().format(DTFormatter));
} else {
throw new P4EventException(dto.getId() + " -- Message received is not a String");
}
processWriteFutures(lastWriteFuture);
} catch (Throwable t) {
String tError = StacktraceUtil.getCustomStackTrace(t);
LOG.error(tError);
} finally {
if (lastWriteFuture != null) {
lastWriteFuture.sync();
}
}
}
private void processWriteFutures(ChannelFuture writeFuture) throws InterruptedException {
// Wait until all messages are flushed before closing the channel.
if (writeFuture != null) {
writeFuture.sync();
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
/**
* Creates a newly configured {#link ChannelPipeline} for a new channel.
*/
public class P4ListenerInitializer extends ChannelInitializer<SocketChannel> {
private static final StringDecoder DECODER = new StringDecoder();
private static final StringEncoder ENCODER = new StringEncoder();
private final SslContext sslCtx;
public P4ListenerInitializer(SslContext sslCtx) {
this.sslCtx = sslCtx;
}
#Override
public void initChannel(SocketChannel ch) {
P4ListenerHandler lh = null;
ChannelPipeline pipeline = ch.pipeline();
if (sslCtx != null) {
P4Listener.LOG.info("{} -- constructing SslContext new handler ", P4Listener.getId());
pipeline.addLast(sslCtx.newHandler(ch.alloc(), P4Listener.HOST, P4Listener.PORT));
} else {
P4Listener.LOG.info("{} -- SslContext null; bypassing adding sslCtx.newHandler(ch.alloc(), P4Listener.HOST, P4Listener.PORT) ", P4Listener.getId());
}
// Add the text line codec combination first,
pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
pipeline.addLast(DECODER);
P4Listener.LOG.debug("{} -- added Decoder ", P4Listener.getId());
pipeline.addLast(ENCODER);
P4Listener.LOG.debug("{} -- added Encoder ", P4Listener.getId());
// and then business logic.
pipeline.addLast(lh = new P4ListenerHandler());
P4Listener.LOG.debug("{} -- added P4ListenerHandler: {} ", P4Listener.getId(), lh.getClass().getSimpleName());
}
}
#Sharable
public class P4ListenerOutboundHandler extends ChannelOutboundHandlerAdapter {
static final Logger LOG = LoggerFactory.getLogger(P4ListenerOutboundHandler.class);
private Dto outBoundDTO = new Dto();
public String getId() {return this.outBoundDTO.getId(); }
#Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) {
try {
ChannelFuture lastWrite = ctx.write(Unpooled.copiedBuffer((String) msg, CharsetUtil.UTF_8));
try {
if (lastWrite != null) {
lastWrite.sync();
promise.setSuccess();
}
} catch (InterruptedException e) {
promise.setFailure(e);
e.printStackTrace();
}
} finally {
ReferenceCountUtil.release(msg);
}
}
}
output from client
Just override channelActive(...) on the handler of the server and trigger a write there.

Spring Cloud - Getting Retry Working In RestTemplate?

I have been migrating an existing application over to Spring Cloud's service discovery, Ribbon load balancing, and circuit breakers. The application already makes extensive use of the RestTemplate and I have been able to successfully use the load balanced version of the template. However, I have been testing the situation where there are two instances of a service and I drop one of those instances out of operation. I would like the RestTemplate to failover to the next server. From the research I have done, it appears that the fail-over logic exists in the Feign client and when using Zuul. It appears that the LoadBalancedRest template does not have logic for fail-over. In diving into the code, it looks like the RibbonClientHttpRequestFactory is using the netflix RestClient (which appears to have logic for doing retries).
So where do I go from here to get this working?
I would prefer to not use the Feign client because I would have to sweep A LOT of code.
I had found this link that suggested using the #Retryable annotation along with #HystrixCommand but this seems like something that should be a part of the load balanced rest template.
I did some digging into the code for RibbonClientHttpRequestFactory.RibbonHttpRequest:
protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
try {
addHeaders(headers);
if (outputStream != null) {
outputStream.close();
builder.entity(outputStream.toByteArray());
}
HttpRequest request = builder.build();
HttpResponse response = client.execute(request, config);
return new RibbonHttpResponse(response);
}
catch (Exception e) {
throw new IOException(e);
}
}
It appears that if I override this method and change it to use "client.executeWithLoadBalancer()" that I might be able to leverage the retry logic that is built into the RestClient? I guess I could create my own version of the RibbonClientHttpRequestFactory to do this?
Just looking for guidance on the best approach.
Thanks
To answer my own question:
Before I get into the details, a cautionary tale:
Eureka's self preservation mode sent me down a rabbit hole while testing the fail-over on my local machine. I recommend turning self preservation mode off while doing your testing. Because I was dropping nodes at a regular rate and then restarting (with a different instance ID using a random value), I tripped Eureka's self preservation mode. I ended up with many instances in Eureka that pointed to the same machine, same port. The fail-over was actually working but the next node that was chosen happened to be another dead instance. Very confusing at first!
I was able to get fail-over working with a modified version of RibbonClientHttpRequestFactory. Because RibbonAutoConfiguration creates a load balanced RestTemplate with this factory, rather then injecting this rest template, I create a new one with my modified version of the request factory:
protected RestTemplate restTemplate;
#Autowired
public void customizeRestTemplate(SpringClientFactory springClientFactory, LoadBalancerClient loadBalancerClient) {
restTemplate = new RestTemplate();
// Use a modified version of the http request factory that leverages the load balacing in netflix's RestClient.
RibbonRetryHttpRequestFactory lFactory = new RibbonRetryHttpRequestFactory(springClientFactory, loadBalancerClient);
restTemplate.setRequestFactory(lFactory);
}
The modified Request Factory is just a copy of RibbonClientHttpRequestFactory with two minor changes:
1) In createRequest, I removed the code that was selecting a server from the load balancer because the RestClient will do that for us.
2) In the inner class, RibbonHttpRequest, I changed executeInternal to call "executeWithLoadBalancer".
The full class:
#SuppressWarnings("deprecation")
public class RibbonRetryHttpRequestFactory implements ClientHttpRequestFactory {
private final SpringClientFactory clientFactory;
private LoadBalancerClient loadBalancer;
public RibbonRetryHttpRequestFactory(SpringClientFactory clientFactory, LoadBalancerClient loadBalancer) {
this.clientFactory = clientFactory;
this.loadBalancer = loadBalancer;
}
#Override
public ClientHttpRequest createRequest(URI originalUri, HttpMethod httpMethod) throws IOException {
String serviceId = originalUri.getHost();
IClientConfig clientConfig = clientFactory.getClientConfig(serviceId);
RestClient client = clientFactory.getClient(serviceId, RestClient.class);
HttpRequest.Verb verb = HttpRequest.Verb.valueOf(httpMethod.name());
return new RibbonHttpRequest(originalUri, verb, client, clientConfig);
}
public class RibbonHttpRequest extends AbstractClientHttpRequest {
private HttpRequest.Builder builder;
private URI uri;
private HttpRequest.Verb verb;
private RestClient client;
private IClientConfig config;
private ByteArrayOutputStream outputStream = null;
public RibbonHttpRequest(URI uri, HttpRequest.Verb verb, RestClient client, IClientConfig config) {
this.uri = uri;
this.verb = verb;
this.client = client;
this.config = config;
this.builder = HttpRequest.newBuilder().uri(uri).verb(verb);
}
#Override
public HttpMethod getMethod() {
return HttpMethod.valueOf(verb.name());
}
#Override
public URI getURI() {
return uri;
}
#Override
protected OutputStream getBodyInternal(HttpHeaders headers) throws IOException {
if (outputStream == null) {
outputStream = new ByteArrayOutputStream();
}
return outputStream;
}
#Override
protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
try {
addHeaders(headers);
if (outputStream != null) {
outputStream.close();
builder.entity(outputStream.toByteArray());
}
HttpRequest request = builder.build();
HttpResponse response = client.executeWithLoadBalancer(request, config);
return new RibbonHttpResponse(response);
}
catch (Exception e) {
throw new IOException(e);
}
//TODO: fix stats, now that execute is not called
// use execute here so stats are collected
/*
return loadBalancer.execute(this.config.getClientName(), new LoadBalancerRequest<ClientHttpResponse>() {
#Override
public ClientHttpResponse apply(ServiceInstance instance) throws Exception {}
});
*/
}
private void addHeaders(HttpHeaders headers) {
for (String name : headers.keySet()) {
// apache http RequestContent pukes if there is a body and
// the dynamic headers are already present
if (!isDynamic(name) || outputStream == null) {
List<String> values = headers.get(name);
for (String value : values) {
builder.header(name, value);
}
}
}
}
private boolean isDynamic(String name) {
return name.equals("Content-Length") || name.equals("Transfer-Encoding");
}
}
public class RibbonHttpResponse extends AbstractClientHttpResponse {
private HttpResponse response;
private HttpHeaders httpHeaders;
public RibbonHttpResponse(HttpResponse response) {
this.response = response;
this.httpHeaders = new HttpHeaders();
List<Map.Entry<String, String>> headers = response.getHttpHeaders().getAllHeaders();
for (Map.Entry<String, String> header : headers) {
this.httpHeaders.add(header.getKey(), header.getValue());
}
}
#Override
public InputStream getBody() throws IOException {
return response.getInputStream();
}
#Override
public HttpHeaders getHeaders() {
return this.httpHeaders;
}
#Override
public int getRawStatusCode() throws IOException {
return response.getStatus();
}
#Override
public String getStatusText() throws IOException {
return HttpStatus.valueOf(response.getStatus()).name();
}
#Override
public void close() {
response.close();
}
}
}
I had the same problem but then, out of the box, everything was working (using a #LoadBalanced RestTemplate). I am using Finchley version of Spring Cloud, and I think my problem was that I was not explicity adding spring-retry in my pom configuration. I'll leave here my spring-retry related yml configuration (remember this only works with #LoadBalanced RestTemplate, Zuul of Feign):
spring:
# Ribbon retries on
cloud:
loadbalancer:
retry:
enabled: true
# Ribbon service config
my-service:
ribbon:
MaxAutoRetries: 3
MaxAutoRetriesNextServer: 1
OkToRetryOnAllOperations: true
retryableStatusCodes: 500, 502

GCM XMPP Server using Smack 4.1.0

I'm trying to adapt the example provided here for Smack 4.1.0. and getting a little confused.
Specifically I'm struggling to understand what the GcmPacketExtension should now extend, how the constructor should work and how the Providermanager.addExtensionProvider should be updated to tie in with it.
I'm sure someone must have done this before but I can't find any examples
and I seem to be going round in circles using just the documentation.
Any help would be much appreciated, I'm sure the answer is very simple!
Current Code (is compiling but not running):
static {
ProviderManager.addExtensionProvider(GCM_ELEMENT_NAME, GCM_NAMESPACE, new ExtensionElementProvider<ExtensionElement>() {
#Override
public DefaultExtensionElement parse(XmlPullParser parser,int initialDepth) throws org.xmlpull.v1.XmlPullParserException,
IOException {
String json = parser.nextText();
return new GcmPacketExtension(json);
}
});
}
and:
private static final class GcmPacketExtension extends DefaultExtensionElement {
private final String json;
public GcmPacketExtension(String json) {
super(GCM_ELEMENT_NAME, GCM_NAMESPACE);
this.json = json;
}
public String getJson() {
return json;
}
#Override
public String toXML() {
return String.format("<%s xmlns=\"%s\">%s</%s>",
GCM_ELEMENT_NAME, GCM_NAMESPACE,
StringUtils.escapeForXML(json), GCM_ELEMENT_NAME);
}
public Stanza toPacket() {
Message message = new Message();
message.addExtension(this);
return message;
}
}
Current exception:
Exception in thread "main" java.lang.NoClassDefFoundError: de/measite/minidns/DNSCache
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.jivesoftware.smack.SmackInitialization.loadSmackClass(SmackInitialization.java:213)
at org.jivesoftware.smack.SmackInitialization.parseClassesToLoad(SmackInitialization.java:193)
at org.jivesoftware.smack.SmackInitialization.processConfigFile(SmackInitialization.java:163)
at org.jivesoftware.smack.SmackInitialization.processConfigFile(SmackInitialization.java:148)
at org.jivesoftware.smack.SmackInitialization.<clinit>(SmackInitialization.java:116)
at org.jivesoftware.smack.SmackConfiguration.getVersion(SmackConfiguration.java:96)
at org.jivesoftware.smack.provider.ProviderManager.<clinit>(ProviderManager.java:121)
at SmackCcsClient.<clinit>(SmackCcsClient.java:58)
Caused by: java.lang.ClassNotFoundException: de.measite.minidns.DNSCache
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 10 more
OK, so I managed to get it working after much reading and pain, so here is a VERY crude server implementation that actually works. Obviously not for production and feel free to correct anything that's wrong. I'm not saying this is the best way to do it but it does work. It will send a message and receive messages but only shows them in the log.
import org.jivesoftware.smack.ConnectionConfiguration.SecurityMode;
import org.jivesoftware.smack.ConnectionListener;
import org.jivesoftware.smack.SmackException;
import org.jivesoftware.smack.SmackException.NotConnectedException;
import org.jivesoftware.smack.XMPPConnection;
import org.jivesoftware.smack.XMPPException;
import org.jivesoftware.smack.packet.DefaultExtensionElement;
import org.jivesoftware.smack.packet.Message;
import org.jivesoftware.smack.packet.Stanza;
import org.jivesoftware.smack.filter.StanzaFilter;
import org.jivesoftware.smack.provider.ProviderManager;
import org.jivesoftware.smack.tcp.XMPPTCPConnection;
import org.jivesoftware.smack.tcp.XMPPTCPConnectionConfiguration;
import org.jivesoftware.smack.util.StringUtils;
import org.json.simple.JSONValue;
import org.json.simple.parser.ParseException;
import org.xmlpull.v1.XmlPullParser;
import org.jivesoftware.smack.*;
import org.jivesoftware.smack.packet.ExtensionElement;
import org.jivesoftware.smack.provider.ExtensionElementProvider;
import org.jivesoftware.smack.roster.Roster;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SSLSocketFactory;
/**
* Sample Smack implementation of a client for GCM Cloud Connection Server. This
* code can be run as a standalone CCS client.
*
* <p>For illustration purposes only.
*/
public class SmackCcsClient {
private static final Logger logger = Logger.getLogger("SmackCcsClient");
private static final String GCM_SERVER = "gcm.googleapis.com";
private static final int GCM_PORT = 5235;
private static final String GCM_ELEMENT_NAME = "gcm";
private static final String GCM_NAMESPACE = "google:mobile:data";
private static final String YOUR_PROJECT_ID = "<your ID here>";
private static final String YOUR_API_KEY = "<your API Key here>"; // your API Key
private static final String YOUR_PHONE_REG_ID = "<your test phone's registration id here>";
static {
ProviderManager.addExtensionProvider(GCM_ELEMENT_NAME, GCM_NAMESPACE, new ExtensionElementProvider<ExtensionElement>() {
#Override
public DefaultExtensionElement parse(XmlPullParser parser,int initialDepth) throws org.xmlpull.v1.XmlPullParserException,
IOException {
String json = parser.nextText();
return new GcmPacketExtension(json);
}
});
}
private XMPPTCPConnection connection;
/**
* Indicates whether the connection is in draining state, which means that it
* will not accept any new downstream messages.
*/
protected volatile boolean connectionDraining = false;
/**
* Sends a downstream message to GCM.
*
* #return true if the message has been successfully sent.
*/
public boolean sendDownstreamMessage(String jsonRequest) throws
NotConnectedException {
if (!connectionDraining) {
send(jsonRequest);
return true;
}
logger.info("Dropping downstream message since the connection is draining");
return false;
}
/**
* Returns a random message id to uniquely identify a message.
*
* <p>Note: This is generated by a pseudo random number generator for
* illustration purpose, and is not guaranteed to be unique.
*/
public String nextMessageId() {
return "m-" + UUID.randomUUID().toString();
}
/**
* Sends a packet with contents provided.
*/
protected void send(String jsonRequest) throws NotConnectedException {
Stanza request = new GcmPacketExtension(jsonRequest).toPacket();
connection.sendStanza(request);
}
/**
* Handles an upstream data message from a device application.
*
* <p>This sample echo server sends an echo message back to the device.
* Subclasses should override this method to properly process upstream messages.
*/
protected void handleUpstreamMessage(Map<String, Object> jsonObject) {
// PackageName of the application that sent this message.
String category = (String) jsonObject.get("category");
String from = (String) jsonObject.get("from");
#SuppressWarnings("unchecked")
Map<String, String> payload = (Map<String, String>) jsonObject.get("data");
payload.put("ECHO", "Application: " + category);
// Send an ECHO response back
String echo = createJsonMessage(from, nextMessageId(), payload,
"echo:CollapseKey", null, false);
try {
sendDownstreamMessage(echo);
} catch (NotConnectedException e) {
logger.log(Level.WARNING, "Not connected anymore, echo message is not sent", e);
}
}
/**
* Handles an ACK.
*
* <p>Logs a INFO message, but subclasses could override it to
* properly handle ACKs.
*/
protected void handleAckReceipt(Map<String, Object> jsonObject) {
String messageId = (String) jsonObject.get("message_id");
String from = (String) jsonObject.get("from");
logger.log(Level.INFO, "handleAckReceipt() from: " + from + ",messageId: " + messageId);
}
/**
* Handles a NACK.
*
* <p>Logs a INFO message, but subclasses could override it to
* properly handle NACKs.
*/
protected void handleNackReceipt(Map<String, Object> jsonObject) {
String messageId = (String) jsonObject.get("message_id");
String from = (String) jsonObject.get("from");
logger.log(Level.INFO, "handleNackReceipt() from: " + from + ",messageId: " + messageId);
}
protected void handleControlMessage(Map<String, Object> jsonObject) {
logger.log(Level.INFO, "handleControlMessage(): " + jsonObject);
String controlType = (String) jsonObject.get("control_type");
if ("CONNECTION_DRAINING".equals(controlType)) {
connectionDraining = true;
} else {
logger.log(Level.INFO, "Unrecognized control type: %s. This could happen if new features are " + "added to the CCS protocol.",
controlType);
}
}
/**
* Creates a JSON encoded GCM message.
*
* #param to RegistrationId of the target device (Required).
* #param messageId Unique messageId for which CCS sends an
* "ack/nack" (Required).
* #param payload Message content intended for the application. (Optional).
* #param collapseKey GCM collapse_key parameter (Optional).
* #param timeToLive GCM time_to_live parameter (Optional).
* #param delayWhileIdle GCM delay_while_idle parameter (Optional).
* #return JSON encoded GCM message.
*/
public static String createJsonMessage(String to, String messageId,
Map<String, String> payload, String collapseKey, Long timeToLive,
Boolean delayWhileIdle) {
Map<String, Object> message = new HashMap<String, Object>();
message.put("to", to);
if (collapseKey != null) {
message.put("collapse_key", collapseKey);
}
if (timeToLive != null) {
message.put("time_to_live", timeToLive);
}
if (delayWhileIdle != null && delayWhileIdle) {
message.put("delay_while_idle", true);
}
message.put("message_id", messageId);
message.put("data", payload);
return JSONValue.toJSONString(message);
}
/**
* Creates a JSON encoded ACK message for an upstream message received
* from an application.
*
* #param to RegistrationId of the device who sent the upstream message.
* #param messageId messageId of the upstream message to be acknowledged to CCS.
* #return JSON encoded ack.
*/
protected static String createJsonAck(String to, String messageId) {
Map<String, Object> message = new HashMap<String, Object>();
message.put("message_type", "ack");
message.put("to", to);
message.put("message_id", messageId);
return JSONValue.toJSONString(message);
}
/**
* Connects to GCM Cloud Connection Server using the supplied credentials.
*
* #param senderId Your GCM project number
* #param apiKey API Key of your project
*/
public void connect(String senderId, String apiKey)
throws XMPPException, IOException, SmackException {
XMPPTCPConnectionConfiguration config =
XMPPTCPConnectionConfiguration.builder()
.setServiceName(GCM_SERVER)
.setHost(GCM_SERVER)
.setCompressionEnabled(false)
.setPort(GCM_PORT)
.setConnectTimeout(30000)
.setSecurityMode(SecurityMode.disabled)
.setSendPresence(false)
.setSocketFactory(SSLSocketFactory.getDefault())
.build();
connection = new XMPPTCPConnection(config);
//disable Roster as I don't think this is supported by GCM
Roster roster = Roster.getInstanceFor(connection);
roster.setRosterLoadedAtLogin(false);
logger.info("Connecting...");
connection.connect();
connection.addConnectionListener(new LoggingConnectionListener());
// Handle incoming packets
connection.addAsyncStanzaListener(new MyStanzaListener() , new MyStanzaFilter() );
// Log all outgoing packets
connection.addPacketInterceptor(new MyStanzaInterceptor(), new MyStanzaFilter() );
connection.login(senderId + "#gcm.googleapis.com" , apiKey);
}
private class MyStanzaFilter implements StanzaFilter
{
#Override
public boolean accept(Stanza arg0) {
// TODO Auto-generated method stub
if(arg0.getClass() == Stanza.class )
return true;
else
{
if(arg0.getTo()!= null)
if(arg0.getTo().startsWith(YOUR_PROJECT_ID) )
return true;
}
return false;
}
}
private class MyStanzaListener implements StanzaListener{
#Override
public void processPacket(Stanza packet) {
logger.log(Level.INFO, "Received: " + packet.toXML());
Message incomingMessage = (Message) packet;
GcmPacketExtension gcmPacket =
(GcmPacketExtension) incomingMessage.
getExtension(GCM_NAMESPACE);
String json = gcmPacket.getJson();
try {
#SuppressWarnings("unchecked")
Map<String, Object> jsonObject =
(Map<String, Object>) JSONValue.
parseWithException(json);
// present for "ack"/"nack", null otherwise
Object messageType = jsonObject.get("message_type");
if (messageType == null) {
// Normal upstream data message
handleUpstreamMessage(jsonObject);
// Send ACK to CCS
String messageId = (String) jsonObject.get("message_id");
String from = (String) jsonObject.get("from");
String ack = createJsonAck(from, messageId);
send(ack);
} else if ("ack".equals(messageType.toString())) {
// Process Ack
handleAckReceipt(jsonObject);
} else if ("nack".equals(messageType.toString())) {
// Process Nack
handleNackReceipt(jsonObject);
} else if ("control".equals(messageType.toString())) {
// Process control message
handleControlMessage(jsonObject);
} else {
logger.log(Level.WARNING,
"Unrecognized message type (%s)",
messageType.toString());
}
} catch (ParseException e) {
logger.log(Level.SEVERE, "Error parsing JSON " + json, e);
} catch (Exception e) {
logger.log(Level.SEVERE, "Failed to process packet", e);
}
}
}
private class MyStanzaInterceptor implements StanzaListener
{
#Override
public void processPacket(Stanza packet) {
logger.log(Level.INFO, "Sent: {0}", packet.toXML());
}
}
public static void main(String[] args) throws Exception {
SmackCcsClient ccsClient = new SmackCcsClient();
ccsClient.connect(YOUR_PROJECT_ID, YOUR_API_KEY);
// Send a sample hello downstream message to a device.
String messageId = ccsClient.nextMessageId();
Map<String, String> payload = new HashMap<String, String>();
payload.put("Message", "Ahha, it works!");
payload.put("CCS", "Dummy Message");
payload.put("EmbeddedMessageId", messageId);
String collapseKey = "sample";
Long timeToLive = 10000L;
String message = createJsonMessage(YOUR_PHONE_REG_ID, messageId, payload,
collapseKey, timeToLive, true);
ccsClient.sendDownstreamMessage(message);
logger.info("Message sent.");
//crude loop to keep connection open for receiving messages
while(true)
{;}
}
/**
* XMPP Packet Extension for GCM Cloud Connection Server.
*/
private static final class GcmPacketExtension extends DefaultExtensionElement {
private final String json;
public GcmPacketExtension(String json) {
super(GCM_ELEMENT_NAME, GCM_NAMESPACE);
this.json = json;
}
public String getJson() {
return json;
}
#Override
public String toXML() {
return String.format("<%s xmlns=\"%s\">%s</%s>",
GCM_ELEMENT_NAME, GCM_NAMESPACE,
StringUtils.escapeForXML(json), GCM_ELEMENT_NAME);
}
public Stanza toPacket() {
Message message = new Message();
message.addExtension(this);
return message;
}
}
private static final class LoggingConnectionListener
implements ConnectionListener {
#Override
public void connected(XMPPConnection xmppConnection) {
logger.info("Connected.");
}
#Override
public void reconnectionSuccessful() {
logger.info("Reconnecting..");
}
#Override
public void reconnectionFailed(Exception e) {
logger.log(Level.INFO, "Reconnection failed.. ", e);
}
#Override
public void reconnectingIn(int seconds) {
logger.log(Level.INFO, "Reconnecting in %d secs", seconds);
}
#Override
public void connectionClosedOnError(Exception e) {
logger.info("Connection closed on error.");
}
#Override
public void connectionClosed() {
logger.info("Connection closed.");
}
#Override
public void authenticated(XMPPConnection arg0, boolean arg1) {
// TODO Auto-generated method stub
}
}
}
I also imported the following external JARs:
(they might not all be required but most are!)
json-simple-1.1.1.jar
jxmpp-core-0.4.1.jar
jxmpp-util-cache-0.5.0-alpha2.jar
minidns-0.1.3.jar
commons-logging-1.2.jar
httpclient-4.3.4.jar
xpp3_xpath-1.1.4c.jar
xpp3-1.1.4c.jar
For the Client I used the GCM sample project here. (Scroll to the bottom of the page for the source link)
Hope this helps someone!
[23-Oct-2015] I'm editing this answer for others who are using Gradle ... below is all the dependencies that I needed to get this to compile (add to the bottom of your build.gradle file).
dependencies {
compile 'com.googlecode.json-simple:json-simple:1.1.1'
compile 'org.igniterealtime.smack:smack-java7:4.1.4'
compile 'org.igniterealtime.smack:smack-tcp:4.1.4'
compile 'org.igniterealtime.smack:smack-im:4.1.4'
compile 'org.jxmpp:jxmpp-core:0.5.0-alpha6'
compile 'org.jxmpp:jxmpp-util-cache:0.5.0-alpha6'
}
There are two reference implementations provided by Google for the GCM Cloud Connection Server (XMPP endpoint).
Both are here:
https://github.com/googlesamples/friendlyping/tree/master/server
The Java server uses the Smack XMPP library.
The Go server uses Google's own go-gcm library - https://github.com/google/go-gcm
The Go server is also used in the GCM Playground example - https://github.com/googlesamples/gcm-playground - so it seems like the Go server may be preferred by Google. Being Go, it can be deployed without any dependencies, which is an advantage over the Java server.