ConnectionPoolTimeoutException: timeout waiting for connection from pool - httpclient

I have app that keep calling an api for json data, and pretty quickly i saw this 'Timeout waiting for connection from pool' exception, I googled around and found that's a connection leak caused by content not consumed, so i updated the code to close the inputstream after consume the jsondata, but still got those connection timeout exception, here is my code:
InputStream is = ApiUtil.getAsStream(Api.get(bookUrl).param("limit","500").param("bookId", bid).enable(Options.LongRunning), 3);
List<JsonBook> books = mapper.readValue(is, BOOK_TYPE);
is.close()
Api:
private JsonHttpClient client;
public APIForGet get(String endpoint) {
return new APIForGet(this.client, endpoint);
}
ApiUtil:
public static InputStream getAsStream(APIForGet get, Iterable<Long>retries) {
return get.asStream();
}
APIForGet:
private JsonHttpClient client;
public InputStream asStream() {
return this.client.getAsStream(this.hostname, this.port, this.endpoint, params, optionsArray());
}
JsonHttpClientImpl:
public InputStream getAsStream(Optional<String> host, Optional<Integer> port,String path, Multimap<String, String> param, Options ... options) {
HttpResponse reponse;
try {
response = request.execute();
if (response.isSuccessStatusCode()) {
return response.getContent();
}
} catch (Exception e) {
throw new Exception();
}
}
the wrapping logic here is kind of complicated, but eventually i think by closing the inputstream should work, any thoughts? Thanks!

try {
response = request.execute();
if (response.isSuccessStatusCode()) {
return response.getContent();
}
} catch (Exception e) {
throw new Exception();
}finally{
//TODO release conn or abort
}

Related

ConnectionFactory throwing errors when shared

I've a very simple application that adds messages to a queue and reads them using a MessagerListener.
Edit: I was testing this on a single instance of Artemis that had been setup as part of a two instance cluster on docker.
I want to create the ConnectionFactory once and reuse it for all producers and consumers in the application.
I have created the ConnectionFactory and stored it in a static variable (singleton) so it can be accessed from anywhere.
The aim is that the client use this shared connection factory to create a new connection when required.
However, I have noticed that doing this causes a "Failed to create session factory" when trying to create a new connection.
javax.jms.JMSException: Failed to create session factory
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:886)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:299)
at com.test.artemistest.jms.QueueTest2.getMessagesFromQueue(QueueTest2.java:137)
at com.test.artemistest.jms.QueueTest2.access$000(QueueTest2.java:61)
at com.test.artemistest.jms.QueueTest2$1.run(QueueTest2.java:75)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ219007: Cannot connect to server(s). Tried with all available servers.]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:690)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:884)
If I create a connection factory per call this error does not occur.
Doing this seems very inefficient.
I've recreated a similar issue below.
If I create the connection factory in the main method the error occurs.
However if created just before use in a method it works as expected.
If I add two listeners the error occurs even though they are in separate threads. Could it be linked to the fact the connections are not closed in the consumers but are in the producers?
Why is this the case and do you recommend sharing the connection factory?
Thanks
public class QueueTest2 {
private static boolean shutdown = false;
private static ConnectionFactory cf;
public static void main(String[] args) {
// uncomment below for error to occur
// QueueTest2.getConnectionFactory("localhost", 61616);
ExecutorService executor = Executors.newCachedThreadPool();
executor.execute(new Runnable() {
#Override
public void run() {
getMessagesFromQueue("localhost", 61616);
while (!shutdown) {
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("getMessagesFromQueue shutdown");
}
});
addMessagesToQueue("localhost", 61616);
// uncommenting below also causes the issue
// executor.execute(new Runnable() {
// #Override
// public void run() {
// getMessagesFromQueue("localhost", 61616);
// while (!shutdown) {
// try {
// Thread.sleep(1000L);
// } catch (InterruptedException e) {
// e.printStackTrace();
// }
// }
// System.out.println("getMessagesFromQueue shutdown");
// }
// });
addMessagesToQueue("localhost", 61616);
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
e.printStackTrace();
}
shutdown = true;
executor.shutdownNow();
}
private static void addMessagesToQueue(String host, int port) {
ConnectionFactory cf2 = getConnectionFactory(host, port);
Connection connection = null;
Session sessionQueue = null;
try {
connection = cf2.createConnection("artemis", "password");
connection.setClientID("Producer");
sessionQueue = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
Queue orderQueue = sessionQueue.createQueue("exampleQueue");
MessageProducer producerQueue = sessionQueue.createProducer(orderQueue);
connection.start();
// send 100 messages
for (int i = 0; i < 100; i++) {
TextMessage message = sessionQueue.createTextMessage("This is an order: " + i);
producerQueue.send(message);
}
} catch (JMSException ex) {
Logger.getLogger(QueueTest2.class.getName()).log(Level.SEVERE, null, ex);
} finally {
try {
if (sessionQueue != null) {
sessionQueue.close();
}
} catch (JMSException ex) {
Logger.getLogger(QueueTest2.class.getName()).log(Level.SEVERE, null, ex);
}
try {
if (connection != null) {
connection.close();
}
} catch (JMSException ex) {
Logger.getLogger(QueueTest2.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
private static void getMessagesFromQueue(String host, int port) {
ConnectionFactory cf2 = getConnectionFactory(host, port);
Connection connection2 = null;
Session sessionQueue2;
try {
connection2 = cf2.createConnection("artemis", "password");
connection2.setClientID("Consumer2");
sessionQueue2 = connection2.createSession(false, Session.CLIENT_ACKNOWLEDGE);
Queue orderQueue = sessionQueue2.createQueue("exampleQueue");
MessageConsumer consumerQueue = sessionQueue2.createConsumer(orderQueue);
consumerQueue.setMessageListener(new MessageHandlerTest2());
connection2.start();
Thread.sleep(5000);
} catch (JMSException ex) {
Logger.getLogger(QueueTest2.class.getName()).log(Level.SEVERE, null, ex);
} catch (InterruptedException ex) {
Logger.getLogger(QueueTest2.class.getName()).log(Level.SEVERE, null, ex);
}
}
private static ConnectionFactory getConnectionFactory(String host, int port) {
if (cf == null) {
Map<String, Object> connectionParams2 = new HashMap<String, Object>();
connectionParams2.put(TransportConstants.PORT_PROP_NAME, port);
connectionParams2.put(TransportConstants.HOST_PROP_NAME, host);
TransportConfiguration transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class
.getName(), connectionParams2);
cf = ActiveMQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.CF, transportConfiguration);
}
return cf;
}
}
class MessageHandlerTest2 implements MessageListener {
#Override
public void onMessage(Message message) {
try {
System.out.println("new message: " + ((TextMessage) message).getText());
message.acknowledge();
} catch (JMSException ex) {
Logger.getLogger(MessageHandlerTest2.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
I've run your code, but I don't see any errors. My guess is that there may be a timing issue related to concurrency. Try adding synchronized to your getConnectionFactory method since it can theoretically be called concurrently by multiple threads in your application, e.g.:
private synchronized static ConnectionFactory getConnectionFactory(String host, int port)
I have found a solution that works on a clustered environment and docker.
It involves using the "pooled-jms" connection pool. Something I had planned to use anyway.
Although it does not explain the issues I was seeing above, it is at least a work around until I can investigate further.
The "WARN: AMQ212064: Unable to receive cluster topology " mentioned above appears to have been a red herring as it went away as quickly as it appeared.

Netty connection pool not sending messages to server

I have a simple netty connection pool and a simple HTTP endpoint to use that pool to send TCP messages to ServerSocket. The relevant code looks like this, the client (NettyConnectionPoolClientApplication) is:
#SpringBootApplication
#RestController
public class NettyConnectionPoolClientApplication {
private SimpleChannelPool simpleChannelPool;
public static void main(String[] args) {
SpringApplication.run(NettyConnectionPoolClientApplication.class, args);
}
#PostConstruct
public void setup() throws Exception {
EventLoopGroup group = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap();
bootstrap.group(group);
bootstrap.channel(NioSocketChannel.class);
bootstrap.option(ChannelOption.SO_KEEPALIVE, true);
bootstrap.remoteAddress(new InetSocketAddress("localhost", 9000));
bootstrap.handler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
pipeline.addLast(new StringDecoder());
pipeline.addLast(new StringEncoder());
pipeline.addLast(new DummyClientHandler());
}
});
simpleChannelPool = new SimpleChannelPool(bootstrap, new DummyChannelPoolHandler());
}
#RequestMapping("/test/{msg}")
public void test(#PathVariable String msg) throws Exception {
Future<Channel> future = simpleChannelPool.acquire();
future.addListener((FutureListener<Channel>) f -> {
if (f.isSuccess()) {
System.out.println("Connected");
Channel ch = f.getNow();
ch.writeAndFlush(msg + System.lineSeparator());
// Release back to pool
simpleChannelPool.release(ch);
} else {
System.out.println("not successful");
}
});
}
}
and the Server (ServerSocketRunner)
public class ServerSocketRunner {
public static void main(String[] args) throws Exception {
ServerSocket serverSocket = new ServerSocket(9000);
while (true) {
Socket socket = serverSocket.accept();
new Thread(() -> {
System.out.println("New client connected");
try (PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
BufferedReader in = new BufferedReader(
new InputStreamReader(socket.getInputStream()));) {
String inputLine, outputLine;
out.println("Hello client!");
do {
inputLine = in.readLine();
System.out.println("Received: " + inputLine);
} while (!"bye".equals(inputLine));
System.out.println("Closing connection...");
socket.close();
} catch (Exception e) {
e.printStackTrace();
}
}).start();
}
}
}
DummyChannelPoolHandler and DummyClientHandler just print out events that happen, so they are not relevant. When the server and the client are started and I send a test message to test endpoint, I can see the server prints "New client connected" but the message sent by client is not printed. None of the consecutive messages sent by client are printed by the server.
If I try telnet, everything works fine, the server prints out messages. Also it works fine with regular netty client with same bootstrap config and without connection pool (SimpleNettyClientApplication).
Can anyone see what is wrong with my connection pool, I'm out of ideas
Netty versioin: 4.1.39.Final
All the code is available here.
UPDATE
Following Norman Maurer advice. I added
ChannelFuture channelFuture = ch
.writeAndFlush(msg + System.lineSeparator());
channelFuture.addListener(writeFuture -> {
System.out
.println("isSuccess(): " + channelFuture.isSuccess() + " : " + channelFuture.cause());
});
This prints out
isSuccess: false : java.lang.UnsupportedOperationException: unsupported message type: String (expected: ByteBuf, FileRegion)
To fix it, I just converted String into ByteBuf
ch.writeAndFlush(Unpooled.wrappedBuffer((msg + System.lineSeparator()).getBytes()));
You should check what the status of the ChannelFuture is that is returned by writeAndFlush(...). I suspect it is failed.

xmpp file upload using smack

i am unable to upload file on xmpp using smack client android. slot.puturl() returns "https://localhost:7443/httpfileupload/27c97df7-dbbf-47ff-b19a-3ac624e51cf0/1.jpg"
HttpFileUploadManager manager = HttpFileUploadManager.getInstanceFor(mConnection);
try {
Slot slot = manager.requestSlot(path, 10000);
uploadFileToSlot(new File(path), slot);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (XMPPException.XMPPErrorException e) {
e.printStackTrace();
} catch (SmackException e) {
e.printStackTrace();
}
I got solution for after very deep research.
That HttpFileUploadManager is only for requesting slot from server.
Once you got slot request url upload file using httpclient or okhttpclient.
For okhttpclient:
You need to configure sslSocketFactory by mtm or certificatepinning.
OkHttpClient.Builder okHttpClientBuilder = new OkHttpClient().newBuilder();
SSLContext sslContext = JavaPinning.forPin(<PINNING_VALUE>);
okHttpClientBuilder.writeTimeout(5, TimeUnit.MINUTES)
.connectTimeout(5, TimeUnit.MINUTES)
.readTimeout(5, TimeUnit.MINUTES);
okHttpClientBuilder.sslSocketFactory(sslContext.getSocketFactory(), JavaPinning.trustManagerForPin(<PINNING_VALUE>));
OkHttpClient client = okHttpClientBuilder.build();
initiate okhttpclient and add file like.
Request request = new Request.Builder()
.url(slot.getPutUrl())
.put(RequestBody.create(MediaType.parse("application/octet-stream"), files))
.build();
Now lets begin to upload.
client.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(final Call call, final IOException e) {
// Handle the error
Log.i(log, "error " + e);
}
#Override
public void onResponse(final Call call, final Response response) throws IOException {
if (!response.isSuccessful()) {
// Handle the error
Log.i(log, "errored " + response);
}
Log.i(log, "success " + response);
// Upload successful
}
});
Hope it helps you.

Propogate errors to UI with Spring 3 MVC / REST

When /api/upload REST endpoint is accessed I have a UploadController that uses a service UploadService to upload a file to an ftp server with org.apache.commons.net.ftp.FTPClient. I would like to be able to send information back to the user if the ftp client was unable to connect or timed out, or successfully sent the file. I have some IOException handling, but I don't know how to turn that around and send it back to the front-end. Any help appreciated, thanks!
public void upload(InputStream inputStream) {
String filename = "file.txt"
client = new FTPClient();
try {
client.connect("ftpsite");
client.login("username", "password");
client.storeFile(filename, inputStream);
} catch (IOException ioe) {
ioe.printStackTrace();
} finally {
try {
if (inputStream!= null) {
inputStream.close();
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
return null;
}
You should throw a new Exception in your catch statement.
For example, you could create a RequestTimeoutException class:
#ResponseStatus(HttpStatus.REQUEST_TIMEOUT)
public class RequestTimeoutException extends RuntimeException { }
and then throw it when need be:
catch (IOException ioe) {
//do some logging while you're at it
throw new RequestTimeoutException();
}

linking my applet to a server dirctory to recieve or save a file from there?

I' m looking for a code to save the files created in a applet normally text files i want to save them on a server directory how can i do so.
Here is an example of how to send a String. In fact any Object can be sent this method so long as it's serializable and the same version of the Object exists on both the applet and the servlet.
To send from the applet
public void sendSomeString(String someString) {
ObjectOutputStream request = null;
try {
URL servletURL = new URL(getCodeBase().getProtocol(),
getCodeBase().getHost(),
getCodeBase().getPort(),
"/servletName");
// open the connection
URLConnection con = servletURL.openConnection();
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestProperty("Content-Type", "application/octet-stream");
// send the data
request =
new ObjectOutputStream(
new BufferedOutputStream(con.getOutputStream()));
request.writeObject(someString);
request.flush();
// performs the connection
new ObjectInputStream(new BufferedInputStream(con.getInputStream()));
} catch (Exception e) {
System.err.println("" + e);
} finally {
if (request != null) {
try {
request.close();
} catch (Exception e) {
System.err.println("" + e);
};
}
}
}
To retrieve on the server side
#Override
public void doPost(HttpServletRequest request, HttpServletResponse response) {
try {
// get the input stream
ObjectInputStream inputStream = new ObjectInputStream(
new BufferedInputStream(request.getInputStream()));
String someString = (String)inputStream.readObject();
ObjectOutputStream oos = new ObjectOutputStream(
new BufferedOutputStream(response.getOutputStream()));
oos.flush();
// handle someString....
} catch (SocketException e) {
// ignored, occurs when connection is terminated
} catch (IOException e) {
// ignored, occurs when connection is terminated
} catch (Exception e) {
log.error("Exception", e);
}
}
No one is going to hand you this on a plate. You have to write code in your applet to make a socket connection back to your server and send the data. One way to approach this is to push the data via HTTP, and use a library such as commons-httpclient. That requires your server to handle the appropriate HTTP verb.
There are many other options, and the right one will depend on the fine details of the problem you are trying to solve.