Debugging a standalone jetty server - how to specify single threaded mode? - scala

I have successfully created a standalone Scalatra / Jetty server, using the official instructions from Scalatra ( http://www.scalatra.org/2.3/guides/deployment/standalone.html )
I am debugging it under Ensime, and would like to limit the number of threads handling messages to a single one - so that single-stepping through the servlet methods will be easier.
I used this code to achieve it:
package ...
import org.eclipse.jetty.server.Server
import org.eclipse.jetty.servlet.{DefaultServlet, ServletContextHandler}
import org.eclipse.jetty.webapp.WebAppContext
import org.scalatra.servlet.ScalatraListener
import org.eclipse.jetty.util.thread.QueuedThreadPool
import org.eclipse.jetty.server.ServerConnector
object JettyLauncher {
def main(args: Array[String]) {
val port =
if (System.getenv("PORT") != null)
System.getenv("PORT").toInt
else
4080
// DEBUGGING MODE BEGINS
val threadPool = new QueuedThreadPool()
threadPool.setMaxThreads(8)
val server = new Server(threadPool)
val connector = new ServerConnector(server)
connector.setPort(port)
server.setConnectors(Array(connector))
// DEBUGGING MODE ENDS
val context = new WebAppContext()
context setContextPath "/"
context.setResourceBase("src/main/webapp")
context.addEventListener(new ScalatraListener)
context.addServlet(classOf[DefaultServlet], "/")
server.setHandler(context)
server.start
server.join
}
}
It works fine - except for one minor detail...
I can't tell Jetty to use 1 thread - the minimum value is 8!
If I do, this is what happens:
$ sbt assembly
...
$ java -jar ./target/scala-2.11/CurrentVersions-assembly-0.1.0-SNAPSHOT.jar
18:13:27.059 [main] INFO org.eclipse.jetty.util.log - Logging initialized #41ms
18:13:27.206 [main] INFO org.eclipse.jetty.server.Server - jetty-9.1.z-SNAPSHOT
18:13:27.220 [main] WARN o.e.j.u.component.AbstractLifeCycle - FAILED org.eclipse.jetty.server.Server#1ac539f: java.lang.IllegalStateException: Insufficient max threads in ThreadPool: max=1 < needed=8
java.lang.IllegalStateException: Insufficient max threads in ThreadPool: max=1 < needed=8
...which is why you see setMaxThreads(8) instead of setMaxThreads(1) in my code above.
Any ideas why this happens?

The reason is that the size of the threadpool also depends on th enumber of connectors you've got defined. If you look at the source code of the jetty server you'll see this:
// check size of thread pool
SizedThreadPool pool = getBean(SizedThreadPool.class);
int max=pool==null?-1:pool.getMaxThreads();
int selectors=0;
int acceptors=0;
if (mex.size()==0)
{
for (Connector connector : _connectors)
{
if (connector instanceof AbstractConnector)
acceptors+=((AbstractConnector)connector).getAcceptors();
if (connector instanceof ServerConnector)
selectors+=((ServerConnector)connector).getSelectorManager().getSelectorCount();
}
}
int needed=1+selectors+acceptors;
if (max>0 && needed>max)
throw new IllegalStateException(String.format("Insufficient threads: max=%d < needed(acceptors=%d + selectors=%d + request=1)",max,acceptors,selectors));
So the minimum with a single serverconnector is 2. It looks like you've got a couple of other default connectors or selectors running.

Related

testcontainer initializationError while running a test suite

I have multiple test classes running the same docker-compose with testcontainer.
The suite fails with initializationError although each test passes when performed separately.
Here is the relevant part of the stacktrace occuring during the second test.
./gradlew e2e:test -i
io.foo.e2e.AuthTest > initializationError FAILED
org.testcontainers.containers.ContainerLaunchException: Container startup failed
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:330)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
at org.testcontainers.containers.DockerComposeContainer.startAmbassadorContainers(DockerComposeContainer.java:331)
at org.testcontainers.containers.DockerComposeContainer.start(DockerComposeContainer.java:178)
at io.foo.e2e.bases.BaseE2eTest$Companion.beforeAll$e2e(BaseE2eTest.kt:62)
at io.foo.e2e.bases.BaseE2eTest.beforeAll$e2e(BaseE2eTest.kt)
...
Caused by:
org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323)
... 83 more
Caused by:
org.testcontainers.containers.ContainerLaunchException: Could not create/start container
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:497)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
... 84 more
Caused by:
org.testcontainers.containers.ContainerLaunchException: Aborting attempt to link to container btraq5fzahac_worker_1 as it is not running
at org.testcontainers.containers.GenericContainer.applyConfiguration(GenericContainer.java:779)
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:359)
... 86 more
It seems to me that the second test doesn't wait for the first to shutdown previous containers.
Here the base class that all tests inherit from. It is responsible for spinning up the containers.
open class BaseE2eTest {
...
companion object {
const val A = "containera_1"
const val B = "containerb_1"
const val C = "containerc_1"
val dockerCompose: KDockerComposeContainer by lazy {
defineDockerCompose()
.withLocalCompose(true)
.withExposedService(A, 8080, Wait.forListeningPort())
.withExposedService(B, 8081)
.withExposedService(C, 5672, Wait.forListeningPort())
}
class KDockerComposeContainer(file: File) : DockerComposeContainer<KDockerComposeContainer>(file)
private fun defineDockerCompose() = KDockerComposeContainer(File("../docker-compose.yml"))
#BeforeAll
#JvmStatic
internal fun beforeAll() {
dockerCompose.start()
}
#AfterAll
#JvmStatic
internal fun afterAll() {
dockerCompose.stop()
}
}
}
docker-compose version 1.27.4, build 40524192
testcontainer 1.15.2
testcontainers:junit-jupiter:1.15.2
After watching this talk, I realized that my testcontainers instantiation approach with Junit5 was wrong.
Here is the working code:
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
open class BaseE2eTest {
...
val A = "containera_1"
val B = "containerb_1"
val C = "containerc_1"
val dockerCompose: KDockerComposeContainer by lazy {
defineDockerCompose()
.withLocalCompose(true)
.withExposedService(A, 8080, Wait.forListeningPort())
.withExposedService(B, 8081)
.withExposedService(C, 5672, Wait.forListeningPort())
}
class KDockerComposeContainer(file: File) : DockerComposeContainer<KDockerComposeContainer>(file)
private fun defineDockerCompose() = KDockerComposeContainer(File("../docker-compose.yml"))
#BeforeAll
fun beforeAll() {
dockerCompose.start()
}
#AfterAll
fun afterAll() {
dockerCompose.stop()
}
}
Now the test suite passes.
For anyone stumbling upon this, I had a similar problem that suddenly occurred after months of working perfectly fine. It was caused by this problem: Docker "ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"
After removing all unused networks the problem was fixed

Asterisk-Java Unable to load Properties file

I'm developing a very simple Asterisk-Java IVR based program that greets the caller, retrieves some information from a web service, reads up the retrieved data to the caller, and finally hangs up.
What steps I followed:
Added the following line is entered on extensions_custom.conf:
exten => 1000,n,Agi(agi://192.168.0.8/ivryobi.agi)
Created the following file structure inside C:\Project\target\classes\
Runnable.java
IvrYobi.java
Runnable.class
IvrYobi.class
fastagi-mapping.properties
Inside fastagi-mapping.properties I have:
ivryobi.agi = main.IvrYobi
The contens of IvrYoby are:
public class IvrYobi extends BaseAgiScript {
public void service(AgiRequest request, AgiChannel channel) throws AgiException {
String callerMsisdn = request.getCallerIdNumber();
}
When it works normally
Running the following command in the console
C:\Project\target\classes>java -cp
asterisk-java.jar;commons-lang3-3.10.jar;commons-logging-1.2.jar;httpclient-4.5.12.jar;httpcore-4.4.13.jar;mysql-connector-java-8.0.20.jar;.
org.asteriskjava.fastagi.DefaultAgiServer
As you can see on the following console output, works perfectly
jun 30, 2020 6:09:04 PM org.asteriskjava.fastagi.DefaultAgiServer
startup INFORMACIËN: Listening on *:4573. jun 30, 2020 6:09:09 PM
org.asteriskjava.fastagi.AbstractAgiServer getPool INFORMACIËN: Thread
pool started. jun 30, 2020 6:09:09 PM
org.asteriskjava.fastagi.ResourceBundleMappingStrategy
loadResourceBundle INFORMACIËN: Added mapping for 'ivryobi.agi' to
class IvrYobi ...
When the problem appears
When I run the very same code, but insted of the console I use Runnable.java
Here are the contents of Runnable.java:
DefaultAgiServer server = new DefaultAgiServer();
public MyRunnable() {
ClassNameMappingStrategy strategy = new ClassNameMappingStrategy(false);
server.setMappingStrategy(strategy);
}
public void run() {
try {
server.startup();
} catch (IllegalStateException | IOException e) {
e.printStackTrace();
server.shutdown();
}
}
public void stop() {
server.shutdown();
}
We can observe the following error on Eclipse's console:
0 [main] DEBUG org.asteriskjava.fastagi.DefaultAgiServer - Using
channelFactory
org.asteriskjava.fastagi.internal.DefaultAgiChannelFactory 9 [main]
INFO org.asteriskjava.fastagi.DefaultAgiServer - Listening on *:4573.
4806 [main] DEBUG org.asteriskjava.fastagi.DefaultAgiServer -
Received connection from /192.168.0.254 4810 [main] INFO
org.asteriskjava.fastagi.DefaultAgiServer - Thread pool started. 4849
[AJ DaemonPool-1.1] DEBUG
org.asteriskjava.fastagi.ClassNameMappingStrategy - Unable to create
AgiScript instance of type ivryobi.agi: Class not found, make sure the
class exists and is available on the CLASSPATH 4849 [AJ
DaemonPool-1.1] ERROR
org.asteriskjava.fastagi.internal.FastAgiConnectionHandler - No
script configured for URL 'agi://192.168.0.8/ivryobi.agi' (script
'ivryobi.agi')
Attempted troubleshooting
I already made sure that fastagi-mapping.properties is on the CLASSPATH.
Tried different name and case
Copied the .properties file on the java Execution Path
Compiled the project as an executable jar
Added / removed packages inside eclipse (ex: com.test.IvrYobi) and also applied the changes on the .properties file.
I checked the code inside asterisk-java-3.5.0.jar, looks like that in case that the configuration file is not found, it just continues without throwing any warning. Since is packed inside the jar I'm unable to modify that code.
Please, do you have any other ideas I can try?
Finally found the solution by myself
I had to recompile asterisk-java.jar using the project's source code
On DefaultAgiServer.java change the line:
resourceBundle = ResourceBundle.getBundle(configResourceBundleName);
With:
FileInputStream fis = new FileInputStream("myivr.properties");
resourceBundle = new PropertyResourceBundle(fis);
logger.info("resourceBundle cargado ok!");
On the catch, replace the
return;
with a more decent response, so you will know if the resource could not be loaded
logger.info("resourceBundle cargado ok!");
}
catch (MissingResourceException | IOException e)
{
logger.error("No existe el recurso invocado: " + e.getMessage());
return;
}

Bridge startup error connecting to Zookeeper

Just getting the following error trying to setup the Bridge component using Zookeeper, according to the steps described in https://docs.corda.r3.com/website/releases/3.1/bridge-configuration-file.html?highlight=zookeeper.
> java -jar corda-bridgeserver-3.1.jar
BridgeSupervisorService: active = false
[ERROR] 20:59:31-0300 [main-EventThread] imps.EnsembleTracker.processConfigData - Invalid config event received: {server.1=10.102.32.104:2888:3888:participant, version=100000000, server.3=10.102.32.108:2888:3888:participant, server.2=10.102.32.107:2888:3888:participant}
[ERROR] 20:59:32-0300 [main-EventThread] imps.EnsembleTracker.processConfigData - Invalid config event received: {server.1=10.102.32.104:2888:3888:participant, version=100000000, server.3=10.102.32.108:2888:3888:participant, server.2=10.102.32.107:2888:3888:participant}
My bridge.conf:
bridgeMode = BridgeInner
outboundConfig {
artemisBrokerAddress = "10.102.32.97:10010"
alternateArtemisBrokerAddresses = [ "10.102.32.98:10010" ]
}
bridgeInnerConfig {
floatAddresses = ["10.102.32.103:12005", "10.102.32.105:12005"]
expectedCertificateSubject = "CN=Float Local,O=Local Only,L=London,C=GB"
customSSLConfiguration {
keyStorePassword = "bridgepass"
trustStorePassword = "trustpass"
sslKeystore = "./bridgecerts/bridge.jks"
trustStoreFile = "./bridgecerts/trust.jks"
crlCheckSoftFail = true
}
}
haConfig {
haConnectionString = "zk://10.102.32.104:2181,zk://10.102.32.107:2181,zk://10.102.32.108:2181"
}
networkParametersPath = ./network-parameters
Any thoughts?
This error is harmless. It indicates that the Dockerised Zookeeper has bad IP addresses, so when the Apache Curator is sent the dynamic topology, some checks fail. It does not invalidate the static configuration and everything should work fine.
Note that as of Corda Enterprise 3.2, you must use the Zookeeper version that is compatible with the Apache Curator library, which is 3.5.3-beta, and NOT the latest version.

How to publish data to using MQTT

I used this docker image to install Mosquitto MQTT.
Now it's running and showing the following message in the terminal:
1515680808: mosquitto version 1.4.14 (build date Mon, 10 Jul 2017 23:48:43 +0100) starting
1515680808: Config loaded from /mqtt/config/mosquitto.conf.
1515680808: Opening websockets listen socket on port 9001.
1515680808: Opening ipv4 listen socket on port 1883.
1515680808: Opening ipv6 listen socket on port 1883.
Then I created a simple Maven project:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-mqtt_2.11</artifactId>
<version>1.6.3</version>
</dependency>
I tried to publish some data to a topic using the code shown below. I point to localhost:1883 as the MqttBrokerUrl and a topic test. However, I get this error:
Exception in thread "main" java.lang.NullPointerException at
org.eclipse.paho.client.mqttv3.MqttConnectOptions.validateURI(MqttConnectOptions.java:457)
at
org.eclipse.paho.client.mqttv3.MqttAsyncClient.(MqttAsyncClient.java:273)
at
org.eclipse.paho.client.mqttv3.MqttAsyncClient.(MqttAsyncClient.java:167)
at
org.eclipse.paho.client.mqttv3.MqttClient.(MqttClient.java:224)
at org.test.MQTTPublisher$.main(MQTTPublisher.scala:37) at
org.test.MQTTPublisher.main(MQTTPublisher.scala)
Code:
package org.test
import org.apache.log4j.{Level, Logger}
import org.eclipse.paho.client.mqttv3._
import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence
import org.apache.spark.storage.StorageLevel
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.streaming.mqtt._
import org.apache.spark.SparkConf
object MQTTPublisher {
def main(args: Array[String]) {
if (args.length < 2) {
System.err.println("Usage: MQTTPublisher <MqttBrokerUrl> <topic>")
System.exit(1)
}
// Set logging level if log4j not configured (override by adding log4j.properties to classpath)
if (!Logger.getRootLogger.getAllAppenders.hasMoreElements) {
Logger.getRootLogger.setLevel(Level.WARN)
}
val Seq(brokerUrl, topic) = args.toSeq
var client: MqttClient = null
try {
val persistence = new MemoryPersistence()
client = new MqttClient("localhost:1883", MqttClient.generateClientId(), persistence)
client.connect()
val msgtopic = client.getTopic(topic)
val msgContent = "test test test"
val message = new MqttMessage(msgContent.getBytes("utf-8"))
while (true) {
try {
msgtopic.publish(message)
println(s"Published data. topic: ${msgtopic.getName()}; Message: $message")
} catch {
case e: MqttException if e.getReasonCode == MqttException.REASON_CODE_MAX_INFLIGHT =>
Thread.sleep(10)
println("Queue is full, wait for to consume data from the message queue")
}
}
} catch {
case e: MqttException => println("Exception Caught: " + e)
} finally {
if (client != null) {
client.disconnect()
}
}
}
}
The MqttClient() constructor takes a URI.
What you have provided is just a hostname and port number (localhost:1883), it's missing a protocol section which should be tcp:// (which is what the library is expecting and getting null back. This really should throw a better error.)
You need to change the line to be
client = new MqttClient("tcp://localhost:1883", MqttClient.generateClientId(), persistence);
I think you are giving the wrong Url i.e you are not specifying the protocol over which it has to connect that is my hunch.
Try changing the url to :
tcp://localhost:1883
I think it would work ! Rest all seems fine to me.
For a working example See this : https://github.com/shiv4nsh/scala-mqtt-client-rasberrypi-starter-kit/blob/master/src/main/scala/com/knoldus/MQTTPublisher.scala

Scalate ResourceNotFoundException in Scalatra

I'm trying the following based on scalatra-sbt.g8:
class FooWeb extends ScalatraServlet with ScalateSupport {
beforeAll { contentType = "text/html" }
get("/") {
templateEngine.layout("/WEB-INF/scalate/templates/hello-scalate.jade")
}
}
but I'm getting the following exception (even though the file exists) - any clues?
Could not load resource: [/WEB-INF/scalate/templates/hello-scalate.jade]; are you sure it's within [null]?
org.fusesource.scalate.util.ResourceNotFoundException: Could not load resource: [/WEB-INF/scalate/templates/hello-scalate.jade]; are you sure it's within [null]?
FWIW, the innermost exception is coming from org.mortbay.jetty.handler.ContextHandler.getResource line 1142: _baseResource==null.
Got an answer from the scalatra mailing list. The problem was that I was starting the Jetty server with:
import org.mortbay.jetty.Server
import org.mortbay.jetty.servlet.{Context,ServletHolder}
val server = new Server(8080)
val root = new Context(server, "/", Context.SESSIONS)
root.addServlet(new ServletHolder(new FooWeb()), "/*")
server.start()
I needed to insert this before start():
root.setResourceBase("src/main/webapp")