Enable JBoss Web Native (APR) on JBoss AS 6 - jboss

Question

I am trying to enable APR to improve performance of the AJP connector we use between apache and jboss.
I have downloaded the correct .tar file from http://jbossweb.jboss.org/downloads/jboss-native-2-0-10 and extracted the files in accordance with the wiki instructions here : https://developer.jboss.org/wiki/HowToAddAprToJBoss
Putting the .so files in the {jboss}/bin/native/lib64/ directory
The server already has the startup parameter:
-Djava.library.path=/opt/jboss/bin/native/lib64
But whenever I start the server I get the line:
[AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /opt/jboss/bin/native/lib64
Can anyone offer any advice? I know there are many questions regarding AS7 (and EAP 6) but I cannot see ANY documentation for AS 6
I have tried for both our development (OS/X) and staging (Debian) servers using the corresponding .tar files but no luck with either
As requested, here is result of ls -l on /{jboss}/bin/native/lib64
drwxr-xr-x 2 jboss jbossgroup 4096 Nov 9 14:29 engines
-rw-r--r-- 1 jboss jbossgroup 53584 Aug 17 2010 libHornetQAIO64.so
lrwxrwxrwx 1 jboss jbossgroup 18 Nov 9 14:29 libapr-1.so -> libapr-1.so.0.3.12
lrwxrwxrwx 1 jboss jbossgroup 18 Nov 9 14:29 libapr-1.so.0 -> libapr-1.so.0.3.12
-rwxr-xr-x 1 jboss jbossgroup 760761 Nov 9 14:29 libapr-1.so.0.3.12
lrwxrwxrwx 1 jboss jbossgroup 18 Nov 9 14:29 libcrypto.so -> libcrypto.so.0.9.8
lrwxrwxrwx 1 jboss jbossgroup 18 Nov 9 14:29 libcrypto.so.0 -> libcrypto.so.0.9.8
-r-xr-xr-x 1 jboss jbossgroup 2461453 Nov 9 14:29 libcrypto.so.0.9.8
lrwxrwxrwx 1 jboss jbossgroup 15 Nov 9 14:29 libssl.so -> libssl.so.0.9.8
lrwxrwxrwx 1 jboss jbossgroup 15 Nov 9 14:29 libssl.so.0 -> libssl.so.0.9.8
-r-xr-xr-x 1 jboss jbossgroup 606710 Nov 9 14:29 libssl.so.0.9.8
lrwxrwxrwx 1 jboss jbossgroup 23 Nov 9 14:29 libtcnative-1.so -> libtcnative-1.so.0.1.22
lrwxrwxrwx 1 jboss jbossgroup 23 Nov 9 14:29 libtcnative-1.so.0 -> libtcnative-1.so.0.1.22
-rwxr-xr-x 1 jboss jbossgroup 972435 Nov 9 14:29 libtcnative-1.so.0.1.22
And after enabling DEBUG level logging on org.apache.catalina.core I get the following boot.log output:
JBoss Bootstrap Environment
JBOSS_HOME: /opt/jboss
JAVA: java
JAVA_OPTS: -server -d64 -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled -Xverify:none -Dapp.mode=staging -Djava.rmi.server.hostname=ec2-11-11-111-111.eu-west-1.compute.amazonaws.com -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=6789 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Djboss.platform.mbeanserver -Djavax.management.builder.initial=org.jboss.system.server.jmx.MBeanServerBuilderImpl -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dorg.jboss.logging.Logger.pluginClass=org.jboss.logging.logmanager.LoggerPluginImpl -Djava.net.preferIPv4Stack=true -Dprogram.name=run.sh -Djava.library.path=/opt/jboss/bin/native/lib64
CLASSPATH: :/opt/jboss-6.0.0.Final/lib/jboss-logmanager.jar:/opt/jboss/bin/run.jar
=========================================================================
Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future
23:12:22,466 INFO [AbstractJBossASServerBase] Server Configuration:
JBOSS_HOME URL: file:/opt/jboss-6.0.0.Final/
Bootstrap: $JBOSS_HOME/server/default/conf/bootstrap.xml
Common Base: $JBOSS_HOME/common/
Common Library: $JBOSS_HOME/common/lib/
Server Name: default
Server Base: $JBOSS_HOME/server/
Server Library: $JBOSS_HOME/server/default/lib/
Server Config: $JBOSS_HOME/server/default/conf/
Server Home: $JBOSS_HOME/server/default/
Server Data: $JBOSS_HOME/server/default/data/
Server Log: $JBOSS_HOME/server/default/log/
Server Temp: $JBOSS_HOME/server/default/tmp/
23:12:22,501 INFO [AbstractServer] Starting: JBossAS [6.0.0.Final "Neo"]
23:12:26,130 INFO [ServerInfo] Java version: 1.6.0_22,Sun Microsystems Inc.
23:12:26,138 INFO [ServerInfo] Java Runtime: Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
23:12:26,139 INFO [ServerInfo] Java VM: Java HotSpot(TM) 64-Bit Server VM 17.1-b03,Sun Microsystems Inc.
23:12:26,139 INFO [ServerInfo] OS-System: Linux 3.2.0-4-amd64,amd64
23:12:26,139 INFO [ServerInfo] VM arguments: -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled -Xverify:none -Dapp.mode=staging -Djava.rmi.server.hostname=ec2-11-11-111-111.eu-west-1.compute.amazonaws.com -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.rmi.port=9010 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=6789 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Djboss.platform.mbeanserver -Djavax.management.builder.initial=org.jboss.system.server.jmx.MBeanServerBuilderImpl -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dorg.jboss.logging.Logger.pluginClass=org.jboss.logging.logmanager.LoggerPluginImpl -Djava.net.preferIPv4Stack=true -Dprogram.name=run.sh -Djava.library.path=/opt/jboss/bin/native/lib64 -Djava.endorsed.dirs=/opt/jboss/lib/endorsed
23:12:26,256 INFO [JMXKernel] Legacy JMX core initialized
23:12:38,901 INFO [AbstractServerConfig] JBoss Web Services - Stack CXF Server 3.4.1.GA
23:12:40,353 INFO [JSFImplManagementDeployer] Initialized 3 JSF configurations: [Mojarra-1.2, MyFaces-2.0, Mojarra-2.0]
23:12:56,530 WARNING [FileConfigurationParser] AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
23:13:16,702 INFO [PersistenceUnitValueMetaData] iDependOn persistence.unit:unitName=jboss-ejb3-timerservice-mk2.jar#timerdb
23:13:17,121 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#1944676091{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/SwimtagActions.jar/}
23:13:17,122 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#1944676091{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/SwimtagActions.jar/}
23:13:17,122 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#1617437744{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/SwimtagEntities.jar/}
23:13:17,122 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#1617437744{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/SwimtagEntities.jar/}
23:13:17,123 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#655052855{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/jboss-seam.jar/}
23:13:17,135 INFO [Ejb3DependenciesDeployer] Encountered deployment AbstractVFSDeploymentContext#655052855{vfs:///opt/jboss-6.0.0.Final/server/default/deploy/Swimtag.ear/jboss-seam.jar/}
23:13:21,055 WARNING [FileConfigurationParser] AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
23:13:21,451 INFO [JMXConnector] starting JMXConnector on host 0.0.0.0:1090
23:13:21,718 INFO [MailService] Mail Service bound to java:/Mail
23:13:24,038 INFO [HornetQServerImpl] live server is starting..
23:13:24,211 INFO [JournalStorageManager] Using NIO Journal
23:13:25,638 INFO [NettyAcceptor] Started Netty Acceptor version 3.2.1.Final-r2319 0.0.0.0:5445 for CORE protocol
23:13:25,655 INFO [NettyAcceptor] Started Netty Acceptor version 3.2.1.Final-r2319 0.0.0.0:5455 for CORE protocol
23:13:25,680 INFO [HornetQServerImpl] HornetQ Server version 2.1.2.Final (Colmeia, 120) started
23:13:25,878 INFO [WebService] Using RMI server codebase: http://ec2-11-11-111-111.eu-west-1.compute.amazonaws.com:8083/
23:13:26,402 INFO [jbossatx] ARJUNA-32010 JBossTS Recovery Service (tag: JBOSSTS_4_14_0_Final) - JBoss Inc.
23:13:26,406 INFO [arjuna] ARJUNA-12324 Start RecoveryActivators
23:13:26,491 INFO [arjuna] ARJUNA-12296 ExpiredEntryMonitor running at Wed, 12 Nov 2014 23:13:26
23:13:26,681 INFO [arjuna] ARJUNA-12310 Recovery manager listening on endpoint 0.0.0.0:4712
23:13:26,682 INFO [arjuna] ARJUNA-12344 RecoveryManagerImple is ready on port 4712
23:13:26,682 INFO [jbossatx] ARJUNA-32013 Starting transaction recovery manager
23:13:26,712 INFO [arjuna] ARJUNA-12163 Starting service com.arjuna.ats.arjuna.recovery.ActionStatusService on port 4713
23:13:26,761 INFO [arjuna] ARJUNA-12337 TransactionStatusManagerItem host: 0.0.0.0 port: 4713
23:13:26,777 INFO [arjuna] ARJUNA-12170 TransactionStatusManager started on port 4713 and host 0.0.0.0 with service com.arjuna.ats.arjuna.recovery.ActionStatusService
23:13:26,830 INFO [jbossatx] ARJUNA-32017 JBossTS Transaction Service (JTA version - tag: JBOSSTS_4_14_0_Final) - JBoss Inc.
23:13:26,892 INFO [arjuna] ARJUNA-12202 registering bean jboss.jta:type=ObjectStore.
23:13:27,372 INFO [AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /opt/jboss/bin/native/lib64
23:13:27,794 INFO [ModClusterService] Initializing mod_cluster 1.1.0.Final
23:13:27,816 INFO [TomcatDeployment] deploy, ctxPath=/invoker
23:13:28,657 INFO [RARDeployment] Required license terms exist, view vfs:/opt/jboss-6.0.0.Final/server/default/deploy/jboss-local-jdbc.rar/META-INF/ra.xml
23:13:28,689 INFO [RARDeployment] Required license terms exist, view vfs:/opt/jboss-6.0.0.Final/server/default/deploy/jboss-xa-jdbc.rar/META-INF/ra.xml
23:13:28,696 INFO [RARDeployment] Required license terms exist, view vfs:/opt/jboss-6.0.0.Final/server/default/deploy/jms-ra.rar/META-INF/ra.xml
23:13:28,730 INFO [HornetQResourceAdapter] HornetQ resource adaptor started
23:13:28,787 INFO [RARDeployment] Required license terms exist, view vfs:/opt/jboss-6.0.0.Final/server/default/deploy/mail-ra.rar/META-INF/ra.xml
23:13:28,804 INFO [RARDeployment] Required license terms exist, view vfs:/opt/jboss-6.0.0.Final/server/default/deploy/quartz-ra.rar/META-INF/ra.xml
23:13:28,912 INFO [SimpleThreadPool] Job execution threads will use class loader of thread: Thread-2
23:13:28,949 INFO [SchedulerSignalerImpl] Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
23:13:28,949 INFO [QuartzScheduler] Quartz Scheduler v.1.8.3 created.
23:13:28,951 INFO [RAMJobStore] RAMJobStore initialized.
23:13:28,952 INFO [QuartzScheduler] Scheduler meta-data: Quartz Scheduler (v1.8.3) 'JBossQuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
Using the following in jboss-logging.xml
<logger category="org.apache.catalina.core">
<level name="DEBUG"/>
</logger>

Answer 1

Jboss AS 6.x in run.sh script try set value of java.library.path
If you look in the run.sh file:
# Setup JBoss Native library path
#
if [ -d "$JBOSS_HOME/../native/lib" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/../native" && pwd`
elif [ -d "$JBOSS_HOME/native/lib" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/native" && pwd`
elif [ -d "$JBOSS_HOME/../native/lib64" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/../native" && pwd`
elif [ -d "$JBOSS_HOME/native/lib64" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/native" && pwd`
elif [ -d "$JBOSS_HOME/native/bin" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/native" && pwd`
elif [ -d "$JBOSS_HOME/bin/native" ]; then
JBOSS_NATIVE_DIR=`cd "$JBOSS_HOME/bin/native" && pwd`
fi
if [ -d "$JBOSS_NATIVE_DIR" ]; then
if $cygwin; then
JBOSS_NATIVE_DIR="$JBOSS_NATIVE_DIR/bin"
export PATH="$JBOSS_NATIVE_DIR:$PATH"
JBOSS_NATIVE_LIBPATH=`cygpath --path --windows "$JBOSS_NATIVE_DIR"`
else
IS_64_BIT_JVM=`"$JAVA" $JVM_OPTVERSION 2>&1 | $GREP -i 64-bit`
if [ "x$IS_64_BIT_JVM" != "x" ]; then
JBOSS_NATIVE_DIR="$JBOSS_NATIVE_DIR/lib64"
else
JBOSS_NATIVE_DIR="$JBOSS_NATIVE_DIR/lib"
fi
LD_LIBRARY_PATH="$JBOSS_NATIVE_DIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH
JBOSS_NATIVE_LIBPATH=$LD_LIBRARY_PATH
fi
JAVA_OPTS="$JAVA_OPTS -Djava.library.path=$JBOSS_NATIVE_LIBPATH"
fi
This overwrites the value in startup, you must install the native components in bin/native or comment this line.
JAVA_OPTS="$JAVA_OPTS -Djava.library.path=$JBOSS_NATIVE_LIBPATH"