Using Liquibase Mongodb extension with Quarkus - mongodb

Trying to use liquibase-mongodb extension with Quarkus. Without any sucess.
Anyone able to guide me to some working example?
application.yaml contents:
quarkus:
mongodb:
connection-string: mongodb://localhost:27017
write-concern:
journal: false
database: foo1
liquibase:
migrate-at-start: true
change-log: db/changeLog.yaml
db/changeLog.yaml contents:
databaseChangeLog:
- include:
file: changesets/foo.json
build.gradle contains:
implementation "io.quarkus:quarkus-liquibase"
implementation "org.liquibase.ext:liquibase-mongodb:${liquibaseVersion}"
implementation "org.mongodb:mongodb-driver-sync:${mongodbVersion}"
output:
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Slf4jLoggerFactory]
__ ____ __ _____ ___ __ ____ ______
--/ __ \/ / / / _ | / _ \/ //_/ / / / __/
-/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2021-03-07 12:26:39,254 WARN [io.qua.dep.QuarkusAugmentor] (main) Using Java versions older than 11 to build Quarkus applications is deprecated and will be disallowed in a future release!
2021-03-07 12:26:39,573 WARN [io.qua.agr.dep.AgroalProcessor] (build-24) The Agroal dependency is present but no JDBC datasources have been defined.
2021-03-07 12:26:40,583 INFO [org.mon.dri.cluster] (Quarkus Main Thread) Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms'}
2021-03-07 12:26:40,595 INFO [org.mon.dri.cluster] (Quarkus Main Thread) Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms'}
2021-03-07 12:26:40,617 INFO [org.mon.dri.connection] (cluster-rtt-ClusterId{value='6044b870c8687d11c71dfb0b', description='null'}-localhost:27017) Opened connection [connectionId{localValue:1, serverValue:5}] to localhost:27017
2021-03-07 12:26:40,617 INFO [org.mon.dri.connection] (cluster-ClusterId{value='6044b870c8687d11c71dfb0b', description='null'}-localhost:27017) Opened connection [connectionId{localValue:2, serverValue:6}] to localhost:27017
2021-03-07 12:26:40,617 INFO [org.mon.dri.cluster] (cluster-ClusterId{value='6044b870c8687d11c71dfb0b', description='null'}-localhost:27017) Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=11510711}
2021-03-07 12:26:40,620 INFO [org.mon.dri.connection] (cluster-ClusterId{value='6044b870c8687d11c71dfb0c', description='null'}-localhost:27017) Opened connection [connectionId{localValue:3, serverValue:8}] to localhost:27017
2021-03-07 12:26:40,620 INFO [org.mon.dri.cluster] (cluster-ClusterId{value='6044b870c8687d11c71dfb0c', description='null'}-localhost:27017) Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=3728082}
2021-03-07 12:26:40,621 INFO [org.mon.dri.connection] (cluster-rtt-ClusterId{value='6044b870c8687d11c71dfb0c', description='null'}-localhost:27017) Opened connection [connectionId{localValue:4, serverValue:7}] to localhost:27017
2021-03-07 12:26:40,704 INFO [io.quarkus] (Quarkus Main Thread) foo-app 0.0.1-SNAPSHOT on JVM (powered by Quarkus 1.11.3.Final) started in 1.524s. Listening on: http://localhost:8080
2021-03-07 12:26:40,705 INFO [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
2021-03-07 12:26:40,705 INFO [io.quarkus] (Quarkus Main Thread) Installed features: [agroal, cdi, config-yaml, liquibase, mongodb-client, mongodb-panache, mongodb-rest-data-panache, mutiny, narayana-jta, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-openapi, swagger-ui]
so liquibase is known to quarkus, but mongodb changesets are not executed.

The Quarkus Liquibase extension only targets JDBC datasources for now.
Probably worth opening an enhancement request in our tracker so that we track this need, if it hasn't already been done.

Due to lack of official support, ended up with custom implementation, that is all I needed:
import java.util.Optional;
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import io.quarkus.runtime.StartupEvent;
import liquibase.Liquibase;
import liquibase.database.Database;
import liquibase.database.DatabaseFactory;
import liquibase.ext.mongodb.database.MongoLiquibaseDatabase;
import liquibase.resource.ClassLoaderResourceAccessor;
import lombok.SneakyThrows;
#ApplicationScoped
public class MongoDBMigration {
#ConfigProperty(name = "quarkus.mongodb.connection-string")
String connectionString;
#ConfigProperty(name = "quarkus.mongodb.credentials.username")
Optional<String> username;
#ConfigProperty(name = "quarkus.mongodb.credentials.password")
Optional<String> password;
#ConfigProperty(name = "quarkus.liquibase.migrate-at-start")
boolean liquibaseEnabled;
#SneakyThrows
void onStart(#Observes StartupEvent ev) {
if (liquibaseEnabled) {
Database database = (MongoLiquibaseDatabase) DatabaseFactory.getInstance().openDatabase(connectionString, username.orElse(null), password.orElse(null), null, null);
Liquibase liquiBase = new Liquibase("db/changeLog.json", new ClassLoaderResourceAccessor(), database);
liquiBase.update("");
}
}
}
having application.yaml:
quarkus:
mongodb:
connection-string: mongodb://localhost:27017/foo?socketTimeoutMS=1000&connectTimeoutMS=1000&serverSelectionTimeoutMS=1000
write-concern:
journal: false
database: foo
liquibase:
migrate-at-start: true
change-log: db/changeLog.xml
and build.gradle:
dependencies {
implementation "io.quarkus:quarkus-liquibase"
implementation "org.liquibase.ext:liquibase-mongodb:${liquibaseVersion}"
implementation "org.mongodb:mongodb-driver-sync:${mongodbVersion}"

Related

ActiveMQ Artemis in Docker shows empty screen

Short version
When I run ActiveMQ Artemis in docker I see this basically empty screen:
That doesn't look right... I was expecting this, like I get when using the zip file:
Regardless of whether I use docker or the zip file, it doesn't matter what username or password I enter, I just get logged in regardless, which is a little concerning...
What am I doing wrong?
Longer Version
I'm attempting a "Hello World" style installation of ActiveMQ. It sounds like ActiveMQ Artemis is what I should be using. We'll be using this on Kubernetes, so I found and have followed https://artemiscloud.io/. There is a Quickly deploy a basic Container image that runs the broker right there on the front page. It suggests:
docker run -e AMQ_USER=admin -e AMQ_PASSWORD=admin -p80:8161 --name artemis quay.io/artemiscloud/activemq-artemis-broker:dev.latest
I changed the port to -p8161:8161 and run it using Docker for Windows. Visiting http://localhost:8161/console/ shows me the first screenshot above.
If I instead download from https://activemq.apache.org/components/artemis/download/ and follow https://activemq.apache.org/components/artemis/documentation/latest/using-server.html I get the second screenshot.
Regardless of whether I use docker or the zip file, it doesn't matter what username or password I enter, I just get logged in regardless.
Full docker console log
ยป docker run -e AMQ_USER=admin -e AMQ_PASSWORD=admin -p8161:8161 --name artemis quay.io/artemiscloud/activemq-artemis-broker:latest
Creating Broker with args --user XXXXX --password XXXXX --role admin --name broker --allow-anonymous --http-host 172.17.0.3 --host 172.17.0.3 --force
Creating ActiveMQ Artemis instance at: /home/jboss/broker
Auto tuning journal ...
done! Your system can make 1.44 writes per millisecond, your journal-buffer-timeout will be 696000
You can now start the broker by executing:
"/home/jboss/broker/bin/artemis" run
Or you can run the broker in the background using:
"/home/jboss/broker/bin/artemis-service" start
Running Broker
_ _ _
/ \ ____| |_ ___ __ __(_) _____
/ _ \| _ \ __|/ _ \ \/ | |/ __/
/ ___ \ | \/ |_/ __/ |\/| | |\___ \
/_/ \_\| \__\____|_| |_|_|/___ /
Apache ActiveMQ Artemis 2.17.0
2021-05-27 21:23:44,556 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server
2021-05-27 21:23:44,605 INFO [org.apache.activemq.artemis.core.server] AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=false,journalDirectory=data/journal,bindingsDirectory=data/bindings,largeMessagesDirectory=data/large-messages,pagingDirectory=data/paging)
2021-05-27 21:23:44,650 INFO [org.apache.activemq.artemis.core.server] AMQ221012: Using AIO Journal
2021-05-27 21:23:44,694 INFO [org.apache.activemq.artemis.core.server] AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size (-Xmx). being defined as 1,073,741,824
2021-05-27 21:23:44,712 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
2021-05-27 21:23:44,712 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
2021-05-27 21:23:44,713 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2021-05-27 21:23:44,713 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT
2021-05-27 21:23:44,713 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE
2021-05-27 21:23:44,714 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
2021-05-27 21:23:44,759 INFO [org.apache.activemq.artemis.core.server] AMQ221034: Waiting indefinitely to obtain live lock
2021-05-27 21:23:44,760 INFO [org.apache.activemq.artemis.core.server] AMQ221035: Live Server Obtained live lock
2021-05-27 21:23:44,832 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address DLQ supporting [ANYCAST]
2021-05-27 21:23:44,839 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue DLQ on address DLQ
2021-05-27 21:23:44,880 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST]
2021-05-27 21:23:44,882 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue
2021-05-27 21:23:45,148 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 172.17.0.3:61616 for protocols [CORE,MQTT,AMQP,STOMP,HORNETQ,OPENWIRE]
2021-05-27 21:23:45,152 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 172.17.0.3:5445 for protocols [HORNETQ,STOMP]
2021-05-27 21:23:45,155 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 172.17.0.3:5672 for protocols [AMQP]
2021-05-27 21:23:45,158 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 172.17.0.3:1883 for protocols [MQTT]
2021-05-27 21:23:45,162 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 172.17.0.3:61613 for protocols [STOMP]
2021-05-27 21:23:45,167 INFO [org.apache.activemq.artemis.core.server] AMQ221007: Server is now live
2021-05-27 21:23:45,168 INFO [org.apache.activemq.artemis.core.server] AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.17.0 [broker, nodeID=d1043ae4-bf31-11eb-8c49-0242ac110003]
2021-05-27 21:23:45,379 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized activemq-branding plugin
2021-05-27 21:23:45,413 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized artemis-plugin plugin
2021-05-27 21:23:45,625 INFO [io.hawt.HawtioContextListener] Initialising hawtio services
2021-05-27 21:23:45,636 INFO [io.hawt.system.ConfigManager] Configuration will be discovered via system properties
2021-05-27 21:23:45,638 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to Hawtio 2.11.0
2021-05-27 21:23:45,643 INFO [io.hawt.web.auth.AuthenticationConfiguration] Starting hawtio authentication filter, JAAS realm: "activemq" authorized role(s): "admin" role principal classes: "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
2021-05-27 21:23:45,656 INFO [io.hawt.web.proxy.ProxyServlet] Proxy servlet is disabled
2021-05-27 21:23:45,660 INFO [io.hawt.web.servlets.JolokiaConfiguredAgentServlet] Jolokia overridden property: [key=policyLocation, value=file:/home/jboss/broker/etc/jolokia-access.xml]
2021-05-27 21:23:45,727 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://172.17.0.3:8161
2021-05-27 21:23:45,727 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://172.17.0.3:8161/console/jolokia
2021-05-27 21:23:45,728 INFO [org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://172.17.0.3:8161/console
2021-05-27 21:23:54,289 INFO [io.hawt.web.auth.LoginServlet] Hawtio login is using 1800 sec. HttpSession timeout
2021-05-27 21:23:54,968 INFO [io.hawt.web.auth.keycloak.KeycloakServlet] Keycloak integration is disabled
2021-05-27 21:24:00,270 INFO [io.hawt.web.auth.LoginServlet] Logging in user: admin
The ArtemisCloud container image for ActiveMQ Artemis is designed to run inside a container so the container IP address should be used to access to the console or to other resources.
The container IP address can be obtained by using the command docker inspect or by reading the container log, ie:
[org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://172.17.0.3:8161/console

Artemis Master-Slave failover and sync error message in logs

After struggling with Artemis 2.11 and an older Java version I decided to update my whole system to the "latest greatest software" that is currently available. So I am using Artemis 2.14 and Java 14.0.2 on two Ubuntu 18.04 VM with 4 Cores an 16 GB RAM.
I configured the master-slave ha-policy like this:
MASTER:
<ha-policy>
<replication>
<master>
<check-for-live-server>true</check-for-live-server>
</master>
</replication>
</ha-policy>
SLAVE
<ha-policy>
<replication>
<slave>
<allow-failback>true</allow-failback>
</slave>
</replication>
</ha-policy>
And I am using the cluster-connection like this:
MASTER
<cluster-connections>
<cluster-connection name="artemis-cluster">
<connector-ref>Artemis-Node-A-Sync</connector-ref>
<retry-interval>500</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<static-connectors>
<connector-ref>Node-B-Sync</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
SLAVE
<cluster-connections>
<cluster-connection name="artemis-cluster">
<connector-ref>Node-B-Sync</connector-ref>
<retry-interval>500</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<static-connectors>
<connector-ref>Node-A-Sync</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
My problem is that I get this ERROR message from at SLAVE...
2020-08-07 12:45:37,548 INFO [io.hawt.system.ConfigManager] Configuration will be discovered via system properties
2020-08-07 12:45:37,550 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to hawtio 1.5.12 : http://hawt.io/ : Don't cha wish your console was hawt like me? ;-)
2020-08-07 12:45:37,552 INFO [io.hawt.jmx.UploadManager] Using file upload directory: /opt/mybroker-broker/tmp/uploads
2020-08-07 12:45:37,565 INFO [io.hawt.web.AuthenticationFilter] Starting hawtio authentication filter, JAAS realm: "activemq" authorized role(s): "amq" role principal classes: "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
2020-08-07 12:45:37,585 INFO [io.hawt.web.JolokiaConfiguredAgentServlet] Jolokia overridden property: [key=policyLocation, value=file:/opt/mybroker-broker/etc/jolokia-access.xml]
2020-08-07 12:45:37,605 INFO [io.hawt.web.RBACMBeanInvoker] Using MBean [hawtio:type=security,area=jmx,rank=0,name=HawtioDummyJMXSecurity] for role based access control
2020-08-07 12:45:37,703 INFO [io.hawt.system.ProxyWhitelist] Initial proxy whitelist: [localhost, 127.0.0.1, *.*.*.*, *.*.*.*, localhost.localdomain]
2020-08-07 12:45:37,966 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://0.0.0.0:8161
2020-08-07 12:45:37,967 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://0.0.0.0:8161/console/jolokia
2020-08-07 12:45:37,967 INFO [org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://0.0.0.0:8161/console
2020-08-07 12:45:46,905 WARN [org.apache.activemq.artemis.core.client] AMQ212037: Connection failure to 192.168.144.75/192.168.144.75:22522 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
2020-08-07 12:45:50,678 INFO [org.apache.activemq.artemis.core.server] AMQ221066: Initiating quorum vote: LiveFailoverQuorumVote
2020-08-07 12:45:50,678 INFO [org.apache.activemq.artemis.core.server] AMQ221084: Requested 0 quorum votes
2020-08-07 12:45:50,679 INFO [org.apache.activemq.artemis.core.server] AMQ221083: ignoring quorum vote as max cluster size is 1.
2020-08-07 12:45:50,679 INFO [org.apache.activemq.artemis.core.server] AMQ221071: Failing over based on quorum vote results.
2020-08-07 12:45:50,720 ERROR [org.apache.activemq.artemis.core.server] AMQ224000: Failure in initialisation: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE message=AMQ229026: Backup Server was not yet in sync with live]
at org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run(SharedNothingBackupActivation.java:310) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:3946) [artemis-server-2.14.0.jar:2.14.0]
The logfile at master is:
2020-08-07 12:44:58,292 INFO [org.apache.activemq.artemis.core.server] AMQ221014: 21% loaded
2020-08-07 12:44:58,540 INFO [org.apache.activemq.artemis.core.server] AMQ221014: 42% loaded
2020-08-07 12:44:59,020 INFO [org.apache.activemq.artemis.core.server] AMQ221014: 64% loaded
2020-08-07 12:44:59,416 INFO [org.apache.activemq.artemis.core.server] AMQ221014: 85% loaded
2020-08-07 12:45:12,143 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address DLQ supporting [ANYCAST]
2020-08-07 12:45:12,145 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue DLQ on address DLQ
2020-08-07 12:45:12,151 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST]
2020-08-07 12:45:12,152 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue
2020-08-07 12:45:12,382 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 0.0.0.0:1883 for protocols [MQTT]
2020-08-07 12:45:12,385 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 0.0.0.0:22522 for protocols [CORE,MQTT,AMQP,STOMP,HORNETQ,OPENWIRE]
2020-08-07 12:45:12,387 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 0.0.0.0:8883 for protocols [MQTT]
2020-08-07 12:45:12,388 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started EPOLL Acceptor at 0.0.0.0:18884 for protocols [MQTT]
2020-08-07 12:45:12,388 INFO [org.apache.activemq.artemis.core.server] AMQ221007: Server is now live
2020-08-07 12:45:12,389 INFO [org.apache.activemq.artemis.core.server] AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.14.0 [0.0.0.0, nodeID=95f808d9-d641-11ea-9c48-005056073c33]
2020-08-07 12:45:12,650 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized activemq-branding plugin
2020-08-07 12:45:12,686 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized artemis-plugin plugin
2020-08-07 12:45:13,281 INFO [io.hawt.HawtioContextListener] Initialising hawtio services
2020-08-07 12:45:13,304 INFO [io.hawt.system.ConfigManager] Configuration will be discovered via system properties
2020-08-07 12:45:13,306 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to hawtio 1.5.12 : http://hawt.io/ : Don't cha wish your console was hawt like me? ;-)
2020-08-07 12:45:13,311 INFO [io.hawt.jmx.UploadManager] Using file upload directory: /opt/mybroker-broker/tmp/uploads
2020-08-07 12:45:13,325 INFO [io.hawt.web.AuthenticationFilter] Starting hawtio authentication filter, JAAS realm: "activemq" authorized role(s): "amq" role principal classes: "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
2020-08-07 12:45:13,344 INFO [io.hawt.web.JolokiaConfiguredAgentServlet] Jolokia overridden property: [key=policyLocation, value=file:/opt/mybroker-broker/etc/jolokia-access.xml]
2020-08-07 12:45:13,373 INFO [io.hawt.web.RBACMBeanInvoker] Using MBean [hawtio:type=security,area=jmx,rank=0,name=HawtioDummyJMXSecurity] for role based access control
2020-08-07 12:45:13,472 INFO [io.hawt.system.ProxyWhitelist] Initial proxy whitelist: [localhost, 127.0.0.1, *.*.*.*, 192.168.144.75, localhost.localdomain]
2020-08-07 12:45:13,680 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://0.0.0.0:8161
2020-08-07 12:45:13,681 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://0.0.0.0:8161/console/jolokia
2020-08-07 12:45:13,681 INFO [org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://0.0.0.0:8161/console
2020-08-07 12:45:46,608 WARN [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page, message=/opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page (Too many open files): ActiveMQIOErrorException[errorType=IO_ERROR message=/opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page (Too many open files)]
at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:156) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:98) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.Page.open(Page.java:483) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.openNewPage(PagingStoreImpl.java:1136) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.forceAnotherPage(PagingStoreImpl.java:606) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.getPageInformationForSync(JournalStorageManager.java:738) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.startReplication(JournalStorageManager.java:665) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation$2.run(SharedNothingLiveActivation.java:179) [artemis-server-2.14.0.jar:2.14.0]
at java.base/java.lang.Thread.run(Thread.java:832) [java.base:]
Caused by: java.io.FileNotFoundException: /opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page (Too many open files)
at java.base/java.io.RandomAccessFile.open0(Native Method) [java.base:]
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347) [java.base:]
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:261) [java.base:]
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:216) [java.base:]
at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:143) [artemis-journal-2.14.0.jar:2.14.0]
... 8 more
2020-08-07 12:45:46,615 WARN [org.apache.activemq.artemis.core.server] AMQ222251: Unable to start replication: java.io.FileNotFoundException: /opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page (Too many open files)
at java.base/java.io.RandomAccessFile.open0(Native Method) [java.base:]
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347) [java.base:]
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:261) [java.base:]
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:216) [java.base:]
at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:143) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:98) [artemis-journal-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.Page.open(Page.java:483) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.openNewPage(PagingStoreImpl.java:1136) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.forceAnotherPage(PagingStoreImpl.java:606) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.getPageInformationForSync(JournalStorageManager.java:738) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.startReplication(JournalStorageManager.java:665) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation$2.run(SharedNothingLiveActivation.java:179) [artemis-server-2.14.0.jar:2.14.0]
at java.base/java.lang.Thread.run(Thread.java:832) [java.base:]
2020-08-07 12:45:46,693 WARN [org.apache.activemq.artemis.utils.actors.OrderedExecutor] Server locator is closed (maybe it was garbage collected): java.lang.IllegalStateException: Server locator is closed (maybe it was garbage collected)
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.assertOpen(ServerLocatorImpl.java:1844) [artemis-core-client-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:644) [artemis-core-client-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:545) [artemis-core-client-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:524) [artemis-core-client-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.core.server.cluster.ClusterController$ConnectRunnable.run(ClusterController.java:433) [artemis-server-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.14.0.jar:2.14.0]
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.14.0.jar:2.14.0]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [java.base:]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [java.base:]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.14.0.jar:2.14.0]
2020-08-07 12:45:47,407 INFO [io.hawt.HawtioContextListener] Destroying hawtio services
2020-08-07 12:45:47,409 INFO [io.hawt.web.AuthenticationFilter] Destroying hawtio authentication filter 2020-08-07 12:45:47,442 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Destroyed artemis-plugin plugin
2020-08-07 12:45:47,444 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Destroyed activemq-branding plugin
2020-08-07 12:45:47,476 INFO [org.apache.activemq.artemis.core.server] AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.14.0 [95f808d9-d641-11ea-9c48-005056073c33] stopped, uptime 1 minute
tail: /opt/mybroker-broker/log/artemis.log: file truncated
2020-08-07 12:46:07,212 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server
2020-08-07 12:46:07,242 INFO [org.apache.activemq.artemis.core.server] AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=data/journal,bindingsDirectory=data/bindings,largeMessagesDirectory=data/large-messages,pagingDirectory=data/paging)
2020-08-07 12:46:09,516 INFO [org.apache.activemq.artemis.core.server] AMQ221012: Using AIO Journal
2020-08-07 12:46:09,542 INFO [org.apache.activemq.artemis.core.server] AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size (-Xmx). being defined as 5,368,709,120
2020-08-07 12:46:09,589 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
2020-08-07 12:46:09,590 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
2020-08-07 12:46:09,591 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2020-08-07 12:46:09,592 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT
2020-08-07 12:46:09,592 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE
2020-08-07 12:46:09,593 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
2020-08-07 12:46:09,776 WARN [org.apache.activemq.artemis.core.server] AMQ222035: Directory /opt/mybroker-broker/data/paging/9fd77264-d6db-11ea-bb8e-005056073c33 did not have an identification file address.txt
2020-08-07 12:46:13,695 WARN [org.apache.activemq.artemis.core.server] AMQ222035: Directory /opt/mybroker-broker/data/paging/9fd77264-d6db-11ea-bb8e-005056073c33 did not have an identification file address.txt
...when I do the following:
Stop the SLAVE
Restart the MASTER
Start the SLAVE
Using Jolokia I see that the SLAVE is connected to the MASTER after some time.
Questions:
What does the ERROR message mean?
Is there an CLI command to get some information about synchronization?
Is there an CLI command to get some system/cluster status information?
The fundamental problem is noted in the log from the master broker:
2020-08-07 12:45:46,615 WARN [org.apache.activemq.artemis.core.server] AMQ222251: Unable to start replication: java.io.FileNotFoundException: /opt/mybroker-broker/data/paging/9fbeba62-d6db-11ea-bb8e-005056073c33/000000003.page (Too many open files)
This is an environmental error indicating that the operating system will not allow the process (i.e. the broker) to open any more files.
Since the broker can't open all the files it needs then it can't properly synchronize its data with the backup. Furthermore, the inability to open additional files is interpreted as a "critical" error which forces the broker to shut itself down. When the master broker shuts down the slave activates, but since the master was not able to complete the initial replication with the slave the slave states:
2020-08-07 12:45:50,720 ERROR [org.apache.activemq.artemis.core.server] AMQ224000: Failure in initialisation: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE message=AMQ229026: Backup Server was not yet in sync with live]
The slave is simply logging the fact that it didn't complete the initial synchronization/replication process with the master before the master was shut down.
Ultimately you need to increase the file limit for the user running the broker. A quick web search should help you with the specifics of that task for your particular operating system.

Spark program is not able to connect to MySql hive context through eclipse

I have set the hive metastore in mySql and same can be accessed through hive and create database and tables. If I try to access hive table through spark-shell then able to get the tables info correctly by getting from mysql hive metastore. But it is not fetching from Mysql if execute from eclipse.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.0.0
/_/
Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_151)
Type in expressions to have them evaluated.
Type :help for more information.
scala> val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
warning: there was one deprecation warning; re-run with -deprecation for details
sqlContext: org.apache.spark.sql.hive.HiveContext = org.apache.spark.sql.hive.HiveContext#2b9e69fb
scala> sqlContext.sql("show databases");
res0: org.apache.spark.sql.DataFrame = [databaseName: string]
But if I try to access through eclipse, then it is not point to MySql. Instead of that it is Derby. Please find the below log and hive-site.xml for an idea.
Note: hive-site.xml is same in hive/conf and spark/conf paths.
Spark code that is executing from eclipse:
package events
import org.apache.spark.{SparkContext, SparkConf}
import org.apache.spark.sql.SQLContext
import org.apache.spark.sql
object DataContent {
def main(args:Array[String])
{
val conf = new SparkConf()
conf.setAppName("Word Count2").setMaster("local")
val sc = new SparkContext(conf)
println("Hello to Spark World")
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
var query = sqlContext.sql("show databases");
query.collect()
println("Bye to Spark example2")
}
}
Spark output log:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
18/02/17 12:03:15 INFO SparkContext: Running Spark version 2.0.0
18/02/17 12:03:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/02/17 12:03:18 WARN Utils: Your hostname, ubuntu resolves to a loopback address: 127.0.1.1; using 192.168.189.136 instead (on interface ens33)
18/02/17 12:03:18 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
18/02/17 12:03:18 INFO SecurityManager: Changing view acls to: vm4learning
18/02/17 12:03:18 INFO SecurityManager: Changing modify acls to: vm4learning
18/02/17 12:03:18 INFO SecurityManager: Changing view acls groups to:
18/02/17 12:03:18 INFO SecurityManager: Changing modify acls groups to:
18/02/17 12:03:18 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(vm4learning); groups with view permissions: Set(); users with modify permissions: Set(vm4learning); groups with modify permissions: Set()
18/02/17 12:03:20 INFO Utils: Successfully started service 'sparkDriver' on port 45637.
18/02/17 12:03:20 INFO SparkEnv: Registering MapOutputTracker
18/02/17 12:03:20 INFO SparkEnv: Registering BlockManagerMaster
18/02/17 12:03:20 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-37db59bd-c12a-4603-ba9f-8e8fec88cc29
18/02/17 12:03:20 INFO MemoryStore: MemoryStore started with capacity 881.4 MB
18/02/17 12:03:21 INFO SparkEnv: Registering OutputCommitCoordinator
18/02/17 12:03:23 INFO Utils: Successfully started service 'SparkUI' on port 4040.
18/02/17 12:03:23 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.189.136:4040
18/02/17 12:03:23 INFO Executor: Starting executor ID driver on host localhost
18/02/17 12:03:23 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 38313.
18/02/17 12:03:23 INFO NettyBlockTransferService: Server created on 192.168.189.136:38313
18/02/17 12:03:23 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.189.136, 38313)
18/02/17 12:03:23 INFO BlockManagerMasterEndpoint: Registering block manager 192.168.189.136:38313 with 881.4 MB RAM, BlockManagerId(driver, 192.168.189.136, 38313)
18/02/17 12:03:23 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.189.136, 38313)
Hello to Spark World
18/02/17 12:03:30 INFO HiveSharedState: Warehouse path is 'file:/home/vm4learning/workspace/Acumen/spark-warehouse'.
18/02/17 12:03:30 INFO SparkSqlParser: Parsing command: show databases
18/02/17 12:03:32 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
18/02/17 12:03:34 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
18/02/17 12:03:34 INFO ObjectStore: ObjectStore, initialize called
18/02/17 12:03:36 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
18/02/17 12:03:36 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored
18/02/17 12:03:41 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
18/02/17 12:03:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
18/02/17 12:03:47 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
18/02/17 12:03:49 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
18/02/17 12:03:49 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
18/02/17 12:03:49 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery#0" since the connection used is closing
18/02/17 12:03:49 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
18/02/17 12:03:49 INFO ObjectStore: Initialized ObjectStore
18/02/17 12:03:50 INFO HiveMetaStore: Added admin role in metastore
18/02/17 12:03:50 INFO HiveMetaStore: Added public role in metastore
18/02/17 12:03:50 INFO HiveMetaStore: No user is added in admin role, since config is empty
18/02/17 12:03:51 INFO HiveMetaStore: 0: get_all_databases
18/02/17 12:03:51 INFO audit: ugi=vm4learning ip=unknown-ip-addr cmd=get_all_databases
18/02/17 12:03:51 INFO HiveMetaStore: 0: get_functions: db=default pat=*
18/02/17 12:03:51 INFO audit: ugi=vm4learning ip=unknown-ip-addr cmd=get_functions: db=default pat=*
18/02/17 12:03:51 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table.
18/02/17 12:03:52 INFO SessionState: Created local directory: /tmp/5c825a73-be72-4bd1-8bfc-966d8a095919_resources
18/02/17 12:03:52 INFO SessionState: Created HDFS directory: /tmp/hive/vm4learning/5c825a73-be72-4bd1-8bfc-966d8a095919
18/02/17 12:03:52 INFO SessionState: Created local directory: /tmp/vm4learning/5c825a73-be72-4bd1-8bfc-966d8a095919
18/02/17 12:03:52 INFO SessionState: Created HDFS directory: /tmp/hive/vm4learning/5c825a73-be72-4bd1-8bfc-966d8a095919/_tmp_space.db
18/02/17 12:03:52 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is file:/home/vm4learning/workspace/Acumen/spark-warehouse
18/02/17 12:03:52 INFO SessionState: Created local directory: /tmp/32b99842-2ac2-491e-934d-9726a6213c37_resources
18/02/17 12:03:52 INFO SessionState: Created HDFS directory: /tmp/hive/vm4learning/32b99842-2ac2-491e-934d-9726a6213c37
18/02/17 12:03:52 INFO SessionState: Created local directory: /tmp/vm4learning/32b99842-2ac2-491e-934d-9726a6213c37
18/02/17 12:03:52 INFO SessionState: Created HDFS directory: /tmp/hive/vm4learning/32b99842-2ac2-491e-934d-9726a6213c37/_tmp_space.db
18/02/17 12:03:52 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is file:/home/vm4learning/workspace/Acumen/spark-warehouse
18/02/17 12:03:53 INFO HiveMetaStore: 0: create_database: Database(name:default, description:default database, locationUri:file:/home/vm4learning/workspace/Acumen/spark-warehouse, parameters:{})
18/02/17 12:03:53 INFO audit: ugi=vm4learning ip=unknown-ip-addr cmd=create_database: Database(name:default, description:default database, locationUri:file:/home/vm4learning/workspace/Acumen/spark-warehouse, parameters:{})
18/02/17 12:03:55 INFO HiveMetaStore: 0: get_databases: *
18/02/17 12:03:55 INFO audit: ugi=vm4learning ip=unknown-ip-addr cmd=get_databases: *
18/02/17 12:03:56 INFO CodeGenerator: Code generated in 1037.20509 ms
Bye to Spark example2
18/02/17 12:03:56 INFO SparkContext: Invoking stop() from shutdown hook
18/02/17 12:03:57 INFO SparkUI: Stopped Spark web UI at http://192.168.189.136:4040
18/02/17 12:03:57 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
18/02/17 12:03:57 INFO MemoryStore: MemoryStore cleared
18/02/17 12:03:57 INFO BlockManager: BlockManager stopped
18/02/17 12:03:57 INFO BlockManagerMaster: BlockManagerMaster stopped
18/02/17 12:03:57 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
18/02/17 12:03:57 INFO SparkContext: Successfully stopped SparkContext
18/02/17 12:03:57 INFO ShutdownHookManager: Shutdown hook called
18/02/17 12:03:57 INFO ShutdownHookManager: Deleting directory /tmp/spark-6addf0da-f076-4dd1-a5eb-38dca93a2ad6
hive-site.xml:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>hive.metastore.uris</name>
<value>thrift://localhost:9083</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/hcatalog?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>vm4learning</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>hive.hwi.listen.host</name>
<value>localhost</value>
</property>
<property>
<name>hive.hwi.listen.port</name>
<value>9999</value>
</property>
<property>
<name>hive.hwi.war.file</name>
<value>lib/hive-hwi-0.11.0.war</value>
</property>
</configuration>

How do i control mongo java driver logging using java.util.logging properties

I am using the java mongodb driver 3.2.2 (compile group: 'org.mongodb', name: 'mongo-java-driver', version:'3.2.2) and can't seem to turn OFF the logging that is coming from the driver.
My program is as follows:
public static void main(String args[]) {
Enumeration<String> names = LogManager.getLogManager().getLoggerNames();
Logger l = Logger.getLogger( "org.mongodb.driver" );
l.info("Hello INFO!");
l.warning("Hello WARNING!");
SoundDB db = new SoundDB();
db.doMain(args);
while (names.hasMoreElements())
System.out.println("Name = " + names.nextElement());
l.info("Hello INFO!");
l.warning("Hello WARNING!");
}
and when started with -Djava.util.logging.config.file=logging.properties, produces
Oct 12, 2016 7:44:22 PM com.ibm.watson.iot.sound.tools.SoundDB main
WARNING: Hello WARNING!
Loading caa properties from file:/C:/Users/IBM_ADMIN/git/iot-sound/IoT-Sound/caa.properties
19:44:23.889 [main] INFO org.mongodb.driver.cluster - Cluster created with settings {hosts=[localhost:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
19:44:23.971 [main] DEBUG org.mongodb.driver.cluster - Updating cluster description to {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]
19:44:24.030 [main] INFO org.mongodb.driver.cluster - No server chosen by ReadPreferenceServerSelector{readPreference=primary} from cluster description ClusterDescription{type=UNKNOWN, connectionMode=SINGLE, all=[ServerDescription{address=localhost:27017, type=UNKNOWN, state=CONNECTING}]}. Waiting for 30000 ms before timing out
19:44:24.042 [cluster-ClusterId{value='57fecad73df6efadcc807d9e', description='null'}-localhost:27017] INFO org.mongodb.driver.connection - Opened connection [connectionId{localValue:1, serverValue:1261}] to localhost:27017
19:44:24.042 [cluster-ClusterId{value='57fecad73df6efadcc807d9e', description='null'}-localhost:27017] DEBUG org.mongodb.driver.cluster - Checking status of localhost:27017
19:44:24.044 [cluster-ClusterId{value='57fecad73df6efadcc807d9e', description='null'}-localhost:27017] INFO org.mongodb.driver.cluster - Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 2, 4]}, minWireVersion=0, maxWireVersion=4, maxDocumentSize=16777216, roundTripTimeNanos=1672627}
19:44:24.046 [cluster-ClusterId{value='57fecad73df6efadcc807d9e', description='null'}-localhost:27017] DEBUG org.mongodb.driver.cluster - Updating cluster description to {type=STANDALONE, servers=[{address=localhost:27017, type=STANDALONE, roundTripTime=1.7 ms, state=CONNECTED}]
...
Name = javax.management.monitor
Name = javax.management.mlet
Name = org.bson.ObjectId
Name = global
Name = org.mongodb.driver
Name = javax.management
Name = javax.management.mbeanserver
Name =
Oct 12, 2016 7:44:24 PM com.ibm.watson.iot.sound.tools.SoundDB main
WARNING: Hello WARNING!
logging.properties contains
.level=WARNING
handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.ConsoleHandler.level=ALL
The org.mongodb.driver logger has its level set correctly to WARNING since only my warning messages are being printed out and not the info messages. There is no change (as I would expect) if I add the following to the properties:
org.bson.ObjectId.level=WARNING
org.mongodb.driver.level=WARNING
So, does anyone have any idea what I'm doing wrong? Thanks.
From: http://mongodb.github.io/mongo-java-driver/3.2/driver/reference/management/logging/
"By default, logging is enabled via the popular SLF4J API. The use of SLF4J is optional; the driver will use SLF4J if the driver detects the presence of SLF4J in the classpath. Otherwise, the driver will fall back to JUL (java.util.logging)"
Make sure that you have no slf4j dependency in your classpath (directly or through other libs). In case you have slf4j you need to configure slf4j instead of java logging to set-up log level.
slf4j is just logging API, actual logging could be backed by any implementation (JUG, Log4J, logback). See https://dzone.com/articles/how-configure-slf4j-different for additional info. What is actually using is depends on your classpath. If you use maven you could find it by getting dependency hierarchy.

404 Error while deploying simple web-app in JBoss AS 6 and JBoss AS 7?

I followed this blog for injecting EJB in REST layer.
Here is the code that I tried deploying in JBOSS AS 6 and 7 using Eclipse:
REST:
package com.example.rest;
import javax.ejb.EJB;
import javax.ejb.Stateless;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
#Stateless
#Path("current")
public class ServiceFacade {
#EJB
ServiceImpl service;
#GET
public String getDate(){
return service.getCurrentDate().toString();
}
}
EJB:
import java.util.Date;
import javax.ejb.Stateless;
#Stateless
public class ServiceImpl {
public Date getCurrentDate(){
return new Date();
}
}
#ApplicationPath("rest")
public class RestApplication extends Application {
}
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.sample.rest</groupId>
<artifactId>restejb</artifactId>
<version>0.0.1-SNAPSHOT</version>
<description>simplet project to test ejb injection in rest</description>
<dependencies>
<dependency>
<groupId>org.jboss.spec</groupId>
<artifactId>jboss-javaee-6.0</artifactId>
<version>1.0.0.Final</version>
<packaging>war</packaging>
<type>pom</type>
<scope>provided</scope>
</dependency>
</dependencies>
</project>
when I access http://localhost:8080/restejb/rest/current, I get 404 page NOT found error.
Here is the log from deployment to JBOSS AS 6:
11:19:23,701 INFO [AbstractJBossASServerBase] Server Configuration:
JBOSS_HOME URL: file:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/
Bootstrap: $JBOSS_HOME/server/default/conf/bootstrap.xml
Common Base: $JBOSS_HOME/common/
Common Library: $JBOSS_HOME/common/lib/
Server Name: default
Server Base: $JBOSS_HOME/server/
Server Library: $JBOSS_HOME/server/default/lib/
Server Config: $JBOSS_HOME/server/default/conf/
Server Home: $JBOSS_HOME/server/default/
Server Data: $JBOSS_HOME/server/default/data/
Server Log: $JBOSS_HOME/server/default/log/
Server Temp: $JBOSS_HOME/server/default/tmp/
11:19:23,706 INFO [AbstractServer] Starting: JBossAS [6.1.0.Final "Neo"]
11:19:27,412 INFO [ServerInfo] Java version: 1.7.0_71,Oracle Corporation
11:19:27,412 INFO [ServerInfo] Java Runtime: Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
11:19:27,413 INFO [ServerInfo] Java VM: Java HotSpot(TM) 64-Bit Server VM 24.71-b01,Oracle Corporation
11:19:27,413 INFO [ServerInfo] OS-System: Mac OS X 10.9.5,x86_64
11:19:27,414 INFO [ServerInfo] VM arguments: -Dprogram.name=JBossTools: JBoss AS 6.x -Xms256m -Xmx768m -XX:MaxPermSize=256m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.endorsed.dirs=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/lib/endorsed -Djava.library.path=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/bin/native -Dlogging.configuration=file:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/bin/logging.properties -Dfile.encoding=UTF-8
11:19:27,483 INFO [JMXKernel] Legacy JMX core initialized
11:19:35,849 INFO [AbstractServerConfig] JBoss Web Services - Stack CXF Server 3.4.1.GA
11:19:36,679 INFO [JSFImplManagementDeployer] Initialized 3 JSF configurations: [Mojarra-1.2, MyFaces-2.0, Mojarra-2.0]
11:19:47,865 WARNING [FileConfigurationParser] AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
11:19:48,389 INFO [JMXConnector] starting JMXConnector on host localhost:1090
11:19:48,609 INFO [MailService] Mail Service bound to java:/Mail
11:19:49,990 INFO [HornetQServerImpl] live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/data/hornetq/journal,bindingsDirectory=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/data/hornetq/bindings,largeMessagesDirectory=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/data/hornetq/largemessages,pagingDirectory=/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/data/hornetq/paging)
11:19:49,992 INFO [HornetQServerImpl] Waiting to obtain live lock
11:19:50,105 INFO [JournalStorageManager] Using NIO Journal
11:19:50,140 WARNING [HornetQServerImpl] Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.
11:19:50,468 INFO [FileLockNodeManager] Waiting to obtain live lock
11:19:50,469 INFO [FileLockNodeManager] Live Server Obtained live lock
11:19:51,283 INFO [NettyAcceptor] Started Netty Acceptor version 3.2.3.Final-r${buildNumber} localhost:5445 for CORE protocol
11:19:51,287 INFO [NettyAcceptor] Started Netty Acceptor version 3.2.3.Final-r${buildNumber} localhost:5455 for CORE protocol
11:19:51,290 INFO [HornetQServerImpl] Server is now live
11:19:51,291 INFO [HornetQServerImpl] HornetQ Server version 2.2.5.Final (HQ_2_2_5_FINAL_AS7, 121) [251821f6-c6bb-11e4-9df3-60334b2115c1] started
11:19:51,386 INFO [WebService] Using RMI server codebase: http://localhost:8083/
11:19:51,699 INFO [jbossatx] ARJUNA-32010 JBossTS Recovery Service (tag: JBOSSTS_4_14_0_Final) - JBoss Inc.
11:19:51,711 INFO [arjuna] ARJUNA-12324 Start RecoveryActivators
11:19:51,745 INFO [arjuna] ARJUNA-12296 ExpiredEntryMonitor running at Tue, 10 Mar 2015 11:19:51
11:19:51,903 INFO [arjuna] ARJUNA-12310 Recovery manager listening on endpoint 127.0.0.1:4712
11:19:51,904 INFO [arjuna] ARJUNA-12344 RecoveryManagerImple is ready on port 4712
11:19:51,905 INFO [jbossatx] ARJUNA-32013 Starting transaction recovery manager
11:19:51,933 INFO [arjuna] ARJUNA-12163 Starting service com.arjuna.ats.arjuna.recovery.ActionStatusService on port 4713
11:19:51,934 INFO [arjuna] ARJUNA-12337 TransactionStatusManagerItem host: 127.0.0.1 port: 4713
11:19:51,937 INFO [arjuna] ARJUNA-12170 TransactionStatusManager started on port 4713 and host 127.0.0.1 with service com.arjuna.ats.arjuna.recovery.ActionStatusService
11:19:52,007 INFO [jbossatx] ARJUNA-32017 JBossTS Transaction Service (JTA version - tag: JBOSSTS_4_14_0_Final) - JBoss Inc.
11:19:52,103 INFO [arjuna] ARJUNA-12202 registering bean jboss.jta:type=ObjectStore.
11:19:52,481 INFO [AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/bin/native
11:19:52,764 INFO [TomcatDeployment] deploy, ctxPath=/invoker
11:19:53,207 INFO [ModClusterService] Initializing mod_cluster 1.1.0.Final
11:19:53,286 INFO [RARDeployment] Required license terms exist, view vfs:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/deploy/jboss-local-jdbc.rar/META-INF/ra.xml
11:19:53,308 INFO [RARDeployment] Required license terms exist, view vfs:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/deploy/jboss-xa-jdbc.rar/META-INF/ra.xml
11:19:53,322 INFO [RARDeployment] Required license terms exist, view vfs:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/deploy/jms-ra.rar/META-INF/ra.xml
11:19:53,347 INFO [HornetQResourceAdapter] HornetQ resource adaptor started
11:19:53,359 INFO [RARDeployment] Required license terms exist, view vfs:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/deploy/mail-ra.rar/META-INF/ra.xml
11:19:53,382 INFO [RARDeployment] Required license terms exist, view vfs:/Users/sridhar1982AQ/Documents/EE7_servers/jboss-6.1.0.Final/server/default/deploy/quartz-ra.rar/META-INF/ra.xml
11:19:53,522 INFO [SimpleThreadPool] Job execution threads will use class loader of thread: Thread-2
11:19:53,574 INFO [SchedulerSignalerImpl] Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
11:19:53,575 INFO [QuartzScheduler] Quartz Scheduler v.1.8.3 created.
11:19:53,579 INFO [RAMJobStore] RAMJobStore initialized.
11:19:53,583 INFO [QuartzScheduler] Scheduler meta-data: Quartz Scheduler (v1.8.3) 'JBossQuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
11:19:53,583 INFO [StdSchedulerFactory] Quartz scheduler 'JBossQuartzScheduler' initialized from an externally opened InputStream.
11:19:53,583 INFO [StdSchedulerFactory] Quartz scheduler version: 1.8.3
11:19:53,584 INFO [QuartzScheduler] Scheduler JBossQuartzScheduler_$_NON_CLUSTERED started.
11:19:54,133 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=DefaultDS' to JNDI name 'java:DefaultDS'
11:19:54,533 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' to JNDI name 'java:JmsXA'
11:19:54,715 INFO [xnio] XNIO Version 2.1.0.CR2
11:19:54,733 INFO [nio] XNIO NIO Implementation Version 2.1.0.CR2
11:19:55,100 INFO [remoting] JBoss Remoting version 3.1.0.Beta2
11:19:55,279 INFO [TomcatDeployment] deploy, ctxPath=/
11:19:55,417 INFO [HornetQServerImpl] trying to deploy queue jms.queue.ExpiryQueue
11:19:55,462 INFO [HornetQServerImpl] trying to deploy queue jms.queue.DLQ
11:19:55,508 INFO [service] Removing bootstrap log handlers
11:19:55,616 INFO [org.apache.coyote.http11.Http11Protocol] Starting Coyote HTTP/1.1 on http-localhost%2F127.0.0.1-8080
11:19:55,624 INFO [org.apache.coyote.ajp.AjpProtocol] Starting Coyote AJP/1.3 on ajp-localhost%2F127.0.0.1-8009
11:19:55,625 INFO [org.jboss.bootstrap.impl.base.server.AbstractServer] JBossAS [6.1.0.Final "Neo"] Started in 31s:909ms
11:19:56,100 INFO [org.jboss.web.tomcat.service.deployers.TomcatDeployment] deploy, ctxPath=/restejb
I am checking one of my projects using rest and in the annotation I use a slash / before the resource name both in #ApplicationPath("/rest") as in the rest service `#Path("/current"), so your EJB seems like must be something like:
#Stateless
#Path("/current")
public class ServiceImpl {
#GET
public Date getCurrentDate(){
return new Date();
}
}
And your activator class something like:
#ApplicationPath("/rest")
public class RestApplication extends Application {
}